RMAN restore of ODA X6-2-HA RAC database
A client that I was supporting asked me to complete a dry run of a database restore on their newly built ODA. Below are the steps that I performed to complete the database restore.
Prerequisites:
I built the RAC database using OAKCLI.
I backed up the database with RMAN incremental level 0 + cumulative level 1.
I dropped the database using SQLPLUS commands vs. OAKCLI as to not remove the ACFS directories for the database.
1. Startup the database instance in NOMOUNT.
RMAN> startup nomount force;
startup failed: ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '/u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/labtest/spfilelabtest.ora'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
starting Oracle instance without parameter file for retrieval of spfile
Oracle instance started
Total System Global Area 1073741824 bytes
Fixed Size 2932632 bytes
Variable Size 415236200 bytes
Database Buffers 650117120 bytes
Redo Buffers 5455872 bytes
2. Restore the spfile from backup taken.
RMAN> restore spfile from '/backup/LABTEST/ctrl_c-1771719914-20180118-00';
Starting restore at 18-JAN-18
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=2 device type=DISK
channel ORA_DISK_1: restoring spfile from AUTOBACKUP /backup/LABTEST/ctrl_c-1771719914-20180118-00
channel ORA_DISK_1: SPFILE restore from AUTOBACKUP complete
Finished restore at 18-JAN-18
3. Set the DBID for the database being restored.
RMAN> set dbid 1771719914;
executing command: SET DBID
4. Restore the controlfile from autobackup.
RMAN> restore controlfile from '/backup/LABTEST/ctrl_c-1771719914-20180118-00';
Starting restore at 18-JAN-18
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=181 instance=labtest1 device type=DISK
channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:02
output file name=/u01/app/oracle/oradata/datastore/labtest/LABTEST/controlfile/o1_mf_f5zk5rxs_.ctl
Finished restore at 18-JAN-18
5. Mount the database with the controlfile that was just restored.
RMAN> alter database mount;
using target database control file instead of recovery catalog
Statement processed
RMAN> list backup summary;
List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
2 B 0 A DISK 17-JAN-18 1 1 NO FULL_011718_172826
6 B A A DISK 17-JAN-18 1 1 NO FULL_011718_172826
7 B A A DISK 17-JAN-18 1 1 NO INCR_011718_220001
8 B 1 A DISK 17-JAN-18 1 1 NO INCR_011718_220001
10 B 1 A DISK 17-JAN-18 1 1 NO INCR_011718_220001
11 B 1 A DISK 17-JAN-18 1 1 NO INCR_011718_220001
12 B A A DISK 17-JAN-18 1 1 NO INCR_011718_220001
13 B F A DISK 18-JAN-18 1 1 NO TAG20180118T151334
RMAN> report schema;
Starting implicit crosscheck backup at 18-JAN-18
allocated channel: ORA_DISK_1
Crosschecked 8 objects
Finished implicit crosscheck backup at 18-JAN-18
Starting implicit crosscheck copy at 18-JAN-18
using channel ORA_DISK_1
Finished implicit crosscheck copy at 18-JAN-18
searching for all files in the recovery area
cataloging files...
cataloging done
List of Cataloged Files
=======================
File Name: /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_14_f6201oyo_.arc
RMAN-06139: WARNING: control file is not current for REPORT SCHEMA
Report of database schema for database with db_unique_name LABTEST
List of Permanent Datafiles
===========================
File Size(MB) Tablespace RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1 0 SYSTEM *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f5zk5vs5_.dbf
2 0 SYSAUX *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f5zk5y1n_.dbf
3 0 UNDOTBS1 *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f5zk5yyz_.dbf
4 0 UNDOTBS2 *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f5zk677x_.dbf
5 0 USERS *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f5zk67kc_.dbf
List of Temporary Files
=======================
File Size(MB) Tablespace Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1 20 TEMP 32767 /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_temp_f5zk5z7r_.tmp
6. Restore the database.
RMAN> restore database;
Starting restore at 18-JAN-18
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f5zk5vs5_.dbf
channel ORA_DISK_1: restoring datafile 00002 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f5zk5y1n_.dbf
channel ORA_DISK_1: restoring datafile 00003 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f5zk5yyz_.dbf
channel ORA_DISK_1: restoring datafile 00004 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f5zk677x_.dbf
channel ORA_DISK_1: restoring datafile 00005 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f5zk67kc_.dbf
channel ORA_DISK_1: reading from backup piece /backup/LABTEST/FULL_LABTEST_1771719914_20180117_03sotse8_1_1.bkp
channel ORA_DISK_1: piece handle=/backup/LABTEST/FULL_LABTEST_1771719914_20180117_03sotse8_1_1.bkp tag=FULL_011718_172826
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:03
Finished restore at 18-JAN-18
7. Recover the database.
RMAN> recover database;
Starting recover at 18-JAN-18
using channel ORA_DISK_1
channel ORA_DISK_1: starting incremental datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
destination for restore of datafile 00001: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f621nm7q_.dbf
destination for restore of datafile 00002: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f621nm8v_.dbf
destination for restore of datafile 00003: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f621nm9n_.dbf
destination for restore of datafile 00004: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f621nmb4_.dbf
destination for restore of datafile 00005: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f621nmbm_.dbf
channel ORA_DISK_1: reading from backup piece /backup/LABTEST/INCR_LABTEST_1771719914_20180117_09soucbg_1_1.bkp
channel ORA_DISK_1: piece handle=/backup/LABTEST/INCR_LABTEST_1771719914_20180117_09soucbg_1_1.bkp tag=INCR_011718_220001
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
starting media recovery
archived log for thread 1 with sequence 8 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_1_8_f603g8f0_.arc
archived log for thread 1 with sequence 9 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_9_f61w5hxq_.arc
archived log for thread 1 with sequence 10 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_10_f61w5jxs_.arc
archived log for thread 1 with sequence 11 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_11_f61ybnd6_.arc
archived log for thread 1 with sequence 12 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_12_f61ybod8_.arc
archived log for thread 1 with sequence 13 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_13_f61zwfo3_.arc
archived log for thread 1 with sequence 14 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_14_f6201oyo_.arc
archived log for thread 2 with sequence 7 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_2_7_f603g8lb_.arc
archived log for thread 2 with sequence 8 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_8_f61ybo5h_.arc
archived log for thread 2 with sequence 9 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_9_f61zwgcx_.arc
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_1_8_f603g8f0_.arc thread=1 sequence=8
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_2_7_f603g8lb_.arc thread=2 sequence=7
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_9_f61w5hxq_.arc thread=1 sequence=9
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_8_f61ybo5h_.arc thread=2 sequence=8
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_10_f61w5jxs_.arc thread=1 sequence=10
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_11_f61ybnd6_.arc thread=1 sequence=11
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_12_f61ybod8_.arc thread=1 sequence=12
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_9_f61zwgcx_.arc thread=2 sequence=9
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_13_f61zwfo3_.arc thread=1 sequence=13
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_14_f6201oyo_.arc thread=1 sequence=14
unable to find archived log
archived log thread=2 sequence=10
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 01/18/2018 15:43:12
RMAN-06054: media recovery requesting unknown archived log for thread 2 with sequence 10 and starting SCN of 683417
8. Attempted to open the database via resetlogs but received an error about block change tracking. Therefore, I disabled block change tracking in order to successfully open the database.
RMAN> alter database open resetlogs;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of sql statement command at 01/18/2018 15:44:11
ORA-19751: could not create the change tracking file
ORA-19750: change tracking file: '/u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/changetracking/o1_mf_f5zm3sf3_.chg'
ORA-27040: file create error, unable to create file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 1
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
RMAN> alter database disable block change tracking;
Statement processed
RMAN> select open_mode from v$database;
OPEN_MODE
--------------------
MOUNTED
RMAN> alter database open;
Statement processed
RMAN> select open_mode from v$database;
OPEN_MODE
--------------------
READ WRITE
9. Shutdown the database.
RMAN> shutdown immediate
database closed
database dismounted
Oracle instance shut down
RMAN> exit
Recovery Manager complete.
10. I checked the status of and started the database via srvctl. Restore completed.
[oracle@dzpubpoda0 ~]$ srvctl status database -d labtest
Instance labtest1 is not running on node dzpubpoda0
Instance labtest2 is not running on node dzpubpoda1
[oracle@dzpubpoda0 ~]$ srvctl start database -d labtest
[oracle@dzpubpoda0 ~]$ srvctl status database -d labtest
Instance labtest1 is running on node dzpubpoda0
Instance labtest2 is running on node dzpubpoda1
[oracle@dzpubpoda0 ~]$ rman target /
Recovery Manager: Release 12.1.0.2.0 - Production on Thu Jan 18 15:47:22 2018
Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.
connected to target database: LABTEST (DBID=1771719914)
RMAN> report schema;
using target database control file instead of recovery catalog
Report of database schema for database with db_unique_name LABTEST
List of Permanent Datafiles
===========================
File Size(MB) Tablespace RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1 700 SYSTEM YES /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f621nm7q_.dbf
2 600 SYSAUX NO /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f621nm8v_.dbf
3 300 UNDOTBS1 YES /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f621nm9n_.dbf
4 200 UNDOTBS2 YES /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f621nmb4_.dbf
5 5 USERS NO /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f621nmbm_.dbf
List of Temporary Files
=======================
File Size(MB) Tablespace Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1 20 TEMP 32767 /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_temp_f621t812_.tmp
RMAN>
Prerequisites:
I built the RAC database using OAKCLI.
I backed up the database with RMAN incremental level 0 + cumulative level 1.
I dropped the database using SQLPLUS commands vs. OAKCLI as to not remove the ACFS directories for the database.
1. Startup the database instance in NOMOUNT.
RMAN> startup nomount force;
startup failed: ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '/u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/labtest/spfilelabtest.ora'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
starting Oracle instance without parameter file for retrieval of spfile
Oracle instance started
Total System Global Area 1073741824 bytes
Fixed Size 2932632 bytes
Variable Size 415236200 bytes
Database Buffers 650117120 bytes
Redo Buffers 5455872 bytes
2. Restore the spfile from backup taken.
RMAN> restore spfile from '/backup/LABTEST/ctrl_c-1771719914-20180118-00';
Starting restore at 18-JAN-18
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=2 device type=DISK
channel ORA_DISK_1: restoring spfile from AUTOBACKUP /backup/LABTEST/ctrl_c-1771719914-20180118-00
channel ORA_DISK_1: SPFILE restore from AUTOBACKUP complete
Finished restore at 18-JAN-18
3. Set the DBID for the database being restored.
RMAN> set dbid 1771719914;
executing command: SET DBID
4. Restore the controlfile from autobackup.
RMAN> restore controlfile from '/backup/LABTEST/ctrl_c-1771719914-20180118-00';
Starting restore at 18-JAN-18
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=181 instance=labtest1 device type=DISK
channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:02
output file name=/u01/app/oracle/oradata/datastore/labtest/LABTEST/controlfile/o1_mf_f5zk5rxs_.ctl
Finished restore at 18-JAN-18
5. Mount the database with the controlfile that was just restored.
RMAN> alter database mount;
using target database control file instead of recovery catalog
Statement processed
RMAN> list backup summary;
List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
2 B 0 A DISK 17-JAN-18 1 1 NO FULL_011718_172826
6 B A A DISK 17-JAN-18 1 1 NO FULL_011718_172826
7 B A A DISK 17-JAN-18 1 1 NO INCR_011718_220001
8 B 1 A DISK 17-JAN-18 1 1 NO INCR_011718_220001
10 B 1 A DISK 17-JAN-18 1 1 NO INCR_011718_220001
11 B 1 A DISK 17-JAN-18 1 1 NO INCR_011718_220001
12 B A A DISK 17-JAN-18 1 1 NO INCR_011718_220001
13 B F A DISK 18-JAN-18 1 1 NO TAG20180118T151334
RMAN> report schema;
Starting implicit crosscheck backup at 18-JAN-18
allocated channel: ORA_DISK_1
Crosschecked 8 objects
Finished implicit crosscheck backup at 18-JAN-18
Starting implicit crosscheck copy at 18-JAN-18
using channel ORA_DISK_1
Finished implicit crosscheck copy at 18-JAN-18
searching for all files in the recovery area
cataloging files...
cataloging done
List of Cataloged Files
=======================
File Name: /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_14_f6201oyo_.arc
RMAN-06139: WARNING: control file is not current for REPORT SCHEMA
Report of database schema for database with db_unique_name LABTEST
List of Permanent Datafiles
===========================
File Size(MB) Tablespace RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1 0 SYSTEM *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f5zk5vs5_.dbf
2 0 SYSAUX *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f5zk5y1n_.dbf
3 0 UNDOTBS1 *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f5zk5yyz_.dbf
4 0 UNDOTBS2 *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f5zk677x_.dbf
5 0 USERS *** /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f5zk67kc_.dbf
List of Temporary Files
=======================
File Size(MB) Tablespace Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1 20 TEMP 32767 /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_temp_f5zk5z7r_.tmp
6. Restore the database.
RMAN> restore database;
Starting restore at 18-JAN-18
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f5zk5vs5_.dbf
channel ORA_DISK_1: restoring datafile 00002 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f5zk5y1n_.dbf
channel ORA_DISK_1: restoring datafile 00003 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f5zk5yyz_.dbf
channel ORA_DISK_1: restoring datafile 00004 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f5zk677x_.dbf
channel ORA_DISK_1: restoring datafile 00005 to /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f5zk67kc_.dbf
channel ORA_DISK_1: reading from backup piece /backup/LABTEST/FULL_LABTEST_1771719914_20180117_03sotse8_1_1.bkp
channel ORA_DISK_1: piece handle=/backup/LABTEST/FULL_LABTEST_1771719914_20180117_03sotse8_1_1.bkp tag=FULL_011718_172826
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:03
Finished restore at 18-JAN-18
7. Recover the database.
RMAN> recover database;
Starting recover at 18-JAN-18
using channel ORA_DISK_1
channel ORA_DISK_1: starting incremental datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
destination for restore of datafile 00001: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f621nm7q_.dbf
destination for restore of datafile 00002: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f621nm8v_.dbf
destination for restore of datafile 00003: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f621nm9n_.dbf
destination for restore of datafile 00004: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f621nmb4_.dbf
destination for restore of datafile 00005: /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f621nmbm_.dbf
channel ORA_DISK_1: reading from backup piece /backup/LABTEST/INCR_LABTEST_1771719914_20180117_09soucbg_1_1.bkp
channel ORA_DISK_1: piece handle=/backup/LABTEST/INCR_LABTEST_1771719914_20180117_09soucbg_1_1.bkp tag=INCR_011718_220001
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
starting media recovery
archived log for thread 1 with sequence 8 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_1_8_f603g8f0_.arc
archived log for thread 1 with sequence 9 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_9_f61w5hxq_.arc
archived log for thread 1 with sequence 10 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_10_f61w5jxs_.arc
archived log for thread 1 with sequence 11 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_11_f61ybnd6_.arc
archived log for thread 1 with sequence 12 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_12_f61ybod8_.arc
archived log for thread 1 with sequence 13 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_13_f61zwfo3_.arc
archived log for thread 1 with sequence 14 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_14_f6201oyo_.arc
archived log for thread 2 with sequence 7 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_2_7_f603g8lb_.arc
archived log for thread 2 with sequence 8 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_8_f61ybo5h_.arc
archived log for thread 2 with sequence 9 is already on disk as file /u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_9_f61zwgcx_.arc
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_1_8_f603g8f0_.arc thread=1 sequence=8
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_17/o1_mf_2_7_f603g8lb_.arc thread=2 sequence=7
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_9_f61w5hxq_.arc thread=1 sequence=9
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_8_f61ybo5h_.arc thread=2 sequence=8
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_10_f61w5jxs_.arc thread=1 sequence=10
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_11_f61ybnd6_.arc thread=1 sequence=11
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_12_f61ybod8_.arc thread=1 sequence=12
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_2_9_f61zwgcx_.arc thread=2 sequence=9
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_13_f61zwfo3_.arc thread=1 sequence=13
archived log file name=/u01/app/oracle/fast_recovery_area/datastore/labtest/LABTEST/archivelog/2018_01_18/o1_mf_1_14_f6201oyo_.arc thread=1 sequence=14
unable to find archived log
archived log thread=2 sequence=10
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 01/18/2018 15:43:12
RMAN-06054: media recovery requesting unknown archived log for thread 2 with sequence 10 and starting SCN of 683417
8. Attempted to open the database via resetlogs but received an error about block change tracking. Therefore, I disabled block change tracking in order to successfully open the database.
RMAN> alter database open resetlogs;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of sql statement command at 01/18/2018 15:44:11
ORA-19751: could not create the change tracking file
ORA-19750: change tracking file: '/u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/changetracking/o1_mf_f5zm3sf3_.chg'
ORA-27040: file create error, unable to create file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 1
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
RMAN> alter database disable block change tracking;
Statement processed
RMAN> select open_mode from v$database;
OPEN_MODE
--------------------
MOUNTED
RMAN> alter database open;
Statement processed
RMAN> select open_mode from v$database;
OPEN_MODE
--------------------
READ WRITE
9. Shutdown the database.
RMAN> shutdown immediate
database closed
database dismounted
Oracle instance shut down
RMAN> exit
Recovery Manager complete.
10. I checked the status of and started the database via srvctl. Restore completed.
[oracle@dzpubpoda0 ~]$ srvctl status database -d labtest
Instance labtest1 is not running on node dzpubpoda0
Instance labtest2 is not running on node dzpubpoda1
[oracle@dzpubpoda0 ~]$ srvctl start database -d labtest
[oracle@dzpubpoda0 ~]$ srvctl status database -d labtest
Instance labtest1 is running on node dzpubpoda0
Instance labtest2 is running on node dzpubpoda1
[oracle@dzpubpoda0 ~]$ rman target /
Recovery Manager: Release 12.1.0.2.0 - Production on Thu Jan 18 15:47:22 2018
Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.
connected to target database: LABTEST (DBID=1771719914)
RMAN> report schema;
using target database control file instead of recovery catalog
Report of database schema for database with db_unique_name LABTEST
List of Permanent Datafiles
===========================
File Size(MB) Tablespace RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1 700 SYSTEM YES /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_system_f621nm7q_.dbf
2 600 SYSAUX NO /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_sysaux_f621nm8v_.dbf
3 300 UNDOTBS1 YES /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs1_f621nm9n_.dbf
4 200 UNDOTBS2 YES /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_undotbs2_f621nmb4_.dbf
5 5 USERS NO /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_users_f621nmbm_.dbf
List of Temporary Files
=======================
File Size(MB) Tablespace Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1 20 TEMP 32767 /u02/app/oracle/oradata/datastore/.ACFS/snaps/labtest/LABTEST/datafile/o1_mf_temp_f621t812_.tmp
RMAN>
Comments