1. 1 Chapter 17 Troubleshooting RMAN

2. 2 Background Authors thought this topic was often glazed over or not covered well Knew that every topic can’t be covered, that could be a very long list Tried to cover the most common types of RMAN issues

3. 3 Sources of Information Backup and recovery problems are stressful Usually you can find somebody else who has already encountered the issue Start at bottom of error stack and work your way up Syntax issues are common Check alert.log and trace files

4. 4 Sources of Information (continued) Ask for help Search Oracle’s documentation Use RMAN forum Open a SR with Oracle Enable output logging (Oracle Support will often request this)

5. 5 Resolving Connection Issues Common problem Usually caused by not using an authenticated user If using a password file, ensure correct username/password are being used Attempt to connect to SQL*Plus to troubleshoot

6. 6 Handling Disk Space Issues Usually caused by running out of disk space To correct: – Change location of the backup – Add disk space (if possible) – If using a FRA, either add space or move – Change retention policy – Delete old files

7. 7 Dealing with RMAN-06059 Error Common problem, occurs when RMAN doesn’t know where old archive redo log files are located Due to switching from user managed to RMAN Due to not using RMAN to delete old files “loss of archived log compromises recoverability”... Yikes! RMAN> crosscheck archivelog all;

8. 8 Terminating RMAN Processes Sometimes necessary when backup or restore hangs Try control + C Manually kill process from operating system Kill RMAN session from SQL*Plus Query v$session_longops to see if the job is actually making progress

9. 9 Diagnosing NLS Character Set Issues ORA-12705 error Sometimes caused by a mismatch between NLS character set on client and server Sometimes caused by an erroneously set NLS-related operating system parameter SQL> select value from v$nls_parameters where parameter = 'NLS_CHARACTERSET';

10. 10 Logging RMAN Output Extremely helpful for debugging/diagnosing issues From OS prompt: $ rman target / log=rman_output.log From within RMAN: RMAN> spool log to rman_output.log RMAN> set echo on RMAN> backup database; RMAN> spool log off;

11. 11 Viewing RMAN Command History Use v$rman_output SQL> select sid, recid, output from v$rman_output order by recid; Holds 32,768 rows

12. 12 Enabling Debug Output You can turn on debugging in several different ways: – From the OS prompt – When allocating a channel – When configuring a channel – From the RMAN command-line prompt

13. 13 Enabling Debug Output Enable all debugging $ rman target / debug=all log=rman_output.log Enables debugging just for I/O activities: $ rman target / debug=io

14. 14 Information Types of Debugging io sql plsql rcvman rpc

15. 15 Enabling Granular Time Reporting Oracle Support will often recommend that you enable this before sending them output Allows you to view down to the second when operations occurred Authors recommend that you always enable this Korn shell example: $ export NLS_DATE_FORMAT='dd-mon-yyyy hh24:mi:ss'

16. 16 Working with Oracle Support Some problems will require help from Oracle Support To facilitate response: – Test case that illustrates step-by-step details to reproduce the problem – Complete RMAN script or command(s) that were run – Complete RMAN output log – Database alert.log file – Remote Diagnostic Agent (RDA) output (optional)

17. 17 Resolving RMAN Compatibility Issues Ideally, all databases and catalog will be at the same level Reality, almost never the case Most up to date source RMAN compatibility matrix information in MetaLink note 73431.1 Checking the recovery catalog version (if using): SQL> select * from rcver;

18. 18 Dealing with an ORA-19511 Error Vexing issue Caused by MML mis-configuration Caused by incorrectly set OS variables Work with your MML vendor to resolve Check sbtio.log file

19. 19 Dealing with an ORA-27211 Error Vexing issue Usually related to MML not being correctly installed Work with MML vendor to resolve Look in log and trace files for more information

20. 20 Dealing with an ORA-04031 Error Enabled I/O slaves RMAN will throw this if it can’t allocate enough memory Setup a large pool area Use ASMM See MetaLink note 73354.1

21. 21 Managing Files in an ASM Environment Background: Technical editor wrote this material in the book, often received questions regarding ASM at Oracle Support Using ASM (Automatic Storage Management) Need to view RMAN files Use ASMCMD Command line utility $ asmcmd –p Unix like commands