2 Copyright © 2007, Oracle. All rights reserved. Realistic Testing Using Database Replay

Copyright © 2007, Oracle. All rights reserved Objectives After completing this lesson, you should be able to: Identify the benefits of using Database Replay List the steps involved in Database Replay Use Enterprise Manager to record and replay workloads

Copyright © 2007, Oracle. All rights reserved Managing Change in Oracle Database 11g Oracle Database 11g dramatically reduces the cost of system changes by providing: Database Replay –Captures actual database workload on the production system –Replays the captured workload (with same concurrency) on a test system –Identifies performance changes and errors SQL Performance Analyzer –Identifies SQL regressions and fixes them

Copyright © 2007, Oracle. All rights reserved Managing Change Effectively Confidence in the success of a system change in minimizing system disruption Ease of upgrade to a different RDBMS release or server configuration Performing capacity planning Recording and replaying sessions when debugging issues Developing autonomic features on real-life workloads Preventing degradations that are potentially caused by a change in the production environment

Copyright © 2007, Oracle. All rights reserved Why Use Database Replay? System changes such as hardware and software upgrades are a fact of life. Customers want to identify the full impact of a change before going live. Extensive testing and validation can be expensive in time and money. Despite expensive testing, success rates can be low: –Many issues may go undetected. –Changes can impact system availability and performance negatively. Database Replay makes it possible to test with real-world production workloads.

Copyright © 2007, Oracle. All rights reserved Database Replay Re-create actual production database workload in test environment. Identify, analyze, and fix potential instabilities before making changes to production. Capture workload in production: –Capture full production workload with real load and concurrency. –Move the captured workload to test system. Replay workload in test: –Make the desired changes in test system. –Replay workload with production load and concurrency. –Honor commit ordering. Analyze and report: –Errors –Data divergence –Performance divergence

Copyright © 2007, Oracle. All rights reserved System Architecture: Capture Database stack Production database Database backup Capture directory Recording infrastructure Shadow Shadow capture file Background

Copyright © 2007, Oracle. All rights reserved System Architecture: Processing the Workload Database stack Production database Database backup Capture directory Shadow capture file Process capture files Background Process capture

Copyright © 2007, Oracle. All rights reserved System Architecture: Replay Database stack Test database Database backup Capture directory Shadow capture file Process capture files Replay client Shadow … … Background Test system with changes Replay system

Copyright © 2007, Oracle. All rights reserved The Big Picture Production database Database backup Capture directory Shadow capture file Process capture files Database restore Pre-change production system Post-change test system Replay system Can use snapshot standby as test system Test system with changes Production system Clients/app servers

Copyright © 2007, Oracle. All rights reserved Pre-Change Production System Supported changes: Database upgrades, patches Schema, parameters RAC nodes, interconnect OS platforms, OS upgrades CPU, memory Storage Changes not supported Production database Production system Clients/app servers

Copyright © 2007, Oracle. All rights reserved Workloads Supported Supported –All SQL (DML, DDL, PLSQL) with practically all types of binds –Full LOB functionality (cursor based and direct OCI) –Local transactions –Login/Logoffs –Session switching –Limited PLSQL calls (database interaction) Limitations –Direct path load, import/export –OCI based object navigation (ADTs) and REF binds –Streams, non-PLSQL based AQ –Distributed transactions, remote describe, or commit operations

Copyright © 2007, Oracle. All rights reserved Capture Considerations The goal of capture planning is to ensure that: The database can be restored to the SCN when capture starts ( StartSCN ) The system has enough resources for the capture You find a suitable period for restarting the database before capture starts (if selected) You can specify filters to capture a subset of the workload Users have SYSDBA or SYSOPER privileges and appropriate OS privileges

Copyright © 2007, Oracle. All rights reserved Replay Considerations Preprocess captured workload: –One-time action –On same database version as replay –Can be performed anywhere (production, test system, or other system) if versions match Restore database and then perform the change: –Upgrade –Schema changes –OS change –Hardware change –Add instance

Copyright © 2007, Oracle. All rights reserved Replay Considerations Manage external interactions: –Remap connection strings to be used for the workload — One-to-one: Allows simple instance-to-instance remapping — Many-to-one: Use of load balancer (single node to RAC) –Modify external references (such as database links and directory objects) that point to production systems Set up one or more replay clients: –Multithreaded clients that can each drive multiple workload sessions

Copyright © 2007, Oracle. All rights reserved Replay Analysis Data divergence –Number of rows compared for each call (queries, DMLs) Error divergence –New errors –Mutated errors –Errors disappear Performance –Capture and replay report –ADDM report –ASH report for skew analysis –AWR report

Copyright © 2007, Oracle. All rights reserved Replay Data Divergence Workload characteristics that increase data or error divergence: Implicit session dependencies by the application (for example, the use of dbms_pipe ) Extensive use of multiple commits in PL/SQL User locks Use of nonrepeatable functions or system-dependent data External interactions via URLs or database links

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Capture Start Raw captured data Workload recording Plan Shutdown - startup restricted Capture...

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Capture

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Capture

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Capture

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Capture

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay End Raw captured data (from production system) Workload replay Process capture Prepare Replay Analyze Initialize replay Replay report Replay files & metadata

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay References to resolve:

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay %> wrc userid= password= server= replaydir= workdir= $ wrc REPLAYDIR=/home/oracle/solutions/dbreplay USERID=system PASSWORD=oracle Workload Replay Client: Release Production on Tue … Copyright (c) 1982, 2007, Oracle. All rights reserved. Wait for the replay to start (21:47:01) Replay started (21:48:14)

Copyright © 2007, Oracle. All rights reserved Using Enterprise Manager for Workload Replay

Copyright © 2007, Oracle. All rights reserved Database Replay: PL/SQL Example exec DBMS_WORKLOAD_CAPTURE.ADD_FILTER(fname => 'sessfilt',- fattribute => USER,- fvalue => 'JFV'); exec DBMS_WORKLOAD_CAPTURE.START_CAPTURE(name => 'june_peak',- dir => 'jun07'); Execute your workload exec DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE(); exec DBMS_WORKLOAD_REPLAY.PROCESS_CAPTURE(capture_dir => 'jun07'); wrc userid=system password=oracle replaydir=/dbreplay exec DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY(replay_name => 'j_r',- replay_dir => 'jun07');

Copyright © 2007, Oracle. All rights reserved Database Replay: PL/SQL Example exec DBMS_WORKLOAD_REPLAY.REMAP_CONNECTION(connection_id => 101,- replay_connection => 'edlin44:3434/bjava21'); exec DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY(synchronization => TRUE,- think_time_scale=> 2); exec DBMS_WORKLOAD_REPLAY.START_REPLAY (); DECLARE cap_id NUMBER; rep_id NUMBER; rep_rpt CLOB; BEGIN cap_id := DBMS_WORKLOAD_REPLAY.GET_REPLAY_INFO(dir => 'jun07'); /* Get the latest replay for that capture */ SELECT max(id) INTO rep_id FROM dba_workload_replays WHERE capture_id = cap_id; rep_rpt := DBMS_WORKLOAD_REPLAY.REPORT(replay_id => rep_id, format => DBMS_WORKLOAD_REPLAY.TYPE_TEXT); END;

Copyright © 2007, Oracle. All rights reserved Calibrating Replay Clients $ wrc mode=calibrate replaydir=/dbreplay Workload Replay Client: Release Beta on Tue … Copyright (c) 1982, 2007, Oracle. All rights reserved. Report for Workload in: /dbreplay Recommendation: Consider using at least 1 clients divided among 1 CPU(s). Workload Characteristics: - max concurrency: 4 sessions - total number of sessions: 11 Assumptions: - 1 client process per 50 concurrent sessions - 4 client process per CPU - think time scale = connect time scale = synchronization = TRUE $

Copyright © 2007, Oracle. All rights reserved Packages and Procedures DBMS_WORKLOAD_CAPTURE – START_CAPTURE – FINISH_CAPTURE – ADD_FILTER – DELETE_FILTER – DELETE_CAPTURE_INFO – GET_CAPTURE_INFO() – EXPORT_AWR – IMPORT_AWR() – REPORT() DBMS_WORKLOAD_REPLAY – PROCESS_CAPTURE – INITIALIZE_REPLAY – PREPARE_REPLAY – START_REPLAY – CANCEL_REPLAY – DELETE_REPLAY_INFO – REMAP_CONNECTION – EXPORT_AWR – IMPORT_AWR – GET_REPLAY_INFO – REPORT

Copyright © 2007, Oracle. All rights reserved Data Dictionary Views: Database Replay DBA_WORKLOAD_CAPTURES : Lists all the workload captures performed in the database DBA_WORKLOAD_FILTERS : Lists all the workload filters defined in the database DBA_WORKLOAD_REPLAYS : Lists all the workload replays that have been performed in the database DBA_WORKLOAD_REPLAY_DIVERGENCE: Is used to monitor workload divergence DBA_WORKLOAD_CONNECTION_MAP : Is used to review all connection strings used by workload replays V$WORKLOAD_REPLAY_THREAD : Monitors the status of external replay clients

Copyright © 2007, Oracle. All rights reserved Summary In this lesson, you should have learned how to: Identify the benefits of using Database Replay List the steps involved in Database Replay Use Enterprise Manager to record and replay workloads

Copyright © 2007, Oracle. All rights reserved