CERN IT Department CH-1211 Genève 23 Switzerland t Load testing & benchmarks on Oracle RAC Romain Basset – IT PSS DP

CERN IT Department CH-1211 Genève 23 Switzerland t Load testing & Benchmarking Framework for Oracle RAC - 2 Outline Motivations DB load testing & benchmarks –Existing solutions The application –Description –Examples Conclusion

CERN IT Department CH-1211 Genève 23 Switzerland t Motivations Database workload is rising until LHC startup. → Database architecture should be tested to study performance and availability issues. Internal software (COOL, CORAL) has to be tested in production-like environment. Load testing & Benchmarking Framework for Oracle RAC - 3

CERN IT Department CH-1211 Genève 23 Switzerland t Atlas example Load testing & Benchmarking Framework for Oracle RAC - 4

CERN IT Department CH-1211 Genève 23 Switzerland t DB load testing & benchmarks To ensure the quality of service, three main aspects should be looked at : –Performance –Scalability –Reliability Benchmarking is focusing on the first two. Load testing aims at proving the last one. Load testing & Benchmarking Framework for Oracle RAC - 5

CERN IT Department CH-1211 Genève 23 Switzerland t TPC TPC : Transaction Processing Performance Council Benchmark specifications –TPC-C : Order-Entry system (OLTP) select, update, insert & delete –TPC-H : Decision support system complex select queries & few insert/delete statements No out-of-the-box implementation Simulated behaviors do not match with our use cases Load testing & Benchmarking Framework for Oracle RAC - 6

CERN IT Department CH-1211 Genève 23 Switzerland t HammerORA Tcl/Tk open source software Possibility to replay a workload from Oracle trace files (10046) Load testing Implementation of TPC-like benchmarks –TPC-C and TPC-H Master/Slave module out of order No real monitoring / reporting feature No C++ support Load testing & Benchmarking Framework for Oracle RAC - 7

CERN IT Department CH-1211 Genève 23 Switzerland t Swingbench Free load generator & benchmarks Implementation of OLTP-like benchmarks Comprehensive graphical toolset for performance metrics and monitoring Distributed execution Not open source No C++ support Load testing & Benchmarking Framework for Oracle RAC - 8

CERN IT Department CH-1211 Genève 23 Switzerland t SwingBench Load testing & Benchmarking Framework for Oracle RAC - 9

CERN IT Department CH-1211 Genève 23 Switzerland t Requirements Possibility to plug new tests (C++) Distributed execution Monitoring feature Performance report for analysis Load testing & Benchmarking Framework for Oracle RAC - 10

CERN IT Department CH-1211 Genève 23 Switzerland t Technologies Python AFS SSH XML HTML, PHP, JPGraph CORAL LEMON Oracle Load testing & Benchmarking Framework for Oracle RAC - 11

CERN IT Department CH-1211 Genève 23 Switzerland t Typical scenario 1.Spawn the clients 2.Query the DB 3.Write results on files (on AFS) 4.Gather results 5.Query the DB for statistics Load testing & Benchmarking Framework for Oracle RAC - 12

CERN IT Department CH-1211 Genève 23 Switzerland t Configuration Configuration file : - node1 node2 node3 server1 server2 - Object inserted Memory usage in MB - client.py testConf.xml Authentication.xml (CORAL): Dblookup.xml (CORAL): - Load testing & Benchmarking Framework for Oracle RAC - 13

CERN IT Department CH-1211 Genève 23 Switzerland t Monitoring (RAC CPU) RAC CPU monitoring Get the values from Oracle : GV$SYSMETRIC Load testing & Benchmarking Framework for Oracle RAC - 14

CERN IT Department CH-1211 Genève 23 Switzerland t Monitoring (wait events) Average active sessions (OEM style) Get the values from Oracle : GV$WAITCLASSMETRIC Load testing & Benchmarking Framework for Oracle RAC - 15

CERN IT Department CH-1211 Genève 23 Switzerland t LEMON LEMON plots for client and server nodes –Cpu –Load –Network (in & out) Linked or downloaded from LEMON web pages Load testing & Benchmarking Framework for Oracle RAC - 16

CERN IT Department CH-1211 Genève 23 Switzerland t Performance (client) Metric measured by the clients Get the values from the log files Load testing & Benchmarking Framework for Oracle RAC - 17

CERN IT Department CH-1211 Genève 23 Switzerland t Performance (aggregated) Aggregated measures from the clients Get the values from the log files Number of sessions (GV$SESSION) Load testing & Benchmarking Framework for Oracle RAC - 18

CERN IT Department CH-1211 Genève 23 Switzerland t Possibility to write new tests No specific language No specific metrics –Dynamic, time dependant (throughput) –Static, time independant (response time) Specific input –Client ID –Result file path –Result output frequency Specific output (CSV file) –Timestamp –Time elapsed since last result –Values for the different metrics Load testing & Benchmarking Framework for Oracle RAC - 19

CERN IT Department CH-1211 Genève 23 Switzerland t Examples CORAL : generic relational database access API for Physics applications COOL based on CORAL, defines a schema and provides C++ API to store/retrieve contitions data Load testing & Benchmarking Framework for Oracle RAC - 20

CERN IT Department CH-1211 Genève 23 Switzerland t Examples COOL performance tests –Insertions : storeObject() –Retrievals : findObjects(), browseObjects() –Benchmarking (using different COOL versions) –Tests on production sized dataset (using 1TB database ~1-year Atlas PVSS data) Stress testing using COOL insertion test –10 clients –4-node RAC –1TB of data in ~10 hours Load testing & Benchmarking Framework for Oracle RAC - 21

CERN IT Department CH-1211 Genève 23 Switzerland t Conlusion Tests already conducted : –Performance when changing Oracle configuration –Performance in COOL package –Stress testing on 11g (by Balys) Tutorial available (on PSS wiki) Future work –Comparison of 2 test runs (new report) –Integration with COOL performance suite –Automatic checking of performance issues before a software release Load testing & Benchmarking Framework for Oracle RAC - 22

CERN IT Department CH-1211 Genève 23 Switzerland t References Pro Oracle Database 10g RAC on Linux, J. Dyke and S. Shaw, chap. 5 & 23. TPC, HammerORA, SwingBench, CORAL, COOL, Oracle Load Testing Tools Documentation, Tutorial, Load testing & Benchmarking Framework for Oracle RAC - 23