1. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t COOL Performance Tests ATLAS Conditions Database example Romain Basset, IT-DM October 2008

2. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Outline Description –ATLAS conditions database –Different performance tests Database population Results Conclusion

3. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Description ATLAS COOL reference workloads Source : https://twiki.cern.ch/twiki/bin/view/Atlas/CoolRefWork, R. Hawkings.https://twiki.cern.ch/twiki/bin/view/Atlas/CoolRefWork Folder(s) N-Folders N-channels Data #per run kB/run ID/POOL/POOLDIR_n232string(160 char)110 CALO/POOL/POOLDIR_n1732string(160 char)187 MUON/MDT/RT11174CLOB 4.5k0.1500 MUON/MDT/T011174CLOB 3k0.1350 MUON/POOL/POOLDIR132string(160 char)15 GLOBAL/DETSTATUS/TISUMM1503 floats63.6 TDAQ/DCS/TESTDCS_n1020025 floats122400 TDAQ/DCS/TESTDCS_n5100025 floats126000

4. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Performance tests Two main cases : –Insertion from online or from calibration jobs –Data readback from reconstruction jobs Two types of data in the DB schema : –Simple version folders (DCS) –Multi-version folders (tags, user tags) The tests should cover both cases with a mix of both types of data.

5. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Performance tests 2 other questions with performance implications: –Is the Oracle partitioning a possible solution to ease data management in few years ? –Is index compression a good solution to decrease the size of the DB ? To answer these questions, we will also compare the performance of partitioned tables and compressed index to the normal schema.

6. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Database population A test DB had to be generated from scratch. The AthenaDBTestRec package provided in the ATLAS software was used to insert and retrieve the data. Several clients were running in parallel, but it was not possible to split the work for the DCS data. The duplicates for partitioned tables and index compression were created using Data Pump to accelerate the process.

7. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Test datasets List of the different datasets : Data typeAmount (year)Size (GB)Rows ATLAS COOL DB1154- DCS12460,000,000 DCS5120300,000,000 DCS10230600,000,000 DCS partitioned w/ global index10211600,000,000 DCS partitioned w/ local index10260600,000,000 DCS w/ comp. index10200600,000,000 MV w/ CLOB10.51,100,000 MV w/ CLOB52.55,500,000 MV w/ CLOB10511,000,000 MV CLOB w/ comp. index103.711,000,000

8. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t 10-year dataset : SV folder The equivalent of a 10 years old SV folder was populated in about 5 days.

9. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Inserting in a SV folder As seen in previous slide, insertion in a SV folder remains stable over time. Only 1% increase after 10 years.

10. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t 10-year dataset : MV folder The equivalent of a 10 years old MV folder was populated in about 2 days.

11. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Inserting in a MV folder As seen in previous slide, insertion in a MV folder remains stable over time. The increase is less than 2% after 10 years.

12. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Data retrieval in SV folder

13. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Data retrieval in MV folder

14. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Table partitioning To ease the data management on the long term, we proposed to use Oracle partitioning (eg: to put old data offline). The partitioning should be transparent to the software to reduce the implementation cost. Several problems were found with this idea (IOV overlap…) as well as performance issues.

15. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Partitioning problem Readback test on a partitioned table

16. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Conclusion The tests show no performance problem when scaling up to a 10-year dataset. The parallel readback tests are the next in line. The preliminary tests by R. Hawkings are good so we are confident. Partitioning and index compression are still being tested.

17. CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Questions ? / Discussion ?