1. AMGA-Bookkeeping Carmine Cioffi Department of Physics, Oxford University UK Metadata Workshop Oxford, 05 July 2006

2. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 2 Outline What is AMGA-Bookkeeping. Why AMGA-Bookkeeping? Logical view of metadata. Performance: –Why AMGA v JDBC (oracke thin driver 10.2). –Test bed peculiarity. –How the test was performed. –Charts. –Comments. Conclusions

3. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 3 What is AMGA-Bookkeeping The AMGA Bookkeeping is an implementation of the LHCB bookkeeping that use AMGA to manage File-Metadata. All the metadata are accessed via gLite/ARDA metadata interface. From the coding point of view the JDBC API are replaced by the AMGA API (with related query translated). All the metadata are not anymore organized in relational table but in a file–system like structure.

4. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 4 Why AMGA- Bookkeeping? AMGA implements the gLite/ARDA interface. Security. Different security protocols available (SSL, Grid Certificate, VOMS). DB replication and DB independence. It can use different DBMS (Oracle, MySQL, PostGres). It is also web service oriented (SOAP). Many client implementations available (C++,Java, Python.Perl). All clients are standalone applications. They don’t need any external library. Monitoring. The AMGA server can be monitored. …

5. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 5 Logical view of metadata. The picture show how the metadata are translated from the relational table to a file system like structure (WS):

6. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 6 Logical view of metadata.

7. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 7 Performance: Why AMGA v JDBC The Bookkeeping is using JDBC to access the DB. It would tell us how much is efficient AMGA against a direct connection to the DB. This would set the lower limit for the Bookkeeping performance.

8. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 8 Performance: Test bed peculiarity The AMGA server 1.0 was running on a dual Xeon (2.40Ghz ) machine with 1GB RAM The server machine wasn’t a dedicated machine. The clients where executed on the lxplus system (dual Xeon 2.80Ghz). DB –4 nodes cluster 2-xeon 2.8GHz. –storage arrays with 16 SATA disks of 300GB. –… Java program to read/insert data to the DB. Python script to create threads.

9. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 9 Performance: How the test was performed The python script generates threads which have the only duty of connecting to one of the lxplus machine and start the java program. The java program execute an SQL query continuously for 10 minutes reporting about the number of entries read/inserted and the time when the connection started/terminated.

10. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 10 Performance: How the test was performed Both clients, AMGA and JDBC, were reading the same data from the same table. The table contains a mapping between GUID and LFN. The throughput was calculated over the interval time when all the client where connected.

11. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 11 Performance: How the test was performed

12. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 12 Performance: Comments All the following behaviour was observed on the AMGA server machine: –The memory usage was very low. –Unfortunately the server load was very high, up to 10. The high load could explain why for some operations, like bulk read, AMGA behave very well on few client connections and then get worst as the number of clients goes up.

13. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 13 Performance: C harts

14. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 14 Performance: C harts

15. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 15 Performance: C harts

16. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 16 Performance: C harts

17. UK Metadata Workshop Oxford 5 July 2006 AMGA-Bookkeeping 17 Performance: C harts