1. DBS Monitor and DAN CD Projects Report July 9, 2003

2. Leader(s) Lee Lueking Abstract and Goals Database Infrastructure : Common Monitoring 1. Evaluate the differences and similarities between CDF central logger and DAN statistics gathering and mining. 2. Develop a common framework to provide a) alarms, b)plotting, c) archiving to mysql, oracle, xml or other, d)moving through hierarchy of servers, and 3) plotting and display packages. http://dbsmon.fnal.gov Comment Stakeholder Organizations CDF, D0, Others Participants and Effort Jim K, Steve W., Yuyi G., Eric W., Margherita V., John W. Lee L. Timeline and Schedule First brainstorming session Jan 14 with exploratory assignments. Define project tasks Jan 28. Complete MySQL databases, Statistics parsers, and plotting packages. Generate static plots daily with cron. Deliverables Common framework for archiving, presenting, and alarming on stat data. Daily static plots. Usage history for trend analysis. Plans Change Control LL – 01/28/03, LL – 07/08/03 Store daily summary information and produce historical trend plots. Establish live parsing of status information to generate near real-time Charts, and alarms.

3. DBSMON Status and Plans Daily plots are available and being used –http://dbsmon.fnal.govhttp://dbsmon.fnal.gov –Daily plots are archived Continued project plans –Provide additional plots for SAM project –Add historical statistics for trend analysis –Real-time parsing for event charting and alarms –Additional work to build dynamic web pages and active plots as needed. –Some additional work to place D0 MySQL server in final location (currently on d0ora3) when that is determined.

4. Abstract and GoalsDAN is the architecture which underlies Python based database servers. Its supplies the classes and functions to perform operations common to all database servers. Leader(s) Stephen White Stakeholder Organizations Dzero & CD Participants and Effort Stephen White 50%, Margherita Vittone 10% Timeline and Schedule 02/2003 - Deployment of remote proxy server code and server bootstrap. 02/2003 - Resolution of memory footprint. Deliverables Core class support for concurrent transactions multithreaded ORB memory cache file cache remote proxy support statistical data collection database reconnection client identification server bootstrap (restarting) http://www-cdserver/cd_public/ods/dbserver_design/dbserver_project.doc Plans Complete the proxy testing and deploy related classes. Address the memory footprint of a server. The size needs to be reduced. At times a server will lock up for unknown reasons producing many child processes. This must be addressed. Collected Statistical data must be displayed with a set of graphs. (Plan to leverage as joint working project.) Documentation providing an over view of the features must be provided. Comment

5. DAN Status and Plans (1) Current deployment is documented at http://d0db.fnal.gov/d0dbsrv/d0_db_servers_vers.html http://d0db.fnal.gov/d0dbsrv/d0_db_servers_vers.html Operational stats are being plotted using the DBSmon tools. Plots available at http://dbsmon.fnal.gov, Example from archive: http://dbsmon.fnal.gov/d0/All/2003-07-08/All_QryVsTime.pnghttp://dbsmon.fnal.gov http://dbsmon.fnal.gov/d0/All/2003-07-08/All_QryVsTime.png Operational for user and Farms production work. Have had several problems relating to yet do be resolved threading latencies, and memory cache size. Memory caching bloat problem for SMT –Resolved by using a C++ cache and client connection manager that runs in tandem with the Python server. –This reduces the memory requirement for each SMT set from 300 MB down to 25MB. Considered interim solution.

6. DAN Status and Plans (2) Exploring other possible options to improve performance and/or simplify the server. –Options include 1) C++ rewrite and 2) Web services approach using Apache server (preferred). –Anticipate an 8 week exploratory project to start mid July –D0, CDF, and CMS are interested and providing requirements. CHEP03 paper describing the project is available at http://arXiv.org/ftp/cs/papers/0307/0307001.pdf http://arXiv.org/ftp/cs/papers/0307/0307001.pdf

7. Other DBS Group Involvement Minos computing: Minos Freeware help and Oracle "main repository" evaluation (Dennis and Yuyi) - CDF DB Tasklist: Several CDF Tasklist items including –Better design of generalized connection code to replace separate otl/mysql/etc. modules. (Dennis) –Codegen – finish and fix Java compilation problems, test and deploy (Dennis) –Slow controls queries w/ Larry Kirsch (Margherita) Run II SAM: SAM dbserver changes, and updates, DFC to SAM migration (Steve, Carmenita) Shahkar: Working on RunJob (a.k.a. Shahkar) (Eric) Greg Graham is the project lead. USCMS VO Management: Evaluating VOX Vo Management (John). 23rd July is a taking stock meeting. CMS Offline Calibration: CMS offline calibration prototype. (Dennis) USCMS briefing for more extensive project plan is July 30