1. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 1 IT-AIS-HRMarch 9 th 2006 AisMonitor Giovanni Chierico & Camilla Stenersen

2. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 2 IT-AIS-HRMarch 9 th 2006 Background Multiple remarks and demands from both IT-AIS and IT- DES after various system failures which often had cascade effects on other systems: Difficult to monitor (everybody has his own monitoring system) Difficult to know potential cascade effects (were there two separate problems or one causing the other) IT-AIS GL and DGL triggered the project using available TECH

3. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 3 IT-AIS-HRMarch 9 th 2006 Goals Create a common framework to monitor AIS applications Simple to integrate in the existing AIS & IT environments Reliable Easy to extend Generic Keep history

4. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 4 IT-AIS-HRMarch 9 th 2006 Process Gather requirements Interviews with all the AIS Section leaders Meetings with IT-DES (Artur Wiecek) Meetings with IT-FIO (Sebastian Lopienski) Meeting with IT-AIS and IT-DES GLs Implement full app stack to monitor some “sample” apps. DB Messaging Applications “hooks” Interface Refine the application and implement “hooks” in AIS apps

5. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 5 IT-AIS-HRMarch 9 th 2006 Suggestions Keep it simple Don’t reinvent the wheel Push & Pull monitoring Communicate via DB APIs, HTTP, HTTPS, POP, SMTP, JMS, … Aware of scheduled downtimes “Digest” alarms Monitor batch jobs Map dependencies Monitor business processes - +

6. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 6 IT-AIS-HRMarch 9 th 2006 Architecture Everything is a service Services are defined by two parameters The service level (L) The expected service level (E) The quality of service is Q=L-E Service hierarchy defines an oriented weighted graph, with no closed loops. Services communicate with the world via messages to modify their actual (L) and expected (E) service level. Level changes are propagated to the children An alarm defines a Q threshold for a given service, and is triggered accordingly

7. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 7 IT-AIS-HRMarch 9 th 2006 Architecture DB API HTTP WS HTTPS POP Message Service alarm Message contains Payload Service level Expected service level Comment Authorization Service name Auth password

8. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 8 IT-AIS-HRMarch 9 th 2006 Internals Repository Java Bus&Logic Pull: Http(s), POP Push: HTTP, WS, SMTP Push: DB APIs Alerts: SMTP Export: XML MetaData InterfaceDisplay Interface

9. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 9 IT-AIS-HRMarch 9 th 2006 Example Expected level (E) Actual Level (L) Trigger alarm if threshold equals zero time level

10. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 10 IT-AIS-HRMarch 9 th 2006 Architecture Key Points Low “entry cost” No big initial effort to map dependencies, or modify applications. Monitor and map what you need. Positive feedback The more you invest (map deps, create app hooks), the more you get in return The more value we give back, the more likely the mapped structure will stay up-to-date Give control to the developers The developer knows best! No need to depend on others to monitor your app. Easily manage the alerts and the downtimes

11. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 11 IT-AIS-HRMarch 9 th 2006 For the AIS Developer PULL (AisMonitor checks the service): web apps Expose the test (e.g. on HTTP) Create AisMonitor service Configure AisMonitor PULL (where, when, …) PUSH (App sends events to AisMonitor): batch jobs Create AisMonitor service Write the app specific test, or hook into existing process Sending the message can be usually done in 2-3 lines of code

12. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 12 IT-AIS-HRMarch 9 th 2006 Push examples Java MuleClient client = new MuleClient(); client.dispatch(“axis:http://monitor/?", Payload, null); or HttpClient client = new HttpClient(); GetMethod method = new GetMethod(“http://monitor/”); //set params client.executeMethod(method); MonitorPkg.push@link(name, pwd, level, expected, comment); or SELECT utl_http.request('http://monitor/?params') FROM dual; PL/SQL

13. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 13 IT-AIS-HRMarch 9 th 2006 Roadmap What we have today Push: Run servers and receive messages over HTTP, HTTPS, and file system. Pull: messages from HTTP and HTTPS, even behind AisLogin Propagate messages on Service hierarchy Persistence (Services, Dependencies, Messages, Statuses saved on DB) Export service status to xml, accessible over HTTP (needed by IT monitor) In a week Basic web interface (ApEx) to manage the services and the pulling params Pull messages via POP from ais.monitor@cern.ch Send alerts (SMTP) In two weeks Monitor implemented in production apps (CTA & GAD) to test it with real data

14. CERN – European Organization for Nuclear Research Administrative Information Services AisMonitor 14 IT-AIS-HRMarch 9 th 2006