1. Information Services Andrew Brown Jon Ludwig Elvis Montero http://blackrose02.rit.edu/wiki/lib/exe/fetch.php?media= grid:seminar1:lectures:seminar-grid-1-information-services.ppt

2. Overview Globus o MDS o WSRF o WS MDS Deployment o ESG o MonaLISA o Inca Ganglia o Daemons o Web Frontend

3. MDS Monitoring and Discovery System Versions o MDS2 -> Pre-WS MDS o MDS4 -> WS MDS Capabilities o Queries o Subscriptions o Triggers [Globus01]

4. WSRF Background o OASIS specifications for web services o Globus, IBM major contributors Model o Clients communicate with resource services o Include identifier to specify resource o Similar to objects and methods [OASIS], [Globus02]

5. WS MDS - Overview Implementation of MDS in WSRF Components o Foundation  Aggregator Framework  Information Providers o Components  Index Service  Trigger Service  WebMDS [Globus01]

6. WS MDS - Foundation Aggregator Framework o Aggregator Sources  Java interface to collect information  What to get, where to get it from  Types  Query  Subscription  Execution o Aggregator Services  Collection of Aggregator Sources [Globus01]

7. WS MDS - Basic Components Index Service o Collects information o Stores as resource properties o Publishes in single location o Supports queries and subscriptions o Indices can be linked together Trigger Service o Collects information o Matches rules and fires actions WebMDS o View information in web browser o Used by end users [Globus01]

8. WS MDS - Advanced Components Information Providers o External applications used to collect data o Collection of Aggregation Services o Examples  Condor - hosts, processors, memory  Ganglia - hosts, processors, memory  WS GRAM - queue, CPUs, memory  RFT - server, transfers [Globus01]

9. WS MDS - Use Index Service o wsrf-query http://url [expression] o wsrf-get-property http://url --key Trigger Service o mds-servicegroup-add http://url config.xml WebMDS o http://hostname:port/webmd [Globus01], [IBM]

10. Deployment - ESG [ESG]

11. Deployment - MonALISA [MonALISA]

12. Deployment - Inca User Level [Inca]

13. Ganglia Why monitor your systems? o Health o Performance / Bottlenecks o Trends Remotely view live or historical statistical data High performance o Low overhead per node o High concurrency o Scalable Designed for federated clusters Graphical view via website XML data from Ganglia Meta Daemon

14. Ganglia Monitor Daemon - gmond Runs on each node that you wish to monitor Minimal footprint - In-memory hashtable Responsibilities: o Monitors changes o Listens for and records the state of other Ganglia nodes o Reports state of the node  Broadcast when the state changes  Periodically, if the state > threshold  Replies to requests for state information o XML + TCP (telnet?!) o XDR + UDP (Multicast / Unicast) Need a different metric? - gmetric Fault Tolerance

15. Ganglia Monitor Daemon - gmond [GDT2002]

16. Ganglia Meta Daemon - gmetad Periodically polls child data sources to gather information about cluster states o Unicast TCP + XML o Redundant sources o gmond or gmetad Saves data in a database o rrdtool Collect data from individual nodes or from clusters o Hierarchical

17. Ganglia Meta Daemon - gmetad

18. Ganglia Web Frontend Collects the tree of data from gmetad and displays it to the user via a website Demo o http://monitor.millennium.berkeley.edu/ http://monitor.millennium.berkeley.edu/ o http://cluster.rit.edu/ganglia/ http://cluster.rit.edu/ganglia/

19. References [Globus01] GT 4.0: Information Services. http://www.globus.org/toolkit/docs/4.0/info/. [OASIS] OASIS Web Services Resource Framework (WSRF) TC. http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrf. [Globus02] WSRF: The Web Services Resource Framework. http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html. [Atkinson] Atkinson, Malcolm et al. Web Service Grids: An Evolutionary Approach. http://www.omii.ac.uk/dissemination/paper_web_service_grids.pdf. [Wikipedia01] Web Services Resource Framework. http://en.wikipedia.org/wiki/Web_Services_Resource_Framework.

20. References cont. [Foster] Foster, Ian. A Globus Primer. http://www.globus.org/toolkit/docs/4.0/key/GT4_Primer_0.6.pdf. [Globus03] Deploying WS MDS in a Virtual Organization. http://www.globus.org/toolkit/docs/4.0/info/key/deployment_overview.html. [Globus04] GT 4.0 WS MDS WebMDS: System Administrator’s Guide. http://www.globus.org/toolkit/docs/4.0/info/webmds/admin-index.html. [Globus05] GT 4.0 WS MDS Trigger Service: System Administrator’s Guide. http://www.globus.org/toolkit/docs/4.0/info/trigger/admin-index.html. [Wikipedia02] Ganglia (software). http://en.wikipedia.org/wiki/Ganglia_%28software%29.

21. References cont. [GDT2002] Ganglia Toolkit. http://www.msg.ucsf.edu/local/ganglia/ganglia_docs/index.html [Massie2004Ganglia] "The Ganglia Distributed Monitoring System: Design, Implementation, and Experience". Matthew L. Massie, Brent N. Chun, and David E. Culler. Parallel Computing, Vol. 30, Issue 7, July 2004. [ESG] The Earth System Grid (ESG) monitoring architecture. http://www.globus.org/solutions/system_monitoring/architecture.php. [MonaLISA] MonaLISA's Architecture. http://monalisa.cacr.caltech.edu/monalisa_e__System_Design.htm. [Inca] Inca's architecture and screenshots. http://inca.sdsc.edu/drupal/node/20. [IBM] WebMDS Screen Shot. http://www.ibm.com/developerworks/cn/grid/gr- gt4mds/webMDSRFT.jpg