Information Services Andrew Brown Jon Ludwig Elvis Montero grid:seminar1:lectures:seminar-grid-1-information-services.ppt

Overview Globus o MDS o WSRF o WS MDS Deployment o ESG o MonaLISA o Inca Ganglia o Daemons o Web Frontend

MDS Monitoring and Discovery System Versions o MDS2 -> Pre-WS MDS o MDS4 -> WS MDS Capabilities o Queries o Subscriptions o Triggers [Globus01]

WSRF Background o OASIS specifications for web services o Globus, IBM major contributors Model o Clients communicate with resource services o Include identifier to specify resource o Similar to objects and methods [OASIS], [Globus02]

WS MDS - Overview Implementation of MDS in WSRF Components o Foundation  Aggregator Framework  Information Providers o Components  Index Service  Trigger Service  WebMDS [Globus01]

WS MDS - Foundation Aggregator Framework o Aggregator Sources  Java interface to collect information  What to get, where to get it from  Types  Query  Subscription  Execution o Aggregator Services  Collection of Aggregator Sources [Globus01]

WS MDS - Basic Components Index Service o Collects information o Stores as resource properties o Publishes in single location o Supports queries and subscriptions o Indices can be linked together Trigger Service o Collects information o Matches rules and fires actions WebMDS o View information in web browser o Used by end users [Globus01]

WS MDS - Advanced Components Information Providers o External applications used to collect data o Collection of Aggregation Services o Examples  Condor - hosts, processors, memory  Ganglia - hosts, processors, memory  WS GRAM - queue, CPUs, memory  RFT - server, transfers [Globus01]

WS MDS - Use Index Service o wsrf-query [expression] o wsrf-get-property --key Trigger Service o mds-servicegroup-add config.xml WebMDS o [Globus01], [IBM]

Deployment - ESG [ESG]

Deployment - MonALISA [MonALISA]

Deployment - Inca User Level [Inca]

Ganglia Why monitor your systems? o Health o Performance / Bottlenecks o Trends Remotely view live or historical statistical data High performance o Low overhead per node o High concurrency o Scalable Designed for federated clusters Graphical view via website XML data from Ganglia Meta Daemon

Ganglia Monitor Daemon - gmond Runs on each node that you wish to monitor Minimal footprint - In-memory hashtable Responsibilities: o Monitors changes o Listens for and records the state of other Ganglia nodes o Reports state of the node  Broadcast when the state changes  Periodically, if the state > threshold  Replies to requests for state information o XML + TCP (telnet?!) o XDR + UDP (Multicast / Unicast) Need a different metric? - gmetric Fault Tolerance

Ganglia Monitor Daemon - gmond [GDT2002]

Ganglia Meta Daemon - gmetad Periodically polls child data sources to gather information about cluster states o Unicast TCP + XML o Redundant sources o gmond or gmetad Saves data in a database o rrdtool Collect data from individual nodes or from clusters o Hierarchical

Ganglia Meta Daemon - gmetad

Ganglia Web Frontend Collects the tree of data from gmetad and displays it to the user via a website Demo o o

References [Globus01] GT 4.0: Information Services. [OASIS] OASIS Web Services Resource Framework (WSRF) TC. [Globus02] WSRF: The Web Services Resource Framework. [Atkinson] Atkinson, Malcolm et al. Web Service Grids: An Evolutionary Approach. [Wikipedia01] Web Services Resource Framework.

References cont. [Foster] Foster, Ian. A Globus Primer. [Globus03] Deploying WS MDS in a Virtual Organization. [Globus04] GT 4.0 WS MDS WebMDS: System Administrator’s Guide. [Globus05] GT 4.0 WS MDS Trigger Service: System Administrator’s Guide. [Wikipedia02] Ganglia (software).

References cont. [GDT2002] Ganglia Toolkit. [Massie2004Ganglia] "The Ganglia Distributed Monitoring System: Design, Implementation, and Experience". Matthew L. Massie, Brent N. Chun, and David E. Culler. Parallel Computing, Vol. 30, Issue 7, July [ESG] The Earth System Grid (ESG) monitoring architecture. [MonaLISA] MonaLISA's Architecture. [Inca] Inca's architecture and screenshots. [IBM] WebMDS Screen Shot. gt4mds/webMDSRFT.jpg