1. hLS Overview Jason Zurawski perfSONAR-PS Developer Meeting April 30, 2009

2. Agenda Architecture DBXML Data Structure Daemon Messages Summarization Cleaning gLS Interaction Summarization

3. Architecture Oracle DBXML Data Storage Summarization Storage perfSONAR-PS Daemon Handles Messages Maintenance ‘Cleaning’ Summarization Registration to gLS

4. Architecture - DBXML http://www.oracle.com/technology/produ cts/berkeley-db/xml/index.htmlhttp://www.oracle.com/technology/produ cts/berkeley-db/xml/index.html 2.3.X series currently We re-package RPM form for distribution ‘Best Effort’ Database Issues TBD later on

5. Architecture - DBXML Storage Environment Directory where a DBXML instance lives Configuration DB_CONFIG file in the environment Items needed for operation of database (e.g. log strategy, cache size, etc.) Container Data storage May be many per environment

6. Architecture - DBXML Storage ‘mapping’ is the same as the MDM (‘store’ files that contain things) To support DBXML, do not use a single store file per container, each item is wrapped in a store with the appropriate ‘type’ LSStore LSStore-control LSStore-summary

7. Architecture - DBXML Data Storage glsstore.dbxml Different ‘types’ for the store file LSStore –Service Metadata + Service Data LSStore-control –Service ‘control’ structure (Metadata w/ expiration time)

8. Architecture - DBXML Summary Storage glsstore-summary.dbxml Different ‘types’ for the store file LSStore –Service Metadata + Summarization result (summary of all metadata for a particular service LSStore-control –Service ‘control’ structure (Metadata w/ expiration time) LSStore-summary –hLS service name + ‘Summary of Summaries’ –We pass this to the gLS

9. Architecture – perfSONAR-PS Daemon Messages LSRegister LSDeregister LSKeepalive LSQuery LSKey LSDiscovery

10. Architecture – perfSONAR-PS Daemon LSRegister(Request/Response) Register content into the hLS Metadata(s) = Service Structure Data(s) = Metadata associated with service Different operations based on context ‘New’ – If the service is not in the LS, it is added to the LS as a new entity ‘Update’ – Use the ‘key’ to add new items to the LS ‘Clobber’ – Send the key and changes to the service element (N.B. this nukes existing dataset)

11. Architecture – perfSONAR-PS Daemon LSDeregister(Request/Response) Remove content from the hLS Metadata = Key for a service Data = nothing, or specific service metadata ‘nothing’ case is a data trigger ‘Selective’ case is a pS-PS addition (EU *LS code does not support this)

12. Architecture – perfSONAR-PS Daemon LSKeepalive(Request/Response) Updates the time in the control structure to allow this service to still live in the gLS Metadata = key Data = nothing (trigger)

13. Architecture – perfSONAR-PS Daemon LSQuery(Request/Response) Query the database in a ‘raw’ format (need to know a little about how the data is structured) Metadata = XQuery/XPath statement Data = nothing (trigger)

14. Architecture – perfSONAR-PS Daemon LSKey(Request/Response) Request a key (normally given at registration time) for a service Metadata = Service description Data = nothing (trigger) Currently not ‘safe’ due to no AA infrastructure

15. Architecture – perfSONAR-PS Daemon LSDiscovery(Request/Response) Similar to query, search discovery set for specific items Metadata = Discovery Items Domains Keywords Addresses EventTypes Data = nothing (trigger) More valuable in gLS case (N.B. discovery on an hLS works only on the hLS dataset)

16. Architecture – perfSONAR-PS Daemon Maintenance Periodic events the daemon schedules maintenance_interval = 2 hours on new release ‘Cleaning’ Looks at LSStore-control elements in containers Removes ‘expired’ services and data Summarization Summarize the data set of each registered service Summarize the dataset of the hLS (e.g. summary of all services

17. Architecture – perfSONAR-PS Daemon Registration Use bootstrapping procedure (hints) to find close gLS (currently limited to 3 in pS-PS, unsure what MDM does [maybe just one]) Send ‘summary of summaries’ message to these gLS instances hLS will parse response message, but from it’s point of view the work is done.

18. Summarization Process Storage Use gLS Registration Queries

19. Summarization - Process Periodic Event maintenance_interval = 2hrs in current release Foreach registered service, foreach registered metadata description Extract domain/eventType/keyword/ip info Hack up each Create a ‘summary’ set Summarize the summaries Create a ‘summary’ set over all the summaries

20. Summarization - Process Summarization – Domain Given a domain, break off the subclasses For host moonshine.damsl.cis.udel.edu: edu udel.edu cis.udel.edu damsl.cis.udel.edu Each will be stored

21. Summarization - Process Summarization – Keywords Adopted form = project:LHC Similar to URIs – discussion on format/use not active currently Starting to overload (communities, projects, could be used in DCN too…) Used in GUIs We simply gather these up (do not unroll or break apart URI [yet])

22. Summarization - Process Summarization – eventType Typical format http://some/data/stuff/2.0/ Treat same as keywords (gather – do not break apart)

23. Summarization - Process Summarization – IP Addresses Collect all we can find from topology Using CIDR Summaries, create an IPTrie Find the ‘cut point’ that best describes the data (e.g. where can we claim we control the IP space based on included data) Ongoing See: http://code.google.com/p/perfsonar- ps/wiki/IPSummarization http://code.google.com/p/perfsonar- ps/wiki/IPSummarization

24. Summarization - Storage Service Summaries glsstore-summary.dbxml LSStore container hLS Summary glsstore-summary.dbxml LSStore-sumary container Still controlled via ‘control’ elements, but these are nuked each time we summarize (ensure valid data + freshness)

25. Summarization - Use Registration ‘summary of summaries’ is sent to the gLS periodically ls_registration_interval = 3 hours in current release Query LSQuery for ‘raw’ operations LSDiscovery for structured query of summary set

26. Next Global Lookup Framework gLS As a service gLS/hLS as a system Known Problems Discussion