1 OGSA Framework for Higher-Level and International Collaboration UK e-Science Core Programme Meeting Malcolm Atkinson 31 st January 2005

2 Contents & Story Standards vital middleware, applications development & deployment Why standards matter Interoperability, replaceability, portability & direction Why architecture (OGSA) matters Integration, completeness, abstraction & cooperation UK’s investment in standards Annual investment – people and funds OGSA is making this more effective Recommendations  Collaborate

3 Standards are Vital What do we want from standards Foundation for inter-domain operations My software will work with your software At the level of our business – policies, concepts, data & processes Foundation for replacement I was using X’s Y-system – I can replace it with Z’s Y-system Everything else is unchanged Foundation for Portability My application code works unchanged  on all e-Infrastructures My (upper) middleware code works unchanged  With alternative supporting implementations of the M/W it uses We get standards only if we work hard enough good standards require sustained effort, insight & negotiation To be useful standards must match requirement s and be widely adopted

4 Standards are Vital What do we want from standards Foundation for inter-domain operations My software will work with your software At the level of our business – policies, concepts, data & processes Foundation for replacement I was using X’s Y-system – I can replace it with Z’s Y-system Everything else is unchanged Foundation for Portability My application code works unchanged  on all e-Infrastructures My (upper) middleware code works unchanged  With alternative supporting implementations of the M/W it uses E-Science is inter- domain, internationa l & multi- platform E-Science requires interoperabilit y to function & inter- manageability to be efficient

5 Standards are Vital What do we want from standards Foundation for inter-domain operations My software will work with your software At the level of our business – policies, concepts, data & processes Foundation for replacement I was using X’s Y-system – I can replace it with Z’s Y-system Everything else is unchanged Foundation for Portability My application code works unchanged  on all e-Infrastructures My (upper) middleware code works unchanged  With alternative supporting implementations of the M/W it uses Required to improve while sustaining function & service Opportunity for competition & market forces

6 Standards are Vital What do we want from standards Foundation for inter-domain operations My software will work with your software At the level of our business – policies, concepts, data & processes Foundation for replacement I was using X’s Y-system – I can replace it with Z’s Y-system Everything else is unchanged Foundation for Portability My application code works unchanged  on all e-Infrastructures My (upper) middleware code works unchanged  With alternative supporting implementations of the M/W it uses Agreed APIs & semantics save porting, testing, maintenance, documentation & training costs Recent OGSA-DAI experience shows costs are serious Lack of portability will drive developers of scientific code away

7 Importance of Standards - caveats Waiting for standards Doesn’t keep pioneering users & projects happy Competing standards Almost as bad as no standards Partially adopted standards – no enforcement A real nuisance – try SQL – see swissSQL Industry doesn’t always have the same goals Waiting for their decisions / endorsement Backs their business agenda Against our collaboration agenda Commercially supported implementations May not happen for every standard we need May be too late May choose different performance trade offs Open source implementations are needed Join in improving them – don’t keep starting again! If you think standards are expensive, try chaos to misquote Roy Crock, Founder of M c Donalds

8 Health Warning We build e-Infrastructure To make it easier to conduct research Multi-purpose infrastructure Multi-domain collaborations Autonomous and heterogeneous providers Addressing challenge of distributed systems Large scale Long-lived Coordination and policy alignment part of the story Transfer work & responsibility From client and application developer To infrastructure & infrastructure deployers Community will be unforgiving if we base our standards on an unrealistic simplification because the scale of research interaction has not yet exposed well understood features of such distributed systems

9 Why architecture (OGSA) matters

10 Why Invest in OGSArchitecture Integration E-Infrastructure assembled from many standards They must work together – to meet requirements OGSA provides context To identify and integrate the requirements To assimilate and integrate current experience To review, initiate & steer standards So they can be integrated to deliver an e-Infrastructure Completeness Abstraction Cooperation

11 Why Invest in OGSArchitecture 2 Integration Completeness Analysis identifies missing subsystems  Not yet standardised  Not yet finished  Not yet started  Not completely filling the expected niche Consider  Existing requirements  Existing standards  Existing implementations  Existing knowledge of eventual requirements of distributed systems Abstraction Cooperation

12 Architecture: Status GRID COMPUTING UTILITY COMPUTING DISTRIBUTED COMPUTING Core Services Base Profile WS-Addressing Privacy WS-BaseNotification CIM/JSIM WSRF-RAP WSDM WS-Security Naming OGSA-EMSOGSA Self Mgmt GFD-C.16 GGF-UR Data Model HTTP(S)/SOAP Discovery SAML/XACML WSDL WSRF-RL Trust WS-DAI VO Management Information Distributed query processing ASP Data Centre Use Cases & Applications CollaborationMulti MediaPersistent Archive Data Transport WSRF-RP X.509 StandardEvolvingGapHole Slide from Dave Snelling & Mark Linesch November 2005

13 Why Invest in OGSArchitecture 3 Integration Completeness Abstraction Require higher-level abstractions  More easily explained to users  Better support for application developers  Better isolation from underlying / changing platforms  Better opportunity for e-Infrastructure optimisation Analysis & re-factoring of larger function space  Select recurrent structures  Push for higher-level standards Cooperation

14 Why Invest in OGSArchitecture 4 Integration Completeness Abstraction Cooperation OGSA partitions the e-Infrastructure implementation Encourages independent concurrent & coordinated  Development or evaluation of each part’s standards  R&D on implementation of each part Promises assembly of the parts Basic profile provides context for concurrent R&D  Context for each M/W developer to build against  Reduced interdependence – each can deliver if others don’t Focus effort on reaching minimum threshold that makes this work

15 Current Release Schedule In order to allow more time for performance, stability and other usability improvements, the Globus Alliance is extending the GT4 testing cycle. l 17 December 2004 –GT4 Alpha development release (3.9.4) –full set of core GT components –significant performance and usability improvements l 31 January 2005 –GT4 Alpha development release (3.9.5) l March 2005 –GT4 Beta development release (3.9.6) –to contain the final set of public interfaces for 4.0 l April 2005 –GT 4.0 Final

16 Contributions to Apache l All Globus software under an Apache-like licence l Contributed over past 2 years –Many bug fixes and extensions to Axis & Tomcat, etc >E.g. WS-Addressing & WS-Security –Several GA developers on repository commit team l Two new core WS Apache Incubator projects –Apollo – the WSRF code >Java, C & Perl –Hermes – the WS-Notification code l Test process with these before contributing more

Pre-WS Authentication Authorization Data Management Security Common Runtime Execution Management Information Services GridFTP Web Services Components Non-WS Components Grid Resource Allocation Mgmt (Pre-WS GRAM) Monitoring & Discovery System (MDS2) C Common Libraries GT2GT2 WS Authentication Authorization Reliable File Transfer (RFT) OGSA-DAI [Tech Preview] Grid Resource Allocation Mgmt (WS GRAM) Monitoring & Discovery System (MDS4) Java WS Core CAS GT3GT3 Replica Location Service (RLS) XIO GT3GT3 Credential Management GT4GT4 Python WS Core [contribution] C WS Core Community Scheduler Framework [contribution] Delegation Service GT4GT4 GT Release History

18 UK’s investment in standards

19 Gridnet 1 Data – Three years Active in 42 GGF Groups 19 (co) chairs 1 Member GFAC 5 Members GFSG 4 Area Directors Active in IETF, OASIS & W3C Authors on 9 out of 41 GGF documents Many report high value Community + Platinum sponsor Much imminent output OGSA + SAGA JSDL, DFDL, InfoD DAIS … >6 person-years/year Not including industry Not GridNet CCLRC, GridPP, … Reference implementation 47 active via GridNet £130K/year GridNet 13 universities No academic staff are directly funded for standards work

20 UK Contribution to OGSA OGSA History First OGSA meeting November 2001 ATF Recommend UK adoption April 2002 OGSA-DAI Proposed December 2001 Started February 2002 Adopted OGSA model Followed OGSA path 2000 downloads proto- reference implementation Extensive consultation with data community Limited by time available Needs more investment OGSA-WG Leadership Dave Snelling Dave Berry data area team Authorship  Dave Berry, Abdeslam Djaoui, … Standards for OGSA JSDL DFDL, DAIS & InfoD Naming scheme, GSM, … NextGRID Next architecture Led by Dave Snelling OGSA as basis OGSA base profile as first step

21 Recommendations

22 UK Should Back OGSA more Invest effort in OGSA Investigating, evaluating, contributing, commenting Implementing profiles Using it It is the ONLY show in town Which offers  integration, completeness, abstraction  A foundation for collaboration Support UK focus on Data Design Team Support new UK efforts in other design teams EMS, Grid markets, JSDL, GSM, mySpace, …

23 Use OGSA for Collaboration “All Hands” to Reach OGSA Basic Profile Sufficient platform, context & framework For safely partitioning further R&D Agree a division of work Upgrading / alternative trade-off components New components Higher level facilities Minimise duplication Maximise combined efforts to deliver Function, Stability, Quality & Abstraction Bury the egos, project competitio n & national pride silos

24 Comments & Questions

Reserve Slides

26 What’s This About a Globus Company? l Univa was announced December 13, 2004 – – l Steve Tuecke is CEO l Carl Kesselman and Ian Foster are in advisory role –Remain at USC/ISI and Argonne l Basic concept: “Redhat Linux for Globus” l This will NOT affect the GT open source policy –On the contrary, Univa is already contributing resources to open source software l This WILL allow greater industrial involvement and investment in Grids

27 How to Get Involved Become a GT4 Friend! l Open group of people from various organizations working with GT4 pre-release code and documents –Reporting problems in code and documents –Contributing ideas, tests, documentation –Building GT4-enabled applications l Weekly telephone calls l Discussion list –To subscribe to the GT4 friends list, send an to which contains the words “subscribe gt4-friends” in the message body Significant input from Terry Harmer and his UK ETF evaluation team

28

29 Java Services in Apache Axis Plus GT Libraries and Handlers Your Java Service Your Python Service Your Java Service RFT GRAM Delegation Index Trigger Archiver pyGlobus WS Core Your C Service C WS Core RLSPre-WS MDS CAS Pre-WS GRAM SimpleCAMyProxy OGSA-DAI GTCP GridFTP C Services using GT Libraries and Handlers SERVER CLIENT Interoperable WS-I-compliant SOAP messaging Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client X.509 credentials = common authentication Python hosting, GT Libraries

30 Custom Web Services WS-Addressing, WSRF, WS-Notification Custom WSRF Web Services GT4 WSRF Web Services WSDL, SOAP, WS-Security User Applications Registry Administration GT4 Container