1. 30 August 2015 NSF Middleware Initiative Building for the Future Kevin Thompson, NSF Ken Klingenstein, Internet2 Tom Garritano, Grids Center Mary Fran Yafchak, SURA Ann West, EDUCAUSE/Internet2

2. 10 April 2003 2 Topics  NMI Overview and New Awards  NMI-EDIT  GRIDS Center  NMI Integration Testbed  NMI and NMI-EDIT Outreach

3. 10 April 2003 3 NSF Middleware Initiative  Purpose  To design, develop, deploy and support a set of reusable, expandable set of middleware functions and services that benefit applications in a networked environment

4. 10 April 2003 4 To allow scientists and engineers the ability to transparently use and share distributed resources, such as computers, data, and instruments To develop effective collaboration and communications tools such as Grid technologies, desktop video, and other advanced services to expedite research and education, and To develop a working architecture and approach which can be extended to Internet users around the world. Middleware is the stuff that makes “transparently use” happen, providing persistency, consistency, security, privacy and capability A Vision for Middleware

5. 10 April 2003 5 NMI Status  21 Active Awards (Prior to new awards) 3 Cooperative Agreements 18 individual research awards  Focus on integration and deployment of grid and middleware for S&E Near-term deliverables (working code) coordination and persuasion rather than standards Significant effort on interoperability, testing, inclusion  NMI Software Releases, best practices, white papers NMI Release 1 – May, 2002 NMI Release 2 – Oct, 2002 NMI Release 3 – Apr, 2003 NMI Release 4 - est. Dec, 2003

6. 10 April 2003 6 NMI Organization GRIDS Center – ISI, NCSA, U Chicago, UCSD & U Wisconsin EDIT Team (Enterprise and Desktop Integration Technologies) –EDUCAUSE, Internet2 & SURA Several additions in 2003 Core NMI Team Grants for R & D Year 1 -- 9 grants Year 2 -- 9 grants Year 3 (New) -- 10 grants

7. 10 April 2003 7 2003 NSF Middleware Initiative Program Awards  20 awards totaling $9M 10 “System Integrator” awards –Focus – to further develop the integration and support infrastructure of middleware for the longer term –Themes - extending and deepening current activities, and expanding into new areas 10 smaller awards focused on near-term capabilities and tool development –Focus – to encourage the development of additional new middleware components and capabilities for the NMI program

8. 10 April 2003 8 2003 New System Integrator Awards Butler (UIUC) Disseminating and Supporting Middleware Infrastructure: Engaging and Expanding Scientific Grid Communities Kesselman (USC/ISI) Designing and Building a National Middleware Infrastructure (NMI-2) Klingenstein (UCAID) Extending Integrated Middleware to Collaborative Environments in Research and Education Livny (U Wisc) An Integrative Testing Framework for Grid Middleware and Grid Environments McMullen (Indiana) Instruments and Sensors as Network Services: Instruments as First Class Members of the Grid Pierce, Alameda, Severance, Thomas, and von Laszewski Collaborative Proposal: Middleware for Grid Portal Development

9. 10 April 2003 9 Other New Awards in 2003 Chase (Duke), Ramakrishnan (MCNC) Collaborative Research: A Grid Service for Dynamic Virtual Clusters Gemmil (UAB)NMI-Enabled Open Source Collaboration Tools for Virtual Organizations Karonis (Northern Illinois)Critical Globus-enabled Implementation of the MPI-2 Standard Lumsdaine (Indiana)Scalable Fault Tolerance for MPI Ramachandran (Ga Tech)Exploration of Middleware Technologies for Ubiquitous Computing with Applications to Grid Computing Saltz (Ohio St. URF)GridDB-Lite: Database Support for Data-Driven Scientific Applications in the Grid Wright (U Wisc.), Linderoth (Lehigh) Collaborative Research: MW: Master-Worker Middleware for Grids Arzberger (UCSD)Pacific Rim Application and Grid Middleware Assembly

10. 10 April 2003 10 Looking Ahead  There will be an NMI solicitation in 2004  Exact funding level not set  NMI program is expected to be a primary focus area under CISE’s new division - Shared CyberInfrastructure  October 23 Review at NSF for existing activities among the Grids Center and EDIT teams

11. 10 April 2003 11 Enterprise and Desktop Integration Technologies (EDIT) Consortium Ken Klingenstein Director, Internet2 Middleware Initiative kjk@internet2.edu

12. 10 April 2003 12 Overview  NMI-EDIT Overview  Research Findings  NMI Release Components  Building on the Future

13. 10 April 2003 13 NMI-EDIT Consortium  Enterprise and Desktop Integration Technologies Consortium Internet2 EDUCAUSE, SURA Almost all funding passed through to campuses for work  Project Goals Create a common, persistent and robust core middleware infrastructure for the R&E community Provide tools and services in support of inter-institutional and inter-realm collaborations

14. 10 April 2003 14 A Map of Middleware Land

15. 10 April 2003 15 NMI-EDIT Findings  Consensus on inter-institutional middleware standards and maturing architecture to support collaborative applications  Widespread interest in Shibboleth within R&E communities  Credential mapping from core enterprise to Grid service  Grid adoption of SAML in Open Grid Services Architecture (OGSA)

16. 10 April 2003 16 NMI-EDIT Findings (cont.)  Creation and maintenance of a heavily referenced set of design and best practices documents  Effective linkages with International research communities  Discovery and development of campus IT staff, leading to  Influence on both federal and commercial standards  Direct outreach to over 320 institutions

17. 10 April 2003 17 NMI-EDIT Components from Three NMI Releases  Authentication: WebISO solution, credential mapping from Kerberos to PKI, policy documents, registry service  Enterprise Directories: Schemas; operational monitoring and schema analysis tools; practices in design, groups, and metadirectories  Authorization: Architecture and related software and libraries for multi- institution collaboration

18. 10 April 2003 18 NMI-EDIT Components from Three NMI Releases (cont.)  Integration Activities: Credential mapping from campus to Grid environment, GLUE schema analysis tool  Education: Venues for learning about enterprise middleware including CAMPs and on-line deployment materials for directories

19. 10 April 2003 19 NMI-EDIT: Building for the Future  Authorization infrastructure Develop authorization architecture to support –campus applications, both legacy and productivity –network services –inter-realm collaboration efforts –virtual organizations Deliverables –An Enterprise Authorization Design and Implementation Guide, with case histories –A set of tools for use with Authz, including registry templates for policies, roles, and authorities, a GROUPER group manager, end user interfaces, etc.

20. 10 April 2003 20 NMI-EDIT: Building for the Future (cont.)  Middleware end-to-end diagnostics How do I resolve a permission denied problem? –From the user perspective –From the user’s help-desk perspective –From the technical support perspective Deliverables first year –Develop an expert group to design an interim architecture and functionality –Build liaisons to adjacent efforts by other groups –Design standard, extensible logging formats –Create a harvesting tool that can assemble end-end middleware flows from logs –Work with KX.509, Shibboleth, etc…

21. 10 April 2003 21 New Partnerships  Work with JISC on Virtual Organizations A key cross-stitch among enterprises for inter- institutional collaborations VO’s range from Grids to digital libraries, from earthquake engineering to collaborative curation, from managing observatories to managing rights  Interworkings with Australian, Swiss, French universities  Corporate interactions with MS, Sun, Liberty Alliance, etc

22. 10 April 2003 22 NMI-EDIT: Next Generation Architecture

23. 10 April 2003 23 The pieces fit together…  Campus infrastructure Developing and encouraging the deployment of identity management components, tools, and support services  Inter-realm infrastructure Leveraging the local organizational infrastructure to enable access to the broader community though –Building on campus identity management infrastructures –Extending them to contain standard schemas and data definitions –Enabling the exchange of access information in a private and secure way –Developing diagnostic tools to make complex middleware interactions easier to understand

24. 10 April 2003 24 The GRIDS Center: Defining and Deploying Grid Middleware presented by Tom Garritano University of Chicago and Argonne National Laboratory

25. 10 April 2003 25 NSF Middleware Initiative (NMI)  GRIDS is one of two original teams, the other being EDIT  New NMI teams just announced (Grid portals and instrumentation)  GRIDS releases well-tested, deployed and supported middleware based on common architectures that can be extended to Internet users around the world  NSF support of GRIDS leverages investment by DOE, NASA, DARPA, UK e-Science Program, and private industry

26. 26 GRIDS Center  GRIDS = Grid Research Integration Development & Support  Partnership of leading teams in Grid computing University of Chicago and Argonne National Lab Information Sciences Institute at USC NCSA at the University of Illinois at Urbana-Champaign SDSC at the University of California at San Diego University of Wisconsin at Madison Plus other software contributors (to date: UC Santa Barbara, U. of Michigan)  GRIDS develops, tests, deploys and supports standard tools for: Authentication, authorization, policy Resource discovery and directory services Remote access to computers, data, instruments

27. 27 The Grid: What is it?  “Resource-sharing technology with software and services that let people access computing power, databases, and other tools securely online across corporate, institutional, and geographic boundaries without sacrificing local autonomy.”  Three key Grid criteria: coordinates distributed resources using standard, open, general-purpose protocols and interfaces to deliver qualities of service not possible with pre-Grid technologies

28. 10 April 2003 28 R R R R R R R R R R R R Virtual Organizations Distributed resources and people

29. 10 April 2003 29 R R R R R R R R R R R R Virtual Organizations Distributed resources and people Linked by networks, crossing administrative domains

30. 10 April 2003 30 R R R R R R R R R R Virtual Organizations R R Distributed resources and people Linked by networks, crossing administrative domains Sharing resources, common goals VO-B VO-A

31. 10 April 2003 31 R R R R R R R R R R Virtual Organizations Distributed resources and people Linked by networks, crossing administrative domains Sharing resources, common goals Dynamic VO-B VO-A R R

32. 10 April 2003 32 R R R R R R R R R R R R VO-A VO-B Virtual Organizations Distributed resources and people Linked by networks, crossing administrative domains Sharing resources, common goals Dynamic Fault tolerant

33. 10 April 2003 33 GRIDS Center Software Suite  Globus Toolkit ®. The de facto standard for Grid computing, an open-source "bag of technologies" to simplify collaboration across organizations. Includes tools for authentication, scheduling, file transfer and resource description.  Condor-G. Enhanced version of the core Condor software optimized to work with GT for managing Grid jobs.  Network Weather Service (NWS). Periodically monitors and dynamically forecasts performance of network and computational resources.  Grid Packaging Tools (GPT). XML-based packaging data format defines complex dependencies between components.

34. 10 April 2003 34 GRIDS Center Software Suite (cont.)  GSI-OpenSSH. Modified version adds support for Grid Security Infrastructure (GSI) authentication and single sign-on capability.  MyProxy. Repository lets users retrieve a proxy credential on demand, without managing private key and certificate files across sites and applications.  MPICH-G2. Grid-enabled implementation of the Message Passing Index (MPI) standard, based on the popular MPICH library.  GridConfig. Manages the configuration of GRIDS components, letting users regenerate configuration files in native formats and ensure consistency.  KX.509 and KCA. A tool from EDIT that bridges Kerberos and PKI infrastructure.

35. 35 E-Science Benefits Substantially from GRIDS Components  Large-scale IT deployment projects rely on GRIDS components and architecture for core services BIRN, the Bioinformatics Research Network GEON, the Geoscience Network GriPhyN, Particle Physics Data Grid, International Virtual Data Grid Laboratory NEESgrid, part of the Network for Earthquake Engineering and Simulation International projects such as the UK e-Science Program and EU DataGrid  GRIDS standard tools let projects avoid building their own infrastructure Increases interoperability, efficiency Prevents “balkanization” of applications BIRN MRI Data for Brain Imaging

36. 36 Industrial and International Leaders Move to Grid Services  GRIDS leaders engage a worldwide community in defining specifications for Grid services Very active working through Global Grid Forum Over a dozen leading companies (IBM, HP, Platform) have committed to Globus-based Grid services for their products  NMI-R4 in December will include Globus Toolkit 3.0 GT3 is the first full-scale deployment of new Open Grid Services Infrastructure (OGSI) spec Significant contributions from new international partners (University of Edinburgh and Swedish Royal Institute of Technology) for database access and security UK Council for the Central Laboratory of the Research Councils (CCLRC) users rank deployment of GT3 as their #1 priority

37. 37 Acclaim for GRIDS Components  On July 15, the New York Times noted the “far-sighted simplicity” of the Grid services architecture  The Globus Toolkit has earned: R&D 100 Award Federal Laboratory Consortium Award for Excellence in Technology Transfer  MIT Technology Review named Grid one of “Ten Technologies That Will Change the World”  InfoWorld list of 2003’s top 10 innovators includes two GRIDS PIs  GRIDS co-PI Ian Foster named “Innovator of the Year” for 2003 by R&D Magazine

38. 10 April 2003 38 Future GRIDS Plans  GRIDS is completing its second year in October Original three-year award, through Fall 2004 Very successful in establishing processes, meeting twice/year release schedule, defining broadly accepted Grid middleware standards, and increasing public awareness of Grid computing  GRIDS Center 2 plans Further develop and refine core NMI releases and processes Deploy tools based on Open Grid Services Architecture Expand testing capability Create a federated bug-tracking facility Public databases: Grid Projects and Deployments System and Grid Technology Repository Increase outreach to communities at all levels: –Existing major Grid projects (e.g., TeraGrid, NEESgrid) –Major projects that should use Grid more (e.g., SEEK, NEON) –New communities not yet using Grid (e.g., Computer-Aided Diagnosis)

39. 10 April 2003 39 Upcoming Tutorials  GRIDS is extremely well-represented at SC03, the supercomputing conference Tutorials, technical papers, BoFs, demonstrations Phoenix, AZ, November 15-21 http://www.sc-conference.org/sc2003  GlobusWORLD 2004 conference Co-sponsored by GRIDS San Francisco, CA, January 20-23 Academia and Industry both well-represented http://www.globusworld.org

40. 10 April 2003 40 For More Information The GRIDS Center http://www.grids-center.org/ http://www.grids-center.org/ NSF Middleware Initiative http://www.nsf-middleware.org/ http://www.nsf-middleware.org/ The Globus Alliance http://www.globus.org/ http://www.globus.org/

41. 10 April 2003 41 NMI Integration Testbed Mary Fran Yafchak maryfran@sura.org NMI Integration Testbed Manager SURA IT Program Coordinator Southeastern Universities Research Association

42. 10 April 2003 42 USERS Implementers Target Communities NMI Integration Testbed About the NMI Testbed NMI Participation Developed and managed by SURA Evaluate NMI components upon release Real life contexts - research projects, enterprise applications and infrastructure ? future expansion UAB UAH UFL FSU GSU UMich TACC UVA Sites NMI Integration Testbed (USC) DEVELOPERSSUPPORTERSCONTRIBUTORS http://www.nsf-middleware.org/testbed

43. 10 April 2003 43 Activities this year 1)Evaluation of NMI Releases 2 & 3 2)Continued project & enterprise integration 3)Addition of REU student positions 4)Workshops & Presentations 5)Firing up of intra-Testbed grid

44. 10 April 2003 44 Evaluation of NMI R2 & R3 Once upon a time…  NMI R1, completed September 2002. 18 components, 61 reports, focused “across the board” This past year…  NMI R2, completed February 2003. 25 components, 59 reports, focused “across the board”  NMI R3, completed August 2003. 30 components, 57 reports, heavy on grids and newer authn/authz components Trend towards more “practical” evaluation

45. 10 April 2003 45 Project/Enterprise Integration Catalyzing advanced infrastructure Seven sites providing centralized identity & authentication for multiple active applications –Examples: UAB Faculty/Class emailer, UFL Wireless VPN control Four sites implementing campus grids (various stages) –MGrid, USCGrid, UTGrid (w/TIGRE), GridGroup@GSU

46. 10 April 2003 46 Project/Enterprise Integration Extending access for existing projects  Examples:  UAH: Grid-based applications with NCSA MEAD expedition and NASA Marshall Space Flight CenterNCSA MEAD expedition  UMich, GSU: Access for physicists to DOE experiments (Particle Physics Data Grid, ATLAS, CERN LHC)Particle Physics Data GridATLASLHC Expanded access for new audiences  Examples:  GSU: Student access to graphic rendering capability  GSU: Distributed muon detector in collaboration with GSU physicists and Georgia state high schools.  NSF REU (Research Experiences for Undergraduates)

47. 10 April 2003 47 NSF REU in NMI  SURA provides admin & management; sites provide experience and mentoring  5 positions at 4 sites: GSU, UMich, TACC/UVa GSU –Muon particle detector GRID for K-12 –GRID-enabled Applications Catalog UMich –NMI components in ATLAS and MGRID TACC/UVA –Grid Portals for the NMI Integration Testbed

48. 10 April 2003 48 Intra-Testbed Grid  Rationale - Testbed sites’ interest, expertise and position to contribute  Two foci - Demonstration (showcase) & Research Showcase lead: Art Vandenberg at GSU Research leads; Marty Humphrey & Jim Jokl at UVa  Key deliverables (through August 2004) - Working showcase grid and application catalog Identification and documentation of cross-campus issues (barriers to scalability) and recommendations for resolution

49. 10 April 2003 49 NMI Testbed Workshops  1st Testbed Results workshop April 2003 at Internet2 Spring Meeting  SURA NMI PACS workshops Small group training in enterprise directories and related applications (Aug & Sept. 2003)  2nd Testbed Results workshop Monday, Nov. 3, preceding EDUCAUSE 2003 See Testbed Web site for more detailsTestbed Web site Registration closes this Friday! (October 17)

50. 10 April 2003 50 NMI Testbed Presentations  At I2 meetings (Spring 2003, Fall 2003) –“Taking Grids out of the Lab and onto the Campus,” Wednesday, 10/14, 10:30 - 11:45 a.m. –“Advanced Directory Services and Applications,” Thursday, 10/15, 8:45 - 10 a.m.  At EDUCAUSE 2003 annual conference –“Experiences in Middleware Deployment: Teach a Man to Fish…,” Thursday, 11/6, 3:55 - 4:15 p.m.  At GlobusWorld 2004? –Proposed presentation on the intra-Testbed grid. Waiting to hear… Tomorrow Next

51. 10 April 2003 51 NMI and NMI-EDIT Outreach Ann West EDUCAUSE/Internet2

52. 10 April 2003 52 Topics  NMI participation model  NMI-EDIT goals, products, and results  Education opportunities

53. 10 April 2003 53 NMI Outreach Participation Overview

54. 10 April 2003 54 NMI-EDIT Outreach Goals  Informing higher-education constituents about enterprise middleware and available resources.  Guiding a broad selection of campuses interested in implementing enterprise middleware towards deployment and service integration, and assisting them in joining the emerging middleware community to benefit from peer support and developments in that community.  Serving the technical community researching and developing enterprise middleware infrastructures.  Establishing information and support persistence.

55. 10 April 2003 55 NMI-EDIT Outreach Products  Awareness presentations 22 presentations last year  Workshops and tutorials 1060 total attendance. 718 distinct participants 323 distinct organizations  Content development Articles Directory Roadmap  Community work Minority serving institutions and small colleges Registrars (AACRAO) and CFOs (NACUBO) Higher-education systems  Persistence

56. 10 April 2003 56

57. 10 April 2003 57

58. 10 April 2003 58 Education Opportunities  CAMPs Directory CAMP Feb 3-6, 2004 in Tempe AZ CAMP/Research CAMP June and November 2004  EDUCAUSE regional and annual meetings  Getting started? Check out getting started link on www.nmi-edit.org

59. 10 April 2003 59 More Information…  NMI nsf-middleware.org  NMI-EDIT www.nmi-edit.org middleware.internet2.edu  GRIDS Center www.grids-center.org

60. 10 April 2003 60