1. Grid-BGC: A Grid Enabled Carbon Cycle Modeling Environment Jason Cope and Matthew Woitaszek University of Colorado, Boulder Jason.Cope@colorado.edu Matthew.Woitaszek@colorado.edu Unidata Seminar Monday 22 January 2006

2. Department of Computer Science University of Colorado at Boulder 2 Motivation: NCAR as an Integrator qScientific workflows are becoming too complicated for manual (or semi-manual) implementation. qNot reasonable to expect a scientist to routinely: q Design simulation solutions by chaining together application software packages q Manage the data lifecycle (check out, analysis, publishing, and check in) q Do this in an evolving computational and information environment qNCAR must provide the software infrastructure to allow scientists to seamlessly (and painlessly) implement workflows

3. Department of Computer Science University of Colorado at Boulder 3 Motivation: Robust Modeling Environments qOur goal is to develop a simple, production quality modeling environment for NCAR and the geoscience community that insulates scientists from the technical details of the execution environment q Cyberinfrastructure q System and software integration q Data archiving qGrid-BGC is an example of such an environment and is the first of these environments developed for NCAR q Learning as we develop and deploy q Tasked by the geoscience community, but developed services are applicable to other collaborative research projects

4. Department of Computer Science University of Colorado at Boulder 4 Outline qIntroduction qCarbon Cycle Modeling qService Oriented Architecture for the Earth Sciences qGrid-BGC System Architecture qRe-tasking the services for other Earth Science applications qFuture Work

5. Department of Computer Science University of Colorado at Boulder 5 Introduction: Participants qThis is a collaborative project between the National Center for Atmospheric Research (NCAR) and the University of Colorado at Boulder (CU) qNASA has provided funding for three years via the Advanced Information Systems Technology (AIST) program qResearchers: q Peter Thornton (PI), NCAR q Henry Tufo (co-PI), CU q Luca Cinquini, NCAR q Jason Cope, CU q Craig Hartsough, NCAR q Rich Loft, NCAR q Sean McCreary, CU q Don Middleton, NCAR q Nate Wilhelmi, NCAR q Matthew Woitaszek, CU

6. Department of Computer Science University of Colorado at Boulder 6 Daymet inputs… …Grid-BGC outputs Carbon Cycle Modeling: Workflow

7. Department of Computer Science University of Colorado at Boulder 7 Carbon Cycle Modeling: Workflow qDaymet model interpolates a high resolution grid of weather observations for a region qBiome BGC model calculates carbon cycle parameters at each grid point qModels originally intended for analysis of small geographic regions. qAnalysis of larger regions is accomplished by simulating its composite regions

8. Department of Computer Science University of Colorado at Boulder 8 Carbon Cycle Modeling: Grid-BGC Motivation qRequires managing multiple simultaneously executing workflows q Task creation q Execution management q Data management qDistributed resource access across multiple organizations q Data archive and front-end portal are located at NCAR q Execution resources are located at CU and possibly other sites qReuse of software infrastructure q Extending the Grid-BGC workflow q Enabling other NCAR scientific applications and workflows Goal: Create an easy to use computational environment for scientists running large scale carbon cycle simulations.

9. Department of Computer Science University of Colorado at Boulder 9 Service Oriented Architecture for the Earth Sciences: Desired Service Overview qUser interface services q Portal q GUI q Command line client qData services q Mass storage service q File transfer service q Data publishing service qExecution services q Model execution service q Workflow control service q Resource allocation service qMetadata services q Registry / Index Service q Resource brokerage service

10. Department of Computer Science University of Colorado at Boulder 10 Grid-BGC: System Overview qSystem goals q Easy to use q Efficient and productive science qDevelopment summary q Prototype developed with GT 3.2 q Current system redeveloped with GT4 q Integrates resources from NCAR and CU qArchitecture Implementation q Production system is not a pure service oriented architecture q Research and development system is a service oriented architecture

11. Department of Computer Science University of Colorado at Boulder 11 Service Oriented Architecture for the Earth Sciences: Implemented Services qUser interface services q Portal q GUI q Command line client qData services q Mass storage service q File transfer service q Data publishing service qExecution services q Model execution service q Workflow control service q Resource allocation service qMetadata services q Registry / Index Service q Resource brokerage service

12. Department of Computer Science University of Colorado at Boulder 12 Service Oriented Architecture for the Earth Sciences: Implemented Services qUser interface services q Portal q GUI q Command line client qData services q Mass storage service q File transfer service q Data publishing service qExecution services q Model execution service q Workflow control service q Resource allocation service qMetadata services q Registry / Index Service q Resource brokerage service

13. Department of Computer Science University of Colorado at Boulder 13 Grid-BGC: System Architecture

14. Department of Computer Science University of Colorado at Boulder 14 Grid-BGC Portal qWeb interface to Grid-BGC qJSP / Tomcat implementation using CoG Kit qComposed of logical services

15. Department of Computer Science University of Colorado at Boulder 15 Grid-BGC Execution Services qExecution service contains all functionality needed to run a model and is aware only of those models qProvides interface to request and initialize a model run q Creates directory structure q Creates model initialization files q Registers file transfers and executables with the workflow manager qProvides interfaces to query, terminate, and cleanup requested model runs

16. Department of Computer Science University of Colorado at Boulder 16 Workflow Control Service and Workflow Manager qWorkflow Control Service q Provides functions to register workflow tasks, model executions, and file transfers q Execution service uses the workflow control service functions to register its tasks q Workflow control service stores the workflow metadata in a persistent database qWorkflow Manager q Periodically queries the workflow metadata database for new tasks to execute q Delegates file transfers to the Reliable File Transfer service (RFT) and job executions to the Grid Resource and Allocation Management Service (GRAM)

17. Department of Computer Science University of Colorado at Boulder 17 Example Grid-BGC Workflow

18. Department of Computer Science University of Colorado at Boulder 18 Operational Experience qUser Interface has been externally beta tested q Beta testers from q University of Wisconsin q Utah State University q WSL Switzerland q Feedback helped improve users interactions with the portal qGrid computing and modeling environment beta tested internally q Short term productivity gains have been realized using this system

19. Department of Computer Science University of Colorado at Boulder 19 Current Grid Topology

20. Department of Computer Science University of Colorado at Boulder 20 Grid Enabling CAM and POP qCommunity Atmosphere Model (CAM) q Developed by NCAR q Atmospheric component of NCAR’s Community Climate System Model (CCSM) qParallel Ocean Program (POP) q Developed by the DOE at the Los Alamos National Laboratory q Ocean component of CCSM q Grid Enabling CAM and POP q Re-tasked the grid service and workflow subsystem to run CAM and POP q New components q Execution services q Client interfaces for accessing the services q Reused components q Workflow subsystem and service q Service registry q Service communication package

21. Department of Computer Science University of Colorado at Boulder 21 Future Work: Expansion of the Grid-BGC Environment qIntegrate new computational resources q Integrate NASA’s Columbia Supercomputer into the Grid-BGC environment q Integrate resources provided by the system’s users (University of Wisconsin, …) q TeraGrid qContinue to break out the desired services from current system components qContinue to evolve system architecture into a service oriented architecture (SOA) qVisualization

22. Department of Computer Science University of Colorado at Boulder 22 Future Work: Grid Enabling More Earth Science Applications

23. This research was supported in part by the National Aeronautics and Space Administration (NASA) under AIST Grant AIST-02-0036, the National Science Foundation (NSF) under ARI Grant #CDA-9601817, and NSF sponsorship of the National Center for Atmospheric Research. Grid-BGC: A Grid Enabled Carbon Cycle Modeling Environment Questions? Ideas? Comments? Suggestions? http://www.gridbgc.ucar.edu Presenter’s email: Jason.Cope@colorado.edu Jason.Cope@colorado.edu