1. Climate Modeling in a Browser The Development of an Earth System Climate Modeling Science Gateway Carol X. Song, Ph.D. Senior Research Scientist Rosen Center for Advanced Computing Purdue University carolxsong@purdue.edu

2. A Great Team! Purdue: Lan Zhao, Chris Thompson, Madhavan Lakshminarayanan, Wonjun Lee, Han Zhang NCAR: Don Middleton, Nathan Wilhelmi, Eric Nienhouse, Michael Burek NOAA/CIRES: Cecelia DeLuca and Sylvia Murphy Kathy Saint, SGI “Developing an integrated end-to-end TeraGrid climate modeling environment”, TeraGrid 2011 conference, Salt Lake City, UT, July 2011. 9/12/11CAS2K11, Annecy, France2

3. Background Rosen Center for Advanced Computing, part of central IT Community clusters to address computing needs Resource provider to national cyberinfrastructure, e.g., TeraGrid/XSEDE, Open Science Grid (OSG) Scientific Solutions group 9/12/113CAS2K11, Annecy, France

4. Science Gateways Platforms to make applications available, lower barriers, and enable online collaboration  CCSM3 portal, LEAD portal, …  Nanohub.org TeraGrid Science Gateway initiative  Assist gateway developers to integrate/utilize TeraGrid HPC resources  Partially funded the Earth System Climate Model gateway Motivation for a climate modeling gateway  Resource  Expertise  A collaborative online environment 9/12/114CAS2K11, Annecy, France

5. Project Goal Lower barrier by providing an integrated environment for climate modeling, data/metadata archiving, visualization and analysis Builds on existing synergistic community efforts  ESG, ESG-Curator, METAFOR and Purdue CCSM3 portal Develop a prototype end-to-end environmental science gateway on the TeraGrid  Complete model workflow from CESM portal  Comprehensive and standard metadata describing each model run  Publishing data/metadata from CESM portal to ESG/ESGC gateway Support education and research use Collaboration between Purdue, NCAR/NOAA 9/12/11CAS2K11, Annecy, France5

6. Evolutions CESM evolution  CCSM3.0 (June 2004)  CCSM4.0 (April 2010)  CESM1.0.3 (June 2010)  CESM 1.0.x or….  New infrastructure and data model  being instrumented with Earth System Modeling Framework (ESMF) Attributes for metadata storage and output http://www.earthsystemmodeling.orghttp://www.earthsystemmodeling.org  Climate Modeling portals  Released CCSM3 portal two years ago  CESM gateway to be released soon as beta, Release 1 next spring.  TeraGrid  XSEDE 9/12/11CAS2K11, Annecy, France6

7. Metadata is increasingly important in climate modeling  Understand multi-component models with many options and parameters  Distinguish simulations from one another  Archive simulations with full descriptions for preservation and defensibility  Enable run reproducibility ESMF Attributes store, aggregate, and output metadata  Structured as name-value pairs  Arranged in packages by purpose and convention: ISO, Climate and Forecast (CF), METAFOR Common Information Model, etc.  Output in XML and other formats ESMF Attributes in CESM  All CESM components (land, atm, ice, ocean, sea ice) and driver are being instrumented with Attributes  Information captured includes component level, field level, and system level metadata Self-describing Models 9/12/11CAS2K11, Annecy, France7

8. First implementation: CCSM3 Portal CCSM simulation steps  Create case, configure, submit Status monitoring, feedback Basic vs. advanced user functions File transfer Post processing 9/12/11CAS2K11, Annecy, France8

9. System View End-to-end modeling from simulation to data publishing Integrate CESM 1.0 + metadata capabilities + ESG publishing + NCL Browser GUI and web services (SOAP, Restful XML/JSON) 9/12/11CAS2K11, Annecy, France9 TG Cluster Nodes Publication (VM) ESG Gateway Shared Data Repository CESM Portal JBOSS Job ScriptsPublish Scripts SOAP

10. System Architecture 9/12/11CAS2K11, Annecy, France10 Create Case Configure Case Submit Case Authentication/ Authorization Track Status Post-process Debugging Transfer Files Publish Metadata ESG Data Publisher iRODSScratch Storage Token Mgr Data/ Metadata JobsOutput CESM Web Services User requests Job Management CESM portal ESG gateway Output TG MyProxy Account DB Publish Data TG community account and individual account SOAP and Restful XML/JSON

11. CESM Portal - Model Workflow Insulate users from cluster  Hide command line details  Handle scheduling Manage user’s experience  Maintain job status and metrics  Direct movement through process  Bind independent processes into unified workflow CESM Portal JBOSS Job ScriptsPublish Scripts SOAP Create case Configure case Compile case Execute case CESM-WS/ gateway Create Configure Compile Execute Query Status CESM Model create_case./configure.compile.submit Data Post-processing Post-processing Engines AMWG NCL scripts … Configure post- process Run post-process ESG Publication workflow 9/12/201111CAS2K11

12. Model Data Publishing 9/12/11CAS2K11, Annecy, France12 Publication queue check script Wait interval Call publish script Publication Interface Publish Queue New dataset to publish? Exit Portal processes –Configure publish via XML file just like rest of CESM options –Create entry in publish queue –Monitor queue for status ESG data node integration –Lightweight additions to standard ESG publisher node –Monitor shared space for new queue entries & script publishing 1.Parse options & configure publisher 2.Publish dataset 3.Update status to shared data 4.Process metadata Publish Query No Yes ESG Data Node @Purdue publish

13. Metadata Publishing ESG metadata handled through separate ATOM feed server  Runs on ESG publisher node  Moving file containing metadata to staging area  ESG gateway periodically parses feed which pulls from staging area Unobtrusive integration  CESM portal “publish” script: moves metadata file to ATOM area  ATOM server picks up from there 9/12/11CAS2K11, Annecy, France13 Data ATOM Feed Server Publication Process ESG Gateway ESG Data Node @Purdue CIM Documents

14. 9/12/201114CAS2K11

15. 9/12/201115CAS2K11

16. ESG-Curator Metadata Display 9/12/11CAS2K11, Annecy, France16 Metadata linked to model output Hierarchical display organizes information from multiple components Co-developed by Curator, ESG and the METAFOR project Metadata linked to model output Hierarchical display organizes information from multiple components Co-developed by Curator, ESG and the METAFOR project

17. A joint grad class of political science and earth and atmospheric sciences in fall 2009 “Great issues: climate and policy” Instructors: Drs. Matthew Huber (EAS) and Leigh Raymond (POL) Focus on hands-on learning about the model and its data analysis, rather than dealing with systems, commands and syntax. (Couldn’t do without CCSM gateway) Student projects:  100 year simulation runs (Couldn’t run without TG resource)  Study climate change impact on political policies Use case: Political Science class 9/12/1117CAS2K11, Annecy, France

18. 9/12/11CAS2K11, Annecy, France18 Reforestation of tropical regions to increase Precipitation and decrease temperature Vegetation albedo increase -> Lower temperature?

19. Agriculture: Investigation of the impact of wetland drainage on Midwest Hydro-climatology using coupled atmospheric and land surface models in CCSM (Kumar and Merwade) 9/12/1119CAS2K11, Annecy, France Pre- settlement wetland Present wetland extent

20. Next Steps Federated authentication/identity between CESM and ESG gateways Leverage work of LAS integration with ESG and NCL as backend, making a rich set of tools for analysis and visualization available to CESM gateway users Add other resources to backend for better user experience Work with faculty and instructors to use in teaching 9/12/11CAS2K11, Annecy, France20

21. References S. Kumar, V. Merwade, W. Lee, L. Zhao and X.C. Song. “Hydro- climatological Impact of Century Long Drainage in Midwestern United States: CCSM Sensitivity Experiments”, the Journal of Geophysical Research - Atmospheres, 2010 L. Zhao, W. Lee, C. X. Song, M. Huber and A. Goldner. "Bringing High Performance Climate Modeling into the Classroom", TeraGrid 2010 conference, Pittsburgh, PA, August 2010 A. P. Goldner, M. Huber, R. L. Sriver. "Is the Future State of North American Hydroclimatology Controlled by Tropical Cyclones and the Evolution of El Niño?“, AGU Fall meeting, San Francisco, CA, December 2010 9/12/1121CAS2K11, Annecy, France