Tuning GENIE Earth System Model Components using a Grid Enabled Data Management System Andrew Price University of Southampton UK.

Slides:



Advertisements
Similar presentations
QCDgrid User Interfaces James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.
Advertisements

© Geodise Project, University of Southampton, Applying the Semantic Web to Manage Knowledge on the Grid Feng Tao, Colin.
Tuning GENIE Earth System Model Components using a Grid Enabled Data Management System Andrew Price, Gang Xue, Andrew Yool, Dan Lunt, Tim Lenton, Jasmin.
© Geodise Project, University of Southampton, Short Message Service Aims Architecture Tools Future Work.
Grid Enabled Optimisation and Design Search for Engineering (G EODISE ) Prof Simon Cox Southampton University 3 rd Annual Workshop on Linux Clusters for.
© Geodise Project, University of Southampton, Geodise: Taking the Grid to the Engineer Graeme Pound International Summer.
1 OBJECTIVES To generate a web-based system enables to assemble model configurations. to submit these configurations on different.
© Geodise Project, University of Southampton, Applications and Middleware Hakki Eres, Fenglian Xu & Graeme Pound.
Part 1a: Overview of the UM system
The Community Authorisation Service – CAS Dr Steven Newhouse Technical Director London e-Science Centre Department of Computing, Imperial College London.
MTA SZTAKI Hungarian Academy of Sciences Grid Computing Course Porto, January Introduction to Grid portals Gergely Sipos
© Geodise Project, University of Southampton, CFD-based Shape Optimisation Using Geodise Toolkits Application Demo of.
Application of GRID technologies for satellite data analysis Stepan G. Antushev, Andrey V. Golik and Vitaly K. Fischenko 2007.
4b.1 Grid Computing Software Components of Globus 4.0 ITCS 4010 Grid Computing, 2005, UNC-Charlotte, B. Wilkinson, slides 4b.
TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
The SAM-Grid Fabric Services Gabriele Garzoglio (for the SAM-Grid team) Computing Division Fermilab.
QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.
© Geodise Project, University of Southampton, Geodise: A Grid-enabled PSE for design search and optimisation Graeme Pound.
Grid Enabled Optimisation and Design Search for Engineering (GEODISE)
© Geodise Project, University of Southampton, Data Management in Geodise Jasmin Wason, Zhuoan Jiao and Marc Molinari Engineering.
XCAT Science Portal Status & Future Work July 15, 2002 Shava Smallen Extreme! Computing Laboratory Indiana University.
Metadata Creation with the Earth System Modeling Framework Ryan O’Kuinghttons – NESII/CIRES/NOAA Kathy Saint – NESII/CSG July 22, 2014.
Holding slide prior to starting show. A Grid-based Problem Solving Environment for GECEM Maria Lin and David Walker Cardiff University Yu Chen and Jason.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
A Draft Palaeoclimate Strategy for NCAS 1. Why palaeoclimate? 2. Grand challenges 3. Strategy aim 4. Strategy components 5. Wider implications Resources.
Service-enabling Legacy Applications for the GENIE Project Sofia Panagiotidi, Jeremy Cohen, John Darlington, Marko Krznarić and Eleftheria Katsiri.
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
© Geodise Project, University of Southampton, Data Management in Geodise Zhuoan Jiao, Jasmin Wason and Marc Molinari
Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.
QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.
Using ICENI to run parameter sweep applications across multiple Grid resources Murtaza Gulamali Stephen McGough, Steven Newhouse, John Darlington London.
1 Overview of the Application Hosting Environment Stefan Zasada University College London.
© Geodise Project, University of Southampton, GEODISE: Grid-enabled toolkits for the Engineer Andrew Price UK e-Science Programme,
1 All-Hands Meeting 2-4 th Sept 2003 e-Science Centre The Data Portal Glen Drinkwater.
ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.
Wrapping Scientific Applications As Web Services Using The Opal Toolkit Wrapping Scientific Applications As Web Services Using The Opal Toolkit Sriram.
“Grids and eScience” Mark Hayes Technical Director - Cambridge eScience Centre GEFD Summer School 2003.
The UK eScience Grid (and other real Grids) Mark Hayes NIEeS Summer School 2003.
Holding slide prior to starting show. A Portlet Interface for Computational Electromagnetics on the Grid Maria Lin and David Walker Cardiff University.
Applications & a Reality Check Mark Hayes. Applications on the UK Grid Ion diffusion through radiation damaged crystal structures (Mark Calleja, Mark.
© Geodise Project, University of Southampton, Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.
Web Portal Design Workshop, Boulder (CO), Jan 2003 Luca Cinquini (NCAR, ESG) The ESG and NCAR Web Portals Luca Cinquini NCAR, ESG Outline: 1.ESG Data Services.
Building the e-Minerals Minigrid Rik Tyer, Lisa Blanshard, Kerstin Kleese (Data Management Group) Rob Allan, Andrew Richards (Grid Technology Group)
EGEE User Forum Data Management session Development of gLite Web Service Based Security Components for the ATLAS Metadata Interface Thomas Doherty GridPP.
May 6, 2002Earth System Grid - Williams The Earth System Grid Presented by Dean N. Williams PI’s: Ian Foster (ANL); Don Middleton (NCAR); and Dean Williams.
Mercury – A Service Oriented Web-based system for finding and retrieving Biogeochemical, Ecological and other land- based data National Aeronautics and.
© Geodise Project, University of Southampton, Data Management in Geodise Zhuoan Jiao, Jasmin Wason & Marc Molinari { z.jiao,
© Geodise Project, University of Southampton, CFD-based shape optimisation using Geodise toolkits Nacelle Optimisation.
Edinburgh e-Science MSc Bob Mann Institute for Astronomy & NeSC University of Edinburgh.
Testing Grid Software on the Grid Steven Newhouse Deputy Director.
© Geodise Project, University of Southampton, Grid middleware for engineering design search and optimisation Graeme Pound.
Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing
© Geodise Project, University of Southampton, Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003.
© Geodise Project, University of Southampton, Integrating Data Management into Engineering Applications Zhuoan Jiao, Jasmin.
John Kewley e-Science Centre All Hands Meeting st September, Nottingham GROWL: A Lightweight Grid Services Toolkit and Applications John Kewley.
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
Performance guided scheduling in GENIE through ICENI
© Geodise Project, University of Southampton, Geodise Compute Toolbox Functions CommandFunctionCommandFunction gd_certinfo.
© Geodise Project, Scenario: Design optimisation v Model device, discretize, solve, postprocess, optimise Scripting.
Holding slide prior to starting show. Lessons Learned from the GECEM Portal David Walker Cardiff University
On Advanced Scientific Understanding, Model Componentisation and Coupling in GENIE Sofia Panagiotidi, Eleftheria Katsiri and John Darlington.
© Geodise Project, University of Southampton, Workflow Support for Advanced Grid-Enabled Computing Fenglian Xu *, M.
Intersecting UK Grid & EGEE/LCG/GridPP Activities Applications & Requirements Mark Hayes, Technical Director, CeSC.
Grid Execution Management for Legacy Code Architecture Exposing legacy applications as Grid services: the GEMLCA approach Centre.
Simulation Production System Science Advisory Committee Meeting UW-Madison March 1 st -2 nd 2007 Juan Carlos Díaz Vélez.
Grid Enabled Optimisation and Design Search for Engineering (GEODISE) Prof Simon Cox Southampton University
GEODISE: Grid-enabled toolkits for the Engineer
D. Lunt (1), A. Yool (2), R.Marsh(2), P.Valdes(1) , and the GENIE team
Transient simulations of the last 30,000 years, within the GENIE earth-system framework D.J. Lunt (1) M.Williamson (2) A. Price (3) P.J. Valdes (1)
Presentation transcript:

Tuning GENIE Earth System Model Components using a Grid Enabled Data Management System Andrew Price University of Southampton UK e-Science Programme, Booth 2140 SC2004, Pittsburgh, USA 11 th November 2004

Outline Introduction Scientific aims of GENIE e-Science tools –Data Management System –Geodise Toolboxes –OPTIONS Design Search and Optimisation Results Future work Conclusions

Introduction The GENIE project is developing a Grid-based system to: Flexibly couple together state-of- the-art components to form a unified Earth system model Execute the resulting model on the Grid Share the distributed data produced in simulations Provide high-level open access to the system, creating and supporting virtual organisations of Earth system modellers

Scientific Aims ice-age Orbital parameters affect incident radiation and climate Biological and geological processes interact with, and feedback upon, the climate (via, for instance, CO 2 )

The target GENIE Model 3D atmosphere 3D ocean 2D sea ice Atmospheric CO 2 2D land surface Land vegetation Ocean biogeochemistry Ocean sediments 3D ice sheets Atmosphere – Bristol’s IGCM3 Ocean – SOC’s GOLDSTEIN Land – Met. Office’s TRIFFID Land ice – Bristol’s GLIMMER Ocean biogeochemistry and sediments – UEA’s BioGEM

Initial GENIE experiments Initial studies in GENIE performed parameter sweeps to investigate the properties of the model

e-Science Tools Data Management System (augmented version of the Geodise Database System) Matlab scripting environment Geodise Toolboxes XML Toolbox OPTIONS Design Search and Optimisation package Template and Example scripts

Data Management System Java Client Code Apache Axis CoG Jython Functions Globus Server Geodise Database Toolbox Metadata Database ClientGrid SOAP Matlab Functions Database Web Services Authorisation Service Location Service Metadata Archive & Query Services Jython XML Schema GridFTP Portal

Data Management Client

Geodise Toolboxes Geodise Compute Toolbox gd_createproxy.m Creates a Globus proxy certificate for the user's credentials gd_destroyproxy.m Destroys the local copy of the user's Globus proxy certificate gd_jobsubmit.m Submits a compute job to a Globus GRAM job manager gd_jobstatus.m Gets the status of a Globus GRAM job gd_putfile.m Puts a remote file using GridFTP gd_getfile.m Retrieves a remote file using GridFTP gd_rmfile.m Deletes a remote file using GridFTP gd_makedir.m Creates a remote directory using GridFTP gd_rmdir.m Deletes a remote directory using GridFTP Geodise Database Toolbox gd_archive.m Archives a file or data structure to the database gd_query.m Query the database for data matching specified criteria. gd_retrieve.m Retrieves a file or data structure from the database

Grid Computation OptionsMatlab UK National Grid Service (GT2) OxfordLeeds RALManchester Local Resource (GT2) GENIE Database objfun.mobjfun_parse.m optjobparallel.m OptionsMatlab invokes the wrapped binary using parameters provided by the selected optimisation algorithm. Soton Condor Pool (1700 Nodes)

IGCM3 Atmosphere Model The objective function is a weighted sum of the RMS differences between the model state and NCEP data. Compare annual means for a number model fields.

IGCM3 Tuning Results 40% reduction in error statistic compared to default parameters Similar result to a parallel study performed using the Ensemble Kalman Filter Model physics insufficient to perfectly match observational data.

e-Science Summary

Conclusions Provided the environmental scientist with a toolset for tuning GENIE models: –Scripting environment –Database repository –Computational Grid interface –Suite of generic optimisation algorithms A Global minimum can reliably be found in low dimensional problem space. For higher dimensional problems, the tools are appropriate for locating local minima in the state space.

The GENIE Team Coordinator: Tim Lenton– CEH Edinburgh Principal investigator: Paul Valdes – Bristol Research Team and Collaborators: James Annan – FRSGC, Japan Chris Brockwell – UEA Norwich David Cameron – CEH Edinburgh Peter Cox – Hadley Centre (UKMO) Neil Edwards – Bern, Switzerland Murtaza Gulamali – London e-Science Centre Julia Hargreaves – FRSGC, Japan Phil Harris – CEH Wallingford Dan Lunt – Bristol Bob Marsh – SOC Andrew Price – Southampton e-Science Centre Andy Ridgwell – UBC, Canada Ian Rutt – Bristol Gang Xue – Southampton e-Science Centre Andrew Yool – SOC Management Team: Melvin Cannell – CEH Edinburgh Trevor Cooper-Chadwick – Southampton e-Sci. Centre Simon Cox – Southampton e-Sci. Centre John Darlington – London e-Science Centre Richard Harding – CEH Wallingford Tony Payne – Bristol John Shepherd – SOC Andrew Watson – UEA Norwich Thanks to Steven Newhouse