1. DOE BER Climate Modeling PI Meeting, Potomac, Maryland, May 12-14, 2014 Funding for this study was provided by the US Department of Energy, BER Program. Contract # DE-AC02-05CH11231 Analysis SCIENCE DRIVER The CASCADE climate group's charter has two major thrusts: detection & attribution of extreme events and climate model's ability to simulate extreme events, both of which are grounded heavily using statistical methodologies and plan to utilize scalable HPC-aware software infrastructure to handle current and upcoming climate science challenges through a common unified workflow. Addressing emerging requirements for the analysis of extreme events is a growing challenge in the climate science community. The scale of data currently at terabytes will only grow larger, while processing three to six hours of intervals will become a more frequent occurrence, and focus increasingly on high resolution datasets (i.e, 1/4th to 1/8th degree and beyond). The high resolution and high frequency analysis will be several orders of magnitude greater, resulting in critical need for effective utilization of HPC resources and a software infrastructure and workflow designed to take advantage of these resources. CASCADE Workflow DESIGN OF METHODS SCIENCE IMPACT The CASCADE software infrastructure team is tasked with providing a streamlined interoperable infrastructure and expertise in scaling algorithms, simplifying the exercise and coordination of large ensemble runs for analysis and computation of uncertainties, and leading deployment efforts with the focus on modular, extensible components and an emphasis on usability. This section highlights three characteristic instances of the CASCADE unified workflow: handling performance challenges, scalability challenges for model fidelity, and providing scalable statistical analysis routines. These components provide the building blocks of use cases from the members of each of the other CASCADE teams. The performance pipeline highlights effort required to speed up analysis routines exercised hundreds to thousands of times, the model-fidelity pipeline highlights effort required to provide scalable ensemble execution, and finally the statistical analysis pipeline highlights the efforts to parallelize spatio-temporal statistical routines such as extreme value analysis which is then utilized within the model fidelity and detection & attribution. The examples below show how resources and efforts are shared within a more unified construction of work that manages effective utilization of resources, data movement, scheduling, and management. Climate-Centric WorkFlow Environment Identification of Use Cases Extraction of Computational Algorithms Scaling & Optimization of Work Templated Workflow Configurations for Common Use Cases Abstraction of Services to HPC environments Archiving, Distribution, and Verification Strategy TODO.. DEMONSTRATION OF METHODS Anatomy of a Climate Science Workflow CASCADE The CASCADE Team D&A Statistics Model I/O - Spatio-Temporal: One: All Grid + All Time One: All Grid + All Time Multi: All Time, Ensemble Members Multi: All Time, Ensemble Members Parallel Execution: For Each Location Execute R-based Extreme Value Analysis Algorithms Parallel Execution: For Each Location Execute R-based Extreme Value Analysis Algorithms Data Reduction & Output: Output Multiple Results such return values and standard error to NETCDF Data Reduction & Output: Output Multiple Results such return values and standard error to NETCDF DATA TRACKING PERFORMANCE METRICS FAULT TOLERANCE CUSTOM ANALYSIS CUSTOM ANALYSIS BATCH PROCESING I/O: Data Movement and Staging Parallel Execution: Analysis over Yearly|Monthly|Hourly Data Verification and Archiving Distribution: To ESGF Service at NERSC Distribution: To ESGF Service at NERSC Analysis: TECA| EVA| Custom Analysis: TECA| EVA| Custom Parallel Execution: Create Initial Condition Parallel Execution: Create Initial Condition Merging Separate Tasks Reanalysis Data Parallel Execution: Run CESM Parallel Execution: Run CESM Parallel Execution: Run CESM Parallel Execution: Run CESM Analysis Comparative Analysis Hindcast results from CESM runs at varying spatial resolution Comparative Analysis Hindcast results from CESM runs at varying spatial resolution FILE System/HPSS PROVENANCE DATA MANAGEMENT FILE System Repeat for a range of partial simulation resolutions D & A Workflow Model Fidelity Stats Analysis Unified Workflow Service CASCADE Workflow Module Job Scheduler Fault Tolerance Performance Analysis Data Tracking CASCADE Workflow Module Input/Output Numpy Modules TECA EVA, GEV MpiTaskWrapper Archiving, & Distribution TECA EVA, GEV MpiTaskWrapper Archiving, & Distribution User, Data, and Resource Management Remote Clients TECA EVA, GEV MpiWrapper Archiving, & Distribution TECA EVA, GEV MpiWrapper Archiving, & Distribution Resource Configuration ----- Common Usage Templates Resource Configuration ----- Common Usage Templates Data Reduction Validation & Verification Validation & Verification Optimization Strategies DepCache MPI Threading Optimization Strategies DepCache MPI Threading