Atmospheric Science Data Center HPC User Forum Michael Little September 10, 2009 Broomfield, CO

9/9/09NASA ASDC HPCU ForumPage: 2 Agenda – September 09, 2009 Recent NASA Science Mission Directorate Activities SMD Modeling Needs Study 2008/2009 – Tsengdar Lee Decadal Survey Data Systems (DSDS) –Martha Maiden June, Atmospheric Science at NASA Langley Research Center Science Focus Atmospheric Science Data Center NextGen Net-centric Operations

Atmospheric Remote Sensing Instruments &in situ measurements –Earth Radiation Budget (ERBE, CERES) ▫40 years of internally consistent measurements (<1% err) ▫TRMM, AQUA, TERRA, NPP*, NPOESS* –Atmospheric Chemistry (SAGE) –Clouds and aerosols (CALIPSO) Analysis of measurements –Radiation and Clouds in Climate –Clouds properties Data Fusion is key to all higher order data products 9/9/09Page: 3NASA ASDC HPCU Forum

Earth Radiation Budget CERES Measures the Earth’s Radiant Energy Balance 9/9/09Page: 4NASA ASDC HPCU Forum

CALIPSO LIDAR Data 9/9/09Page: 5NASA ASDC HPCU Forum

Climate Science Needs for Computation High accuracy (reduce error to <1% in measurement) –Calibrate, calibrate, calibrate –Current weather measurements are ~15% Re-processing long time series –Internal consistency –100x (100 days of observations re-processed in 1 day of compute time) –Cannot guarantee re-processing will give same data Easy to Use Development Environment –Physical Scientists frequently have weak computer science skills (no formal training) –Resistance to change –Scientists are more valuable than compute cycles –MATLAB and IDL/ENVI 9/9/09Page: 6NASA ASDC HPCU Forum

Atmospheric Science Data Center ESDIS Digital Active Archive Center –Data Sets ▫ERBE, CERES, CALIPSO, SAGE, TES, MISR –6 Functions: Ingest, Archive, Produce, Distribute, Report, Support –Produce/re-process instrument data products ▫ CERES, CALIPSO, MISR, SAGE –Distribute Released Data Products Science Computing Facility –Local research and missions(300 users) –Develop/test code to produce data products –Instrument calibration (including pre-launch ground cal) –Data Product calibration –Analysis of data –Commercial Applications: MatLab and IDL/ENVI 9/9/09Page: 7NASA ASDC HPCU Forum

Recent Changes at ASDC Replace legacy systems with computational environment with high I/O to disk systems –Big endian and little endian due to code migration issues –Increase processor availability with minimum system-wide outages –Fiberchannel interconnect –Improve reliability, maintainability –Reduce SysAdmin workload –Put all interesting data online (2PB) –IBM x3950, x3650, P6 processors –IBM DS4800, DS5300 Disk Arrays with lots of redundancy –Brocade M48k Director –Sun Grid Engine Improve stewardship and data protection –All sites have local environmental risks ▫Key products stored offsite on tape –Tape archive and backup –SunSTK 8500 with Quantum StoreNext Code conversion took > 5 years –Scientifically identical data products 9/9/09Page: 8NASA ASDC HPCU Forum

64 x86 SMP 256 GB ASDC/SCF Fiberchannel Connectivity Fiberchannel Switch 64 x86 SMP 256 GB 56 P6 256GB 56 P6 256GB 56 P6 256GB 56 P6 256GB 220TB DS TB DS TB DS TB DS TB DS x86 SMP 128 GB 16 x86 HS12 64 GB Common SANhead (NFS) (AFP) (Samba) ASDC JBOSS Server & TS-1 & Ingest ASDC SDP Production 64 x86 SMP 256 GB 56 P6 256GB SCF Cluster (Access through Sun Grid Engine) 220TB DS TB DS4800 SCF Disk Array Instrument ERBE-like SRB/Power SARB 14 SCF Interactive Processors ASDC Disk Archive (R/W Production) (R/O SCF) Consodine TISA SOFA Clouds TISA 220TB DS4800 WWW 64 x86 SMP 128 GB Cloud Modeling Cluster (Xu) 64 x86 SMP 128 GB ECS DUE Processing Db Server Db Server Tesla 3755 GPU Experiment (YongHu) Sim Server FC Monitor LB1 9/9/09Page: 9NASA ASDC HPCU Forum

Projected Work at ASDC Improve performance of processing and storage –Tune SGE –Fiberchannel switch configurations –Improve quality of SysAdmin Support Train Users to leverage investment Model computational environment –SIMSCRIPT III Improve Distribution Function –Google Earth –Permit direct access of data by non-local research community –Service Oriented Architecture (SOA) 9/9/09Page: 10NASA ASDC HPCU Forum

NextGen National Airspace System Trajectory Based Operations requires Common Operating Picture among everyone using NAS Avionics Roadmap created in November, 2008 Scope is curb-to-curb instead of gate-to-gate NextGen Joint Planning and Development Office –Created by Public Law , VISION 100 Act –Coordination function, not PMO –3x air traffic, safer, cheaper with less environmental impact by 2025 – –Industry participates via NextGen Institute to avoid Conflict of Interest Governance by Collaboration among Partner Agencies and Companies –FAA, Defense, Homeland Security, Commerce, NASA, EPA –Airlines, Airports 9/9/09Page: 11NASA ASDC HPCU Forum

NextGen Net-centric Operations Leverage DoD experience with Net-centric Operations JPDO Net-centric Operations Division leads collaboration –USAF CIO, Electronic Systems Center, AF Weather Agency –FAA SWIM PM, FAA Telecommunications Infrastructure –NOAA Office of the Federal Coordinator for Meterological Services and Supporting Research –NWS Aviation Weather Office –NASA LaRC, GSFC and eventually ARC and DFRC –DHS Chief Technology Officer and CIO –NASA, DARPA and NSF supporting R&D Industry partnership via Net-centric Working Group –Membership cleared through NextGen Institute – Lead business process re-engineering, info exchange design –Info owners to implement information exchanges –Ensure access to information for all parties in NAS Semantic Service Oriented Architecture (SOA) –Machine-to-machine readability is essential 9/9/09Page: 12NASA ASDC HPCU Forum

NextGen Semantic SOA 4D Weather Cube is first implementation –2013 Initial Operational Capability for Info Exchange Services –Development at MIT Lincoln Labs and NCAR –FAA NNEW and Weather in the Cockpit programs –NWS Aviation Weather Office Other Business Areas with Information for Exchange –Safety –Aircraft –Airports –Flight/Flow –Unmanned Aerial Vehicles Key Issues –Cultural Change –NCO Governance –Cybersecurity –Infrastructure: computation and data communications 9/9/09NASA ASDC HPCU ForumPage: 13 -Integrated Surveillance -Environmental Impact -Airports -Special Use Airspace