Joint Center for Satellite Data Assimilation Overview of activities Lars Peter Riishojgaard, JCSDA Director 1

Overview JCSDA Overview Background, structure, accomplishments Current activities Future plans Summary JCSDA Science Workshop, NCWCP College Park 2October

3 NASA/Earth Science Division US Navy/Oceanographer and Navigator of the Navy and NRL NOAA/NESDISNOAA/NWS NOAA/OAR US Air Force/Director of Weather Mission: …to accelerate and improve the quantitative use of research and operational satellite data in weather, ocean, climate and environmental analysis and prediction models. Vision: An interagency partnership working to become a world leader in applying satellite data and research to operational goals in environmental analysis and prediction JCSDA Partners, Vision, Mission JCSDA Science Workshop, NCWCP College Park October

JCSDA Science Workshop, NCWCP College Park 4 JCSDA Science Priorities Radiative Transfer Modeling (CRTM) Preparation for assimilation of data from new instruments Clouds and precipitation Assimilation of land surface observations Assimilation of ocean surface observations Atmospheric composition; chemistry and aerosol Driving the activities of the Joint Center since 2001, approved by the Science Steering Committee Overarching goal: Help the operational services improve the quality of their prediction products via improved and accelerated use of satellite data and related research October

JCSDA Mode of operation October JCSDA Science Workshop, NCWCP College Park 5 Directed research Carried out by the partners Mixture of new and leveraged funding NOAA appropriation comes through NESDIS/STAR JCSDA plays a coordinating role External research Grants or contracts awarded to by one of the JCSDA parent agencies on a rotating basis Open to the broader research community Funding awarded competitively, based on peer-reviewed proposals Visiting Scientist program

JCSDA Science Workshop, NCWCP College Park JCSDA accomplishments Common assimilation infrastructure (NOAA, GMAO, AFWA) Community radiative transfer model (all partners) Common NOAA/NASA land data assimilation system (NOAA, GSFC, AFWA) Numerous new satellite data assimilated operationally, e.g. MODIS (winds and AOD), AIRS and IASI hyperspectral IR radiances, GPSRO sensors (COSMIC, GRAS, GRACE), SSMI/S, Windsat, Jason- 2,… Advanced sensors tested for operational readiness, e.g. ASCAT, MLS, SEVIRI (radiances),… Ongoing methodology improvement for sensors already assimilated, e.g. AIRS, GPSRO, SSMI/S,… Improved physically based SST analysis Adjoint sensitivity diagnostics 6October

JCSDA Science Workshop, NCWCP College Park JCSDA accomplishments (II) OSSE capability in support of COSMIC-2, JPSS, GOES-R, Decadal Survey and other missions Comprehensive suite of data impact experiments for all major observing systems using NCEP GFS New supercomputer at GSFC (jointly funded by NASA and NOAA, installed and operated by NASA for the Joint Center) Part of NOAA/NESDIS-funded supercomputer (S4) located at UW Madison available for JCSDA investigators Hand-off to NCEP of ATMS data assimilation capability (collaboration between EMC, NESDIS, NASA, JCSDA); implemented in operations on May October

JCSDA Computing Lack of JCSDA computing “major obstacle to success” (JCSDA Advisory Panel Jan 2009) Review of past JCSDA-funded external projects revealed lack of computer resources as significant limitation R2O requires O2R Research community needs access to operational codes and adequate computer resources in order to help Some resources available on NOAA R&D computer No projected growth for JCSDA NASA made initial investment in JCSDA supercomputer to address this problem IBM Linux cluster; 576 Intel Westmere processors Immediately augmented by NOAA (GOES-R) to 3456 processors Located at Goddard, operated by NCCS for the Joint Center NOAA/NESDIS provides scientific software support October JCSDA Science Workshop, NCWCP College Park

Jibb (“Joint Center in a Big Box”) Total System Capabilities Compute – IBM iDataPlex 3,456 total cores; 37.8 TF Peak Computing 288 Compute Nodes Dual-socket, hex-core 2.8 GHz Intel Westmere with 24 GB of RAM Quad Data Rate Infiniband Network (32 Gbps) in a 2-to-1 blocking fabric Storage 8 IBM x3650 Storage Servers 2 IBM DS3512 Storage Subsystems 400 TB Total IBM GPFS File System Ancillary Nodes 2 Login Nodes 2 Management Nodes 9October JCSDA Science Workshop, NCWCP College Park

Jibb (II) System first open to JCSDA users 01/2011 As of 08/2012 ~50 JCSDA users; many JCSDA applications have been ported on to this system GDAS porting completed late 2011; NCEP verified scientific integrity of forecast result (comparable to to forecasts run on NOAA R&D platform “vapor”) Hybrid DA system ported immediately after initial GDAS port Code management plan between NCEP and JCSDA under development to facilitate two-way code transfers (“R2O and O2R”) October JCSDA Science Workshop, NCWCP College Park

Examples of Jibb experimentation NPP transition to operations impact experiments (Casey, Kleist, Collard, …) General data impact experiments (Jung, Garrett, …) Next generation NOAA hybrid data assimilation system port (Jung, …) Data impact experiments (OSE’s) (Jung, Garrett, …) Hyperspectral IR methodology (Zavodsky,…) Hyperspectral IR cloudy radiances (Casey, …) Satellite wind impact experiments (Jung, Santek, Daniels, Nebuda…) Lightning assimilation methodology (Fuelberg, …) 11

12 S4 in Support of JCSDA Activities Overview: –NESDIS, in collaboration with University of Wisconsin is making S4 available to scientists involved in satellite data assimilation and other activities related to it. –Conceptually, half of S4 is dedicated to OSSE (global &regional). –One of the purposes is to consolidate data assimilation activities performed in the research community and to encourage the usage of NOAA operational models/tools/codes installed on it. –Open policy for users (since hosted in academic institution) including JCSDA scientists. –Only open source/non-restricted data allowed on S4. The S4 system, hosted by University of Wisconsin. Brief Technical Description –The S4 system is a Linux cluster (Dell hardware), –3072 CPU cores in 64 compute nodes with 8TB of total RAM –520 TB in 26 storage nodes –Quad data rate (40 Gbps) Infiniband interconnects between all compute and storage nodes, –Lustre high performance filesystem for scratch space (4 x 80TB) and data storage (200 TB). S4 targets primarily the following activities: –Undertake satellite data assimilation experiments at global and/or regional scales and the assessment of their impacts on forecast models skills, using currently flying satellite sensors and –In support of the activity above, undertake all data simulations, calibration, algorithms development/improvement, radiative transfer modeling and validation, quality control (QC) procedures, etc –Perform Observing System Simulation Experiments (OSSEs) for new sensors (such as GOES-R and JPSS).

Examples of S4 Projects Research of Impact Assessment of SNPP/ATMS and CrIS data on the new hybrid data assimilation system Readiness for GOES-R data assimilation (ABI, GLM), for High-Impact weather events GOES-R AMV assimilation readiness OSSE simulations for DWSS and JPSS Development of DA capability for SEVIRI volcanic ash Development of GOES total column ozone assimilation Community Multi-purpose Formatting Tool (CMFT) MODIS-and AVHRR-Derived Polar Winds Experiments Satellite Ocean Data Assimilation Capability Using HYCOM Assimilation of Microwave and Thermal Infrared Observations of Soil Moisture in LIS 13

R2O requires O2R In order to facilitate R2O, JCSDA has to make “operational” assimilation systems available to the research community GDAS/GFS ported to Jibb and S4, skill is benchmarked against NCEP operational platform and NOAA R&D machine Available to internal and external researchers Currently T-574 hybrid system being benchmarked on jibb and S4 October JCSDA Science Workshop, NCWCP College Park 14

JCSDA/NCEP intercomparison VAPOR – NOAA R&D (Identical to NCEP Operations) S4 – JCSDA / NESDIS / UW Linux cluster JIBB – JCSDA / NASA Linux Cluster From Jung et al 2012, submitted to NESDIS Technical Report, pp 66 October JCSDA Science Workshop, NCWCP College Park

500 hPa Anomaly Correlations 15 Aug – 30 Sep No Satellite / No Conventional Data Northern Hemisphere Southern Hemisphere Example of OSE diagnostics (J. Jung, 5 th WMO Impact Workshop, Sedona 2012) October JCSDA Science Workshop, NCWCP College Park

Aim High…Fly, Fight, Win 17 GSI-WRF Integration  Establish working GSI prototype on AFWA HPC Currently running on Dev 8 Producing a 48-hr forecast at 12Z cycle 6-hr init from 6Z cycle Full DA for 6Z and 12Z run  GSI based WRF initial satellite data assimilation Initially planned: COSMIC and AMSU-A -- completed Additionally added: HIRS3, HIRS4, AIRS – completed Actively working: IASI, WINDSAT, and ASCAT (expect inclusion before IOC)

Aim High…Fly, Fight, Win 18 Working Prototype WRFVAR/GSI Comparison Configure Parallel Runs Conduct Parallel Runs Statistical Analysis DTC Community Capability Cost Analysis Community Funding Decision Phase 2 Decision GSI Operational Transition Optimize AFWA Pre-Processing Configure Code for ops efficiency GSI Sustainment Participate in GSI Review Committee Establish New Satellite Data Flows Jan 12 Mar 12 May 12 Jul 12 Sep 12 Nov 12 Jan 13 Mar 13 May 13 Jul 13 Sep 13 GSI Integration Schedule Dec 11 – Mar 12 Apr 12 – 15 May 12 Apr 12 – Fall May 12 – 1 Aug 12 Aug 12 – Sep 12 LEGEND Major Activity Planned/In progress Complete Sep 12 – Jan 13 Sep 12 Sep 12 – Jan 13 Mar 12 – May 12 Sep 12 Phase 1 Phase 2 IOC

19 FNMOC and GMAO Observation Impact Monitoring Current Operations Much larger relative impact of AMVs in Navy system ms/fp/obs_impact/ _main_od.html Slide courtesy of Ron Gelaro (Fifth WMO Impact Workshop, Sedona, May 2012)

Research to Operations – Suomi NPP ATMS  NCEP/STAR/JCSDA agreement for Expedited R2O of Suomi NPP ATMS in the GSI Management support for collaboration, resource and responsibilities allocation  NCEP/EMC integration of ATMS data in GSI Hybrid  ATMS data processing, quality control, spatial averaging  Low-resolution OSE 20 OSE Results (TOP) 500 mb Height Anomaly Correlation for 00z forecasts between Jan. 16, 2012 – April 28, 2012 in the Tropics. (RIGHT) Equitable Threat Score over CONUS hr forecast Satellite Zenith Angle (degrees) First Guess Departure Tb - Sea (K) ATMS Ch. 10 AMSU-A Ch. 9  JCSDA ATMS OSE  Port GSI Hybrid to S4/JIBB supercomputers (with help from NCEP)  High resolution OSE  ATMS data impact on forecast is neutral  ATMS data available since 11/8/2011 from NDE (NESDIS/ESPC support)  OSEs run from Dec 2011-May 2012  ATMS data assimilation capability delivered with GFS/GDAS update to NCO May 22, Scan dependent brightness temperature bias (observed-simulated) for ATMS Ch. 10 (top) and NOAA-19 AMSU-A Ch. 9 (bottom) Observed brightness temperatures for NOAA-19 AMSU-A Ch. 9 (left) and ATMS Ch. 10 (right). Coverage for one GDAS cycle. Courtesy of Kevin Garrett, NESDIS/STAR October JCSDA Science Workshop, NCWCP College Park

Community Radiative Transfer Model (CRTM) Joint Center for Satellite Data Assimilation CRTM Mission Satellite radiance simulation and assimilation for passive MW, IR, and Visible sensors of NOAA,NASA,DoD satellites, and others(200 sensors) CRTM Applications Data assimilation in supporting of weather forecasting Physical retrieval algorithm for satellite products Stability and accuracy monitoring of satellite observations Education and Research: reanalysis, climate studies, air quality forecasting, and a radiative tool for students CRTM Future Development SBUV for ozone data, JPSS ATMS,CrIS,VIIRS,OMPS, GPM,Fy3,Studying feasibility of active sensors including Radar and Lidar space measurements Derber et al., 2007 Courtesy of Quanhua Liu, NESDIS/STAR

500 hPa geopotential height anomaly correction from 01/09/ /22/2008 Example of data impact study (CRTM v. 1 to v. 2 upgrade) October JCSDA Science Workshop, NCWCP College Park 22 New implementation meets basic “do no harm” criterion Large impact in the SH (more sensitive to satellite data) Courtesy of Fuzhong Weng, NESDIS/STAR

Impact of COSMIC methodology update 500 hPa height AC scores as a function of the forecast for the 500 hPa heights in the Southern Hemisphere 40-day experiments: expx (NO COSMIC) cnt (old RO assimilation code - with COSMIC) exp (updated RO assimilation code - with COSMIC) 23 COSMIC provides 8 hours of gain in model forecast skill starting at day 4 October JCSDA Science Workshop, NCWCP College Park Courtesy of Cuccurul, NCEP/EMC

October JCSDA Science Workshop, NCWCP College Park 24 Preparation for data from new sensors Goal is to have operational users ready to assess data from new sensors from day 1 assimilate data from new sensors within one year from launch Current activities include NPP (ATMS, CrIS, VIIRS, OMPS) in close collaboration with NESDIS/STAR, NCEP/EMC Aquarius SMAP GPM GOES-R (ABI, GLM) …

JCSDA Science Workshop, NCWCP College Park Colloquia and Workshops Fifth WMO Workshop on the Impact of Various Observing Systems in NWP Major international venue for data impact studies Hosted by JCSDA for WMO in Sedona, AZ, May participants from 13 countries(strong US/JCSDA representation) 40 scientific presentations, ample discussion of results Recommendations captured in Final Report from Workshop to be published by WMO Second JCSDA Summer Colloquium, July 23 – Aug Major training event in satellite data assimilation Two weeks, 19 participants, 16 lecturers Particpants were Ph.D. students and post-docs, mostly from US educational institutions or employed in the US Formal lectures by invited lecturers and informal presentation by participants 25October

Future plans (I) Hyperspectral IR water vapor radiances in NCEP operations Additional impact experiments CrIS implementation Better utilization of SATWINDs MetOp-B preparation SMAP GPM … October JCSDA Science Workshop, NCWCP College Park 26

Future Plans (II) Additional OSSEs New high-resolution Nature Run required External research announcement by NOAA in FY2013 New model for rotation of External Research Announcement Examine possibilities for updating jibb/S4 to new high-resolution DA systems JCSDA Symposium at 2013 AMS Annual Meeting October JCSDA Science Workshop, NCWCP College Park 27