1. Joint Center for Satellite Data Assimilation Overview of activities Lars Peter Riishojgaard, JCSDA Director 1

2. Overview JCSDA Overview Background, structure, accomplishments Current activities Future plans Summary JCSDA Science Workshop, NCWCP College Park 2October 10-12 2012

3. 3 NASA/Earth Science Division US Navy/Oceanographer and Navigator of the Navy and NRL NOAA/NESDISNOAA/NWS NOAA/OAR US Air Force/Director of Weather Mission: …to accelerate and improve the quantitative use of research and operational satellite data in weather, ocean, climate and environmental analysis and prediction models. Vision: An interagency partnership working to become a world leader in applying satellite data and research to operational goals in environmental analysis and prediction JCSDA Partners, Vision, Mission JCSDA Science Workshop, NCWCP College Park October 10-12 2012

4. JCSDA Science Workshop, NCWCP College Park 4 JCSDA Science Priorities Radiative Transfer Modeling (CRTM) Preparation for assimilation of data from new instruments Clouds and precipitation Assimilation of land surface observations Assimilation of ocean surface observations Atmospheric composition; chemistry and aerosol Driving the activities of the Joint Center since 2001, approved by the Science Steering Committee Overarching goal: Help the operational services improve the quality of their prediction products via improved and accelerated use of satellite data and related research October 10-12 2012

5. JCSDA Mode of operation October 10-12 2012 JCSDA Science Workshop, NCWCP College Park 5 Directed research Carried out by the partners Mixture of new and leveraged funding NOAA appropriation comes through NESDIS/STAR JCSDA plays a coordinating role External research Grants or contracts awarded to by one of the JCSDA parent agencies on a rotating basis Open to the broader research community Funding awarded competitively, based on peer-reviewed proposals Visiting Scientist program

6. JCSDA Science Workshop, NCWCP College Park JCSDA accomplishments Common assimilation infrastructure (NOAA, GMAO, AFWA) Community radiative transfer model (all partners) Common NOAA/NASA land data assimilation system (NOAA, GSFC, AFWA) Numerous new satellite data assimilated operationally, e.g. MODIS (winds and AOD), AIRS and IASI hyperspectral IR radiances, GPSRO sensors (COSMIC, GRAS, GRACE), SSMI/S, Windsat, Jason- 2,… Advanced sensors tested for operational readiness, e.g. ASCAT, MLS, SEVIRI (radiances),… Ongoing methodology improvement for sensors already assimilated, e.g. AIRS, GPSRO, SSMI/S,… Improved physically based SST analysis Adjoint sensitivity diagnostics 6October 10-12 2012

7. JCSDA Science Workshop, NCWCP College Park JCSDA accomplishments (II) OSSE capability in support of COSMIC-2, JPSS, GOES-R, Decadal Survey and other missions Comprehensive suite of data impact experiments for all major observing systems using NCEP GFS New supercomputer at GSFC (jointly funded by NASA and NOAA, installed and operated by NASA for the Joint Center) Part of NOAA/NESDIS-funded supercomputer (S4) located at UW Madison available for JCSDA investigators Hand-off to NCEP of ATMS data assimilation capability (collaboration between EMC, NESDIS, NASA, JCSDA); implemented in operations on May 22 2012 7October 10-12 2012

8. JCSDA Computing Lack of JCSDA computing “major obstacle to success” (JCSDA Advisory Panel Jan 2009) Review of past JCSDA-funded external projects revealed lack of computer resources as significant limitation R2O requires O2R Research community needs access to operational codes and adequate computer resources in order to help Some resources available on NOAA R&D computer No projected growth for JCSDA NASA made initial investment in JCSDA supercomputer to address this problem IBM Linux cluster; 576 Intel Westmere processors Immediately augmented by NOAA (GOES-R) to 3456 processors Located at Goddard, operated by NCCS for the Joint Center NOAA/NESDIS provides scientific software support October 10-12 20128 JCSDA Science Workshop, NCWCP College Park

9. Jibb (“Joint Center in a Big Box”) Total System Capabilities Compute – IBM iDataPlex 3,456 total cores; 37.8 TF Peak Computing 288 Compute Nodes Dual-socket, hex-core 2.8 GHz Intel Westmere with 24 GB of RAM Quad Data Rate Infiniband Network (32 Gbps) in a 2-to-1 blocking fabric Storage 8 IBM x3650 Storage Servers 2 IBM DS3512 Storage Subsystems 400 TB Total IBM GPFS File System Ancillary Nodes 2 Login Nodes 2 Management Nodes 9October 10-12 2012 JCSDA Science Workshop, NCWCP College Park

10. Jibb (II) System first open to JCSDA users 01/2011 As of 08/2012 ~50 JCSDA users; many JCSDA applications have been ported on to this system GDAS porting completed late 2011; NCEP verified scientific integrity of forecast result (comparable to to forecasts run on NOAA R&D platform “vapor”) Hybrid DA system ported immediately after initial GDAS port Code management plan between NCEP and JCSDA under development to facilitate two-way code transfers (“R2O and O2R”) October 10-12 201210 JCSDA Science Workshop, NCWCP College Park

11. Examples of Jibb experimentation NPP transition to operations impact experiments (Casey, Kleist, Collard, …) General data impact experiments (Jung, Garrett, …) Next generation NOAA hybrid data assimilation system port (Jung, …) Data impact experiments (OSE’s) (Jung, Garrett, …) Hyperspectral IR methodology (Zavodsky,…) Hyperspectral IR cloudy radiances (Casey, …) Satellite wind impact experiments (Jung, Santek, Daniels, Nebuda…) Lightning assimilation methodology (Fuelberg, …) 11

12. 12 S4 in Support of JCSDA Activities Overview: –NESDIS, in collaboration with University of Wisconsin is making S4 available to scientists involved in satellite data assimilation and other activities related to it. –Conceptually, half of S4 is dedicated to OSSE (global &regional). –One of the purposes is to consolidate data assimilation activities performed in the research community and to encourage the usage of NOAA operational models/tools/codes installed on it. –Open policy for users (since hosted in academic institution) including JCSDA scientists. –Only open source/non-restricted data allowed on S4. The S4 system, hosted by University of Wisconsin. Brief Technical Description –The S4 system is a Linux cluster (Dell hardware), –3072 CPU cores in 64 compute nodes with 8TB of total RAM –520 TB in 26 storage nodes –Quad data rate (40 Gbps) Infiniband interconnects between all compute and storage nodes, –Lustre high performance filesystem for scratch space (4 x 80TB) and data storage (200 TB). S4 targets primarily the following activities: –Undertake satellite data assimilation experiments at global and/or regional scales and the assessment of their impacts on forecast models skills, using currently flying satellite sensors and –In support of the activity above, undertake all data simulations, calibration, algorithms development/improvement, radiative transfer modeling and validation, quality control (QC) procedures, etc –Perform Observing System Simulation Experiments (OSSEs) for new sensors (such as GOES-R and JPSS).

13. Examples of S4 Projects Research of Impact Assessment of SNPP/ATMS and CrIS data on the new hybrid data assimilation system Readiness for GOES-R data assimilation (ABI, GLM), for High-Impact weather events GOES-R AMV assimilation readiness OSSE simulations for DWSS and JPSS Development of DA capability for SEVIRI volcanic ash Development of GOES total column ozone assimilation Community Multi-purpose Formatting Tool (CMFT) MODIS-and AVHRR-Derived Polar Winds Experiments Satellite Ocean Data Assimilation Capability Using HYCOM Assimilation of Microwave and Thermal Infrared Observations of Soil Moisture in LIS 13

14. R2O requires O2R In order to facilitate R2O, JCSDA has to make “operational” assimilation systems available to the research community GDAS/GFS ported to Jibb and S4, skill is benchmarked against NCEP operational platform and NOAA R&D machine Available to internal and external researchers Currently T-574 hybrid system being benchmarked on jibb and S4 October 10-12 2012 JCSDA Science Workshop, NCWCP College Park 14

15. JCSDA/NCEP intercomparison VAPOR – NOAA R&D (Identical to NCEP Operations) S4 – JCSDA / NESDIS / UW Linux cluster JIBB – JCSDA / NASA Linux Cluster From Jung et al 2012, submitted to NESDIS Technical Report, pp 66 October 10-12 201215 JCSDA Science Workshop, NCWCP College Park

16. 500 hPa Anomaly Correlations 15 Aug – 30 Sep 2010 16 No Satellite / No Conventional Data Northern Hemisphere Southern Hemisphere Example of OSE diagnostics (J. Jung, 5 th WMO Impact Workshop, Sedona 2012) October 10-12 2012 JCSDA Science Workshop, NCWCP College Park

17. Aim High…Fly, Fight, Win 17 GSI-WRF Integration  Establish working GSI prototype on AFWA HPC Currently running on Dev 8 Producing a 48-hr forecast at 12Z cycle 6-hr init from 6Z cycle Full DA for 6Z and 12Z run  GSI based WRF initial satellite data assimilation Initially planned: COSMIC and AMSU-A -- completed Additionally added: HIRS3, HIRS4, AIRS – completed Actively working: IASI, WINDSAT, and ASCAT (expect inclusion before IOC)

18. Aim High…Fly, Fight, Win 18 Working Prototype WRFVAR/GSI Comparison Configure Parallel Runs Conduct Parallel Runs Statistical Analysis DTC Community Capability Cost Analysis Community Funding Decision Phase 2 Decision GSI Operational Transition Optimize AFWA Pre-Processing Configure Code for ops efficiency GSI Sustainment Participate in GSI Review Committee Establish New Satellite Data Flows Jan 12 Mar 12 May 12 Jul 12 Sep 12 Nov 12 Jan 13 Mar 13 May 13 Jul 13 Sep 13 GSI Integration Schedule Dec 11 – Mar 12 Apr 12 – 15 May 12 Apr 12 – Fall 2012 15 May 12 – 1 Aug 12 Aug 12 – Sep 12 LEGEND Major Activity Planned/In progress Complete Sep 12 – Jan 13 Sep 12 Sep 12 – Jan 13 Mar 12 – May 12 Sep 12 Phase 1 Phase 2 IOC

19. 19 FNMOC and GMAO Observation Impact Monitoring Current Operations Much larger relative impact of AMVs in Navy system http://gmao.gsfc.nasa.gov/products/forecasts/syste ms/fp/obs_impact/ http://www.nrlmry.navy.mil/obsens/fnmoc/obsens _main_od.html Slide courtesy of Ron Gelaro (Fifth WMO Impact Workshop, Sedona, May 2012)

20. Research to Operations – Suomi NPP ATMS  NCEP/STAR/JCSDA agreement for Expedited R2O of Suomi NPP ATMS in the GSI Management support for collaboration, resource and responsibilities allocation  NCEP/EMC integration of ATMS data in GSI Hybrid  ATMS data processing, quality control, spatial averaging  Low-resolution OSE 20 OSE Results (TOP) 500 mb Height Anomaly Correlation for 00z forecasts between Jan. 16, 2012 – April 28, 2012 in the Tropics. (RIGHT) Equitable Threat Score over CONUS 36-60 hr forecast Satellite Zenith Angle (degrees) First Guess Departure Tb - Sea (K) ATMS Ch. 10 AMSU-A Ch. 9  JCSDA ATMS OSE  Port GSI Hybrid to S4/JIBB supercomputers (with help from NCEP)  High resolution OSE  ATMS data impact on forecast is neutral  ATMS data available since 11/8/2011 from NDE (NESDIS/ESPC support)  OSEs run from Dec 2011-May 2012  ATMS data assimilation capability delivered with GFS/GDAS update to NCO May 22, 2012. Scan dependent brightness temperature bias (observed-simulated) for ATMS Ch. 10 (top) and NOAA-19 AMSU-A Ch. 9 (bottom) Observed brightness temperatures for NOAA-19 AMSU-A Ch. 9 (left) and ATMS Ch. 10 (right). Coverage for one GDAS cycle. Courtesy of Kevin Garrett, NESDIS/STAR October 10-12 2012 JCSDA Science Workshop, NCWCP College Park

21. Community Radiative Transfer Model (CRTM) Joint Center for Satellite Data Assimilation CRTM Mission Satellite radiance simulation and assimilation for passive MW, IR, and Visible sensors of NOAA,NASA,DoD satellites, and others(200 sensors) CRTM Applications Data assimilation in supporting of weather forecasting Physical retrieval algorithm for satellite products Stability and accuracy monitoring of satellite observations Education and Research: reanalysis, climate studies, air quality forecasting, and a radiative tool for students CRTM Future Development SBUV for ozone data, JPSS ATMS,CrIS,VIIRS,OMPS, GPM,Fy3,Studying feasibility of active sensors including Radar and Lidar space measurements Derber et al., 2007 Courtesy of Quanhua Liu, NESDIS/STAR

22. 500 hPa geopotential height anomaly correction from 01/09/2008-02/22/2008 Example of data impact study (CRTM v. 1 to v. 2 upgrade) October 10-12 2012 JCSDA Science Workshop, NCWCP College Park 22 New implementation meets basic “do no harm” criterion Large impact in the SH (more sensitive to satellite data) Courtesy of Fuzhong Weng, NESDIS/STAR

23. Impact of COSMIC methodology update 500 hPa height AC scores as a function of the forecast for the 500 hPa heights in the Southern Hemisphere 40-day experiments: expx (NO COSMIC) cnt (old RO assimilation code - with COSMIC) exp (updated RO assimilation code - with COSMIC) 23 COSMIC provides 8 hours of gain in model forecast skill starting at day 4 October 10-12 2012 JCSDA Science Workshop, NCWCP College Park Courtesy of Cuccurul, NCEP/EMC

24. October 10-12 2012 JCSDA Science Workshop, NCWCP College Park 24 Preparation for data from new sensors Goal is to have operational users ready to assess data from new sensors from day 1 assimilate data from new sensors within one year from launch Current activities include NPP (ATMS, CrIS, VIIRS, OMPS) in close collaboration with NESDIS/STAR, NCEP/EMC Aquarius SMAP GPM GOES-R (ABI, GLM) …

25. JCSDA Science Workshop, NCWCP College Park Colloquia and Workshops Fifth WMO Workshop on the Impact of Various Observing Systems in NWP Major international venue for data impact studies Hosted by JCSDA for WMO in Sedona, AZ, May 22-25 2012 59 participants from 13 countries(strong US/JCSDA representation) 40 scientific presentations, ample discussion of results Recommendations captured in Final Report from Workshop to be published by WMO Second JCSDA Summer Colloquium, July 23 – Aug 2 2012 Major training event in satellite data assimilation Two weeks, 19 participants, 16 lecturers Particpants were Ph.D. students and post-docs, mostly from US educational institutions or employed in the US Formal lectures by invited lecturers and informal presentation by participants 25October 10-12 2012

26. Future plans (I) Hyperspectral IR water vapor radiances in NCEP operations Additional impact experiments CrIS implementation Better utilization of SATWINDs MetOp-B preparation SMAP GPM … October 10-12 2012 JCSDA Science Workshop, NCWCP College Park 26

27. Future Plans (II) Additional OSSEs New high-resolution Nature Run required External research announcement by NOAA in FY2013 New model for rotation of External Research Announcement Examine possibilities for updating jibb/S4 to new high-resolution DA systems JCSDA Symposium at 2013 AMS Annual Meeting October 10-12 2012 JCSDA Science Workshop, NCWCP College Park 27