TIGGE Data Archive at NCAR 9th GIFS-TIGGE Working Group World Meteorological Organization Geneva 31 Aug – 2 Sep, 2011 Doug Schuster Steven Worley Dave Stepaniak Tom Cram Others

Provider Summary Archive Summary Service Improvements Special Projects Metrics Future Service Additions

Summary of Current Data Providers KMA and BoM stopped providing data JMA now providing 00z analysis files BOM and KMA are missing in 2011.

Archive Summary at NCAR Online Data Period, most recent 3 months ~ 50 TB , public products ~ 10 TB, data preparation, subsetting ~ 180GB Tropical Cyclone Track Data (CXML) Offline Data Full period of record ~ 600 TB, NCAR HPSS system

2010/2011 Service Improvements Increased online data volume 5 TB -> 50 TB 2 weeks -> 3 months Transitioned to new computing hardware Operates on longer data time-series (3 months) Faster request turn-around time Transitioned to new HPSS tape archive Improved response time to access data Strengthened data ingest processing New ldm server System monitoring (disk capacity, LDM status, etc) Failover mechanisms for LDM downtime Script templates are provided for subscription style download. Capitalize ldm, and remove automated.

2010/2011 Special Projects Developed and implemented first version of TIGGE model validation data portal Offers a wide variety of observational and analysis data NCEP PREPBUFR data Used to initialize NCEP models Surface and Upper Air -T,U,V,Q,Z Others to come NCEP precipitation analysis TWG suggestions? Script templates are provided for subscription style download. Capitalize ldm, and remove automated. ECMWF SYNOP station precip obs. What’s in prepbufr? –upper air and surface.

2010/2011 Special Projects Developed and implemented first version of TIGGE model validation data portal Integrated with TIGGE model data subsetting portal Check box to add model validation data based on model subset request parameters NetCDF and ASCII output formats Compatible with Mettools verification software http://www.dtcenter.org/mrt/users Stand alone portal for independent requests Sends notification email with download instructions Script templates are provided for subscription style download. Capitalize ldm, and remove automated. ECMWF SYNOP station precip obs. What’s in prepbufr? –upper air and surface.

TIGGE Model Validation Portal Script templates are provided for subscription style download. Capitalize ldm, and remove automated.

TIGGE Model Validation Portal Script templates are provided for subscription style download. Capitalize ldm, and remove automated.

NCAR TIGGE Model Data Archive Usage More data available for subsetting YEAR GB Accessed GB Downloaded Active Users / Unique Users 2006 5 0 / 5 2007 900 10 / 50 2008 1,300 1,100 10 / 20 2009 13,700 1,900 10 / 30 2010 48,200 2,200 12 / 32 2011 125,000 17,000 10 / 34

NCAR TIGGE CXML Archive Usage YEAR GB Downloaded Files Downloaded # Active Users / # Unique Users 2008 1 860 7 / 30 2009 26 12,300 17 / 35 2010 0.5 500 3 / 6 2011 147 19,500 6 / 21

NCAR TIGGE Model Validation Data Archive Usage YEAR GB Downloaded # of Requests # Active Users / # Unique Users 2011 200 140 6 / 23

Planned Service Improvements Improved Online Storage High performance centralized disk to facilitate subsetting services Expanded/Updated Computing Transition subsetting processing to HPC environment ~late 2012 – 2013. Faster “turn-around” time on user sub-set requests Potential for subscription services? Web services based requests? Special Projects Support -Hopson Meningitis Proposal? Pending THORPEX Program Office Approval 15 TB for processed request output. Capitalize ldm. South America subset automatically for CPTEC?

Planned Service Improvements Model Validation Data Portal Add more data source options Ocean based observations (ICOADS) Global tropical cyclone position and intensity data Atmospheric Reanalysis (JRA, ERA, NNRR, CFSR) Global PCP datasets Suggested TWG sets Data sub-set and output format selection capability. Add output format options -Integrated with TIGGE portal, automatically offer validation data for selected TIGGE forecast times. (domain and time specific for study region) -Increased burden of updated verification datasets. Condense slide, change colors.

http://tigge.ucar.edu/

Data Delivery (02/2010) Data exchanges using UNIDATA LDM, HTTP and FTP 450 GB/day (ECMWF = 332 GB/day) 1.7+ million 2-D gridded fields/day

Parameter vs Provider Matrix Variable LvL ECWF UKMO JMA NCEP CMA CMC BOM MetF KMA CPTC Geopotential Z PL Specific H T U-velocity V-velocity Potential Vor PT Potential T PV V-Velocity U 10m SL V 10m CAPE Conv. Inhib. Land-sea Mean SLP Orog. Skin T Snow D. H20 Snow F. H20 PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level

Parameter vs Provider Matrix Variable LvL ECWF UKMO JMA NCEP CMA CMC BOM MetF KMA CPTC Soil Moist. SL Soil T Sunshine D. Surf. DPT Surf. ATmax Surf. ATmin Surf. AT Surf. P LW Rad. Out LH flux Net Rad Net Therm. Rad Sensible Rad. Cloud Cov Column Water Precipitation Wilt. Point Field Cap. PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level

Archive Inhomogeneities Focus on new additions, ECMWF resolution increase, MF additional cycle and timesteps, KMA additional timestep.

2010/2011 Supported Special Projects NCAR Research Applications Laboratory (RAL) Google African Meningitis Forecast Initiative Dr. Thomas Hopson Integrate weather and health data to manage African meningitis outbreaks (in Ghana as starting point). Looking for 2-week lead time to position vaccines. Processing ECMWF data from present backwards through the archive for model calibration. Total Precipitation, 2m Temperature, 2m Dew Point. Global Subarea 40 deg N to 40 Deg S. 0.5 x 0.5 degree resolution. Assist RAL and Unidata in providing “Real-time” access to TIGGE based Multi-Model Ensemble Forecast data. http://www.ral.ucar.edu/projects/ghana/ http://www.google.org/predict.html Mention links. Place at bottom, staged in a different color.