Surface Ob Issues in RTMA/URMA Steve Levine Systems Research Group and NOAA/NWS/NCEP/EMC NCEP Production Suite Review 2015

Real Time Mesoscale Analysis (RTMA) Gridded (NDFD resolution) hourly near real time surface fields for situational awareness, verification, ‘analysis of record’ Used for tuning National Blend, initial forecast grid population Use all available surface obs and near-surface satellite wind obs over water Mesonet (MADIS Integrated mesonet feed) METAR Surface Synoptic Surface Ship Buoy ASCAT winds Satellite winds over water and below 850 hPa Assimilate obs with background field (short-term forecast, RAP/NAM/HRRR) Un-Restricted mesoscale analysis (URMA) runs 6 hours later to pick up latent data (also includes min/maxT) Overlap, duplicate, and quality issues with and across all types!! Added near-surface satellite winds, and ASCAT winds

Data Processing for RTMA Satellite ingest scripts BUFR Tanks /dcom Different tanks/dumps for different types - not cross checked GTS NCO/PMB EMC/MMB tranjb dumpjb Gather data decode NWSTG/ TOC/ “Gateway” Station dictionaries Dump Files LDM TNC Satellite ingest scripts “Katz” scripts/EMC tranjb Conventional data Non-conventional Data MADIS LDM PREPDATA Prevents SYNDATA CQCBUFR PROFCQC CQCVAD PREPACQC ACARSQC PREPBUFR Analysis (GSI QC happens here) Parm cards Radar ROC SMS job queries NESDIS servers for new data BUFR mnemonic table Global (not RTMA) Guess (perhaps relocated) NESDIS tcvitals FTP *original slide from Dennis Keyser (EMC) Outside NCEP Inside NCEP

Observational (mostly mesonet) Data Issues Anybody can set up a mesonet site with little training, knowledge, warning CWOP, Weatherbug obs dominate the station counts Lack of consistent siting, equipment standards Also many DOT, University sponsored mesonets (generally more reliable) Many different mesonet providers, but we mostly treat them all the same Should we? All mesonets weighted the same, less than METAR via GTS/TOC (obs weighted by type) Sites MADIS collects overlap with other data sources METAR/AWOS sites, NOS sites, some buoys Duplicate checking tough to do between ob types What’s a bad/unrepresentative ob? On what scale? How do we find them? Who do we call when we find a bad ob? You see obs we don’t (or don’t want to see) We see obs you don’t For non-mesonets, station dictionaries must be updated (requires RFC)

Quality Control in RTMA/URMA right now MADIS MADIS marks of reject (X) are flagged but monitored Prepbufr SDMedit reject list for non-mesonet data (dynamic, updated instantly by/through SDM but requires manual intervention) GSI ‘Gross error’ O-B check (relaxed by terrain, buddy check for temp) Provider/subprovider accept lists and Station-based reject/accept lists for mesonets (static, required RFC to change) (StationID based) Dynamic reject list (O-B last 6 hours) “Variational” QC (problems with 2+ obs/hour/station) Data could also ‘fall on floor’ during decoding or ingest Wrong type, dictionary not updated, etc. Added Provider/subprovider accept lists

Data ‘Falling on the Floor’ WFOs pulling their obs from private/individual providers on their own e.g. We do NOT get data from Weather Underground Obs are not rebroadcast in way NCEP can receive Easiest solution: Put ob into CWOP (run out of MADIS) RTMA cutoff time is XX:30 UTC for XX UTC analysis Generally occurs during mesonet decoding/tanking process Many obs come in late all or some of the time URMA largely solves this issue (cutoff time is XX+6:30 for XX UTC) Processing challenges/issues at provider, MesoWest or MADIS itself Mesonets are ‘data of opportunity’ Ob ‘occasionally’ missing Used in hourly analyses when available, but left out of min/maxT

A Typical RTMA Query: Usual Answers: “Why doesn’t the RTMA/URMA match the value of my favorite ob? How do you expect us to use this when it doesn’t match our obs (like MatchObsAll)?” (sometimes: “Don’t you know we maintain this ASOS site?!”) Usual Answers: We didn’t get your ob (or it didn’t get here in time) Your ob failed a quality control test Your ob got lost somewhere or was mistakenly pulled at a duplicate Your ob was counterweighted by other nearby obs and/or background field Your ob is not perfect (even if it is an ASOS you maintain) “Whaddya mean it’s not your favorite ob too?!”

RTMA over DC/Potomac River All plots are from RTMA, 7/16/15, 05Z

You are here Reagan Nat’l Airport (KDCA) Ft. Belvoir (KDAA)

Now with ALL obs Reagan Nat’l Airport (ASOS)

Background Field BG does not show ob-ob differences here

Increments (Analysis - Background) 1F increment intervals 8-10 km wide Increments between obs are ‘washed out’

Restricted/invisible mesonets! Ob and analysis values Site ID Observed Background Analysis KDCA 72.1 69.9 69.0 AS102 69.7 AXDRA FTWSH 69.2 WSHNP 69.1 68.7 FTBMR 67.9 ALEXD KDAA 70.6 69.5 Restricted/invisible mesonets!

Impact of ob missing from maxT, but included in hourly analyses

Future QC Possibilities CLARUS QC Developed by NCAR/USDOT (see document) To be sent out by MADIS starting mid 2016 (decoder upgrade?) Ob processing changes SDMedit to allow mesonets (but manual effort still needed to change) Time series QC - look at previous analyses? Cross-tank or cross-type duplicate checking Buddy check from WRFDA (implemented by Jacob Carley) Currently lets obs in for temperature that match buddies, ignores O-B Expand usage to throw obs out? Use for other variables? Mesonet metadata database Funded by National Mesonet, includes sensor info where available mySQL format (not immediately compatible with WCOSS) Combine with O-B stats, generate QC lists (already happening some)

Where to submit feedback or ask questions (shameless self-promotion) aor-rtma@infolist.noaa.gov - RTMA/URMA listserver (best option if seen in RTMA) National Blend of Models VLab page/forum (email w/o @noaa.gov to login) national.blend.feedback@noaa.gov - email handle for NBM VLab forum ncep.list.decoders-support@noaa.gov - Decoder/ingest group ncep.list.obsproc_support@noaa.gov - Obs Processing/BUFR issues

Links with further information... NCEP Data Processing (Keyser) RTMA/URMA overview (focus on April 2015 upgrade - made for SOOs/field) Overview of January 2016 upgrade (made for EMC Director) RTMA/URMA and ob NBM viewer (email w/o @noaa.gov to login, click “Daily Analysis Review”) Citizen Weather Observing Program MADIS (see “Integrated Mesonet”) MesoWest (large aggregator, more mesonet research from Univ of Utah)