1. Surface Ob Issues in RTMA/URMA
Steve Levine
Systems Research Group and NOAA/NWS/NCEP/EMC
NCEP Production Suite Review 2015

2. Real Time Mesoscale Analysis (RTMA)
Gridded (NDFD resolution) hourly near real time surface fields for situational awareness, verification, ‘analysis of record’
Used for tuning National Blend, initial forecast grid population
Use all available surface obs and near-surface satellite wind obs over water
Mesonet (MADIS Integrated mesonet feed)
METAR
Surface Synoptic
Surface Ship
Buoy
ASCAT winds
Satellite winds over water and below 850 hPa
Assimilate obs with background field (short-term forecast, RAP/NAM/HRRR)
Un-Restricted mesoscale analysis (URMA) runs 6 hours later to pick up latent data (also includes min/maxT)
Overlap, duplicate, and quality issues with and across all types!!
Added near-surface satellite winds, and ASCAT winds

3. Data Processing for RTMA Satellite ingest scripts
BUFR
Tanks
/dcom
Different tanks/dumps for different types - not cross checked
GTS
NCO/PMB
EMC/MMB
tranjb
dumpjb
Gather data
decode
NWSTG/
TOC/
“Gateway”
Station dictionaries
Dump Files
LDM
TNC
Satellite ingest scripts
“Katz” scripts/EMC
tranjb
Conventional data
Non-conventional Data
MADIS
LDM
PREPDATA
Prevents
SYNDATA
CQCBUFR
PROFCQC
CQCVAD
PREPACQC
ACARSQC
PREPBUFR
Analysis
(GSI QC happens here)
Parm cards
Radar
ROC
SMS job queries NESDIS servers for new data
BUFR mnemonic table
Global (not RTMA) Guess (perhaps relocated)
NESDIS
tcvitals
FTP
*original slide from Dennis Keyser (EMC)
Outside NCEP
Inside NCEP

4. Observational (mostly mesonet) Data Issues
Anybody can set up a mesonet site with little training, knowledge, warning
CWOP, Weatherbug obs dominate the station counts
Lack of consistent siting, equipment standards
Also many DOT, University sponsored mesonets (generally more reliable)
Many different mesonet providers, but we mostly treat them all the same
Should we?
All mesonets weighted the same, less than METAR via GTS/TOC (obs weighted by type)
Sites MADIS collects overlap with other data sources
METAR/AWOS sites, NOS sites, some buoys
Duplicate checking tough to do between ob types
What’s a bad/unrepresentative ob? On what scale? How do we find them?
Who do we call when we find a bad ob?
You see obs we don’t (or don’t want to see)
We see obs you don’t
For non-mesonets, station dictionaries must be updated (requires RFC)

5. Quality Control in RTMA/URMA right now
MADIS
MADIS marks of reject (X) are flagged but monitored
Prepbufr
SDMedit reject list for non-mesonet data (dynamic, updated instantly by/through SDM but requires manual intervention)
GSI
‘Gross error’ O-B check (relaxed by terrain, buddy check for temp)
Provider/subprovider accept lists and Station-based reject/accept lists for mesonets (static, required RFC to change) (StationID based)
Dynamic reject list (O-B last 6 hours)
“Variational” QC (problems with 2+ obs/hour/station)
Data could also ‘fall on floor’ during decoding or ingest
Wrong type, dictionary not updated, etc.
Added Provider/subprovider accept lists

6. Data ‘Falling on the Floor’
WFOs pulling their obs from private/individual providers on their own
e.g. We do NOT get data from Weather Underground
Obs are not rebroadcast in way NCEP can receive
Easiest solution: Put ob into CWOP (run out of MADIS)
RTMA cutoff time is XX:30 UTC for XX UTC analysis
Generally occurs during mesonet decoding/tanking process
Many obs come in late all or some of the time
URMA largely solves this issue (cutoff time is XX+6:30 for XX UTC)
Processing challenges/issues at provider, MesoWest or MADIS itself
Mesonets are ‘data of opportunity’
Ob ‘occasionally’ missing
Used in hourly analyses when available, but left out of min/maxT

7. A Typical RTMA Query: Usual Answers:
“Why doesn’t the RTMA/URMA match the value of my favorite ob? How do you expect us to use this when it doesn’t match our obs (like MatchObsAll)?” (sometimes: “Don’t you know we maintain this ASOS site?!”)
Usual Answers:
We didn’t get your ob (or it didn’t get here in time)
Your ob failed a quality control test
Your ob got lost somewhere or was mistakenly pulled at a duplicate
Your ob was counterweighted by other nearby obs and/or background field
Your ob is not perfect (even if it is an ASOS you maintain)
“Whaddya mean it’s not your favorite ob too?!”

8. RTMA over DC/Potomac River
All plots are from RTMA, 7/16/15, 05Z

9. You
are
here
Reagan
Nat’l
Airport
(KDCA)
Ft. Belvoir (KDAA)

10. Now with ALL obs
Reagan
Nat’l
Airport
(ASOS)

11. Background Field
BG does not show ob-ob differences here

12. Increments (Analysis - Background)
1F increment intervals 8-10 km wide
Increments between obs are ‘washed out’

13. Restricted/invisible mesonets!
Ob and analysis values
Site ID
Observed
Background
Analysis
KDCA
72.1
69.9
69.0
AS102
69.7
AXDRA
FTWSH
69.2
WSHNP
69.1
68.7
FTBMR
67.9
ALEXD
KDAA
70.6
69.5
Restricted/invisible mesonets!

14. Impact of ob missing from maxT, but included in hourly analyses

15. Future QC Possibilities
CLARUS QC
Developed by NCAR/USDOT (see document)
To be sent out by MADIS starting mid 2016 (decoder upgrade?)
Ob processing changes
SDMedit to allow mesonets (but manual effort still needed to change)
Time series QC - look at previous analyses?
Cross-tank or cross-type duplicate checking
Buddy check from WRFDA (implemented by Jacob Carley)
Currently lets obs in for temperature that match buddies, ignores O-B
Expand usage to throw obs out?
Use for other variables?
Mesonet metadata database
Funded by National Mesonet, includes sensor info where available
mySQL format (not immediately compatible with WCOSS)
Combine with O-B stats, generate QC lists (already happening some)

16. Where to submit feedback or ask questions (shameless self-promotion)
- RTMA/URMA listserver (best option if seen in RTMA)
National Blend of Models VLab page/forum ( to login)
- handle for NBM VLab forum
- Decoder/ingest group
- Obs Processing/BUFR issues

17. Links with further information...
NCEP Data Processing (Keyser)
RTMA/URMA overview (focus on April 2015 upgrade - made for SOOs/field)
Overview of January 2016 upgrade (made for EMC Director)
RTMA/URMA and ob NBM viewer ( to login, click “Daily Analysis Review”)
Citizen Weather Observing Program
MADIS (see “Integrated Mesonet”)
MesoWest (large aggregator, more mesonet research from Univ of Utah)