1. Overview of The Weather Company’s Principal Forecasting Methodologies
Peter Neilley, Bruce Rose, Joseph Koval, Todd Hutchinson, Paul Bayer, Jeff McDonald, John Mathews, William Cassanova, Dale Eck, Neil McGillis, Shaun Tanner The Weather Company Eric Floehr Intellovations, LLC or
How Atmospheric and Computer Sciences
have created Forecasts on Demand
NWA 2015, Plenary III

2. Big Data and the Traditional Forecasting Paradigm
Forecast foundational datasets are exploding……
Weather
Obs
Models
Final
Forecast
Users
Post
Proc’s
Forecasters
… which are overwhelming the system
NWA Annual 2015, Neilley et al.

3. TWC’s New Forecasting Paradigm
Therefore, we set out to create a new forecasting paradigm.
Some key tenants of our approach were to:
Take fully-appropriate advantage of the richness of modern NWP output
Allow forecasts to update continuously at the pace of the input data
Preserve human forecaster influence
Be easily adaptable to new input data and new scientific methods
Have globally ubiquitous content
Optimize forecast accuracy
NWA Annual 2015, Neilley et al.

4. Some Key Characteristics
The Weather Company’s Forecasts on Demand Paradigm
Some Key Characteristics
On Demand: Forecasts are created and delivered at the time of request
Human Input: Forecaster influence retained
Fresh: Forecasts always based partly on the latest from all input sources.
Precise: Forecasts built from full resolution input data, not interpolated point forecasts.
Optimized: Various statistical and scientific methods govern optimal forecast assembly
Forecasters
Forecasts On
Demand
Users
NWP
Post
Processors
A true “On Demand” forecast creation system
NWA Annual 2015, Neilley et al.

5. Real-time Publication and Distribution Proprietary Weather Obs
The TWC Forecasts on Demand EcoSystem
Obs on
Demand Engine
Real-time Publication and Distribution
On Demand
Users
Gov’ment Weather Obs
Proprietary Weather Obs
Gov’ment
NWP
Models
Proprietary NWP
Global Weather Data
Forecasts on
NWA Annual 2015, Neilley et al.

6. Forecasts on Demand Engine Human Forecasters “Over the Loop”
The Forecasts on Demand Engine
Forecasts on Demand Engine
For details see:
AMS WAF2015: 12B.8 and AMS16 EIPS: 13.B1
AMS WAF2015: Hutchinson et al., 12B.7
AMS WAF2015: Rose et al., 7B.5
Input Weather Data
1-15 Day Forecast Engine
Forecasts on
Demand
“Core”
Users
0-6 hr Forecast Engine
Human Forecasters “Over the Loop”
NWA Annual 2015, Neilley et al.

7. TWC’s Forecasts-on-Demand 1-15 Day Forecast Engine:
A multi-model, multi-method blending approach
162 inputs, including:
Global and regional models
Individual ensemble members (where available)
Various MOS forecast products
Reforecast-based post- processor outputs.
A statistically-optimized blend is based on recent verification and continuously updated
The blend is unique at each location, time, parameter
System updates with each new input arrival
For details see: AMS WAF2015: 12B.8 Koval et al.,
NWA Annual 2015, Neilley et al.

8. TWC’s Forecasts-on-Demand 0-hr Forecast Engine:
A blended nowcasting approach
Key Characteristics
Multi-method, nowcasting system using 6 inputs
Principal inputs to the precipitation forecasts are:
Radar advection forecasts (where available)
Our internal, HRRR-like global NWP
Human forecasters.
Forecasts update as new input data arrives, no less than once every 15 minutes (in No. Amer and Europe)
See AMS WAF201512B.7 Hutchinson et al.
NWA Annual 2015, Neilley et al.

9. Percent Correct Forecasts
Comparative Forecast Skill: YTD - US Day 1-3
Source: ForecastWatch.com
From ForecastWatch
2015 YTD “Overview” statistic, defined as:
Overview = .25 * MaxT %Cor +
.25 * MinT %Cor +
.50 * Precip %Cor
Computed using about 340K forecasts from each providor from 770 1st order US sites
Not shown here:
Redundant TWC brands
Redundant NDFD
Climo & Persistence
Percent Correct Forecasts
NWA Annual 2015, Neilley et al.

10. US, Precip ETS (>0) Days 1-3
Broader Accuracy Metrics
Source: ForecastWatch.com
Max Temp
Min Temp
Precipitation
US, MaxT MAE Days 1-3
US, MinT MAE Days 1-3
US, Precip ETS (>0) Days 1-3
MAE (F)
MAE (F)
ETS
US, MaxT %Busts Days 1-3
US, MinT %Busts Days 1-3
US, Precip %Cor Days 1-3
Percent
Percent
Percent
NWA Annual 2015, Neilley et al.

11. Overview statistic: Combined Mx/Mn/Pcp %Correct
Global Metrics:
Overview statistic: Combined Mx/Mn/Pcp %Correct
EUROPE US
ASIA
Day 1-3
Days 6-9
Days 3-5
Source: ForecastWatch.com

12. Forecasts on Demand Operations
Deployed in the Amazon cloud in 4 global regions
Typically delivers 11B forecasts/day
Peak load of 26B/day or 250K/sec.
Mean forecast creation time ~ 11 ms.
Mean total request-to-delivery time < 300 ms.
Used by all TWC/WSI consumer forecast systems (The Weather Channel, weather.com, WeatherUnderground, Intellicast, etc.)
Also drives our partner’s weather including Apple, Google, Yahoo!, IBM and a majority of the domestic TV stations.
NWA Annual 2015, Neilley et al.

13. Moving Forward Expansion of input data, particularly internationally
Models, (e.g. UKMO, GEM, JMA, FIM global models),
Additional observations and remote sensing for nowcasting
Non-linear multi-model integration techniques
Forecast PDFs and event/threshold forecasts (e.g. prob,.of 6” of snow)
“Weather object”- based blending techniques
NWA Annual 2015, Neilley et al.

14. From the AMS Weather Anal. and Forecasting Conf, 2015:
For More Details......
From the AMS Weather Anal. and Forecasting Conf, 2015:
7B.5 A Human Over the Loop Forecast Paradigm at The Weather Company (TWC), Rose et al.,
12B Day Weather Forecast Guidance at The Weather Company, Koval et al.
12B hour Weather Forecast Guidance at The Weather Company, Hutchinson et al
Upcoming at the AMS Annual Meeting:
EIPS 13B.1 Recent Advances in The Weather Company’s 1-15 Day Forecasting Guidance Infrastructure, Koval et al.
ProbStats 6.5 Consensus forecasting using constrained, regularized regression. Williams et al.
NWA Annual 2015, Neilley et al.