1. FORMULATION METHODS FOR STATISTICAL APPLICATION IN WEATHER FORECASTING
BPKC : CLM (STATS)

2. FORMULATION METHODS FOR STATISTICAL APPLICATION IN WEATHER FORECASTING
Classical , PPM , MOS Methods
Of the three, only PPM and MOS use NWP model output, while the Classical Method uses only observations.
BPKC : CLM (STATS)

3. CLASSICAL METHODS
Similar to subjective methods, used to predict weather, based on current observations or recent weather conditions.
This approach does not use numerical model forecasts and relies purely on observed data.
Before Dynamical Models, statistical systems were limited to Classical Approach.
To develop equations with Classical Method, observations of the initial and resultant weather conditions are needed.
BPKC : CLM (STATS)

4. CLASSICAL METHODS
The resultant weather conditions are the ‘Predictands’, while the initial conditions are the ‘Predictors’.
For Example, for forecasting Max Temperature for tomorrow, input would consist only of observational data available at the time that the forecast was to be made.
This situation can be expressed as:-
BPKC : CLM (STATS)

5. CLASSICAL METHODS
Here is estimate (forecast) of ‘Predictand’ (dependent variable) ‘Y’ at time ‘t’ and ‘X0’ is a vector of observational data (independent variables) at initial time ‘0’.
Observations are not necessarily made at initial time, but must be available at that time.
Accuracy of any given forecast based on this approach is strongly dependent on significant changes occur in atmosphere between times of predictor observations and time of validity of resultant forecast.
BPKC : CLM (STATS)

6. CLASSICAL METHODS
Greater the forecast projection, greater will be the chance of significant changes.
Hence, this approach is good for short-range forecasts, but its skill falls sharply beyond a few hours. Practically no skill in medium range.
Work best when the weather tends to be persistent (low variability).
This approach is also used for very long-range seasonal forecasts, where there is little skill in numerical model predictions.
BPKC : CLM (STATS)

7. PERFECT PROGNOSIS METHOD
(PPM)
BPKC : CLM (STATS)

8. PERFECT PROGNOSIS METHOD
The need to accurately predict surface weather elements, led to the development of Perfect Prognosis Method (PPM) (Klien et al., 1959).
This is an objective method, in which, a concurrent relation is developed between the parameter to be predicted and the observed circulation around the location of interest, using several years of data.
PPM technique is based on the assumption that numerical model forecasts are “Perfect”.
BPKC : CLM (STATS)

9. 1st Statistical approach for dealing with forecasts from NWP
Perfect Prog (Klein et al. 1959)
Takes NWP model forecasts for future atmosphere assuming them to be perfect
Perfect prog regression equations are similar to classical regression equations except they do not incorporate any time lag.
Example: equations specifying tomorrows predictands are developed using tomorrow’s predictor values.
If the NWP forecasts for tomorrow’s predictors really are perfect, the perfect-prog regression equations should provide very good forecasts.

10. PERFECT PROGNOSIS METHOD

11. PERFECT PROGNOSIS METHOD
Despite numerical models not being perfect, this approach gives an estimate of what to expect, if numerical models are correct
It does not require numerical model data for development, but uses numerical output when equations are applied operationally
Hence, it is important to make sure that variables used as ‘Predictors’ in development of Perfect Prognosis Equations will be available from NWP models, so that equations can be applied operationally
BPKC : CLM (STATS)

12. PERFECT PROGNOSIS METHOD
Extensive sample of upper air observations and surface weather reports is collected
Multiple Linear Regression equations are then derived
In applying PPM equations, Dynamical Model output for a specific forecast projection is substituted for the developmental observations to give a forecast for the appropriate valid time
Multiple Linear Regression equations (relating observed ‘Predictands’ to concurrent or antecedent ‘Predictors’) are then derived
BPKC : CLM (STATS)

13. PERFECT PROGNOSIS METHOD
These specification equations can be used to generate guidance from any dynamical model as long as the model produces the required predictors and the original time relationships among predictors and predictand are preserved
The PPM approach can be expressed numerically as:-
BPKC : CLM (STATS)

14. PERFECT PROGNOSIS METHOD
Here is estimate of ‘Predictand’ ‘Y’ at time ‘0’ and ‘X0’ is a vector of observations of variables that can be predicted by numerical models
Even though is an estimate, it is not a forecast, it is only a specification
In application, is inserted into equation (2) to provide a forecast as:- 
BPKC : CLM (STATS)

15. MODEL OUTPUT STATISTICS
(MOS)
BPKC : CLM (STATS)

16. MODEL OUTPUT STATISTICS
Like PPM, the MOS approach is also an objective type of forecasting technique (Glahn & Lowry, 1972).
In contrast to Classical Approach and Perfect Prognosis Approach (which uses numerical model output only in operational application of the equations), the Model Output Statistics (MOS) approach uses numerical model forecasts for both the development and the operational application of the equations
BPKC : CLM (STATS)

17. 2nd Statistical approach
Model Output Statistics (MOS)
Preferred because it can include directly in the regression eqns the influences of specific characteristics of different NWP models at different projections into the future.
To get MOS forecast eqns you need a developmental data set with historical records of predictand, and records of the forecasts by NWP model.
Separate MOS forecast equations must by made for different forecast projections.
Example: Predictand is tomorrow’s mb thickness as forecast today by a certain NWP model.

18. MODEL OUTPUT STATISTICS

19. MODEL OUTPUT STATISTICS
MOS Approach requires an archive of dynamical model forecasts : usually 2-3 years of data are needed
MOS Approach relates a weather observation (Predictand) to variables forecast by a numerical model (Predictors)
The Predictors from the numerical model are usually forecasts that are valid at about the same time as the Predictand
BPKC : CLM (STATS)

20. MODEL OUTPUT STATISTICS
Some statistical method (usually Multiple Linear Regression) is then used to determine relationships among various observed weather elements and numerical model output variables at projections near (before, at, or amer) the specific valid time of the Predictand.
To make operational forecasts, MOS equations are usually applied to the same dynamical model that provided the developmental sample.
BPKC : CLM (STATS)

21. MODEL OUTPUT STATISTICS
Mathematically MOS can be expressed as:-
where is estimate of Predictand ‘Y’ at time ‘t’ and is a vector of forecasts from numerical models.
The numerical model predictions need not be limited at time ‘t’; however, the projection times of the different variables will usually be grouped around ‘t’.
BPKC : CLM (STATS)

22. MODEL OUTPUT STATISTICS
The equations (4) account for some of the bias and systematic errors found in the dynamical model.
Local topographic and environmental conditions of a location are also automatically accounted for in the forecast system.
MOS technique also recognises the predictability of the model variables by selecting those variables that provide useful forecast information.
BPKC : CLM (STATS)

23. MODEL OUTPUT STATISTICS
Finally, as the dynamical model predictions deteriorate with increasing time, this approach produces forecasts that tend towards the mean of the predictand in the developmental sample.
Drawback of this technique is that a sufficient sample of Model Output is required in order to derive a stable relation. Hence, it cannot be applied immediately when a new model is made operational.
Also, if model undergoes a major change, MOS relations will have to be developed again.
BPKC : CLM (STATS)

24. COMPARISON BETWEEN CLASSICAL,
PPM AND MOS TECHNIQUES
BPKC : CLM (STATS)

25. COMPARISON
Classical Approach is most useful for very short range (0-12 hour), or if NWP model forecast data is not available.
It is relatively simple to use, since it requires only that data, which are usually available without special archiving procedures.
PPM and MOS approaches require NWP model forecast data in operational application of the equations.
BPKC : CLM (STATS)

26. COMPARISON
However, since PPM equations are derived from observed data only, there is usually a longer sample of data available from which to derive the equations.
In contrast, MOS equations need to be developed from a stable sample of numerical model forecast data.
Because most NWP models are constantly evolving, developmental sample for MOS equations is likely to be relatively short.
BPKC : CLM (STATS)

27. COMPARISON
When a model is changed significantly, forecast produced by MOS has less skill and accuracy and equations will have to be redeveloped. In contrast, PPM forecasts usually do not deteriorate with change in model.
Generally an improved numerical model will lead to improved forecasts.
If model is not modified and sufficient sample of stable numerical model output is available, MOS approach will produce the best results.
BPKC : CLM (STATS)

28. COMPARISON Classical PPM MOS Development of equations Application in
Weather element
observed at T0.
Predictors observed at T0-dT.
Predictors observed
at T0.
Predictors Forecast
values valid at T0
from PPM issued at
T0-dT.
Application in
Operational
Forecast
mode
Predictor values
observed now (T0) to give forecast valid at T0+dT.
Predictor values valid
for T0+dT from PPM
issued now to give
forecast valid at
T0+dT.
Comments
dT less than or equal to 6 hours preferable unless persistence works well. Time lag built into equations.
dT can take any
value, for which
Forecast predictors
are available.
Same application as
PPM, but separate
equations used for
each dT.
BPKC : CLM (STATS)

29. COMPARISON Classical PPM MOS Specific Differences
Relationship weaken rapidly as predictor-weather element time lag increases.
Model-independent.
Does not use model output.
Large development sample possible.
Must have access to observed or analysed variables.
Relationships strong because only observed data concurrent in time are used.
Model- independent.
Does not account for model bias – model errors decrease accuracy.
Relationships weaken with increasing projection time due to increasing model error variance.
Model-dependent.
Partially accounts for model bias.
Generally small development samples, depends on frequency of model changes.
Must have access to model output variables that may not be observed.
BPKC : CLM (STATS)

30. Advantages and Disadvantages of Perfect-Prog and MOS:

31. Perfect Prog Advantages: Disadvantages:
Large developmental sample (fit using historical climate data)
Equations developed without NWP info, so changes to NWP models don’t require changes in regression equations
Improving NWP models will improve forecasts
Same equations can be used with any NWP models
Disadvantages:
Potential predictors must be well forecast by the NWP model

32. MOS Advantages: Disadvantages:
Model-calculated, but un-observed quantities can be predictors
Systematic errors in the NWP model are accounted for
Different MOS equations required for different projection times
Method of choice when practical
Disadvantages:
Requires archived records (several years) of forecast from NWP model to develop, and models regularly undergo changes.
Different MOS equations required for different NWP models

33. ANY QUESTION ?
BPKC : CLM (STATS)

34. MOS (Additional Information)

35. Model Output Statistics (MOS)
REVISION
MOS relates observations of the weather element to be predicted (PREDICTANDS) to appropriate variables (PREDICTORS) via a statistical method
Predictors can include:
NWP model output interpolated to observing site
Prior observations
Geoclimatic data – terrain, normals, lat/long, etc.
Current statistical method:
Multiple Linear Regression (forward selection)

36. MOS Development Strategy
CAREFULLY define your predictand
Stratify data as appropriate
Pool data if needed (Single Station / Regional)
Select predictors for equations
AVOID OVERFITTING!

37. Predictand Strategies
Predictands always come from meteorological data and a variety of sources:
Point observations (ASOS, AWOS, Co-op sites)
Satellite data (e.g., SCP data)
Lightning data (NLDN)
Radar data (WSR-88D)
It is very important to quality control predictands before performing a regression analysis.

38. Predictand Strategies
(Quasi-)Continuous Predictands: best for variables with a relatively smooth distribution
Temperature, dew point, wind (u and v components, wind speed)
Quasi-continuous because temperature available usually only to the nearest degree C, wind direction to the nearest 10 degrees, wind speed to the nearest m/s.
Categorical Predictands: observations are reported as categories
Sky Cover (CLR, FEW, SCT, BKN, OVC)
There are four different types of predictands we use. Most weather observations can be considered quasi-continuous, but for reasons I’ll discuss shortly, there are only two groups that we use in a continuous form—temperature and wind.

39. Example of MOS Predictands
Temperature
Dry bulb temperature (every 3 h)
Dew point (every 3 h)
Daytime maximum temperature [0700 – 1900 LST] (every 24 h)
Nighttime minimum temperature [1900 – 0800 LST] (every 24 h)
Wind
U- and V- wind components (every 3 h)
Wind speed (every 3 h)
Sky Cover
Clear, few, scattered, broken, overcast [binary/MECE] (every 3 h)

40. Example of MOS Predictands
PoP/QPF
PoP: accumulation of 0.01” of liquid-equivalent precipitation in a {6/12/24} h period [binary]
QPF: accumulation of {0.10”/0.25”/0.50”/1.00”/2.00”*} CONDITIONAL on accumulation of 0.01” [binary/conditional]
6 h and 12 h guidance every 6 h; 24 h guidance every 12 h
2.00” category not available for 6 h guidance
Thunderstorms
1+ lightning strike in gridbox [binary]
Severe
1+ severe weather report in gridbox [binary]

41. Example of MOS Predictands
Ceiling Height
CH < 200 m, m, m, m, m, m, m, > m [binary/MECE]
Visibility
Visibility < ½ km, < 1 km, < 2 kms, < 3 kms, < 5 kms, < 6 kms [binary]
Obstruction to Vision
Observed fog (fog w/ vis < 1 km), mist (fog w/ vis > 1-2 km), haze (includes smoke and dust), blowing phenomena, or none [binary]

42. Example of MOS Predictands
Precipitation Type
Pure snow (S); freezing rain/drizzle, ice pellets, or anything mixed with these (Z); pure rain/drizzle or rain mixed with snow (R)
Conditional on precipitation occurring
Precipitation Characteristics (PoPC)
Observed drizzle, steady precip, or showery precip
Precipitation Occurrence (PoPO)
Observed precipitation on the hour – does NOT have to accumulate

43. MOS equations are developed season wise
Stratification
Goal: To achieve maximum homogeneity in our developmental datasets, while keeping their size large enough for a stable regression
MOS equations are developed season wise

44. Pooling Data
Generally, this means REGIONALIZATION: collecting nearby stations with similar climatology
Regionalization allows for guidance to be produced at sites with poor, unreliable, or non-existent observation systems
All MOS equations are regional except temperature and wind

45. MOS Development Strategy
MOS equations are multivariate of the form:
Y = c0 + c1*X1 + c2*X2 + … + cN*XN
C’s are constants, X’s are predictors
N is the number of predictors in the equation and is specified when the equations are developed.
Now that we have our predictands settled, and our predictors to offer to the regression, we can talk about developing some MOS equations.

46. MOS Development Strategy
Forward Selection ensures that the “best” or most STATISTICALLY IMPORTANT predictors are chosen first.
First predictor selected accounts for greatest reduction of variance (RV)
Subsequent predictors chosen give greatest RV in conjunction with predictors already selected
STOP selection when max # of terms reached, or when no remaining predictor will reduce variance by a pre-determined amount

47. Post-Processing MOS Guidance
Meteorological consistencies – SOME checks
T > Td; min T < T < max T; dir = 0 if wind speed = 0
Truncation (no probabilities < 0, > 1)
Monotonicity enforced (for elements like QPF)
BUT temporal coherence is only partially checked
Generation of “best categories”

48. Unconditional Probabilities from Conditional
If event B is conditioned upon A occurring:
Prob(B|A)=Prob(B)/Prob(A)
Prob(B) = Prob(A) × Prob(B|A)
Example:
Let A = event of > 10 mm., and B = event of > .25 mm., then if:
Prob (A) = .70, and
Prob (B|A) = .35, then
Prob (B) = .70 × .35 = .245 B A U

49. Truncating Probabilities
Applied to QPF and thunderstorm probabilities
If Prob(A) < 0, Probadj (A)=0
If Prob(A) > 1, Probadj (A)=1.

50. Normalizing MECE Probabilities
Sum of probabilities for exclusive and exhaustive categories must equal 1.0
If Prob (A) < 0, then sum of Prob (B) and Prob (C) = D, and is > 1.0.
Set: Probadj (A) = 0,
Probadj (B) = Prob (B) / D,
Probadj (C) = Prob (C) / D

51. Monotonic Categorical Probabilities
If event B is a subset of event A, then:
Prob (B) should be < Prob (A).
Example: B is > 25 mm; A is > 10 mm
Then, if Prob (B) > Prob (A)
set Probadj (B) = Prob (A).
Now, if event C is a subset of event B, e.g., C is > 0.50 in, and if Prob (C) > Prob (B),
set Probadj (C) = Prob (B)

52. Temporal Coherence of Probabilities
Event A is > 10mm occurring from 12Z-18Z
Event B is > 10 mm occurring from 18Z-00Z
A B is > 10mm occurring from 12Z-00Z
Then P(AB) = P(A) + P(B) – P(AB)
Thus, P(AB) should be:
< P(A) + P(B) and
> maximum of P(A), P(B) A C B

53. MOS Best Category Selection
An example with QPF…
TO MOS GUIDANCE MESSAGES 1 2 3 4 5 6
YES
YES
THRESHOLD
PROBABILITY (%) NO
EXCEEDED? NO NO NO

54. MOS GRID