Development of a Macro Editing Approach Work Session on Statistical Data Editing, Topic v: Editing based on results April 2008 WP 30

Overview Introduction to series of surveys that measures U.S. petroleum product supplied Limitation of micro editing and need for an edit approach at the aggregate level Approach considered for macro editing and the three types of models developed using one product as an example In sample forecast results and out-of-sample forecast performance results Summary and conclusions

The PSRS and Micro Edit Limitations The surveys, respondents and data collected –WPSRS: Weekly, six cut-off sample surveys –MPSRS : Monthly, nine population census surveys –PSA: Annual of revised monthly estimates, population census Limitations –Variability of responses –Lagged population coverage Corrective Measures –Micro editing –Imputation

The Approach Purpose of Study –Develop point and interval forecast at national and regional levels –One-month ahead forecast Approach –Econometric time-series models –Three models : Base, ARMA, and Supplemental Models –Micro editing enhanced by providing capabilities to identify outliers at the aggregate level

Model Development Model at product level –Distillate (Low Sulfur, High Sulfur, Total) –Gasoline Model at two geographic levels –National –Regional (PADD)

Model Forms Base Model: trends and seasonal factors expressed as: ARMA Model: Box-Jenkins approach utilizing AR and MA to capture the variation and seasonal pattern expressed as: Supplemental Model: Base Model with exogenous variables expressed as:

US Distillate Demand:

In-Sample One-Month-Out Forecast Evaluation Statistics Total Distillate Models BaseARMASuppl. RMSE MAE MAPE HSD Models BaseARMASuppl. RMSE MAE MAPE LSD Models BaseARMASuppl. RMSE MAE MAPE Note: There is no evidence of bias in any of the models

U.S. Distillate Demand Best Model Summary Statistics TotalHSDLSD Adjusted R S.E. of Regression Note: Estimation period Jan 1996 through Dec 2006

In-Sample Model Fit: Best Model ( 2 forecast standard errors)

Out-of-Sample Forecast Results: Best Model

Out-of-Sample Forecast Results: Best Model , HSD

Out-of-Sample Results: Best Model , LSD

Regional Models Regions: Petroleum Administration for Defense District Identify exogenous variables to explain regional patterns of distillate demand –Residential heating in the Northeast (PADD 1): Heating Degree-Days –Agriculture in the Midwest (PADD 2): Precipitation HDD DEVPopulation-Weighted Heating Degree-Days: Deviation from Normal PRECIP DEVArea-Weighted Precipitation: Deviation from Long-Term Normal EMP TRANSEmployment in Transportation Industries IPI MFGIndex of Industrial Production for Durable Goods FREIGHT INDXTransportation Services Index for Freight PRICE RATIOAverage monthly spot price ratio: No.2 Fuel Oil / Natural Gas Exogenous Variables Used in Supplemental Distillate Models PADD 1PADD 2PADD 3PADD 5NATIONAL HSDLSDTOTHSDLSDTOTHSDLSDTOTHSDLSDTOTHSDLSDTOT HDD DEVXXXXX PRECIP DEVXXXX EMP TRANSXX IPI MFGX FREIGHT INDXXXXX PRICE RATIOX

Regional Model Details: In-Sample Model Fit, PADD 1 HSD

Regional Model Details: In-Sample Model Fit, PADD 1, LSD

Regional Model Details: In-Sample Model Fit, PADD 2, HSD

Regional Model Details: In-Sample Model Fit, PADD 2, LSD

Regional Model Details: Out-of-Sample Forecast Results, PADD 1, HSD

Regional Model Details: Out-of-Sample Forecast Results, PADD 1, LSD

Regional Model Details: Out-of-Sample Forecast Results, PADD 2, HSD

Regional Model Details: Out-of-Sample Forecast Results, PADD 2, LSD

Benefits & Limitations How does this improve EIA’s current activities? –Establishes a range of expected results at the aggregate level that will alert a reviewer when to investigate possible anomalies in the respondent data –Can identify the region which provides largest contribution to deviation, guiding further editing and imputation activities prior to data release –Reduces risk of revisions to released data Limitations of Modeling –Reasons for deviations are not always readily apparent: respondent error, structural shifts in consumption, or failure of the model to respond to external influences –Regional-level models provide guidance, but not necessarily answers –Ranges may be too large

Future Plans Model improvements –Dynamic adjustments to known issues like shifts –Better exogenous variables Automation of gathering and formatting model inputs –Weather Data –Economic Data –Forecast generation Expand to other key petroleum products –Gasoline and gasoline subcomponents (currently underway) –Residual fuel oil

US Distillate Demand: Best Model * The variable “MONTH” indicates 11 monthly dummy variables to account for seasonality in demand. In each of the models the probability value is obtained from the results of an F-test of the collective significance of the seasonal dummy variables. Total DistillateHSDLSD VariableCoefficient Std. ErrorProb.VariableCoefficient Std. ErrorProb.VariableCoefficient Std. ErrorProb. C C C MONTH **** 0.000L_OCT MONTH **** L_SEP MONTH **** 0.000L_NOV T_MAY HDD_DEV T_MAY T_JAN PR_RAT(-1) TSI_FRT(-1) HDD_DEV AR(7) AR(10) TSI_FRT AR(1) MA(3) AR(4) MA(2) Adj. R^2SE RegD-WAdj. R^2SE RegD-WAdj. R^2SE RegD-W