Model based approach for estimating and forecasting crop statistics: Update, consolidation and improvement of AGROMET model “AGROMET Project” Working Group Meeting on Crop Statistics October 25th, Luxembourg

DevStat Agenda 1. Introduction: Project Goals and current AGROMET model. 2. Solutions: a new model. A.Key features and Improvements B.Imputation of missing data C.Model selection D.Model validation E.Set-Aside effect of A 3. Results and examples. 4. Conclusions and comments.

DevStat Project goals 1. Introduction To provide forecasts for crops, vegetables and fruit production on every country member at the EU, based on historical data on both observables provided to Eurostat periodically: harvest production (H) and arable area (A). To generate a SAS application for the statistical analysis, and use the SAS Enterprise Guide to create its interface. To generate documentation on the application and form users.

DevStat Project Implementation 1. Introduction DEVSTAT and UMH-CIO are the members of the consortium providing services to Eurostat under the Framework contract number S Within this framework contract, the consortium received the request for services ESTAT E0/24 for the updating, consolidation and improvement of current AGROMET model, according to the goals stated in the previous slide. This presentation shows the main features of the project and the current state of the development and implementation of the new model.

DevStat Current Agromet Model: main features Eurostat has been using the AGROMET MODEL, programmed within FAME, to provide forecasts for areas, yield and production on crops, on twelve of the countries at the EU, and also on its aggregate. Yield is calculated by dividing the observable vaiables Production and Area. Estimates of Yield and Area are produced based on the last 10 years, by using: Linear regression Quadratic regression ARIMA(1,1,1) Production estimate is obtained by multiplying Yield and Area estimates. 1. Introduction

DevStat Current Agromet Model: limitations The current AGROMET model: Constraints forecasts from fixed estimation models for each product, whatever the country and prediction year. Constraints some forecasts to predictions based on lines or parables, also assuming independency of the historical available data series (inconsistent assumption on time correlated data series) Excludes forecasting in all these cases (combinations product-country) with 2 or more missing values in the data series from the last 10 years. Focuses on forecasting the non-observable variable YIELD to provide forecasts on the observable HARVEST PRODUCTION. Does not provide forecasting error measures. Only covers crops on 12 from the 27 current UE members. 1. Introduction

DevStat Key features of the new model 2. Solutions HARVEST PRODUCTION (H) and AREA (A) are the relevant observable variables in the analysis and model fitting. Prediction data: 10 years time series. IMPROVEMENTS An Imputation procedure for missing data values has been developed. Trend models reasonably substitute fixed Agromet models. Relationship between the observables A and H is used for H prediction whenever possible. Inclusion of an automatic model selection criteria for different fitted models. Error measures are provided on the forecasts. Set-aside specifications on A can be used to predict the effect on H.

DevStat Imputation procedure 1) Regression/Inverse regression of H on A. 2) Moving Average Estimate of lag 1 2. Solutions

DevStat Imputation procedure 2. Solutions HARVEST AREA 1) Regression/Inverse regression of H on A.

DevStat Imputation procedure 2. Solutions 2) Moving Average Estimate of lag 1

DevStat Imputation results 2. Solutions

DevStat Model especification Case 1 Case 2 Case 3 4 or more NA data Average of last 3 years 3 or less NA data ARIMA (p,d,q) no NA data on H and A A: ARIMA (p,d,q) Regression models for H on A with ARMA (p,q) errors 2. Solutions

DevStat Model selection Automatic selection rule for the best model among the ARIMA(p,d,q) and ARMA(p,q) models. Flexible and optimal adaptation to data, whatever the product, country and behaviour along time. Intrinsic validation mechanism: the best model provides minimum deviation between observed and estimated. These facts are distinctive improvements with respect to the previous AGROMET model. 2. Solutions

DevStat Validation The validation index VAL is defined as the “percentage of cases whose confidence or credible interval does get the observed value at the prediction year”. This validation index is assessed at two levels: COUNTRY-TYPE-PREDICTION YEAR (country VAL index) PRODUCT-PREDICTION YEAR (product VAL index) 2. Solutions

DevStat Set-aside on Area 2. Solutions In order to produce forecasts of HARVEST under land restrictions, two possibilities are considered: When a regression model is available, to predict HARVEST based in the regression model, with AREA, as explanatory variable. If not, to predict HARVEST on a proportional basis, by applying the same percentage of reduction for arable land.

DevStat Interface in SAS E-G 3. Results Selection of Country

DevStat Interface in SAS E-G 3. Results Selection of Type of product

DevStat Interface in SAS E-G 3. Results Selection of product

DevStat Interface in SAS E-G 3. Results Selection of Prediction year

DevStat Interface in SAS E-G 3. Results Selection of Variable to predict

DevStat Estimation results: example case 1 Case 1 4 or more NA data on A and H Average of last 3 years 3. Results

DevStat Estimation results: example case 2 Case 2 3 or less NA data on A and H H: ARIMA (p,d,q) A: ARIMA (p,d,q) 3. Results

DevStat Estimation results: example case 3 Case 3 no NA data on A and H ARIMA(p,d,q) on A Regression models for H on A with ARMA (p,q) errors 3. Results

DevStat Validation Results: product VAL index 3. Results

DevStat Validation Results: country VAL index 3. Results

DevStat Conclusions 4. Conclusions The AGROMET model has been improved: It uses more appropriate time series models It adapts the available data, whatever the country, product or prediction year, always with the optimal model selected. It provides forecasting errors. Its interface allows for selection on type of product, product, country, prediction year, set-aside. It accommodates a set-aside restriction on A It has been programmed under SAS Enterprise Guide.

DevStat Comments 4. Conclusions THANKS FOR YOUR ATTENTION