Facultad de Ingeniería, Centro de Cálculo Application of ANN to the prediction of missing daily precipitation records, and comparison against linear methodologies Carlos López-Vázquez Facultad de Ingeniería, Centro de Cálculo Montevideo URUGUAY

Organization of the presentation The missing value problem The experiment Data and test area description Methods tested Results Conclusions

The missing value problem Weather records typically have missing values Many applications require complete databases Well established linear methods for interpolate spatial observations exist Their performance is poor for daily rain records

The experiment We created missing values at random locations in our dataset We imputated them using all test methods We computed RMS, mean and some percentiles of the absolute deviation between the original and the imputated value We performed a number of simulations, and averaged the results

Data and test area description 20 years of daily records for 10 stations were available 30 % of the events have missing values More than 80% of the readings are of zero rain, evenly distributed in the year Annual averages ranges from 1600 to 500 mm/day; time correlation is low

Methods tested: linear All follow the general equation We restricted ourselves to weights w and bias b constant in time; its values depends on the method itself Some commonly used methods will be briefly described “Best among linear” methods can be obtained depending on the error measure used

Linear methods: Cressman being dij stands for the distance between station i and j It does not use historical information Only requires the station coordinates

Linear methods: Nearest neighbor Uses only data from the closest available station “closest” might be geometrical, or might be determined by an expert; we tested both Due to its simplicity it is widely used

Linear methods: Optimum interpolation It is the standard interpolation method in meteorology It is also known as kriging in geostatistics Requires the evaluation of the covariance matrix of the sample, and a model of the spatial correlation. It is designed in order to minimize the expected RMSE

Linear methods: “The Best ones” Given an objective measure of the error, it can be minimized by tuning the parameters w Least sum of squares, Least average of the absolute error, Least 95 percentile, etc. are examples. The optimum might led to different vectors w in each case. Calculating the parameters w might be a heavy nonlinear optimization problem

Non-linear methods: ANN We used ANN as interpolators, with 9 inputs and 1 output The training was performed with one third of the dataset using backpropagation and minimizing the RMSE Some different architectures were considered (one and two hidden layers; different number of neurons, etc.) as well as some transformations of the data

Results The results are poor, irrespective of the method; daily rain is a very difficult problem Both ANN gave only slightly better results than linear methods, but were more robust The training algorithm failed to escape from local minima; that leds to a significant CPU cost

Conclusions Linear methods with constant coefficients have limited possibilities Adding information of the day before improves to some extent the results ANN gave interesting results, but with too much CPU requirements Better training algorithms are required