1. Reconstruction of missing data in satellite and in situ data sets with DINEOF A. Alvera-Azcárate, A. Barth, J.-M. Beckers GHER – University de Liège FNRS – National Fund for the Scientific Research, Belgium

2. Reconstruction method for gappy data based on an EOF decomposition Parameter-free, no need of a priori information resolves time and space much faster than objective analysis Determines optimal number of EOFs by cross-validation. Error estimation Lanczos method for EOF decomposition (Toumazou and Crétaux, 2001): large matrices Characteristics of DINEOF DINEOF: Data Interpolating Empirical Orthogonal Functions

3. N EOFs ● ● ● ● ● ● ● 2nd: EOF decomposition with N=1 EOF Calculate missing values: Improved guess for missing values Convergence: best value for missing data with 1 EOF cross validation: error EOF decomposition with N=2 EOFs Calculate missing values Improved guess for missing values Then we repeat with N= 3 EOFs and so on… 1st: Demeaned matrix: missing data flagged and set to zero Some data are set aside for cross-validation error How it works

4. SST Chlorophyll Wind-induced filaments in Adriatic Sea Mississippi river plume advected by Loop Current Some Examples Alvera-Azcárate et al, 2005

5. Multivariate DINEOF: examples Radial currents: positive values: current towards the antenna negative values: current away from the antenna ˚C m/s SST + HF-Radar currents

6. Time series reconstruction Hourly air temperature at Izas catchment (Pyrenees) ‏ ˚C hours months

7. Original data, ˚C Reconstruction, ˚C Time series reconstruction:original vs. reconstruction

8. Error maps are calculated for SST reconstructions using: i) the EOF basis from DINEOF as background covariance ii) the location of valid data Outliers (pixels with value larger than the statistically expected misfit calculated during the analysis) will be objectively identified and removed from initial data Error maps and outlier detection

9. Spike removal in temporal EOFs When too few data are present: temporal EOFs poorly constrained: unrealistic discontinuities Sharp transition

10. Spike removal in temporal EOFs We apply a filter to the temporal covariance matrix Alvera-Azcárate et al, 2009 F is a Laplacian filter Filter on C instead of X: C is much smaller and less sensitive to missing data Filter applied iteratively: more iterations, further reach of the filter days

11. Spike removal in temporal EOFs

12. Conclusions DINEOF is a reliable, easy to use technique to reconstruct missing data Widely tested and results validated with in situ data A variety of data and platforms can be used SST, chlorophyll, TSM, HF-Radar…. Univariate and Multivariate approaches Satellite data and time series Error maps and outliers Generation of error maps to assess the confidence of the reconstruction Outliers (cloud edges, haze...) can be removed from original data Time filter to remove spikes Code is freely available; installation instructions; mailing list http://modb.oce.ulg.ac.be/mediawiki/index.php/DINEOF

13. References (chronologic) ‏ J.-M. Beckers and M Rixen. EOF calculations and data filling from incomplete oceanographic data sets. Journal of Atmospheric and Oceanic Technology, 20(12):1839-1856, 2003. A. Alvera-Azcárate, A. Barth, M. Rixen, and J. M. Beckers. Reconstruction of incomplete oceanographic data sets using Empirical Orthogonal Functions. Application to the Adriatic Sea. Ocean Modelling, 9:325-346, 2005. J.-M. Beckers, A. Barth, and A. Alvera-Azcárate. DINEOF reconstruction of clouded images including error maps. Application to the Sea Surface Temperature around Corsican Island. Ocean Science, 2(2):183- 199, 2006. A. Alvera-Azcarate, A. Barth, J. M. Beckers, and R. H. Weisberg. Multivariate reconstruction of missing data in sea surface temperature, chlorophyll and wind satellite fields. Journal of Geophysical Research, 112:C03008, 2007. A. Alvera-Azcárate, A. Barth, D. Sirjacobs, J.-M. Beckers. Enhancing temporal correlations in EOF expansions for the reconstruction of missing data using DINEOF. Ocean Science, 5, 475-485, 2009. A. Alvera-Azcárate, A. Barth, D. Sirjacobs, F.Lenartz, J.-M. Beckers. Data Interpolating Empirical Orthogonal Functions (DINEOF): a tool for geophysical data analyses. Submitted to Mediterranean Marine Science. 2009. U. Ganzedo, A. Alvera-Azcarate, G.Esnaola, A. Ezcurra and J. Saenz. 2009. Reconstruction of sea surface temperature by means of DINEOF. A case study during the fishing season in the Bay of Byscay. Accepted in International Journal of Remote Sensing. D. Sirjacobs, A. Alvera-Azcárate, A. Barth, G. Lacroix, Y. Park, B. Nechad, K. Ruddick, J.-M. Beckers. Cloud filling of ocean color and sea surface temperature remote sensing products over the Southern North Sea by the Data Interpolating Empirical Orthogonal Functions methodology. Accepted in Journal of Sea Research, 2010. GHER

14. References Y.Z. Ding, Z.H. Wei, Z.H. Mao, X.F. Wang, D.L. Pan. Reconstruction of incomplete satellite SST data sets based on EOF method. Acta Oceanologia Sinica, 28 (2): 36-44 2009. E. Mauri, P.M. Poulain, Južnič-Zonta, Z. MODIS chlorophyll variability in the northern Adriatic Sea and relationship with forcing parameters. Journal of Geophysical Research, vol 112, C03S11, 2007. E. Mauri, P.M. Poulain, G. Notarstefano. Spatial and temporal variability of the sea surface temperature in the Gulf of Trieste between January 2000 and December 2006. Journal of Geophysical Research. Vol. 113, No. C10, C10012. 2008. T.N. Miles, R. He, M. Li. Characterizing the South Atlantic Bight seasonal variability and cold-water event in 2003 using a daily cloud-free SST and chlorophyll analysis. Geophysical Research Letters, 36: Art. No. L02604 JAN 22 2009. N.P. Nezlin, K. Kamer, J. Hyde, E.D. Stein, ED. Dissolved oxygen dynamics in a eutrophic estuary, Upper Newport Bay, California. Estuarine, Coastal and Shelf Science, 82 (1): 139-151 MAR 20 2009. R. F. Sánchez, P. Relvas, M. Delgado. Coupled ocean wind and sea surface temperature patterns off the western Iberian Peninsula. Journal of Marine Systems, 68, 103-127. 2007. … and other applications: