Correction of daily values for inhomogeneities P. Štěpánek Czech Hydrometeorological Institute, Regional Office Brno, Czech Republic

Slides:

Advertisements

Similar presentations

Statistical modelling of precipitation time series including probability assessments of extreme events Silke Trömel and Christian-D. Schönwiese Institute.

Advertisements

Zentralanstalt für Meteorologie und Geodynamik 1. Comparison of HOM, SPLIDHOM and INTERP 2. Ideas for the daily benchmark dataset (temperature) Christine.

Computational Statistics. Basic ideas  Predict values that are hard to measure irl, by using co-variables (other properties from the same measurement.

Literature Review Kathryn Westerman Oliver Smith Enrique Hernandez Megan Fowler.

FTP Biostatistics II Model parameter estimations: Confronting models with measurements.

Homogenization of monthly Benchmark temperature series of network no. 3 – using ProClimDB software COST Benchmark meeting in Zürich September 2010.

STAT 497 APPLIED TIME SERIES ANALYSIS

Vector Generalized Additive Models and applications to extreme value analysis Olivier Mestre (1,2) (1) Météo-France, Ecole Nationale de la Météorologie,

Experimental Modeling

How should these data be modelled?. Identification step: Look at the SAC and SPAC Looks like an AR(1)- process. (Spikes are clearly decreasing in SAC.

Benchmark database based on surrogate climate records Victor Venema.

Curve-Fitting Regression

Environmental Data Analysis with MatLab Lecture 24: Confidence Limits of Spectra; Bootstraps.

2008 Chingchun 1 Bootstrap Chingchun Huang ( 黃敬群 ) Vision Lab, NCTU.

Benchmark database inhomogeneous data, surrogate data and synthetic data Victor Venema.

1 Econometric Load Forecasting Peak and Energy Forecast 06/14/2005 Econometric Load Forecasting Peak and Energy Forecast 06/14/2005.

1 4. Multiple Regression I ECON 251 Research Methods.

1 An Introduction to Nonparametric Regression Ning Li March 15 th, 2004 Biostatistics 277.

Daily Stew Kickoff – 27. January 2011 First Results of the Daily Stew Project Ralf Lindau.

A Regression Model for Ensemble Forecasts David Unger Climate Prediction Center.

Classification and Prediction: Regression Analysis

NIPRL Chapter 2. Random Variables 2.1 Discrete Random Variables 2.2 Continuous Random Variables 2.3 The Expectation of a Random Variable 2.4 The Variance.

Business Forecasting Chapter 4 Data Collection and Analysis in Forecasting.

Investigating the long- and short-term variations in meteorology in Atlanta Lucas Henneman 25 April, 2013.

Physics 114: Lecture 15 Probability Tests & Linear Fitting Dale E. Gary NJIT Physics Department.

AR- MA- och ARMA-.

Temperature correction of energy consumption time series Sumit Rahman, Methodology Advisory Service, Office for National Statistics.

Chapter 15 Modeling of Data. Statistics of Data Mean (or average): Variance: Median: a value x j such that half of the data are bigger than it, and half.

Dr. Richard Young Optronic Laboratories, Inc..  Uncertainty budgets are a growing requirement of measurements.  Multiple measurements are generally.

Utskifting av bakgrunnsbilde: -Høyreklikk på lysbildet og velg «Formater bakgrunn» -Under «Fyll», velg «Bilde eller tekstur» og deretter «Fil…» -Velg ønsket.

Xavier Sala-i-Martin Columbia University June 2008.

Diving into the Deep: Advanced Concepts Module 9.

Random Sampling, Point Estimation and Maximum Likelihood.

Regression analysis Control of built engineering objects, comparing to the plan Surveying observations – position of points Linear regression Regression.

Benchmark dataset processing P. Štěpánek, P. Zahradníček Czech Hydrometeorological Institute (CHMI), Regional Office Brno, Czech Republic, COST-ESO601.

COSTOC Olivier MestreMétéo-FranceFrance Ingebor AuerZAMGAustria Enric AguilarU. Rovirat i VirgiliSpain Paul Della-MartaMeteoSwissSwitzerland Vesselin.

Gridding Daily Climate Variables for use in ENSEMBLES Malcolm Haylock, Climatic Research Unit Nynke Hofstra, Mark New, Phil Jones.

Computer Vision - A Modern Approach Set: Tracking Slides by D.A. Forsyth The three main issues in tracking.

1 Statistical Distribution Fitting Dr. Jason Merrick.

SIXTH SEMINAR FOR HOMOGENIZATION AND QUALITY CONTROL IN CLIMATOLOGICAL DATABASES AND COST ES-0601 “HOME” ACTION MANAGEMENT COMMITTEE AND WORKING GROUPS.

Curve-Fitting Regression

Analysis of Residuals Data = Fit + Residual. Residual means left over Vertical distance of Y i from the regression hyper-plane An error of “prediction”

Experiences with homogenization of daily and monthly series of air temperature, precipitation and relative humidity in the Czech Republic, P.

Gridded Rainfall Estimation for Distributed Modeling in Western Mountainous Areas 1. Introduction Estimation of precipitation in mountainous areas continues.

HOME-ES601WG-1 Report to the 2nd MC, Vienna 23/11/2007 WG1 REPORT TO THE 2nd MC Enric Aguilar URV, Tarragona, Spain

Quality control and homogenization of the COST benchmark dataset Petr Štěpánek Pavel Zahradníček Czech Hydrometeorological Institute, regional office Brno.

Quality control of daily data on example of Central European series of air temperature, relative humidity and precipitation P. Štěpánek (1), P. Zahradníček.

Adjustment of Global Gridded Precipitation for Orographic Effects Jennifer C. Adam 1 Elizabeth A. Clark 1 Dennis P. Lettenmaier 1 Eric F. Wood 2 1.Dept.

Benchmark database inhomogeneous data, surrogate data and synthetic data Victor Venema.

Lecture 6: Point Interpolation

A novel methodology for identification of inhomogeneities in climate time series Andrés Farall 1, Jean-Phillipe Boulanger 1, Liliana Orellana 2 1 CLARIS.

Sea Level in month i =α 2 sin t i + α 3 cos t i + α 4 sin2 t i + α 5 cos2 t i seasonality + α 6 SOI i climate indices + α 1 + α 7 time i + (deviations.

ACTION COST-ES0601: Advances in homogenisation methods of climate series: an integrated approach (HOME), WG Meeting, Palma de Mallorca, January, 25-27,

1 Detection of discontinuities using an approach based on regression models and application to benchmark temperature by Lucie Vincent Climate Research.

Assumptions of Multiple Regression 1. Form of Relationship: –linear vs nonlinear –Main effects vs interaction effects 2. All relevant variables present.

ENVIRONMENTAL AGENCY OF THE REPUBLIC OF SLOVENIA COST benchmark dataset homogenisation: issues and remarks of the “Slovenian team” Presentation.

Homogenization of daily data series for extreme climate index calculation Lakatos, M., Szentimey T. Bihari, Z., Szalai, S. Meeting of COST-ES0601 (HOME)

Inhomogeneities in temperature records deceive long-range dependence estimators Victor Venema Olivier Mestre Henning W. Rust Presentation is based on:

Forecast 2 Linear trend Forecast error Seasonal demand.

MODEL DIAGNOSTICS By Eni Sumarminingsih, Ssi, MM.

Forecasting. Model with indicator variables The choice of a forecasting technique depends on the components identified in the time series. The techniques.

Actions & Activities Report PP8 – Potsdam Institute for Climate Impact Research, Germany 2.1Compilation of Meteorological Observations, 2.2Analysis of.

Lecture 9 Forecasting. Introduction to Forecasting * * * * * * * * o o o o o o o o Model 1Model 2 Which model performs better? There are many forecasting.

Statistical Forecasting

Phil Jones CRU, UEA, Norwich, UK

4th Joint EU-OECD Workshop on BCS, Brussels, October 12-13

Statistical Methods For Engineers

Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.

Chapter 2. Random Variables

Time Series introduction in R - Iñaki Puigdollers

Presentation transcript:

Correction of daily values for inhomogeneities P. Štěpánek Czech Hydrometeorological Institute, Regional Office Brno, Czech Republic COST-ESO601 meeting, Tarragona, 9-11 March 2009

Using daily data for inhomogeniety detection, is it meaningful?

Homogenization of daily values – precipitation series working with individual monthly values (to get rid of annual cycle) It is still needed to adapt data to approximate to normal distribution One of the possibilities: consider values above 0.1 mm only Additional transformation of series of ratios (e.g. with square root)

Original values - f ar from normal distribution (ratios tested/reference series)Frequencies Homogenization of precipitation – daily values

Limit value 0.1 mm (ratios tested/reference series)Frequencies

Limit value 0.1 mm, square root transformation (of ratios) (ratios tested/reference series)Frequencies Homogenization of precipitation – daily values

Problem of independence, Precipitation above 1 mm August, Autocorrelations

Problem of independece, T emperature August, Autocorrelations

Problem of independece, T emperature differences (reference – candidate) August, Autocorrelations

Homogenization Detection (preferably on monthly, seasonal and annual values) Correction – for daily values

WP1 SURVEY (Enric Aguilar) Daily data - Correction (WP4) Very few approaches actually calculate special corrections for daily data. Most approaches either –Do nothing (discard data) –Apply monthly factors –Interpolate monthly factors The survey points out several other alternatives that WG5 needs to investigate

Daily data correction methods „Delta“ methods Variable correction methods – one element Variable correction methods – several elements

Daily data correction methods Interpolation of monthly factors –MASH –Vincent et al (2002) - cublic spline interpolation Nearest neighbour resampling models, by Brandsma and Können (2006) Higher Order Moments (HOM), by Della Marta and Wanner (2006) Two phase non-linear regression (O. Mestre) Modified percentiles approach, by Stepanek Using weather types classifications (HOWCLASS), by I. Garcia-Borés, E. Aguilar,...

Adjusting daily values for inhomogeneities, from monthly versus daily adjustments („delta“ method)

Adjusting from monthly data monthly adjustments smoothed with Gaussian low pass filter (weights approximately 1:2:1) smoothed monthly adjustments are then evenly distributed among individual days

Adjusting straight from daily data Adjustment estimated for each individual day (series of 1 st Jan, 2 nd Jan etc.) Daily adjustments smoothed with Gaussian low pass filter for 90 days (annual cycle 3 times to solve margin values)

Air temperature etc.: differences between individual observation hours: be careful when appliying straight to daily AVG

Adjustments ( Delta method ) The same final adjustments may be obtained from either monthly averages or through direct use of daily data (for the daily-values-based approach, it seems reasonable to smooth with a low-pass filter for 60 days. The same results may be derived using a low-pass filter for two months (weights approximately 1:2:1) and subsequently distributing the smoothed monthly adjustments into daily values) (1 – raw adjustments, 2 – smoothed adjustments, 3 – smoothed adjustments distributed into individual days), b) daily-based approach (4 – individual calendar day adjustments, 5 – daily adjustments smoothed by low-pass filter for 30 days, 6 – for 60 days, 7 – for 90 days)

Spline through monthly temperature adjustments („delta“ method) Easy to implement No assumptions about changes in variance Integrated daily adjustments = monthly adjustments But, is it natural?

Variable correction f(C(d)|R), function build with the reference dataset R, d – daily data cdf, and thus the pdf of the adjusted candidate series C*(d) is exactly the same as the cdf or pdf of the original candidate series C(d)

Trewin & Trevitt (1996) method: Use simultaneous observations of old and new conditions Variable correction

1996

The HOM method concept: Fitting a model Locally weighted regression (LOESS) (Cleveland & Devlin,1998) HSP2 HSP1

The HOM method concept: Calculating the binned difference series Decile 1, k=1 Decile 10, k=10

The HOM method concept: The binned differences DELLA-MARTA AND WANNER, JOURNAL OF CLIMATE 19 (2006)

SPLIDHOM ( ), Olivier Mestre SPLIDHOM (SPLIne Daily HOMogenization), Olivier Mestre direct non-linear spline regression approach (x rather than a correction based on quantiles), cubic smoothing splines for estimating regression functions

Variable correction, q-q function Michel Déqué, Global and Planetary Change 57 (2007) 16–26

Our modified percentiles based approach

Our percentiles based approach

Variable correction methods – complex approach (several elements) not yet available …

Comparison of the methods, ProClimDB software

Correction methods comparison

Correction methods comparison, different parameters settings

Correction methods comparison, different parameters settings

Correction of daily values We have some methods … - but we have to validate them -> benchmark dataset on daily data Do we know how inhomogeneites in daily data behave? we should analyse real data who and when?, what method for data comparison?