Modeling Errors in Satellite Data Yudong Tian University of Maryland & NASA/GSFC Sponsored by NASA ESDR-ERR Program.

Slides:



Advertisements
Similar presentations
Precipitation in IGWCO The objectives of IGWCO require time series of accurate gridded precipitation fields with fine spatial and temporal resolution for.
Advertisements

Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.
NORMAL OR GAUSSIAN DISTRIBUTION Chapter 5. General Normal Distribution Two parameter distribution with a pdf given by:
Uncertainty Representation and Quantification in Precipitation Data Records Yudong Tian Collaborators: Ling Tang, Bob Adler, George Huffman, Xin Lin, Fang.
Computational Statistics. Basic ideas  Predict values that are hard to measure irl, by using co-variables (other properties from the same measurement.
Empirical Analysis and Statistical Modeling of Errors in Satellite Precipitation Sensors Yudong Tian, Ling Tang, Robert Adler, and Xin Lin University of.
GlobColour CDR Meeting ESRIN July 2006 Merging Algorithm Sensitivity Analysis ACRI-ST/UoP.
Second order cone programming approaches for handing missing and uncertain data P. K. Shivaswamy, C. Bhattacharyya and A. J. Smola Discussion led by Qi.
Approaches to Data Acquisition The LCA depends upon data acquisition Qualitative vs. Quantitative –While some quantitative analysis is appropriate, inappropriate.
W. McNair Bostick, Oumarou Badini, James W. Jones, Russell S. Yost, Claudio O. Stockle, and Amadou Kodio Ensemble Kalman Filter Estimation of Soil Carbon.
Uncertainty Representation. Gaussian Distribution variance Standard deviation.
Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft NDACC H2O workshop, Bern, July 2006 Water vapour profiles by ground-based FTIR Spectroscopy:
Statistics for Managers Using Microsoft Excel, 4e © 2004 Prentice-Hall, Inc. Chap 7-1 Chapter 7 Confidence Interval Estimation Statistics for Managers.
Basic Business Statistics, 10e © 2006 Prentice-Hall, Inc. Chap 8-1 Chapter 8 Confidence Interval Estimation Basic Business Statistics 10 th Edition.
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
Lecture 24: Thurs. Dec. 4 Extra sum of squares F-tests (10.3) R-squared statistic (10.4.1) Residual plots (11.2) Influential observations (11.3,
Topic 3: Regression.
Syllabus for Engineering Statistics Probability: rules, cond. probability, independence Summarising and presenting data Discrete and continuous random.
Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.
Introduction to Regression Analysis, Chapter 13,
Classification and Prediction: Regression Analysis
Standard error of estimate & Confidence interval.
Statistics for Managers Using Microsoft Excel, 4e © 2004 Prentice-Hall, Inc. Chap 7-1 Chapter 7 Confidence Interval Estimation Statistics for Managers.
Statistics for Managers Using Microsoft Excel, 4e © 2004 Prentice-Hall, Inc. Chap 7-1 Chapter 7 Confidence Interval Estimation Statistics for Managers.
Regression and Correlation Methods Judy Zhong Ph.D.
Quantify prediction uncertainty (Book, p ) Prediction standard deviations (Book, p. 180): A measure of prediction uncertainty Calculated by translating.
1. An Overview of the Data Analysis and Probability Standard for School Mathematics? 2.
Chapter 8 Introduction to Hypothesis Testing
Hypothesis Testing in Linear Regression Analysis
A Kalman Filter Approach to Blend Various Satellite Rainfall Estimates in CMORPH Robert Joyce NOAA/NCEP/CPC Wyle Information Systems Pingping Xie NOAA/NCEP/CPC.
Calibration Guidelines 1. Start simple, add complexity carefully 2. Use a broad range of information 3. Be well-posed & be comprehensive 4. Include diverse.
Chi-squared distribution  2 N N = number of degrees of freedom Computed using incomplete gamma function: Moments of  2 distribution:
CPC Unified Gauge – Satellite Merged Precipitation Analysis for Improved Monitoring and Assessments of Global Climate Pingping Xie, Soo-Hyun Yoo,
ESA DA Projects Progress Meeting 2University of Reading Advanced Data Assimilation Methods WP2.1 Perform (ensemble) experiments to quantify model errors.
Topics: Statistics & Experimental Design The Human Visual System Color Science Light Sources: Radiometry/Photometry Geometric Optics Tone-transfer Function.
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss High-resolution data assimilation in COSMO: Status and.
EQT 373 Chapter 3 Simple Linear Regression. EQT 373 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value.
Applied Quantitative Analysis and Practices LECTURE#23 By Dr. Osman Sadiq Paracha.
Regional climate prediction comparisons via statistical upscaling and downscaling Peter Guttorp University of Washington Norwegian Computing Center
Why Model? Make predictions or forecasts where we don’t have data.
Multiple Regression Petter Mostad Review: Simple linear regression We define a model where are independent (normally distributed) with equal.
Combining CMORPH with Gauge Analysis over
Maths Study Centre CB Open 11am – 5pm Semester Weekdays
. Outline  Evaluation of different model-error schemes in the WRF mesoscale ensemble: stochastic, multi-physics and combinations thereof  Where is.
Functions of random variables Sometimes what we can measure is not what we are interested in! Example: mass of binary-star system: We want M but can only.
Example: Bioassay experiment Problem statement –Observations: At each level of dose, 5 animals are tested, and number of death are observed.
Planck’s Constant and the Photoelectric Effect Lulu Liu Partner: Pablo Solis Junior Lab 8.13 Preparatory Lab September 25 th, 2007.
Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Model Building and Model Diagnostics Chapter 15.
Yan Y. Kagan Dept. Earth and Space Sciences, UCLA, Los Angeles, CA ,
A Global Kalman Filtered CMORPH using TRMM to Blend Satellite Rainfall Robert Joyce NOAA/NCEP/CPC Wyle Information Systems Pingping Xie NOAA/NCEP/CPC John.
NCAF Manchester July 2000 Graham Hesketh Information Engineering Group Rolls-Royce Strategic Research Centre.
Local Predictability of the Performance of an Ensemble Forecast System Liz Satterfield and Istvan Szunyogh Texas A&M University, College Station, TX Third.
Locally Optimized Precipitation Detection over Land Grant Petty Atmospheric and Oceanic Sciences University of Wisconsin - Madison.
More on regression Petter Mostad More on indicator variables If an independent variable is an indicator variable, cases where it is 1 will.
Infrared and Microwave Remote Sensing of Sea Surface Temperature Gary A. Wick NOAA Environmental Technology Laboratory January 14, 2004.
A new high resolution satellite derived precipitation data set for climate studies Renu Joseph, T. Smith, M. R. P. Sapiano, and R. R. Ferraro Cooperative.
The Unscented Kalman Filter for Nonlinear Estimation Young Ki Baik.
Parameter Estimation. Statistics Probability specified inferred Steam engine pump “prediction” “estimation”
G O D D A R D S P A C E F L I G H T C E N T E R TRMM Tropical Rainfall Measuring Mission 2nd GPM GV Workshop TRMM Ground Validation Some Lessons and Results.
Application of Probability Density Function - Optimal Interpolation in Hourly Gauge-Satellite Merged Precipitation Analysis over China Yan Shen, Yang Pan,
Lecturer: Ing. Martina Hanová, PhD..  How do we evaluate a model?  How do we know if the model we are using is good?  assumptions relate to the (population)
Basics of Multivariate Probability
Why Model? Make predictions or forecasts where we don’t have data.
*CPC Morphing Technique
Multiple Imputation using SOLAS for Missing Data Analysis
Information content in ensemble data assimilation
The Gamma PDF Eliason (1993).
Graph Review Skills Needed Identify the relationship in the graph
CS639: Data Management for Data Science
Thomas Smith1 Phillip A. Arkin2 George J. Huffman3 John J. Bates1
Presentation transcript:

Modeling Errors in Satellite Data Yudong Tian University of Maryland & NASA/GSFC Sponsored by NASA ESDR-ERR Program

Optimal combination of independent observations (or how human knowledge grows) 2 Information content

“Conservation of Information Content” 3

Why uncertainty quantification is always needed 4 Information content

5 1. Most commonly, subconsciously used error model: T i : truth, error free. X i : measurements, b: systematic error (bias) 2. A more general additive error model: The additive error model

6 A nonlinear multiplicative measurement error model: T i : truth, error free. X i : measurements With a logarithm transformation, the model is now a linear, additive error model, with three parameters: A=log(α), B=β, x i =log(X i ), t i =log(T i ) The multiplicative error model

7 Correct error model is critical in quantifying uncertainty TiTi XiXi TiTi XiXi TiTi XiXi

8 Additive model does not have a constant variance

9 Additive error model: why variance is not constant? -- systematic errors leaking into random errors

10 The multiplicative error model predicts better

11 Clean separation of systematic and random errors More appropriate for measurements with several orders of magnitude variability Good predictive skills Tian et al., 2012: Error modeling for daily precipitation measurements: additive or multiplicative? to be submitted to Geophys. Rev. Lett. The multiplicative error model has clear advantages

Spatial distribution of the model parameters 12 TMI AMSR-E F16 F17 A B σ(random error)

13 Probability distribution of the model parameters A B σ TMI AMSR-E F16 F17

14 A measurement without uncertainly is meaningless Wrong error models produce wrong uncertainties Multiplicative model is recommended for fine resolution precipitation measurements Tian et al., 2012: Error modeling for daily precipitation measurements: additive or multiplicative? to be submitted to Geophys. Rev. Lett. Summary

Extra slides 15

Summary and Conclusions Created bias-corrected radar data for validation Evaluated biases in PMW imagers: AMSR-E, TMI and SSMIS Constructed an error model to quantify both systematic and random errors 16