A physical basis of GOES SST retrieval Prabhat Koner Andy Harris Jon Mittaz Eileen Maturi.

Slides:



Advertisements
Similar presentations
Pattern Recognition and Machine Learning
Advertisements

NCAS Atmospheric Measurement Summer School, September 2010 Page 1/12 Deriving useful information from satellite data (a remote sensing application) Satellite.
A fast physical algorithm for hyperspectral sounding retrieval Zhenglong Li #, Jun Li #, Timothy J. and M. Paul Menzel # # Cooperative Institute.
Improvements to the NOAA Geostationary Sea Surface Temperature Product Suite Eileen Maturi, NOAA/NESDIS/STAR Andy Harris, Jonathan Mittaz, Prabhat Koner.
1 Low-Dose Dual-Energy CT for PET Attenuation Correction with Statistical Sinogram Restoration Joonki Noh, Jeffrey A. Fessler EECS Department, The University.
Pattern Recognition and Machine Learning
An optimal estimation based retrieval method adapted to SEVIRI infra-red measurements M. Stengel (1), R. Bennartz (2), J. Schulz (3), A. Walther (2,4),
L.M. McMillin NOAA/NESDIS/ORA Regression Retrieval Overview Larry McMillin Climate Research and Applications Division National Environmental Satellite,
TNO orbit computation: analysing the observed population Jenni Virtanen Observatory, University of Helsinki Workshop on Transneptunian objects - Dynamical.
Uncertainty estimates in input (Rrs) and output ocean color data: a brief review Stéphane Maritorena – ERI/UCSB.
Uncertainty Representation. Gaussian Distribution variance Standard deviation.
Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.
Imaging Algorithm – Eric Thiébaut – IAU Optical/IR Interferometry Working Group, 2002 J M M C Image Reconstruction in Optical/IR Aperture Synthesis Eric.
CPI International UV/Vis Limb Workshop Bremen, April Development of Generalized Limb Scattering Retrieval Algorithms Jerry Lumpe & Ed Cólon.
Integral Equations, Inverse methods for the remote sensing of atmospheric parameters Rolando Rizzi Atmospheric Dynamics Group, Department of.
Lecture 8 The Principle of Maximum Likelihood. Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture.
Retrieval Theory Mar 23, 2008 Vijay Natraj. The Inverse Modeling Problem Optimize values of an ensemble of variables (state vector x ) using observations:
REMOTE SENSING & THE INVERSE PROBLEM “Remote sensing is the science and art of obtaining information about an object, area, or phenomenon through the analysis.
Where m is the first order gain, q is the second order gain, is the observed blackbody counts, is the observed space counts, is the radiance of the blackbody,
GOES-13 Science Team Report SST Images and Analyses Eileen Maturi, STAR/SOCD, Camp Springs, MD Andy Harris, CICS, University of Maryland, MD Chris Merchant,
Algorithm Evaluation and Error Analysis class 7 Multiple View Geometry Comp Marc Pollefeys.
Basics of regression analysis
Bootstrapping a Heteroscedastic Regression Model with Application to 3D Rigid Motion Evaluation Bogdan Matei Peter Meer Electrical and Computer Engineering.
Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 NOAA Operational Geostationary Sea Surface Temperature Products from NOAA.
Principles of the Global Positioning System Lecture 10 Prof. Thomas Herring Room A;
NASA MODIS-VIIRS ST Meeting, May 18 – 22, A Deterministic Inverse Method for SST Retrieval from VIIRS: Early Results Andy Harris, Prabhat Koner.
RLSELE Adaptive Signal Processing 1 Recursive Least-Squares (RLS) Adaptive Filters.
1 Satellite Remote Sensing of Particulate Matter Air Quality ARSET Applied Remote Sensing Education and Training A project of NASA Applied Sciences Pawan.
Colorado Center for Astrodynamics Research The University of Colorado STATISTICAL ORBIT DETERMINATION Project Report Unscented kalman Filter Information.
Stats for Engineers Lecture 9. Summary From Last Time Confidence Intervals for the mean t-tables Q Student t-distribution.
COST 723 Training School - Cargese October 2005 OBS 2 Radiative transfer for thermal radiation. Observations Bruno Carli.
CJT 765: Structural Equation Modeling Class 7: fitting a model, fit indices, comparingmodels, statistical power.
23 rd to 24 th October 2006GIST 25, UK Met Office, Exeter In-flight Assessment of the end-to-end spectral responses of the GERB radiometer detectors Glyn.
Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 MAP (Maximum A Posteriori) x is reduced state vector [SST(x), TCWV(w)]
Modern Navigation Thomas Herring
Development and evaluation of Passive Microwave SWE retrieval equations for mountainous area Naoki Mizukami.
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 3: LINEAR MODELS FOR REGRESSION.
A NON-RAINING 1DVAR RETRIEVAL FOR GMI DAVID DUNCAN JCSDA COLLOQUIUM 7/30/15.
SUPA Advanced Data Analysis Course, Jan 6th – 7th 2009 Advanced Data Analysis for the Physical Sciences Dr Martin Hendry Dept of Physics and Astronomy.
CrIS Use or disclosure of data contained on this sheet is subject to NPOESS Program restrictions. ITT INDUSTRIES AER BOMEM BALL DRS EDR Algorithms for.
University of Colorado Boulder ASEN 5070: Statistical Orbit Determination I Fall 2014 Professor Brandon A. Jones Lecture 26: Singular Value Decomposition.
Kernel adaptive filtering Lecture slides for EEL6502 Spring 2011 Sohan Seth.
NASA MODIS-VIIRS ST Meeting, May 18 – 22, New physically based sea surface temperature retrievals for NPP VIIRS Andy Harris, Prabhat Koner CICS,
Colorado Center for Astrodynamics Research The University of Colorado 1 STATISTICAL ORBIT DETERMINATION The Minimum Variance Estimate ASEN 5070 LECTURE.
AIRS (Atmospheric Infrared Sounder) Regression Retrieval (Level 2)
AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M.
INVERSE MODELING TECHNIQUES Daniel J. Jacob. GENERAL APPROACH FOR COMPLEX SYSTEM ANALYSIS Construct mathematical “forward” model describing system As.
Chapter 28 Cononical Correction Regression Analysis used for Temperature Retrieval.
Bayes Theorem The most likely value of x derived from this posterior pdf therefore represents our inverse solution. Our knowledge contained in is explicitly.
An Introduction to Optimal Estimation Theory Chris O´Dell AT652 Fall 2013.
Estimating the uncertainties in the products of inversion algorithms or, how do we set the error bars for our inversion results? Emmanuel Boss, U. of Maine.
MIIDAPS Application to GSI for QC and Dynamic Emissivity in Passive Microwave Data Assimilation.
Chapter 2-OPTIMIZATION G.Anuradha. Contents Derivative-based Optimization –Descent Methods –The Method of Steepest Descent –Classical Newton’s Method.
Status of Reference Network Simulations John Dale ILC-CLIC LET Beam Dynamics Workshop 23 June 2009.
Considerations for the Physical Inversion of Cloudy Radiometric Satellite Observations.
Retrieval Algorithms The derivations for each satellite consist of two steps: 1) cloud detection using a Bayesian Probabilistic Cloud Mask; and 2) application.
Physical retrieval (for MODIS) Andy Harris Jonathan Mittaz Prabhat Koner (Chris Merchant, Pierre LeBorgne)
ESTIMATION METHODS We know how to calculate confidence intervals for estimates of  and  2 Now, we need procedures to calculate  and  2, themselves.
1 Information Content Tristan L’Ecuyer. 2 Degrees of Freedom Using the expression for the state vector that minimizes the cost function it is relatively.
A New Ocean Suite Algorithm for AMSR2 David I. Duncan September 16 th, 2015 AMSR Science Team Meeting Huntsville, AL.
UCLA Vector Radiative Transfer Models for Application to Satellite Data Assimilation K. N. Liou, S. C. Ou, Y. Takano and Q. Yue Department of Atmospheric.
1 Radiative Transfer Models and their Adjoints Paul van Delst.
Computacion Inteligente Least-Square Methods for System Identification.
EARWiG: SST retrieval issues for High Latitudes Andy Harris Jonathan Mittaz NOAA-CICS University of Maryland Chris Merchant U Edinburgh.
EARWiG: SST retrieval issues for TWP Andy Harris Jonathan Mittaz Prabhat Koner NOAA-CICS University of Maryland Eileen Maturi NOAA/NESDIS, Camp Springs,
June 20, 2005Workshop on Chemical data assimilation and data needs Data Assimilation Methods Experience from operational meteorological assimilation John.
Colorado Center for Astrodynamics Research The University of Colorado 1 STATISTICAL ORBIT DETERMINATION Statistical Interpretation of Least Squares ASEN.
Ch3: Model Building through Regression
Component decomposition of IASI measurements
Biointelligence Laboratory, Seoul National University
Presentation transcript:

A physical basis of GOES SST retrieval Prabhat Koner Andy Harris Jon Mittaz Eileen Maturi

Summary One year match up database ( Buoy & Satellite) has been analyzed to understand the retrieval problem of satellite measurement. Various forms of errors and ambiguities for SST retrieval will be discuused. 110,000 night only data (GOES12 & Buoy) has been considered from this database to compare results with different choices of solution processes:  RGR -> Regression  OEM -> (K T S e -1 K+S a -1 ) -1 K T S e -1 ΔY  ML -> (K T S e -1 K) -1 K T S e -1 ΔY  RTLS -> (K T K+λR)K T ΔY (Regularized Total Least square) 3.9 & 10.7 μm -> 2 channel 3.9, 10.7 & 13.3 μm -> 3 channel Two variables of SST & TCWV

Nomenclature X -> state space parameters Y -> Measurements ΔY -> Residual (Measurements – forward model output) ΔX -> Update of state space variables δ Y -> Measurement noise δ X -> Retrieval error S a -> a priori/ background covariance S e -> measurement error covariance K -> Jacobian (derivative of forward model) κ(K) -> condition number of Jacobian (highest singular value/lowest singular value) T -> Brightness temperature in satellite output BT -> Calculated Brightness temperature SSTg-> Sea Surface Temperature First Guess SSTb -> Sea Surface Temperature from Buoy measurement SSTrgr -> Sea Surface Temperature retrieval using regression SD -> Standard Deviation RMSE -> Root Mean Square Error.

SSTrgr=C 0 +∑C i T i ; C=a+b {sec(θ)-1}+… Coefficients are derived from in situ buoy data or L4 bulk SST. Validate with Buoy data Bulk SST from Satellite measurement? Historical Regressed based SST retrieval Alternately we can use radiative transfer physics inverse model

Statistical (OEM) Deterministic (TLS) Data & Measurement both uncertain It can only estimate a posteriori probability density (parameters x’: “Best Guess”) by calculating Maximum likelihood P(x|y). Measurement only uncertain Jacobian error is condidered in cost funtion minimization. Retrieval in pixel level Physical Retrieval A posteriori observation A priori First we will see Pros-con of OEM

Global SST distributions match quite well, but… …large differences between 1 st guess and buoy SST are real Shortcomings of OEM A-priori based cloud screening algorithm (CSA) in place to constrain in image data

Paradox of OEM output OEM =(K T S e -1 K+S a -1 ) -1 K T S e -1 ΔY Information comes from measurement and a priori covariance. COV= (√S a -1 ) -1 ΔY S a = By accident the choice S a, COV retrieval is just adding residual of 3.9 μm channel with FG. Covariance has no physical meaning Add measurement increase noise into retrievals *TCWV 2 To further investigate this issue, we calculate information content

H=-0.5 ln(I-AVK) Two measurement cannot produce more than 2 pieces of information. Big Question? OEM may be valid for linear problem, not applicable for inherently nonlinear RTE.

Condition number of Jacobian The condition number of jacobian of most real life problem is high. Yields δ x <= κ (K) δy K=randn(2); (κ(K) = 118) x=randn(2,1); [ ] y=K*x For ii=1:100 error(:,ii)=0.01*randn(2,1); x rtv (:,ii)=K -1 (y+error(:,ii)); End Apart from model physics and measurement errors, the error due to κ(K) plays a role. (K T S e -1 K+S a -1 ) -1 K T S e -1 ΔY Remedies: Reducing the condition number of inverted matrix Regularization, constraints, scaling, weighting etc. We use RTLS method. Uniquely solved in simulation study. Difficulty solve using satellite data, drive to further investigation

Online monitor ECMWF It is not an instrument calibration problem or bias Fast Forward Model Error Need improvement of fast forward model.

Simulated Retrieval δ x<= κ (K) δy

Quality Flagging Algorithm K=U∑V T -> Singular vector decomposition V T ∑ 1 -1 U ΔY -> Principle Component solution V T ∑ n -1 U ΔY ->Lowest singular value solution Lowest singular value solution increases error in retrieval where measurement noise is high. Difference of two solutions is able identified bad retrieval.

Comparative results Same data and model, results are different due to choice of solution methods Results are based on single iteration. Second iteration may further improve the results for physical retrieval.

Conclusions Radiative transfer can be successful used in retrieval of SST from satellite data if we pursue rigorous physics (accurate RTM) and mathematics There is an ambiguity in the cost function minimization for nonlinear problems, i.e. (Y δ -KX) T S e -1 (Y δ -KX) + (X-X a ) T S a -1 (X-X a ) does not apply if Y ≠ KX

Few Definitions Model: a simplified description of how the ‘real world’ process behaves Forward Model : the set of rules (mathematical functions) that define the behaviour of the process (e.g. a set equation) Forward model of remote sensing  Well understood but mathematically complex function  Analytical derivative is almost impossible.  Stable: On some appropriate scale a small change in the input produces a small change in the output Inverse Model: quantities within the model structure that need to be quantified from observation data Inverse model of remote sensing is ill-posed  Is the solution we find unique?  Observational, numerical and model errors often cause the inverse problem to be unstable: a small change in the input produces a large change in the output

Forward Model Absorption term Emission term Intensity of the background source “Transmittance” between 0 and L Spectral intensity observed at L “Transmittance” between l and L “Optical depth” “Absorption coefficient”

Forward Model Spectral intensity observed at L “Absorption coefficient” “number density” “Cross section” “Volume Mixing Ratio” “Line shape”“Line strength” GENSPEC: line shape: Voigt: line Strength: HITRAN 2004