Venugopal, Basu, and Foufoula-Georgiou, 2005: New metric for comparing precipitation patterns… Verification methods reading group April 4, 2008 D. Ahijevych.

Slides:

Advertisements

Similar presentations

Prediction of Crime/Terrorist Event Locations National Defense and Homeland Security: Anomaly Detection Francisco Vera, SAMSI.

Advertisements

CORRELATION. Overview of Correlation u What is a Correlation? u Correlation Coefficients u Coefficient of Determination u Test for Significance u Correlation.

6th WMO tutorial Verification Martin GöberContinuous 1 Good afternoon! नमस्कार नमस्कार Guten Tag! Buenos dias! до́брый день! до́брыйдень Qwertzuiop asdfghjkl!

Method for Object-based Diagnostic Evaluation (MODE) Fake forecasts.

CORRELATION. Overview of Correlation u What is a Correlation? u Correlation Coefficients u Coefficient of Determination u Test for Significance u Correlation.

Verification Methods for High Resolution Model Forecasts Barbara Brown NCAR, Boulder, Colorado Collaborators: Randy Bullock, John Halley.

NWP Verification with Shape- matching Algorithms: Hydrologic Applications and Extension to Ensembles Barbara Brown 1, Edward Tollerud 2, Tara Jensen 1,

SocalBSI 2008: Clustering Microarray Datasets Sagar Damle, Ph.D. Candidate, Caltech  Distance Metrics: Measuring similarity using the Euclidean and Correlation.

Adaptive Rao-Blackwellized Particle Filter and It’s Evaluation for Tracking in Surveillance Xinyu Xu and Baoxin Li, Senior Member, IEEE.

ASSIMILATION of RADAR DATA at CONVECTIVE SCALES with the EnKF: PERFECT-MODEL EXPERIMENTS USING WRF / DART Altuğ Aksoy National Center for Atmospheric Research.

Chamfer Matching & Hausdorff Distance Presented by Ankur Datta Slides Courtesy Mark Bouts Arasanathan Thayananthan.

Today Today: More Chapter 3 Reading: –Please read Chapter 3 –Suggested Problems: 3.2, 3.9, 3.12, 3.20, 3.23, 3.24, 3R5, 3R9.

z-Scores What is a z-Score? How Are z-Scores Useful? Distributions of z-Scores Standard Normal Curve.

The Expanded UW SREF System and Statistical Inference STAT 592 Presentation Eric Grimit 1. Description of the Expanded UW SREF System (How is this thing.

Introduction to Longitudinal Phase Space Tomography Duncan Scott.

Great Basin Verification Task 2008 Increased Variability Review of 2008 April through July Period Forecast for 4 Selected Basins Determine what verification.

Simple Linear Regression

MSE 600 Descriptive Statistics Chapter 10 in 6 th Edition (may be another chapter in 7 th edition)

Kalman filtering techniques for parameter estimation Jared Barber Department of Mathematics, University of Pittsburgh Work with Ivan Yotov and Mark Tronzo.

Climate Forecasting Unit Second Ph’d training talk Prediction of climate extreme events at seasonal and decadal time scale Aida Pintó Biescas.

(a.k.a: The statistical bare minimum I should take along from STAT 101)

ESA DA Projects Progress Meeting 2University of Reading Advanced Data Assimilation Methods WP2.1 Perform (ensemble) experiments to quantify model errors.

Improvements in Deterministic and Probabilistic Tropical Cyclone Surface Wind Predictions Joint Hurricane Testbed Project Status Report Mark DeMaria NOAA/NESDIS/ORA,

Spatial Forecast Verification Methods Intercomparison Project: Forecast Quality Index Sukanta Basu Texas Tech University & Efi Foufoula-Georgiou University.

Basic Statistics Correlation Var Relationships Associations.

B AD 6243: Applied Univariate Statistics Correlation Professor Laku Chidambaram Price College of Business University of Oklahoma.

© Crown copyright Met Office Preliminary results using the Fractions Skill Score: SP2005 and fake cases Marion Mittermaier and Nigel Roberts.

Ebert-McBride Technique (Contiguous Rain Areas) Ebert and McBride (2000: Verification of precipitation in weather systems: determination of systematic.

Intro: “BASIC” STATS CPSY 501 Advanced stats requires successful completion of a first course in psych stats (a grade of C+ or above) as a prerequisite.

Summary Five numbers summary, percentiles, mean Box plot, modified box plot Robust statistic – mean, median, trimmed mean outlier Measures of variability.

1 Climate Test Bed Seminar Series 24 June 2009 Bias Correction & Forecast Skill of NCEP GFS Ensemble Week 1 & Week 2 Precipitation & Soil Moisture Forecasts.

Chapter 16 Data Analysis: Testing for Associations.

Convection Alison Stirling 21 st September Contents 1.Verification measures for forecasts (Michael Whitall) 2.Representing convection-wave interactions.

Refinement and Evaluation of Automated High-Resolution Ensemble-Based Hazard Detection Guidance Tools for Transition to NWS Operations Kick off JNTP project.

Traditional Verification Scores Fake forecasts  5 geometric  7 perturbed subjective evaluation  expert scores from last year’s workshop  9 cases x.

Correlation. Correlation is a measure of the strength of the relation between two or more variables. Any correlation coefficient has two parts – Valence:

Local Predictability of the Performance of an Ensemble Forecast System Liz Satterfield and Istvan Szunyogh Texas A&M University, College Station, TX Third.

Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory.

1 ARPA-SIM, Bologna, Italy and CIMA, Savona, Italy. Improving the radar data mosaicking procedure by means of a quality descriptor Fornasiero, A., Alberoni,

Diagnostic Evaluation of Mesoscale Models Chris Davis, Barbara Brown, Randy Bullock and Daran Rife NCAR Boulder, Colorado, USA.

Spatial Forecast Methods Inter-Comparison Project -- ICP Spring 2008 Workshop NCAR Foothills Laboratory Boulder, Colorado.

Fourier and Wavelet Transformations Michael J. Watts

Page 1© Crown copyright 2004 The use of an intensity-scale technique for assessing operational mesoscale precipitation forecasts Marion Mittermaier and.

The Normal Distribution: Comparing Apples and Oranges.

On the Challenges of Identifying the “Best” Ensemble Member in Operational Forecasting David Bright NOAA/Storm Prediction Center Paul Nutter CIMMS/Univ.

Nonlinear regression Review of Linear Regression.

13 / 10 / 2006 Uncertainty and regional air quality model diversity: what do we learn from model ensembles? Robert Vautard Laboratoire des Sciences du.

A Random Subgrouping Scheme for Ensemble Kalman Filters Yun Liu Dept. of Atmospheric and Oceanic Science, University of Maryland Atmospheric and oceanic.

NCAR, 15 April Fuzzy verification of fake cases Beth Ebert Center for Australian Weather and Climate Research Bureau of Meteorology.

DOWNSCALING GLOBAL MEDIUM RANGE METEOROLOGICAL PREDICTIONS FOR FLOOD PREDICTION Nathalie Voisin, Andy W. Wood, Dennis P. Lettenmaier University of Washington,

HMS 320 Understanding Statistics Part 2. Quantitative Data Numbers of something…. (nominal - categorical Importance of something (ordinal - rankings)

Overview of SPC Efforts in Objective Verification of Convection-Allowing Models and Ensembles Israel Jirak, Chris Melick, Patrick Marsh, Andy Dean and.

1 Application of MET for the Verification of the NWP Cloud and Precipitation Products using A-Train Satellite Observations Paul A. Kucera, Courtney Weeks,

GIS and Spatial Analysis1 Summary  Parametric Test  Interval/ratio data  Based on normal distribution  Difference in Variances  Differences are just.

Application of the CRA Method Application of the CRA Method William A. Gallus, Jr. Iowa State University Beth Ebert Center for Australian Weather and Climate.

Application of object-oriented verification techniques to ensemble precipitation forecasts William A. Gallus, Jr. Iowa State University June 5, 2009.

Correlation I have two variables, practically „equal“ (traditionally marked as X and Y) – I ask, if they are independent and if they are „correlated“,

Psych 706: stats II Class #4.

Intensity-scale verification technique

Systematic timing errors in km-scale NWP precipitation forecasts

NMME Update High Frequency Data Online – and Growing

Fourier and Wavelet Transformations

Correlation coefficient

Composite Method Results Artificial Cases April 2008

Numerical Weather Prediction Center (NWPC), Beijing, China

Verification of Tropical Cyclone Forecasts

@pbrucemaths Read the paper, annotate questions Check your answers!

Descriptive statistics for groups:

Ryan Kang, Wee Leng Tan, Thea Turkington, Raizan Rahmat

Presentation transcript:

Venugopal, Basu, and Foufoula-Georgiou, 2005: New metric for comparing precipitation patterns… Verification methods reading group April 4, 2008 D. Ahijevych

Forecast Quality Index useful for ensembles uses “surrogate fields” accounts for “close” forecasts One number

Outline Paper overview universal image quality index (UIQI) and modified UIQI components of forecast quality index (FQI) Geometric examples (from Sukanta and Efi) Perturbed “fake” examples (also from S and E) Cases from SPC Spring 2005 surrogates traditional skill scores expert rankings

Paper overview – forecast ensembles filter out similar members, and keep just enough to characterize the probability structure of forecast find “best” member and propagate it forward single measure (like RMSE and EqTh) but has important additional information

Paper overview - UIQI R1 and R2 are fields being compared 3 terms: covariance means standard deviations 3 properties: correlation brightness (bias) distortion (variability)

Paper overview – UIQI, Hausdorff UIQI entirely amplitude-based measure not efficient at telling difference between displaced patterns and amplitude error Distance-based measures Hausdorff distance

Paper Overview - Hausdorff A B h(A,B) forward distance

Paper Overview - Hausdorff A B h(B,A) backward distance

Paper Overview - Hausdorff A B h(B,A) backward distance h(A,B) forward distance

Paper Overview - Hausdorff A B H(A,B)

Paper Overview – partial Hausdorff A B h(A,B) ?

Paper overview - Hausdorff AB a1 a2 a3 b1 b2 h(A,B) forward distance

Paper overview - FQI

Paper Overview - surrogates

Paper overview – illustrative example RMSEEqThFQI 0 vs vs

Geometric examples CSI = 0 for first 4; CSI > 0 for the 5th

PHD 75 mod. UIQI mod. UIQI, including zero pixels

when I did 10 surrogates = 271 +/-27

Perturbed fake cases 1. 3 pts right, -5 pts up 2. 6 pts right, -10 pts up pts right, -20 pts up pts right, -40 pts up pts right, -80 pts up pts right, -20 pts up, times pts right, -20 pts up, minus 0.05”

Spring 2005 SPC cases surrogates pictures example of distribution of forward and backward Hausdorff distances comparison to traditional methods comparison to expert scores

100 surrogates – distribution of Hausdorff distance, solid/forward, dash/backward 75 th percentile Hausdorff distance (in grid spacing units) count

standard error surrogate mean PHD 75 mod. UIQI FQI: PHD 75

first case really bad; experts start out too generous? r = w/o 1 st case

expert scores vs grid stats grid stats agree: first case was bad

Pearson correlation coefficient and Spearman rank correlation coefficient

FQI Discussion application to ensembles adding to MET...