Comparison of Gap Interpolation Methodologies for Water Level Time Series Using Perl/PDL By Aimee Mostella, Alexey Sadovski, Scott Duff, Patrick Michaud,

Slides:



Advertisements
Similar presentations
Decomposition Method.
Advertisements

Ekman Transport and Tidal Variations in the Gulf of Maine Norman J. Shippee Judd Gregg Meteorology Institute at Plymouth State University (
NOAA’s CENTER for OPERATIONAL OCEANOGRAPHIC PRODUCTS and SERVICES Improvements to the CO-OPS Storm QuickLook Product for Real-Time Storm Surge Monitoring.
FTP Biostatistics II Model parameter estimations: Confronting models with measurements.
Regression Analysis Once a linear relationship is defined, the independent variable can be used to forecast the dependent variable. Y ^ = bo + bX bo is.
Neural Network Based Approach for Short-Term Load Forecasting
Analyzing and Forecasting Time Series Data
Introduction This project deals with conversion of Vector based Probable Maximum Precipitation (PMP) data into Raster based PMP data using different interpolation.
Regression line – Fitting a line to data If the scatter plot shows a clear linear pattern: a straight line through the points can describe the overall.
Evaluating Hypotheses
Data Mining CS 341, Spring 2007 Lecture 4: Data Mining Techniques (I)
1 Econometric Load Forecasting Peak and Energy Forecast 06/14/2005 Econometric Load Forecasting Peak and Energy Forecast 06/14/2005.
Short Term Load Forecasting with Expert Fuzzy-Logic System
Time Series and Forecasting
1 Doing Statistics for Business Doing Statistics for Business Data, Inference, and Decision Making Marilyn K. Pelosi Theresa M. Sandifer Chapter 11 Regression.
Statistical Methods For Engineers ChE 477 (UO Lab) Larry Baxter & Stan Harding Brigham Young University.
Multi-Style Language Model for Web Scale Information Retrieval Kuansan Wang, Xiaolong Li and Jianfeng Gao SIGIR 2010 Min-Hsuan Lai Department of Computer.
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Chapter 12 Describing Data.
© 2006 Prentice Hall, Inc.4 – 1 Forcasting © 2006 Prentice Hall, Inc. Heizer/Render Principles of Operations Management, 6e Operations Management, 8e.
1 Reading Report 9 Yin Chen 29 Mar 2004 Reference: Multivariate Resource Performance Forecasting in the Network Weather Service, Martin Swany and Rich.
Chapter 13 Statistics © 2008 Pearson Addison-Wesley. All rights reserved.
A methodology for defining homogeneous water bodies in transitional and coastal waters S.B. Bricker Gulf of Mexico Alliance Governors’
Model of Prediction Error in Chaotic and Web Driven Business Environment Franjo Jović*, Alan Jović ** * Faculty of Electrical Engineering, University of.
A Statistical Comparison of Weather Stations in Carberry, Manitoba, Canada.
Lower Cape Fear River Estuary Model Progress Report Jim Bowen, UNC Charlotte October 11, 2007 Charlotte, NC (via Centra)
Why Is It There? Getting Started with Geographic Information Systems Chapter 6.
Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 MAP (Maximum A Posteriori) x is reduced state vector [SST(x), TCWV(w)]
Linear Regression Least Squares Method: the Meaning of r 2.
T EXAS C OASTAL O CEAN O BSERVATION N ETWORK D IVISION OF N EARSHORE R ESEARCH-DR.GARY JEFFRESS, DIRECTOR T EXAS A & M U NIVERSITY- C ORPUS C HRISTI HISTORY.
TCOON Station Review Dr. Gary Jeffress James Rizzo.
Safety management software for state and local highway agencies: –Improves identification and programming of site- specific highway safety improvements.
CONRAD BLUCHER INSTITUTE ACTIVITIES SUPPORTING TEXAS PORTS AND WATERWAYS OPERATIONS Two Inter-related Services to the Port Community: 1. The Texas Coastal.
Curve Fitting and Regression EEE 244. Descriptive Statistics in MATLAB MATLAB has several built-in commands to compute and display descriptive statistics.
Resolution (degree) and RMSE (cm) Resolution (degree) and RMSE (cm)
Surface Current Mapping in the Lower Chesapeake Bay INTRODUCTION High frequency RADAR antennas are used to observe the surface circulation patterns in.
MBA.782.ForecastingCAJ Demand Management Qualitative Methods of Forecasting Quantitative Methods of Forecasting Causal Relationship Forecasting Focus.
Integration of Statistics and Harmonic Analysis to Predict Water Levels in Estuaries and Shallow Waters of the Gulf of Mexico Texas A&M University - Corpus.
Time Series Analysis and Forecasting
` DIVISION OF NEARSHORE RESEARCH TEXAS A&M UNIVERSITY-CORPUS CHRISTI Key Features:  Located on oil production platforms 3 and 15 nautical miles from Port.
Statistical Tools for Solar Resource Forecasting Vivek Vijay IIT Jodhpur Date: 16/12/2013.
Neural Network Modeling of Spring Levels Linked to a Karst Aquifer: Case Study of the Comal Springs Dawn Lalmansingh, Philippe Tissot, Richard Hay Texas.
Statistical and Neural Network Analysis to Predict Water Levels in Estuaries and Shallow Waters of the Gulf of Mexico Texas A&M University - Corpus Christi.
Your Logo Here Conrad Blucher Institute for Surveying and Science.
Division of Nearshore Research Texas Coastal Ocean Observation Network Dr. Gary Jeffress Mr. James Rizzo October 2003.
Division of Nearshore Research Texas Coastal Ocean Observation Network Dr. Gary Jeffress Dr. Patrick Michaud Jessica Tishmack May 7, 2003.
Division of Nearshore Research TCOON Tides and Tide Forecasting Dr. Patrick Michaud October 27, 2003.
1 Introduction to Data Mining C hapter 1. 2 Chapter 1 Outline Chapter 1 Outline – Background –Information is Power –Knowledge is Power –Data Mining.
Validation of Coastwatch Ocean Color products S. Ramachandran, R. Sinha ( SP Systems NOAA/NESDIS) Kent Hughes and C. W. Brown ( NOAA/NESDIS/ORA,
Linear Prediction Correlation can be used to make predictions – Values on X can be used to predict values on Y – Stronger relationships between X and Y.
Hybrid Load Forecasting Method With Analysis of Temperature Sensitivities Authors: Kyung-Bin Song, Seong-Kwan Ha, Jung-Wook Park, Dong-Jin Kweon, Kyu-Ho.
A methodology for defining homogeneous water bodies in transitional and coastal waters S.B. Bricker A. Newton, J.G. Ferreira, A.M. Nobre, M.T. Simas, M.C.
CO-OPS Expands Meteorological Sensor Network and Quality Control Kathleen Egan, Tom Landon NOAA/NOS/Center for Operational Oceanographic Products and Services.
Validation of Satellite-derived Clear-sky Atmospheric Temperature Inversions in the Arctic Yinghui Liu 1, Jeffrey R. Key 2, Axel Schweiger 3, Jennifer.
1 1 Chapter 6 Forecasting n Quantitative Approaches to Forecasting n The Components of a Time Series n Measures of Forecast Accuracy n Using Smoothing.
ASSESSING SEARCH TERM STRENGTH IN SPOKEN TERM DETECTION Amir Harati and Joseph Picone Institute for Signal and Information Processing, Temple University.
Extra-Tropical Storm Surge (ETSS 1.5) Pre-Implementation Briefing College Park, MD October 8, 2014 Arthur Taylor, Huiqing Liu and Ryan Schuster MDL/NWS/NOAA.
Multi-Area Load Forecasting for System with Large Geographical Area S. Fan, K. Methaprayoon, W. J. Lee Industrial and Commercial Power Systems Technical.
Surface Current Mapping in the Lower Chesapeake Bay INTRODUCTION High frequency RADAR antennas are used to observe the surface circulation patterns in.
Regression Models for Predictions of Water Levels in the Shallow Waters of the Gulf of Mexico Kelly Torres Texas A&M University - Corpus Christi.
Quality Control of Soil Moisture and Temperature For US Climate Reference Network Basic Methodology February 2009 William Collins USCRN.
Review of Mathematical and Statistical Models to Predict (Extrapolate) Water Levels and to Fill Gaps (Interpolate) of the TCOON Data Texas A&M University.
of Temperature in the San Francisco Bay Area
Regression Analysis Part D Model Building
of Temperature in the San Francisco Bay Area
Texas Coastal Ocean Observation Network
Orientation of Oscillating Water Column Wave Energy Converters Haylee Reyna   Texas A&M University Kingsville 600 East Santa Gertrudis Kingsville, TX,
Texas Coastal Ocean Observation Network
Division of Nearshore Research TCOON
ENM 310 Design of Experiments and Regression Analysis Chapter 3
Samsung Austin Semiconductor
Presentation transcript:

Comparison of Gap Interpolation Methodologies for Water Level Time Series Using Perl/PDL By Aimee Mostella, Alexey Sadovski, Scott Duff, Patrick Michaud, Philippe Tissot, Carl Steidley

Necessity of Interpolation of Nearshore Research (TAMUCC-DNR) and Texas Coastal Ocean Observation Network (TCOON) collect, archive and analyze various types of time series including water level time series Gaps in time series limit the types of methods which may be used to study and further our understanding of water level patterns Texas A&M University- Corpus Christi Division

lrwlfill TAMUCC-DNR and TCOON have developed lrwlfill, a Perl script designed to interpolate gaps in water level time series Ease and power with Perl Efficient computation and data storage with the Perl Data Language Module (PDL)

Basic Algorithm Retrieve data according to user provided parameters Search data for missing values Perform linear regression to obtain two sets of coefficients Calculate missing values with coefficients Combine two sets into one Insert new values in place of missing data

Retrieving the Data Retrieve water level values corresponding to these parameters: Time frame  Retrieve from one month before to one month after time provided by the user Station identifier Number of coefficients Method of fitting the resulting data

Water Level RWL = AWL - HWL RWL => Residual Water Level AWL => Actual Water Level HWL => Harmonic Water Level Record the location of gaps in the AWL Record the difference between AWL and HWL as the RWL

Linear Regression For each gap in the data Perform forward and backward linear regression (FLR & BLR, respectively) using hourly data to obtain coefficients Calculate the missing data points with these coefficients

Methods of Combination Combine the results of FLR & BLR using one of the following methods: Convex linear combination  Based on weighted proportion Convex trigonometric combination  Based on trigonometrically weighted proportion Combination at intersection  Fuse together at the intersection

Testing Conditions Full sets of existing data Two station locations: Embayment station Open coast station Three typical weather conditions Periods of calm weather Periods of frequent frontal passages Extreme weather Varying gap sizes and numbers of coefficients

Testing Standards United States National Ocean Service (NOS) standards: Root mean square error (RMSE) Central Frequency (CF) Other statistical measures: Standard deviation (SD) Maximum error (ME)

Results Analyzed Timing and accuracy for varying numbers of coefficients Best procedure to fit FLR and BLR Timing and accuracy as gap size increased Precision as weather conditions changed Accuracy for embayment stations in contrast to open coast stations

Results

Effect of Number of Coefficients Timing was negligible Accuracy peaked and then declined depending upon weather conditions RMSE was used to determine the optimal number of coefficients

Coefficients Figure 1 displays our chosen coefficients according to weather condition Although these coefficients are optimal, the accuracy of interpolation still declines as weather becomes more extreme

Best Fit Convex linear combination demonstrated the highest level of accuracy in fitting the data as shown in Figure 2.

Effects of Gap Size and Weather Conditions Generally, timing was negligible Precision decreases steadily as gap size increases as shown in figure 3 Precision also decreases as the weather becomes more extreme

Embayment vs. Open Coast Overall, data from embayment stations produced better results than data from stations along the open coast (figure 4) The flow of water into the bay dampens the change in water level

Future Direction Convert lrwlfill to a real-time web based implementation Experiment with the amount and orientation of previous data used to calculate coefficients Study the results of using more frequent time series values in the linear regression process as weather becomes more erratic to make up for rapid changes in weather

Bibliography High Performance Computing Development Center, Texas A&M University-Corpus Christi. Mostella, Aimee; Duff, Scott; Michaud, Patrick R. (2001) HARMAN and HARMPRED: Web-based Software to Analyze Tidal Constituents and Tidal Forecasts for the Texas Coast NOAA, 1994: NOAA Technical Memorandum NOS OES 8. National Oceanic and Atmospheric Administration, Silver Spring, Marilyn. Sadovski, Alexey L.; Michaud, Patrick R.; Steidley, Carl; Tishmack, Jessica; Torres, Kelly & Mostella, Aimee L. (2003). Integration of Statistics and Harmonic Analysis to Predict Water Levels in Estuaries and Shallow Waters of the Gulf of Mexico. Presentation at the MATA International Conference (Cancun, Mexico), April Sadovski, Alexey L.; Tissot, Philippe; Michaud, Patrick; Steidley, Carl (2004) Statistical and Neural Network Modeling and Predictions of Tides in the Shallow Waters of the Gulf of Mexico. In “WSEAS Transactions on Systems”, Issue 2, vol. 2, WSEAS Press, pp