EDUCE: WP 6 & 7 D evelopment of an Algorithm to Detect Spikes and Distortions in UV Spectra Charikleia Meleti, Alkis Bais Aristotle University of Thessaloniki.

Slides:



Advertisements
Similar presentations
QA/QC of Solar spectral UV-measurements: Checking for wavelength shifts and spectral shape deviations with focus on error flagging Harry Slaper, PN den.
Advertisements

QA flagging of clouds and atmospheric anomalies Fast UV simulation tool - FastRT Ola Engelsen Norwegian Institute for Air Research (NILU)
Lesson 10: Linear Regression and Correlation
EDUCE: WP1 Climatological UV Maps A. Bais, K. Tourpali, A. Kazantzidis Aristotle University of Thessaloniki Laboratory of Atmospheric Physics.
Workshop: First look, Calibrations & RV standard IAP
Microspectrophotometry Validation. Reasons for Changing Instruments Reduced reliability. Limited efficiency. Limited availability and cost of replacement.
Presentation on OMPS Nadir Mapper Wavelength Shift Adjustment for Earth-view Measurements.
Starter What are the dependent and independent variables from our experiment with photosynthesis? What is the next step now that we have collected the.
La Parguera Hyperspectral Image size (250x239x118) using Hyperion sensor. INTEREST POINTS FOR HYPERSPECTRAL IMAGES Amit Mukherjee 1, Badrinath Roysam 1,
Institut für Umweltphysik/Fernerkundung Physik/Elektrotechnik Fachbereich 1 Retrieval of SCIAMACHY limb measurements: First Results A. Rozanov, V. Rozanov,
Correlation and Regression Analysis
Chapter 13 Analyzing Quantitative data. LEVELS OF MEASUREMENT Nominal Measurement Ordinal Measurement Interval Measurement Ratio Measurement.
Chapter 14 Analyzing Quantitative Data. LEVELS OF MEASUREMENT Nominal Measurement Nominal Measurement Ordinal Measurement Ordinal Measurement Interval.
8-1 Quality Improvement and Statistics Definitions of Quality Quality means fitness for use - quality of design - quality of conformance Quality is.
1 Information-Theoretic Mass Spectral Library Search Arvind Visvanathan CSCE 990 Seminar in Multi-Dimensional Chromatography Systems, Informatics, and.
Analysis of D Band Cloud Flag Jane Hurley Anu Dudhia Graham Ewen University of Oxford.
1 Chapter 17: Introduction to Regression. 2 Introduction to Linear Regression The Pearson correlation measures the degree to which a set of data points.
Control Charts for Attributes
Two and a half problems in homogenization of climate series concluding remarks to Daily Stew Ralf Lindau.
Cophasing activities at Onera
STATISTICS: BASICS Aswath Damodaran 1. 2 The role of statistics Aswath Damodaran 2  When you are given lots of data, and especially when that data is.
Statistics 1 The Basics Sherril M. Stone, Ph.D. Department of Family Medicine OSU-College of Osteopathic Medicine.
Blue: Histogram of normalised deviation from “true” value; Red: Gaussian fit to histogram Presented at ESA Hyperspectral Workshop 2010, March 16-19, Frascati,
LECTURER PROF.Dr. DEMIR BAYKA AUTOMOTIVE ENGINEERING LABORATORY I.
Here it is…. …Your Super Duper Faster than Ever Pooper Scooper Review of Statistics and Data Analysis!
Automated CBC Parameters
First impressions on Brewer processing methods survey Tomi Karppinen COST WG1/WG2 meeting Manchester, 27. September 2013.
Chapter 4: Test administration. z scores Standard score expressed in terms of standard deviation units which indicates distance raw score is from mean.
® Kick off meeting. February 17th, 2011 QUAlity aware VIsualisation for the Global Earth Observation system of systems GEOVIQUA workshop February, the.
LMS Stability, Data Correction and the Radiation Accident within the PrimEx Experiment by LaRay J. Benton M.S. Nuclear Physics May 2006 Graduate North.
Jeroen Pannekoek - Statistics Netherlands Work Session on Statistical Data Editing Oslo, Norway, 24 September 2012 Topic (I) Selective and macro editing.
Estimating Water Optical Properties, Water Depth and Bottom Albedo Using High Resolution Satellite Imagery for Coastal Habitat Mapping S. C. Liew #, P.
AMFIC second progress meeting MariLiza Koukouli & Dimitris Balis Laboratory of Atmospheric Physics Aristotle University of Thessaloniki.
Quality Assurance How do you know your results are correct? How confident are you?
Regression MBA/510 Week 5. Objectives Describe the use of correlation in making business decisions Apply linear regression and correlation analysis. Interpret.
PHYS 205 Analyzing Starlight PHYS 205 Apparent brightness 2 nd century BC  Hipparchus devised 6 categories of brightness. In 1856 Pogson discovered.
Exposing Digital Forgeries in Color Filter Array Interpolated Images By Alin C. Popescu and Hany Farid Presenting - Anat Kaspi.
Chimiometrie 2009 Proposed model for Challenge2009 Patrícia Valderrama
Tom.h.wilson Department of Geology and Geography West Virginia University Morgantown, WV.
OMI ST meeting KNMI Validation of OMI total ozone using ground-based Brewer and Dobson observations D. Balis 1, E. Brinksma 2, M. Kroon 2,V.
Validation Defination Establishing documentary evidence which provides a high degree of assurance that specification process will consistently produce.
GEOGG121: Methods Inversion I: linear approaches Dr. Mathias (Mat) Disney UCL Geography Office: 113, Pearson Building Tel:
Atmospheric phase correction at the Plateau de Bure interferometer IRAM interferometry school 2006 Aris Karastergiou.
Spectrum Sensing In Cognitive Radio Networks
1 UNIT 13: DATA ANALYSIS. 2 A. Editing, Coding and Computer Entry Editing in field i.e after completion of each interview/questionnaire. Editing again.
The Critical Importance of Data Reduction Calibrations In the Interpretability of S-type Asteroid Spectra Michael J. Gaffey Space Studies Department University.
Blind Inverse Gamma Correction (Hany Farid, IEEE Trans. Signal Processing, vol. 10 no. 10, October 2001) An article review Merav Kass January 2003.
NPI-NILU tasks (Ola Engelsen) •Further development of fast simulation tool for UV radiation (Fastrt) •Applications of fast simulation tool for quality.
Spectral subtraction algorithm and optimize Wanfeng Zou 7/3/2014.
Digital Imaging and Remote Sensing Laboratory thermal infrared data 1 Processing of TIMS data to emissivity spectra TIMS bands for this analysis 8.407,
A. Ealet Berkeley, december Spectrograph calibration Determination of specifications Calibration strategy Note in
Aerosol extinction coefficient (Raman method)
Fourth TEMPO Science Team Meeting
Design and Data Analysis in Psychology II
Phisymmetry in 2011 Phisymmetry was used:
PATTERN COMPARISON TECHNIQUES
Correlational Studies
INTRODUCING A cloud detector USING global horizontal irradiances in UV and PAR in Thessaloniki, Greece. Zempila M.M.1,2, Taylor M. 2,*, Fountoulakis I.2,
Absolute calibration of sky radiances, colour indices and O4 DSCDs obtained from MAX-DOAS measurements T. Wagner1, S. Beirle1, S. Dörner1, M. Penning de.
Week 14 Chapter 16 – Partial Correlation and Multiple Regression and Correlation.
Mina Kang1, Myoung-Hwan Ahn1, Quintus Kleipool2 and Pepijn Veefkind2
Wavelength Anomalies in UV-Vis Spectrophometry
Multi-decadal trend analysis of total columnar ozone over Thessaloniki
Component decomposition of IASI measurements
Moving toward inter-calibration using the Moon as a transfer
Chapter 2 Ideal Sampling and Nyquist Theorem
Moving toward inter-calibration using the Moon as a transfer
TRIPLET MEASUREMENTS OF DIRECTIONAL WAVE SPECTRA
Drift Chamber Monitoring
NOISE FILTER AND PC FILTERING
Presentation transcript:

EDUCE: WP 6 & 7 D evelopment of an Algorithm to Detect Spikes and Distortions in UV Spectra Charikleia Meleti, Alkis Bais Aristotle University of Thessaloniki Laboratory of Atmospheric Physics

Goals Independent of other measurements Expose anomalies above statistical noise Run automatically –Producing flags –Correcting spectra Applicable to cloud-induced distortions

Description of Procedure Construction of a Reference spectrum α(λ, i): statistical coefficients to derive E(λ) from its neighboring wavelengths (SZA dependent) w(i-λ): weights inversely proportional to the difference from the central wavelength

Description of Procedure Construction of a Reference spectrum α(λ, i): statistical coefficients to derive E(λ) from its neighboring wavelengths (SZA dependent) w(i-λ): weights inversely proportional to the difference from the central wavelength

Description of Procedure Construction of a Reference spectrum α(λ, i): statistical coefficients to derive E(λ) from its neighboring wavelengths (SZA dependent) w(i-λ): weights inversely proportional to the difference from the central wavelength

Detection of Spikes Compute the ratio between E(λ) and E R (λ) Ratios higher than 1.5 indicate the presence of a spike

Flagging Spectral Distortions Compute the correlation coefficient (r 2 ) between E(λ) and E R (λ) –Accepted spectra  r 2 > 0.99 –Suspicious spectra  0.89 < r 2 < 0.99 Small spikes Distortion by clouds –Highly distorted spectra  r 2 < 0.89 Wavelength shifts Spikes Noise Zeros

Concluding Remarks All spectra flagged “accepted” were actually correct (by visual inspection) Small uncertainties still exist at low wavelengths The method does not depend on absolute calibration so it can be applied to raw data