Oceanography 569 Oceanographic Data Analysis Laboratory Kathie Kelly Applied Physics Laboratory 515 Ben Hall IR Bldg class web site: faculty.washington.edu/kellyapl/classes/ocean569_.

Slides:



Advertisements
Similar presentations
Eigen Decomposition and Singular Value Decomposition
Advertisements

Eigen Decomposition and Singular Value Decomposition
Tensors and Component Analysis Musawir Ali. Tensor: Generalization of an n-dimensional array Vector: order-1 tensor Matrix: order-2 tensor Order-3 tensor.
Dimensionality Reduction PCA -- SVD
1er. Escuela Red ProTIC - Tandil, de Abril, 2006 Principal component analysis (PCA) is a technique that is useful for the compression and classification.
Statistical tools in Climatology René Garreaud
Lecture 19 Singular Value Decomposition
1cs542g-term High Dimensional Data  So far we’ve considered scalar data values f i (or interpolated/approximated each component of vector values.
An introduction to Principal Component Analysis (PCA)
Variance and covariance M contains the mean Sums of squares General additive models.
Notes on Principal Component Analysis Used in: Moore, S.K., N.J. Mantua, J.P. Kellogg and J.A. Newton, 2008: Local and large-scale climate forcing of Puget.
Principal Component Analysis
1 Chapter 3 Multiple Linear Regression Ray-Bing Chen Institute of Statistics National University of Kaohsiung.
Dimension reduction : PCA and Clustering Slides by Agnieszka Juncker and Chris Workman.
Face Recognition Using Eigenfaces
The Terms that You Have to Know! Basis, Linear independent, Orthogonal Column space, Row space, Rank Linear combination Linear transformation Inner product.
Microarray analysis Algorithms in Computational Biology Spring 2006 Written by Itai Sharon.
Lecture 20 SVD and Its Applications Shang-Hua Teng.
Ordinary least squares regression (OLS)
Exploring Microarray data Javier Cabrera. Outline 1.Exploratory Analysis Steps. 2.Microarray Data as Multivariate Data. 3.Dimension Reduction 4.Correlation.
Linear and generalised linear models
What is EOF analysis? EOF = Empirical Orthogonal Function Method of finding structures (or patterns) that explain maximum variance in (e.g.) 2D (space-time)
Linear and generalised linear models Purpose of linear models Least-squares solution for linear models Analysis of diagnostics Exponential family and generalised.
1cs542g-term Notes  Extra class next week (Oct 12, not this Friday)  To submit your assignment: me the URL of a page containing (links to)
Linear regression models in matrix terms. The regression function in matrix terms.
Separate multivariate observations
Summarized by Soo-Jin Kim
Chapter 2 Dimensionality Reduction. Linear Methods
Linear Algebra Review 1 CS479/679 Pattern Recognition Dr. George Bebis.
Next. A Big Thanks Again Prof. Jason Bohland Quantitative Neuroscience Laboratory Boston University.
Oceanography 569 Oceanographic Data Analysis Laboratory Kathie Kelly Applied Physics Laboratory 515 Ben Hall IR Bldg class web site: faculty.washington.edu/kellyapl/classes/ocean569_.
Chapter 15 Modeling of Data. Statistics of Data Mean (or average): Variance: Median: a value x j such that half of the data are bigger than it, and half.
Oceanography 569 Oceanographic Data Analysis Laboratory Kathie Kelly Applied Physics Laboratory 515 Ben Hall IR Bldg class web site: faculty.washington.edu/kellyapl/classes/ocean569_.
Oceanography 569 Oceanographic Data Analysis Laboratory Kathie Kelly Applied Physics Laboratory 515 Ben Hall IR Bldg class web site: faculty.washington.edu/kellyapl/classes/ocean569_.
Chapter 12 Multiple Linear Regression Doing it with more variables! More is better. Chapter 12A.
Statistics and Linear Algebra (the real thing). Vector A vector is a rectangular arrangement of number in several rows and one column. A vector is denoted.
Wolf-Gerrit Früh Christina Skittides With support from SgurrEnergy Preliminary assessment of wind climate fluctuations and use of Dynamical Systems Theory.
1 Dimension Reduction Examples: 1. DNA MICROARRAYS: Khan et al (2001): 4 types of small round blue cell tumors (SRBCT) Neuroblastoma (NB) Rhabdomyosarcoma.
Canonical Correlation Analysis and Related Techniques Simon Mason International Research Institute for Climate Prediction The Earth Institute of Columbia.
N– variate Gaussian. Some important characteristics: 1)The pdf of n jointly Gaussian R.V.’s is completely described by means, variances and covariances.
Paper review EOF: the medium is the message 報告人:沈茂霖 (Mao-Lin Shen) 2015/11/10 Seminar report.
Environment Canada Environnement Canada Effects of elevated CO 2 on modelled ENSO variability Bill Merryfield Canadian Centre for Climate Modelling and.
Introduction to Matrices and Matrix Approach to Simple Linear Regression.
Central limit theorem revisited
Introduction to Linear Algebra Mark Goldman Emily Mackevicius.
EIGENSYSTEMS, SVD, PCA Big Data Seminar, Dedi Gadot, December 14 th, 2014.
Extratropical Sensitivity to Tropical SST Prashant Sardeshmukh, Joe Barsugli, and Sang-Ik Shin Climate Diagnostics Center.
Matrix Factorization & Singular Value Decomposition Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.
Lecture Note 1 – Linear Algebra Shuaiqiang Wang Department of CS & IS University of Jyväskylä
3 “Products” of Principle Component Analysis
Chapter 61 Chapter 7 Review of Matrix Methods Including: Eigen Vectors, Eigen Values, Principle Components, Singular Value Decomposition.
Central limit theorem revisited Throw a dice twelve times- the distribution of values is not Gaussian Dice Value Number Of Occurrences.
Principal Components Analysis ( PCA)
Central limit theorem - go to web applet. Correlation maps vs. regression maps PNA is a time series of fluctuations in 500 mb heights PNA = 0.25 *
Unsupervised Learning II Feature Extraction
Unsupervised Learning II Feature Extraction
EMPIRICAL ORTHOGONAL FUNCTIONS 2 different modes SabrinaKrista Gisselle Lauren.
PRINCIPAL COMPONENT ANALYSIS(PCA) EOFs and Principle Components; Selection Rules LECTURE 8 Supplementary Readings: Wilks, chapters 9.
Principal Component Analysis (PCA)
EMPIRICAL ORTHOGONAL FUNCTIONS
Exploring Microarray data
9.3 Filtered delay embeddings
Lecture 8:Eigenfaces and Shared Features
Principal Component Analysis
Singular Value Decomposition North Atlantic SST
Dimension reduction : PCA and Clustering
Fig. 1 The temporal correlation coefficient (TCC) skill for one-month lead DJF prediction of 2m air temperature obtained from 13 coupled models and.
Principal Component Analysis
Principal Components What matters most?.
Marios Mattheakis and Pavlos Protopapas
Presentation transcript:

Oceanography 569 Oceanographic Data Analysis Laboratory Kathie Kelly Applied Physics Laboratory 515 Ben Hall IR Bldg class web site: faculty.washington.edu/kellyapl/classes/ocean569_ 2014/ Kathie Kelly Applied Physics Laboratory 515 Ben Hall IR Bldg class web site: faculty.washington.edu/kellyapl/classes/ocean569_ 2014/

Exercise 5: Taylor Diagram NCEP2 fluxes have too large amplitudes and slightly poorer correlation with observed OAFlux slightly underestimates the anomalies with similar correlation but smaller normalized error Skills are surprisingly small... it sets a high bar. OAFlux skill is much higher.

Empirical Orthogonal Functions (EOFs) or Principal Component Analysis F – typically a spatial pattern A - typically a time series, “principal component” (PC)

Linear Algebra: special matrices

EOFs by Singular Value Decomposition associate magnitudes S with V (time series or principal components) The squares of the diagonals of matrix S are the variance in each mode

EOFs: Practical Issues EOFs are a statistical tool in which the object is to understand random variables (mean?? periodic signals??) Data matrix can only be 2D (usually, space and time). But space has two dimensions and missing regions (land). What to do?

EOFs: Practical Issues Data matrix can only be 2D (usually, space and time). But space has two dimensions and missing regions (land). 1) Make 2D space into a vector by ordering locations: dat=reshape(data,nx*ny,nt); 2) Save only valid regions tmp=sum(dat,2);indv=find(~isnan(tmp)); D=dat(indv,:);

EOFs: Practical Issues What if the data are vectors? The easy way: enter both components (u,v) into single array D = (with M columns for locations x and 2 N rows for N times) When unpacking the modes U, remember that the top M rows correspond to the u-component and the bottom M rows correspond to the v-component.

EOFs of SSH using SVD EOF 1 mostly one sign EOF 2 “dipole”

EOFs of SST modes can be reversed: change sign of both F i and A i

EOFs of SST EOF 1 (only) reversed: similar to SSH PC 1 (only) reversed

Significance of EOF Modes Fraction of variance in random numbers * 9 significant modes Monte Carlo Simulation 1.find degrees of freedom in data matrix D (N*, M*) 2.generate matrix R(N*,M*) of random numbers 3.compute EOFs on R 4.compare variance in data EOFs with simulated

EOFs Comparing Fields How similar are these amplitudes? Could the longer record of SST be used to infer historical values of SSH?

EOFs by Singular Value Decomposition associate magnitudes S with V (time series or principal components) The squares of the diagonals of matrix S are the variance in each mode

Covariance Matrix Methods for EOFs for data with gaps Diagonals of covariance matrix are the variance But because there are gaps the amplitudes A are found by doing a least-squares fit of the data to the significant modes

Equivalence of EOF Methods For data without gaps the covariance method gives the same result as SVD on data matrix D

Covariance Method Bookkeeping device for covariance computation NN = N*N’covariance: D*D’/NN N number of valid entries at each location (x,y) D data matrix with 0 for invalid data (NaN)

Covariance Matrix Eigenvectors EOFs on wind vectors with gaps EOF computation: Remove seasonal cycle Covariance matrix Find PCs by least- squares fit of data to EOFs (U) Plot only times with at least 3 buoys operational

Different Results from EOFs