© John M. Abowd 2005, all rights reserved Modeling Integrated Data John M. Abowd April 2005.

Slides:

Advertisements

Similar presentations

Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.

Advertisements

Ordinary Least-Squares

A. The Basic Principle We consider the multivariate extension of multiple linear regression – modeling the relationship between m responses Y 1,…,Y m and.

GENERAL LINEAR MODELS: Estimation algorithms

Presenter: Raghu Ranganathan ECE / CMR Tennessee Technological University March 22th, 2011 Smart grid seminar series Yao Liu, Peng Ning, and Michael K.

COMP 116: Introduction to Scientific Programming Lecture 11: Linear Regression.

The General Linear Model. The Simple Linear Model Linear Regression.

Topic 3: Simple Linear Regression. Outline Simple linear regression model –Model parameters –Distribution of error terms Estimation of regression parameters.

Structural Equation Modeling

Modern iterative methods For basic iterative methods, converge linearly Modern iterative methods, converge faster –Krylov subspace method Steepest descent.

The loss function, the normal equation,

Visual Recognition Tutorial

The Simple Linear Regression Model: Specification and Estimation

Factor Analysis Purpose of Factor Analysis

© John M. Abowd 2007, all rights reserved Statistical Tools for Data Integration John M. Abowd April 2007.

Independent Component Analysis (ICA) and Factor Analysis (FA)

© John M. Abowd 2005, all rights reserved Statistical Tools for Data Integration John M. Abowd April 2005.

Ordinary least squares regression (OLS)

Linear and generalised linear models

Linear and generalised linear models

Linear and generalised linear models Purpose of linear models Least-squares solution for linear models Analysis of diagnostics Exponential family and generalised.

PETE 603 Lecture Session #29 Thursday, 7/29/ Iterative Solution Methods Older methods, such as PSOR, and LSOR require user supplied iteration.

Today Wrap up of probability Vectors, Matrices. Calculus

Principles of the Global Positioning System Lecture 11 Prof. Thomas Herring Room A;

By Mary Hudachek-Buswell. Overview Atmospheric Turbulence Blur.

INFO 7470/ILRLE 7400 Statistical Tools: Basic Integrated Data Models John M. Abowd and Lars Vilhuber April 12, 2011.

PATTERN RECOGNITION AND MACHINE LEARNING

CREST-ENSAE Mini-course Microeconometrics of Modeling Labor Markets Using Linked Employer-Employee Data John M. Abowd portions of today’s lecture are.

Linear Regression Andy Jacobson July 2006 Statistical Anecdotes: Do hospitals make you sick? Student’s story Etymology of “regression”

Gu Yuxian Wang Weinan Beijing National Day School.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Deterministic vs. Random Maximum A Posteriori Maximum Likelihood Minimum.

CS 782 – Machine Learning Lecture 4 Linear Models for Classification  Probabilistic generative models  Probabilistic discriminative models.

SUPA Advanced Data Analysis Course, Jan 6th – 7th 2009 Advanced Data Analysis for the Physical Sciences Dr Martin Hendry Dept of Physics and Astronomy.

ECE 8443 – Pattern Recognition LECTURE 10: HETEROSCEDASTIC LINEAR DISCRIMINANT ANALYSIS AND INDEPENDENT COMPONENT ANALYSIS Objectives: Generalization of.

Computer Animation Rick Parent Computer Animation Algorithms and Techniques Optimization & Constraints Add mention of global techiques Add mention of calculus.

Colorado Center for Astrodynamics Research The University of Colorado 1 STATISTICAL ORBIT DETERMINATION ASEN 5070 LECTURE 11 9/16,18/09.

Introduction to Matrices and Matrix Approach to Simple Linear Regression.

Multivariate Statistics Confirmatory Factor Analysis I W. M. van der Veld University of Amsterdam.

Generalised method of moments approach to testing the CAPM Nimesh Mistry Filipp Levin.

Diversity Loss in General Estimation of Distribution Algorithms J. L. Shapiro PPSN (Parallel Problem Solving From Nature) ’06 BISCuit 2 nd EDA Seminar.

INFO 7470/ECON 7400 Statistical Tools: Complex Data Models John M. Abowd and Lars Vilhuber April 22, 2013.

Colorado Center for Astrodynamics Research The University of Colorado 1 STATISTICAL ORBIT DETERMINATION Kalman Filter with Process Noise Gauss- Markov.

G Lecture 71 Revisiting Hierarchical Mixed Models A General Version of the Model Variance/Covariances of Two Kinds of Random Effects Parameter Estimation.

MathematicalMarketing Slide 5.1 OLS Chapter 5: Ordinary Least Square Regression We will be discussing  The Linear Regression Model  Estimation of the.

Computacion Inteligente Least-Square Methods for System Identification.

Estimation Econometría. ADE.. Estimation We assume we have a sample of size T of: – The dependent variable (y) – The explanatory variables (x 1,x 2, x.

Colorado Center for Astrodynamics Research The University of Colorado 1 STATISTICAL ORBIT DETERMINATION Statistical Interpretation of Least Squares ASEN.

STA302/1001 week 11 Regression Models - Introduction In regression models, two types of variables that are studied:  A dependent variable, Y, also called.

I. Statistical Methods for Genome-Enabled Prediction of Complex Traits OUTLINE THE CHALLENGES OF PREDICTING COMPLEX TRAITS ORDINARY LEAST SQUARES (OLS)

Probability Theory and Parameter Estimation I

Ch3: Model Building through Regression

Linear Mixed Models in JMP Pro

CH 5: Multivariate Methods

Probabilistic Models for Linear Regression

Matrices Definition: A matrix is a rectangular array of numbers or symbolic elements In many applications, the rows of a matrix will represent individuals.

G Lecture 6 Multilevel Notation; Level 1 and Level 2 Equations

Collaborative Filtering Matrix Factorization Approach

The Regression Model Suppose we wish to estimate the parameters of the following relationship: A common method is to choose parameters to minimise the.

'Linear Hierarchical Models'

Linear regression Fitting a straight line to observations.

Conjugate Gradient Method

OVERVIEW OF LINEAR MODELS

مدلسازي تجربي – تخمين پارامتر

Lecture 7. Learning (IV): Error correcting Learning and LMS Algorithm

OVERVIEW OF LINEAR MODELS

Nonlinear Fitting.

The loss function, the normal equation,

Solving Equations 3x+7 –7 13 –7 =.

Topic 11: Matrix Approach to Linear Regression

Computer Animation Algorithms and Techniques

Presentation transcript:

© John M. Abowd 2005, all rights reserved Modeling Integrated Data John M. Abowd April 2005

© John M. Abowd 2005, all rights reserved Outline Application of the grouping algorithm Further discussion of the methods Discussion of software

© John M. Abowd 2005, all rights reserved Characteristics of the Groups

© John M. Abowd 2005, all rights reserved Estimation by Direct Solution of Least Squares Once the grouping algorithm has identified all estimable effects, we solve for the least squares estimates by direct minimization of the sum of squared residuals. This method, widely used in animal breeding and genetics research, produces a unique solution for all estimable effects.

© John M. Abowd 2005, all rights reserved Least Squares Conjugate Gradient Algorithm The matrix  is chosen to precondition the normal equations. The data matrices and parameter vectors are redefined as shown.

© John M. Abowd 2005, all rights reserved LSCG (II) The goal is to find  to solve the least squares problem shown. The gradient vector g figures prominently in the equations. The initial conditions for the algorithm are shown. –e is the vector of residuals. –d is the direction of the search.

© John M. Abowd 2005, all rights reserved LSCG (III) The loop shown has the following features: –The search direction d is the current gradient plus a fraction of the old direction. –The parameter vector  is updated by moving a positive amount in the current direction. –The gradient, g, and residuals, e, are updated. –The original parameters are recovered from the preconditioning matrix.

© John M. Abowd 2005, all rights reserved LSCG (IV) Verify that the residuals are uncorrelated with the three components of the model. –Yes: the LS estimates are calculated as shown. –No: certain constants in the loop are updated and the next parameter vector is calculated.

© John M. Abowd 2005, all rights reserved Mixed Effects Assumptions The assumptions above specify the complete error structure with the firm and person effects random. For maximum likelihood or restricted maximum likelihood estimation assume joint normality.

© John M. Abowd 2005, all rights reserved Estimation by Mixed Effects Methods Solve the mixed effects equations Techniques: Bayesian EM, Restricted ML

© John M. Abowd 2005, all rights reserved Relation Between Fixed and Mixed Effects Models Under the conditions shown above, the ME and estimators of all parameters approaches the FE estimator

© John M. Abowd 2005, all rights reserved Correlated Random Effects vs. Orthogonal Design Orthogonal design means that characteristics, person design, firm design are orthogonal. Uncorrelated random effects means that  is diagonal.

© John M. Abowd 2005, all rights reserved Software SAS: proc mixed ASREML aML SPSS: Linear Mixed Models STATA: xtreg, gllamm, xtmixed R: the lme() function S+: linear mixed models Gauss Matlab Genstat: REML