The Problem with Parameter Redundancy Diana Cole, University of Kent.

Slides:

Advertisements

Similar presentations

Questions From Yesterday

Advertisements

Dummy Dependent variable Models

Tests of Hypotheses Based on a Single Sample

Review bootstrap and permutation

Chapter Outline 3.1 Introduction

GENERAL LINEAR MODELS: Estimation algorithms

1 Low-Dose Dual-Energy CT for PET Attenuation Correction with Statistical Sinogram Restoration Joonki Noh, Jeffrey A. Fessler EECS Department, The University.

Lecture 2: Parameter Estimation and Evaluation of Support.

Uncertainty and confidence intervals Statistical estimation methods, Finse Friday , 12.45–14.05 Andreas Lindén.

Workshop on Parameter Redundancy Part II Diana Cole.

Lwando Kondlo Supervisor: Prof. Chris Koen University of the Western Cape 12/3/2008 SKA SA Postgraduate Bursary Conference Estimation of the parameters.

Cox Model With Intermitten and Error-Prone Covariate Observation Yury Gubman PhD thesis in Statistics Supervisors: Prof. David Zucker, Prof. Orly Manor.

Detecting Parameter Redundancy in Ecological State-Space Models Diana Cole and Rachel McCrea National Centre for Statistical Ecology, University of Kent.

Parameter Redundancy and Identifiability in Ecological Models Diana Cole, University of Kent.

Parameter Redundancy and Identifiability Diana Cole and Byron Morgan University of Kent Initial work supported by an EPSRC grant to the National Centre.

Detecting Parameter Redundancy in Complex Ecological Models Diana Cole and Byron Morgan University of Kent.

Maximum likelihood estimates What are they and why do we care? Relationship to AIC and other model selection criteria.

Estimation A major purpose of statistics is to estimate some characteristics of a population. Take a sample from the population under study and Compute.

Approximations and Errors

1 Econometrics 1 Lecture 7 Multicollinearity. 2 What is multicollinearity.

Curve-Fitting Regression

The Islamic University of Gaza Faculty of Engineering Civil Engineering Department Numerical Analysis ECIV 3306 Chapter 3 Approximations and Errors.

Parameter Redundancy in Ecological Models Diana Cole, University of Kent Byron Morgan, University of Kent Rachel McCrea, University of Kent Ben Hubbard,

1/30 Stochastic Models for Yeast Prion Propagation Diana Cole 1, Lee Byrne 2, Byron Morgan 1, Martin Ridout 1, Mick Tuite Institute of Mathematics,

Determining Parameter Redundancy of Multi-state Mark- Recapture Models for Sea Birds Diana Cole University of Kent.

Rao-Cramer-Frechet (RCF) bound of minimum variance (w/o proof) Variance of an estimator of single parameter is limited as: is called “efficient” when the.

Non-Linear Simultaneous Equations

1 Psych 5500/6500 Statistics and Parameters Fall, 2008.

Computer vision: models, learning and inference

Single and Multiple Spell Discrete Time Hazards Models with Parametric and Non-Parametric Corrections for Unobserved Heterogeneity David K. Guilkey.

F-Test ( ANOVA ) & Two-Way ANOVA

Moderation in Structural Equation Modeling: Specification, Estimation, and Interpretation Using Quadratic Structural Equations Jeffrey R. Edwards University.

QBM117 Business Statistics Estimating the population mean , when the population variance  2, is known.

Adding individual random effects results in models that are no longer parameter redundant Diana Cole, University of Kent Rémi Choquet, Centre d'Ecologie.

R. Kass/W03P416/Lecture 7 1 Lecture 7 Some Advanced Topics using Propagation of Errors and Least Squares Fitting Error on the mean (review from Lecture.

Success depends upon the ability to measure performance. Rule #1:A process is only as good as the ability to reliably measure.

CSDA Conference, Limassol, 2005 University of Medicine and Pharmacy “Gr. T. Popa” Iasi Department of Mathematics and Informatics Gabriel Dimitriu University.

Stochastic Linear Programming by Series of Monte-Carlo Estimators Leonidas SAKALAUSKAS Institute of Mathematics&Informatics Vilnius, Lithuania

Geographic Information Science

Detecting Parameter Redundancy in Integrated Population Models Diana Cole and Rachel McCrea National Centre for Statistical Ecology, School of Mathematics,

A Hybrid Symbolic-Numerical Method for Determining Model Structure Diana Cole, NCSE, University of Kent Rémi Choquet, Centre d'Ecologie Fonctionnelle et.

Progress in identification of damping: Energy-based method with incomplete and noisy data Marco Prandina University of Liverpool.

Maximum Likelihood Estimation Methods of Economic Investigation Lecture 17.

PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Department of Computer Engineering Sharif University of Technology Principles of Parameter Estimation.

Methods for Estimating Defects Catherine V. Stringfellow Mathematics and Computer Science Department New Mexico Highlands University October 20, 2000.

1 1 Slide © 2007 Thomson South-Western. All Rights Reserved Chapter 8 Interval Estimation Population Mean:  Known Population Mean:  Known Population.

Estimating age-specific survival rates from historical ring-recovery data Diana J. Cole and Stephen N. Freeman Mallard Dawn Balmer (BTO) Sandwich Tern.

Chapter 5 Multilevel Models

1 Introduction to Statistics − Day 4 Glen Cowan Lecture 1 Probability Random variables, probability densities, etc. Lecture 2 Brief catalogue of probability.

Statistical Data Analysis 2010/2011 M. de Gunst Lecture 10.

Machine Learning 5. Parametric Methods.

- 1 - Calibration with discrepancy Major references –Calibration lecture is not in the book. –Kennedy, Marc C., and Anthony O'Hagan. "Bayesian calibration.

Tutorial I: Missing Value Analysis

Excursions in Modern Mathematics, 7e: Copyright © 2010 Pearson Education, Inc. 13 Collecting Statistical Data 13.1The Population 13.2Sampling.

1 Chapter 5 – Density estimation based on distances The distance measures were originally developed as an alternative to quadrat sampling for estimating.

Parameter Redundancy in Mark-Recapture and Ring-Recovery Models with Missing Data Diana Cole University of Kent.

1 Chapter 8: Model Inference and Averaging Presented by Hui Fang.

Multistate models UF Outline  Description of the model  Data structure and types of analyses  Multistate with 2 and 3 states  Assumptions 

Hypothesis Testing. Statistical Inference – dealing with parameter and model uncertainty  Confidence Intervals (credible intervals)  Hypothesis Tests.

Computacion Inteligente Least-Square Methods for System Identification.

R. Kass/Sp07P416/Lecture 71 More on Least Squares Fit (LSQF) In Lec 5, we discussed how we can fit our data points to a linear function (straight line)

An Adaptive Learning with an Application to Chinese Homophone Disambiguation from Yue-shi Lee International Journal of Computer Processing of Oriental.

Multi-state Occupancy. Multiple Occupancy States Rather than just presence/absence of the species at a sampling unit, ‘occupancy’ could be categorized.

STA248 week 121 Bootstrap Test for Pairs of Means of a Non-Normal Population – small samples Suppose X 1, …, X n are iid from some distribution independent.

Confidential and Proprietary Business Information. For Internal Use Only. Statistical modeling of tumor regrowth experiment in xenograft studies May 18.

Ch3: Model Building through Regression

Extension to the Hybrid Symbolic-Numeric Method for Investigating Identifiability Diana Cole, University of Kent, UK Rémi Choquet, CEFE, CNRS, France.

Parameter Redundancy and Identifiability in Ecological Models

Multistate models Lecture 10.

Presentation transcript:

The Problem with Parameter Redundancy Diana Cole, University of Kent

Parameter Redundancy

Problems with Parameter Redundancy There will be a flat ridge in the likelihood of a parameter redundant model (Catchpole and Morgan, 1997), resulting in more than one set of maximum likelihood estimates. Numerical methods to find the MLE will not pick up the flat ridge, although could be picked up trying multiple starting values and looking at profile log-likelihoods. The Fisher information matrix will be singular (Rothenberg, 1971) and therefore the standard errors will be undefined. However the exact Fisher information matrix is rarely known. Standard errors are typically approximated using a Hessian matrix obtained numerically. Can parameter redundancy be detected from the standard errors?

Is example 1 parameter redundant? ParameterEstimateStandard Error 0.39imaginary imaginary 0.18imaginary

Is example 2 parameter redundant? ParameterEstimateStandard Error

Is example 3 parameter redundant? ParameterEstimateStandard Error

Simulation Study for Example 1/2 52% have defined standard errors ParameterTrue ValueAverage MLESt. Dev. MLE SVD threshold%age SVD test correct % % % %

Computer Packages and Parameter Redundancy MARK (Cooch and Evans, 2014) Counts the number of estimable parameters using a numerical procedure involving a Single Value Decomposition, if “2ndPart” chosen rather than “Hessian” for variance estimation. Using “Hessian” method parameter redundancy is missed and agree with Cooch and Evans (2014)’s recommendation to use the default of “2ndPart”. Standard errors for non-identifiable parameters are either very large or zero and should be ignored. Parameter estimates for non-identifiable parameters are unreliable and should be ignored. Parameter redundancy could be caused by the model or the data. Recommend refitting any parameter redundant model with suitable constraints.

Computer Packages and Parameter Redundancy M-surge / E-surge (Choquet et al, 2004, Choquet et al, 2009) Uses the hybrid-symbolic-numeric method to detect parameter redundancy, but will not be able to tell whether parameter redundancy is caused by the model or the data. (Parameter redundancy caused by the model could be examined if you used simulated data.) Gives which parameters can and cannot be estimated, but cannot find estimable parameter combinations in parameter redundant models (currently only possibly symbolically) Also recommend refitting parameter redundant models with suitable constraints.

Conclusion It is not always possible to tell from model fitting that a model is parameter redundant. Recommend at least using numeric method to check parameter redundancy, but symbolic or hybrid methods are more reliable. Fitting parameter redundant models results in large bias for non-identifiable parameters and can introduce bias in the identifiable parameter models. If a model is parameter redundant it needs to be (re)fitted with constraints, which can be obtained using the symbolic method.

References Catchpole, E. A. and Morgan, B. J. T (1997) Detecting parameter redundancy. Biometrika, 84, Choquet, R. and Cole, D.J. (2012) A Hybrid Symbolic-Numerical Method for Determining Model Structure. Mathematical Biosciences, 236, p117. Choquet, R., Reboulet, A.M., Pradel, R., Gimenez, O. Lebreton, J.D. (2004). M-SURGE: new software specifically designed for multistate capture- recapture models. Animal Biodiversity and Conservation 27(1): Choquet, R., Rouan, L., Pradel, R. (2009). Program E-SURGE: a software application for fitting Multievent models. Series: Environmental and Ecological Statistics, Vol. 3 Thomson, David L.; Cooch, Evan G.; Conroy, Michael J. (Eds.) p Cole, D.J., Morgan, B.J.T., Titterington, D.M. (2010) Determining the Parametric Structure of Non-Linear Models. Mathematical Biosciences, 228, Cooch and Evans (2014) Program Mark. A Gentle Introduction. Rothenberg, T.J. (1971) Identification in parametric models. Econometrica, 39, Viallefont, A., Lebreton, J.D., Reboulet, A.M. and Gory, G. (1998) Parameter Identifiability and Model Selection in Capture-Recapture Models: A Numerical Approach. Biometrical Journal, 40,