PROCESS MODELLING AND MODEL ANALYSIS © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Statistical Model Calibration and Validation.

Slides:



Advertisements
Similar presentations
11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.
Advertisements

A. The Basic Principle We consider the multivariate extension of multiple linear regression – modeling the relationship between m responses Y 1,…,Y m and.
Regression analysis Relating two data matrices/tables to each other Purpose: prediction and interpretation Y-data X-data.
Forecasting Using the Simple Linear Regression Model and Correlation
PROCESS MODELLING AND MODEL ANALYSIS © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Analysis of Dynamic Process Models C13.
CmpE 104 SOFTWARE STATISTICAL TOOLS & METHODS MEASURING & ESTIMATING SOFTWARE SIZE AND RESOURCE & SCHEDULE ESTIMATING.
11 Simple Linear Regression and Correlation CHAPTER OUTLINE
Chapter 10 Curve Fitting and Regression Analysis
Ch11 Curve Fitting Dr. Deshi Ye
Simple Linear Regression
280 SYSTEM IDENTIFICATION The System Identification Problem is to estimate a model of a system based on input-output data. Basic Configuration continuous.
SYSTEMS Identification Ali Karimpour Assistant Professor Ferdowsi University of Mashhad Reference: “System Identification Theory For The User” Lennart.
BCOR 1020 Business Statistics Lecture 28 – May 1, 2008.
Development of Empirical Models From Process Data
Chapter 11 Multiple Regression.
Basics of regression analysis
11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.
Statistics 350 Lecture 17. Today Last Day: Introduction to Multiple Linear Regression Model Today: More Chapter 6.
Lecture 19 Simple linear regression (Review, 18.5, 18.8)
Simple Linear Regression and Correlation
Feedback Control Systems (FCS)
RLR. Purpose of Regression Fit data to model Known model based on physics P* = exp[A - B/(T+C)] Antoine eq. Assumed correlation y = a + b*x1+c*x2 Use.
Correlation & Regression
Regression Analysis Regression analysis is a statistical technique that is very useful for exploring the relationships between two or more variables (one.
Hydrologic Modeling: Verification, Validation, Calibration, and Sensitivity Analysis Fritz R. Fiedler, P.E., Ph.D.
Basic Probability (Chapter 2, W.J.Decoursey, 2003) Objectives: -Define probability and its relationship to relative frequency of an event. -Learn the basic.
SUPA Advanced Data Analysis Course, Jan 6th – 7th 2009 Advanced Data Analysis for the Physical Sciences Dr Martin Hendry Dept of Physics and Astronomy.
Inference for Regression Simple Linear Regression IPS Chapter 10.1 © 2009 W.H. Freeman and Company.
1 Multiple Regression A single numerical response variable, Y. Multiple numerical explanatory variables, X 1, X 2,…, X k.
1 11 Simple Linear Regression and Correlation 11-1 Empirical Models 11-2 Simple Linear Regression 11-3 Properties of the Least Squares Estimators 11-4.
1 Regression Analysis The contents in this chapter are from Chapters of the textbook. The cntry15.sav data will be used. The data collected 15 countries’
Model Selection and Validation. Model-Building Process 1. Data collection and preparation 2. Reduction of explanatory or predictor variables (for exploratory.
Geology 5670/6670 Inverse Theory 21 Jan 2015 © A.R. Lowry 2015 Read for Fri 23 Jan: Menke Ch 3 (39-68) Last time: Ordinary Least Squares Inversion Ordinary.
Ch14: Linear Least Squares 14.1: INTRO: Fitting a pth-order polynomial will require finding (p+1) coefficients from the data. Thus, a straight line (p=1)
Dept. E.E./ESAT-STADIUS, KU Leuven
How Good is a Model? How much information does AIC give us? –Model 1: 3124 –Model 2: 2932 –Model 3: 2968 –Model 4: 3204 –Model 5: 5436.
KNN Ch. 3 Diagnostics and Remedial Measures Applied Regression Analysis BUSI 6220.
PROCESS MODELLING AND MODEL ANALYSIS © CAPE Centre, The University of Queensland Hungarian Academy of Sciences A Model Building Framework.
Least Squares Estimate Additional Notes 1. Introduction The quality of an estimate can be judged using the expected value and the covariance matrix of.
CY3A2 System identification Input signals Signals need to be realisable, and excite the typical modes of the system. Ideally the signal should be persistent.
Regression Analysis1. 2 INTRODUCTION TO EMPIRICAL MODELS LEAST SQUARES ESTIMATION OF THE PARAMETERS PROPERTIES OF THE LEAST SQUARES ESTIMATORS AND ESTIMATION.
September 28, 2000 Improved Simultaneous Data Reconciliation, Bias Detection and Identification Using Mixed Integer Optimization Methods Presented by:
1 Statistical Analysis Professor Lynne Stokes Department of Statistical Science Lecture 9 Review.
Learning Theory Reza Shadmehr Distribution of the ML estimates of model parameters Signal dependent noise models.
Computacion Inteligente Least-Square Methods for System Identification.
732G21/732G28/732A35 Lecture 3. Properties of the model errors ε 4. ε are assumed to be normally distributed
DSP-CIS Part-III : Optimal & Adaptive Filters Chapter-9 : Kalman Filters Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven
Statistics 350 Lecture 2. Today Last Day: Section Today: Section 1.6 Homework #1: Chapter 1 Problems (page 33-38): 2, 5, 6, 7, 22, 26, 33, 34,
Statistics 350 Review. Today Today: Review Simple Linear Regression Simple linear regression model: Y i =  for i=1,2,…,n Distribution of errors.
Modelling & Simulation of Semiconductor Devices Lecture 1 & 2 Introduction to Modelling & Simulation.
Thomas F. Edgar (UT-Austin) RLS – Linear Models Virtual Control Book 12/06 Recursive Least Squares Parameter Estimation for Linear Steady State and Dynamic.
Estimating standard error using bootstrap
Regression and Correlation of Data Summary
HST 583 fMRI DATA ANALYSIS AND ACQUISITION
11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.
ELG5377 Adaptive Signal Processing
Dynamic Models, Autocorrelation and Forecasting
(5) Notes on the Least Squares Estimate
Probability Theory and Parameter Estimation I
STAT 312 Chapter 7 - Statistical Intervals Based on a Single Sample
Inference for Regression
Ch3: Model Building through Regression
Statistical Methods For Engineers
Unfolding Problem: A Machine Learning Approach
6-1 Introduction To Empirical Models
The regression model in matrix form
Statistical Thinking and Applications
Unfolding with system identification
Regression and Correlation of Data
Introductory Statistics
Presentation transcript:

PROCESS MODELLING AND MODEL ANALYSIS © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Statistical Model Calibration and Validation C12

PROCESS MODELLING AND MODEL ANALYSIS 2 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Overview   Grey box models and model calibration   Data analysis and preprocessing   Model parameter and structure estimation:   linear-nonlinear   static-dynamic   Model validation

PROCESS MODELLING AND MODEL ANALYSIS 3 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences A Systematic Modelling Procedure Problem definition Controlling factors Problem data Model construction Model solution Model verification Model calibration & validation

PROCESS MODELLING AND MODEL ANALYSIS 4 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Grey-box Models Process models   developed from first engineering principles (white box part)   part of their parameters and/or structure unknown (black box part) are called grey-box models

PROCESS MODELLING AND MODEL ANALYSIS 5 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Model Calibration Conceptual Problem Statement Given:  grey-box model  calibration data (measured data)  measure of fit (loss function)  Estimate: the parameter values and/or structural elements

PROCESS MODELLING AND MODEL ANALYSIS 6 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Model Calibration Conceptual Steps of Solution  Analysis of model specification  Sampling of continuous time dynamic models  Data analysis and preprocessing  Model parameter and structure estimation  Evaluation of the quality of the estimate

PROCESS MODELLING AND MODEL ANALYSIS 7 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Sampling of Continuous Time Dynamic Models Equi-distant zero-order hold sampling Discrete time input signal: u : {u(k)=u(t k ) | k=1,2,...} output signal: y : {y(k)=y(t k ) | k=1,2,...}

PROCESS MODELLING AND MODEL ANALYSIS 8 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Sampling of Continuous Time Dynamic Models ) Model parameters (1st order approximation ) Discrete time model equations: model parameters:  = I+Ah,  = Bh ( , ,C,D) Continuous time model equations: model parameters: (A,B,C,D)

PROCESS MODELLING AND MODEL ANALYSIS 9 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Data Analysis and Preprocessing -Data Screening-   gross errors   outliers   trends  Data visualization  Outlier tests  Trends, steady state tests  Gross error detection Check measured data for: Methods to be used include:

PROCESS MODELLING AND MODEL ANALYSIS 10 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Data Screening - Visualization Gross errors

PROCESS MODELLING AND MODEL ANALYSIS 11 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Data Screening - Visualization Trends and jumps

PROCESS MODELLING AND MODEL ANALYSIS 12 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Experimental Design for Parameter Estimation Static models   number of measurements   test point spacing   test point sequencing Dynamic models (in addition)   sampling time selection   excitation Pseudo Random Binary Signal (PRBS)

PROCESS MODELLING AND MODEL ANALYSIS 13 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Model Parameter and Structure Estimation   Conceptual problem statement   Least Squares parameter estimation - estimation procedure - properties of the estimate - linear and nonlinear models   Parameter estimation for static models   Parameter estimation for dynamic models

PROCESS MODELLING AND MODEL ANALYSIS 14 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Problem Statement of Model Parameter Estimation Given:   System model:   Set of measured data:   Loss function: Compute: an estimate such that

PROCESS MODELLING AND MODEL ANALYSIS 15 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Problem Statement of Model Structure Estimation Given:   System model: (not parametrized)   Set of measured data:   Loss function: Compute: an estimate such that + “candidate structures” in

PROCESS MODELLING AND MODEL ANALYSIS 16 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Least Squares (LS) Parameter Estimation Given:   System model: linear in p, single y (M)   Measured data:   Loss function: Compute: an estimate such that

PROCESS MODELLING AND MODEL ANALYSIS 17 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Properties of LS Parameter Estimation Estimation: with Gaussian measurement errors :   unbiased:   covariance matrix:

PROCESS MODELLING AND MODEL ANALYSIS 18 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Assessing the Fit Residuals are independent and   residual tests   correlation coefficient measures

PROCESS MODELLING AND MODEL ANALYSIS 19 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Confidence Regions and Intervals Individual confidence intervals:

PROCESS MODELLING AND MODEL ANALYSIS 20 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences LS Parameter Estimation for Nonlinear Models Solution   Transformation into linear form   Solution by (numerical) optimization   Properties has lost its nice properties - non-normally distributed - confidence region and confidence intervals are not symmetric - unbiased

PROCESS MODELLING AND MODEL ANALYSIS 21 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Confidence Interval for Nonlinear Parameter Estimation Sum-of-squares as a function of a parameter

PROCESS MODELLING AND MODEL ANALYSIS 22 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Static Models Linear in Parameters General form Examples

PROCESS MODELLING AND MODEL ANALYSIS 23 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Identification: Model Parameter and Structure Estimation of Dynamic Models Properties of the estimation problem variables (y and x) are time dependent ordered x : present and past inputs and outputs measurement errors on both y and x Steps 1. sampling continuous time models 2. estimation

PROCESS MODELLING AND MODEL ANALYSIS 24 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Parameter Estimation of Dynamic Models Linear in Parameters General form of the input-output model LS parameter estimation with

PROCESS MODELLING AND MODEL ANALYSIS 25 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Parameter Estimation of Nonlinear Dynamic Models

PROCESS MODELLING AND MODEL ANALYSIS 26 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Statistical Model Validation via Parameter Estimation Conceptual Problem Statement Given:  a calibrated model  validation data (measured data)  measure of fit (loss function) Question: Is the calibrated model “good enough” for the purpose? (Does it reproduce the data well?)

PROCESS MODELLING AND MODEL ANALYSIS 27 © CAPE Centre, The University of Queensland Hungarian Academy of Sciences A Systematic Modelling Procedure Problem definition Controlling factors Problem data Model construction Model solution Model verification Model calibration & validation