Issues Related to Parameter Estimation in Model Accuracy Assessment DDDAS: June 6-7, Tom Henderson & Narong Boonsirisumpun ICCS 2013 Barcelona, Spain 6 June 2013 Funded in part by grant AFOSR-FA

Major Objectives: 1.Develop Bayesian Computational Sensor Networks – Detect & identify structural damage – Quantify physical phenomena and sensors – Characterize uncertainty in calculated quantities of interest (real and Boolean) DDDAS: June 6-7, 20132

Major Objectives (cont’d): 2. Develop active feedback methodology using model-based sampling regimes – Embedded and active sensor placement – On-line sensor model validation – On-demand sensor complentarity DDDAS: June 6-7, 20133

Major Objectives (cont’d): 3. Develop rigorous uncertainty models; stochastic uncertainty of: – System states – Model parameters – Sensor network parameters (e.g., location) – Material damage assessments DDDAS: June 6-7, 20134

DDDAS Aspects Addresses 3 of 4 DDDAS components: Applications modeling Advances in mathematics and statistical algorithms, and Application measurement systems and methods DDDAS: June 6-7, 20135

General Framework 1.World2. Observations 3.Model Code 4. Explanations & Predictions Observe, Measure Analyze Control Inform Generate Validate Constrain Verify DDDAS: June 6-7, Uncertainty Quantification

VVUQ for Sensor Networks DDDAS: June 6-7,

Model Validation Issues: Input uncertainty: parameters, initial conditions, etc. Model discrepancy: fails to capture physics, scale, etc. Cost of computation DDDAS: June 6-7, Note: This and next 2 slides based on “Assessing the Reliability of Complex Models,” NRC Report, 2012.

Model Validation Identify sources of uncertainty Identify information sources Assess quality of prediction Determine resources required to improve validity DDDAS: June 6-7, 20139

Model Adequacy Measure Predict: quantity of interest (QoI) with acceptable tolerance for intended application with uncertainty range attached e.g., V(x) = 5 +/- 2 with 90% confidence DDDAS: June 6-7,

Our Long-term Goal Bayesian inference network analysis of: Computational uncertainty results Information from large knowledge bases: – Maintenance log data – Human knowledge DDDAS: June 6-7,

Model Accuracy Assessment DDDAS: June 6-7, (Figure based on Oberkampf [1])

Model Accuracy Assessment (MAA) Compare 7 parameter estimation approaches: – Inverse method – LLS – MLE – EKF – PF – Levenberg-Marquardt – Minimum RMS Error DDDAS: June 6-7,

Model Accuracy Assessment (MAA) Can statistics produced by estimation technique characterize adequacy of the model? – Which method gives the best k estimate? – Which is least sensitive to noise? – Which has lowest time complexity? DDDAS: June 6-7,

PDE Model: 2D Heat Flow DDDAS: June 6-7, Truncation Error:

Inverse Method DDDAS: June 6-7, At each location: Yields global estimate:

Linear Least Squares DDDAS: June 6-7, C is the Laplacian term and d is the temporal derivative: Yields global estimate:

Maximum Likelihood Estimate DDDAS: June 6-7, Take derivative of log likelihood function of T:

Extended Kalman Filter DDDAS: June 6-7, from temperature equation at each point from where

Particle Filter Method DDDAS: June 6-7, Sample p particles from range of distribution - Use weight function to re-calculate particle probabilities - Re-sample particles from new distribution Continue until change in range is small

Levenberg-Marquardt Method DDDAS: June 6-7, Use Jacobian: Solve for k as:

Minimum RMS Method DDDAS: June 6-7,

Model of Phenomenon Simulated Data Measured Data Sensors Algorithms & Code DDDAS: June 6-7, Thermal Data Processing

Model of Phenomenon Simulated Data Measured Data Algorithms & Code Regular Mesh Temperatures DDDAS: June 6-7, Thermal Data Processing Sensors

Model of Phenomenon Test Generation Simulated Data Known Solution Data Algorithms & Code PDE’s, Material Points, other Sequential, parallel, multi-grid, adaptive mesh refinement Verification: Ensure Code Implements Model Noise Models ? DDDAS: June 6-7,

Model of Phenomenon Simulated Data Measured Data Sensors Algorithms & Code Parameter Estimation Adjust Model Parameters Model Parameter Estimation DDDAS: June 6-7,

Model of Phenomenon Simulated Data Measured Data Sensors Algorithms & Code Validation: Make sure Model matches Phenomenon ? Adjust Model DDDAS: June 6-7,

Example Result (LLS, simulated) DDDAS: June 6-7,

EKF Tracking Results DDDAS: June 6-7,

EKF Predictive Results DDDAS: June 6-7,

UQ in LLS Prediction DDDAS: June 6-7,

Flat Heat Plate Schematic DDDAS: June 6-7,

2D Thermal Data Raw Thermal Data DDDAS: June 6-7, FLIR T420 high performance IR camera 320x240 pixel array 170x170 over plate Subsampled (smoothed) down to 17x17 array

K Estimate Results DDDAS: June 6-7,

RMS Error of Prediction DDDAS: June 6-7,

Time Cost of Methods DDDAS: June 6-7,

Extrapolative Prediction DDDAS: June 6-7,

Summary of Results Given the validation criterion that predicted temperature is within 2 degrees of measured temperature, the accuracy requirements are met. Distributions are determined for the thermal diffusivity parameter. DDDAS: June 6-7,

Our Current Work DDDAS: June 6-7, Study of Bayesian Computational Sensor Networks for Structural Health Monitoring Using Ultrasound

Our Current Work (cont’d) DDDAS: June 6-7,

Current High-Level Goals DDDAS: June 6-7, Develop Uncertainty Quantification for data driven structural health analysis process. Dongbin Xiu has joined the University of Utah and we have started discussions on this. 2. Quantify the effect of subject matter judgments with respect to inferences about VVUQ outcomes. Develop Bayesian inference network methods.

Current Specific Goals 1.Determine prior joint pdf’s describing knowledge of model parameter distributions. 2.Provide proof of robustness and stability of models under the various sources of perturbation (algorithmic, data, etc.). DDDAS: June 6-7,

Current Specific Goals 3. Quantify validation processes to assess the appropriateness of the calibrated model for predictions of quantities of interest (e.g., damage existence, damage extent, model and sensor parameter values). 4. Obtain piezoelectric active sensor network experimental results on metallic plates DDDAS: June 6-7,

Questions? DDDAS: June 6-7,