Reducing MCMC Computational Cost With a Two Layered Bayesian Approach Ramin Madarshahian, Doctoral Candidate, mdrshhn@email.sc.edu Juan M. Caicedo, Associate Professor http://sdii.ce.sc.edu/

Outline Introduction Methodology Example Future work

Introduction

Markov Chain Monte Carlo (MCMC) A very powerful algorithm for getting sample from high dimensional and complicated probability distribution function. It gets more sample from high probability regions, and with enough number of samples, histogram of samples take similar shape as probability distribution of interest.

Markov Chain Monte Carlo (MCMC) Von Neumann he contributed to the development of the Monte Carlo method, which allowed solutions to complicated problems to be approximated using random numbers. Metropolis Ulam Paper 1949: Using Markov chain for Monte Carlo approximation Rossenburg Teller Paper 1953: They applied MCMC for a chemical problem

Bayesian modeling Bayesian modeling is good method to deal with uncertainty. Bayesian modeling update our belief about the model and its parameters by considering evidences. Evidence comes from inputs and outputs.

Problem? With a Bayesian model we would like to make an inference about a model and its parameters. MCMC can be used to sample the posterior, but for each sample we need to run the model. What if our model is very computational expensive?

Metamodeling Metamodel: Simply an approximation of the computationally expensive model. Also known as : Response surface, emulators, auxiliary models, etc. Computationally expensive models: Models of multi-scale problems like shear band, models of complicated structures like airplane, modelling of physical and biological phenomena like protein folding, etc.

General approach Few numbers of input samples. Using the expensive model to obtain corresponding output samples. Using these I/O relationships to fit the metamodel. For some methods, using obtained metamodel to select next input sample to better fit the metamodel. Replacing the expensive model with obtained metamodel, and using this new model in the Bayesian process and etc.

Proposed method In our proposed method, the surface of interest is obtained from the posterior of the Bayesian model, instead of direct approximation of the expensive model by the surrogate.

Motivation A posterior is a probability distribution function with all common characteristics of that. It is usually more well-defined in comparison to the expensive model itself. Depending upon the type of study, a researcher can focus on high probability regions (like model updating problems) or focus on tails (Reliability problems). This makes sampling more efficient.

Methodology

Bayesian modelling formulation Model’s parameters Data Posterior Likelihood

Proposed method formulation

Example

SDOF Simulated by assuming normal distribution with 𝜇=1000 𝑁 𝑚 and 𝜎=10 𝑁 𝑚 for K 𝑀=100 𝑘𝑔 then what is 𝐾 using the data in the table?

Inference without using the metamodel Assuming uniform prior for K from Zero to 2000:

Inference without using the metamodel MCMC: Selecting the prior: Assuming uniform prior for K from Zero to 2000 𝑁 𝑚 : A total of 10000 samples are generated and the first 2000 are discarded.

Inference without using the metamodel 𝜇 𝑘 =1034.84 𝑁 𝑚 95% 𝐻𝑖𝑔ℎ𝑒𝑠𝑡 𝑃𝑜𝑠𝑡𝑒𝑟𝑖𝑜𝑟 𝐷𝑒𝑛𝑠𝑖𝑡𝑦 𝐻𝑃𝐷 = 642 ,1428 𝑁 𝑚 𝜎=203 𝑁 𝑚

Proposed method

Proposed method Priors: For 𝑐 1 : a uniform distribution with the lower bound of 0.001 and the upper bound of 1.0. For 𝜇 1 : a uniform distribution with the lower bound of zero and the upper bound of 2000. For 𝜎 1 : a normal distribution with 𝜇=300 𝑁 𝑚 , and 𝜎=80 𝑁 𝑚 A total of 12000 samples were obtained, 4000 of them were considered burning samples and were discarded.

Proposed method

Proposed method Parameter Mean 95% HPD 𝜇 1 1021 𝑁 𝑚 [1011,1031] 𝑁 𝑚 [1011,1031] 𝑁 𝑚 𝜎 1 201 𝑁 𝑚 [190,210] 𝑁 𝑚 𝑐 1 0.051 [0.049,0.053]

Comparison Using obtained mean and standard deviation, 95% HPD will be [627, 1414] 𝑁 𝑚 which is comparable with results without the metamodel, i.e.[642, 1428] 𝑁 𝑚 .

Future work

Future work Considering different types of metamodels like polynomial, kriging, etc. Development of sampling strategies. Considering expensive models and study of computational power of method.

Reducing MCMC Computational Cost With a Two Layered Bayesian Approach Thank you! Reducing MCMC Computational Cost With a Two Layered Bayesian Approach Ramin Madarshahian, Doctoral Candidate, mdrshhn@email.sc.edu Juan M. Caicedo, Associate Professor http://sdii.ce.sc.edu/