1. SAMSI Kickoff 11/9/06Slide 1 Simulators, emulators, predictors – Validity, quality, adequacy Tony O’Hagan

2. www.mucm.group@shef.ac.ukSlide 2 MUCM Managing Uncertainty in Complex Models Large new research project to establish “Basic Technology” Quantify uncertainties in model predictions Quantify how uncertainty is reduced by calibration or data assimilation Identify robust tools that work on a wide range of models Key underlying mechanism is the use of emulators

3. www.mucm.group@shef.ac.ukSlide 3 Simulators, emulators, predictors A simulator is a model, representing (Wendy Parker: standing for) some real world process An emulator is a statistical description of a simulator Not just a fast surrogate Full probabilistic specification of beliefs A predictor is a statistical description of reality Full probabilistic specification of beliefs Emulator + representation of relationship between simulator and reality

4. www.mucm.group@shef.ac.ukSlide 4 All models are wrong, but … The simulator is inevitably wrong There is no meaningful sense in which we can validate it It might be good enough for purpose But simulators are built to be multi-purpose

5. www.mucm.group@shef.ac.ukSlide 5 Validity, quality, adequacy A predictor/emulator is valid if the truth lies appropriately within probability bounds Could be conservative Need severe testing tools for verification The quality of a predictor is determined by how tight those bounds are Refinement versus calibration A predictor is adequate for purpose if the bounds are tight enough If we are satisfied the predictor is valid over the relevant range we can determine adequacy