Role of Experimentation and Experimental Planning Research Methods CPE 401 / 6002 / 6003 Professor Will Zimmerman

Motivation Some people collect lots of experimental data. Some people model systems and make predictions. Do they ever agree? Too often at the comparison stage, the wrong data is collected for validation.

Agenda II. Experimentation A. Why? B. Planning – role of the model. C. How? AIMS Introduce philosophy of experimentation.

Philosophy of experimentation  Identify aims (Why?) Hypothesis testing Model building Performance knowledge Optimization of performance Control system building  Build a preliminary / approximate model for experimental planning purposes and estimation purposes. Dimensional analysis, literature, package  Choose “venue” and predict range of operating conditions needed to test hypothesis and scope of replication and sampling required to assess system variability.  Resource plan (staff time, equipment, materials, consumables)  Collect data  Analyze / interpret data with respect to hypothesis testing and model building.  Report or repeat: Improve model from data assimilation or change structure to reflect observation.

Planning – the role of the model  Experimental planning without a model is “blind.” Intuition is needed to guess the venue, scope of trial (physical parameters), range of operating parameters, and number of replications / samples needed to “characterize” the system.  Without a model, experimenters tend to do as much as they possibly can, but also fix some choices arbitrarily for convenience. (Look at every tree in the forest but since you chose a desert, they are all cactuses!)  With a model, the experienced planner should select parameters where: (i) the model shows greatest parametric sensitivity; (ii) the model shows interesting features that need to be tested or found; (iii) will given optimum information about poorly known parameters through inverse methods.

How to conduct experiments  Carefully! Avoid systematic errors! 1.Preparation: Equipment, techniques, skills 2.Preliminary studies 3.Production phase 4.Repeatability 5.Representation and analysis of data 6.Interpretation 7.Reporting

The virtuous circle with modelling and experimentation Model predicts most informative experiments to run through sensitivity studies. Experiment and data assimilation improve model parameter knowledge or model structure.

Summary  Central to the scientific method is experimentation for hypothesis formation and model building / validation.  Experimental planning with a model can be greatly facilitated, even if the model is crude.  As the model improves, parametric sensitivity studies can be used to guide experimentation stages towards greater utility and information.  Inverse methods can assimilate experimental data to form semi-empirical models with robust predictive properties and identify parameters, which improves the model.