Experimental Design Source: Christine

Definition In general usage: is the design of any information- gathering exercises where variation is present, whether under the full control of the experimenter or not. – wikipedia wikipedia In research: a study design used to test cause-and- effect relationships between variables. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation. – on-line medical dictionary on-line medical dictionary

Originality in Research Science depends on original thinking in two main areas: – The original ‘guess’ – called an hypothesis – The test or experiment to determine likelihood of hypothesis being correct A good scientist relies on ‘inspiration’ in the same way as a good artist. Some feel this point is largely ignored in present day science education. Encourage your students to be creative, original or inspired by the everyday when designing their research!

Being Discerning in Design Examples: – Teaching a dog French – Fortune Teller’s long term success

Elements of a Good Experiment Features common to all good experiments which exist regardless of utilizing advanced equipment or basic technique: – Discrimination – Replication & Generality – Controls – ‘Blind’ Design – Measurement

Discrimination Experiments should be capable of discriminating clearly between different hypotheses. It often turns out that two or more hypotheses give indistinguishable results when tested by poorly-designed experiments.

Replication & Generality Living material is notoriously variable. Usually experiments must be repeated enough times for the results to be analyzed statistically. Similarly, because of biological variability, we must be cautious of generalizing our results either from individual creatures to others of the same species, or to other species. For instance, if our hypothesis is about mammals, it is inadequate simply to carry out our experiments on laboratory rats. Similarly, it is dangerous to extrapolate from healthy students to elite athletes.

Controls The experiment must be well controlled. We must eliminate by proper checks the possibility that other factors in the overall test situation produce the effect we are observing, rather than the factor we are interested in. Example

“Blind” Design Investigators can subconsciously 'fudge' their data if they know what result they want to find. The answer is to do the experiment 'blind', so the investigators (and the subjects, if humans are being studied) do not know which treatment's effect they are observing. This can make the logistics of doing the experiment more complex. Example

Measurement Good experiments usually involve measuring something i.e. - length. Important you know both the accuracy and the precision of your measuring system. These two terms are not synonymous: – 'accuracy' means the ability of the method to give an unbiased answer on average, – 'precision' is an index of the method's reproducibility. Ideally your method should be both – accurate (i.e., give the true mean) – and precise (i.e., have a low standard deviation). Sometimes one is more important than the other. For example, if you were looking for small changes with time in a quantity (such as an athlete's hemoglobin concentration), you would need a precise measure of it rather more than an accurate one. Accuracy and precision together help you to judge the reliability of your data. They also help you to judge to how many significant figures you should quote your results. For example, if you use a balance reading to the nearest gram, you should give the results to the nearest gram and not to the nearest tenth of a gram. Some experiments are very difficult to do because it is not obvious what can be measured. – This is a real problem in animal behavior: for example, there is no obvious unit or measure for 'emotional state'. It is usually necessary to isolate measurable components of behavior. Thus the speed at which a tiger paces up and down a cage can give some indication of the internal state of the animal but can never give a full picture of it.

Measurement Plant Example Detergent Example

Experimental Design & Statistics Good experimental design involves having a clear idea about how we will analyze the results when we get them. That's why statisticians often tell us to think about the statistical tests we will use before we start an experiment.

Examples Example 1. Experiments that yield no useful results because we did not collect enough data 9:3:3:1 GametesABAbaBab ABAABBAABbAaBBAaBb AbAABbAabbAABbAabb aBAaBBAaBbaaBBaaBb abAaBbAabbaaBbaabb

Example 1 (Cont) Chi squared test Poisson distribution Student’s t-Test How many replicates to use?

Examples Example 2. Experiments that seem to give useful results but our procedures let us down! Latin Square vs Randomization Row 1ABCD Row 2DCAB Row 3BADC Row 4CDBA

Example 2 (Cont) Never put all the replicates of one treatment together! To block or not to block – real world dilemmas – Field Examples

More on Eliminating Bias Yale Experimental Design Example

Steps to Follow 1. Define the objectives. – Record (i.e. write down) precisely what you want to test in an experiment. 2. Devise a strategy. – Record precisely how you can achieve the objective. – This includes thinking about the size and structure of the experiment – how many treatments? how many replicates? how will the results be analysed? 3. Set down all the operational details. – How will the experiment be performed in practice? – In what order will things be done? – Should the treatments be randomised or follow a set structure? – Can the experiment be done in a day? – Will there be time for lunch? etc.

SOURCE USED FOR ABOVE INFO The Really Easy Statistics Site by: Jim Deacon, Biology Teaching Organisation, University of Edinburgh

ADDITIONAL RESOURCES Introduction to Experimental Design Lesson Plan with Worksheets By: Daniell Difrancesca; Critical Thinking in Science Concepts of Experimental Design Concepts of Experimental Design (pdf) By: SAS; A SAS White Paper

Questions?