Designing Experiments Diverse applications, common principles

The story of an experiment Aim to find out of ‘Boreproof’ is more resistant to stem borer than M512 D1: plant a field with Boreproof. D2: plant a field of Boreproof and another of M512 Observed damage: 50% (M512), 20% (Boreproof) But we know damage varies widely Objective Comparison Treatments Experimental units

D3: 4 fields of Boreproof and 4 of M512 Observe: –M512: 50, 60, 40, 35 –Boreproof: 20, 10, 40, 30 D4: Unit = 10x10 plot. One pair in each field. M512 on the lefthand plot in each, Boreproof on the right Replication Precision Blocking Allocation

Consistency Confounding Randomisation FieldM512Bore proof Observe Consistent differences. But effect may not be treatment! D5: as D4 but with random allocation of treatments to units

Same principles Field experiments On-farm participatory experiments Lab experiments Social and institutional experiments ….

getting it right Understand and use experimental design principles Look at papers and reports describing how others have done it Refer to experienced researchers working on similar topics or methods (different region) Observe and critique other experiments Do a pilot experiment Use imagination!

Then… Think! Use all available sources of help Show design to others and get comments Envisage the data and the way you will use it to reach conclusions. –Draw empty (or expected) tables and graphs Look at practical implications Iterate Think!

Concepts and mistakes ObjectivesVague ‘Objective is to compare treatments’ Too many conflicting objectives in one trial –eg biophyisical and farmer assessment

Treatments Contrasts Controls Factorial structure Quantitative levels Extra treatments ‘because they might be interesting’ Omitting suitable control or baseline treatments Too many levels of quantitative factors Units Size and shape Design to measure Interference Multiple units Plots too small for realistic application and measurement of treatments Non-independent responses Over-use of split plots

Replication Estimating precision Controlling precision Insurance Extending range of results Using the ‘usual’ number of reps Forgetting hidden replication Insisting on equal replication for all treatments Forgetting the rules apply to all levels of units Assuming sub-samples are replicates

Site(s) Single site Multiple sites Using the default site Ignoring requirements of objectives Blocking Increasing precision by controlling variation Assuming only useful in field experiments Limited use of incomplete block designs RandomisationAssuming only applied to field experiments Omitting to randomise at some levels of design

Management Every aspect of preparing, implementing, measuring… Management not appropriate for objectives (eg level of inputs) Management confounded with treatments Failure to maximise precision through uniform management

The protocol written plan of the experiment

The protocol Written plan of the experiment A protocol should be: –prepared for every experiment, however small –written, not in your head –shared with others who can help improve it experience from similar problems, methods, species, ecozones,... –detailed enough for someone else to take over the trial –kept up to date plans change during the execution a record of what actually done –archived with the data

kept up to date –plans change during the execution –a record of what actually done archived with the data

The protocol contains... Identification (name of trial, people,...) Justification Objectives Treatments Field layout (sites, blocks, plots) Management Measurements Analysis methods Using the results

Check list for planning on-station Agroforestry experiments -rather statistical! Experiments with Farmers: Checklist for Preparing Protocols -better!

Good practice in AF field experiments

Three changes in research priorities 1.Landscape effects 2.Change processes 3.Participation