1. Chris Brien University of South Australia Bronwyn Harch Ray Correll CSIRO Mathematical and Information Sciences Chris.brien@unisa.edu.au Design and analysis of experiments with a laboratory phase subsequent to an initial phase

2. 2 Outline 1.Designing two-phase experiments a)A biodiversity example b)When first-phase factors do not divide lab factors 2.Trend adjustment in the biodiversity example 3.Taking trend into account in design 4.Duplicates

3. 3 Notation Factor relationships A*B factors A and B are crossed A/B factor B is nested within A Generalized factor A  B is the ab-level factor formed from the combinations of A with a levels and B with b levels Symbolic mixed model Fixed terms : random terms (A*B : Blocks/Runs) A*B = A + B + A  B A/B/C = A + A  B + A  B  C Sources in ANOVA table A#B a source for the interaction of A and B B[A] a source for the effects of B nested within A

4. 4 1.Designing two-phase experiments Two-phase experiments as introduced by McIntyre (1955):  Consider special case of second phase a laboratory phase

5. 5 General considerations Need to randomize laboratory phase so involve two randomizations:  1 st -phase treatments to 1 st -phase, unrandomized factors  latter to unrandomized, laboratory factors Often have a sequence of analyses to be performed and how should one group these over time. Fundamental difference between 1 st and 2 nd randomizations  1 st has randomized factors crossed and nested  2 nd has two sets of factors and all combinations of the two sets are not observable; within sets are crossed or nested  tendency to ignore 1 st phase, unrandomized factors. Categories of designs  Lab phase factors purely hierarchical or involve crossed rows and columns;  Two-phase randomizations are composed or randomized-inclusive (Brien & Bailey, 2006); related to whether 1 st -phase, unrandomized factors divide laboratory unrandomized factors  Treatments added in laboratory phase or not  Lab duplicates included or not

6. 6 1a) A Biodiversity example Effect of tillage treatments on bacterial and fungal diversity Two-phase experiment: field and laboratory phase Field phase: 2 tillage treatments assigned to plots using RCBD with 4 blocks 2 soil samples taken at each of 2 depths  2  4  2  2 = 32 samples (Harch et al., 1997)

7. 7 Laboratory phase: Then analysed soil samples in the lab using Gas Chromatography - Fatty Acid Methyl Ester (GC-FAME) analysis 2 preprocessing methods randomized to 2 samples in each Plot  Depth All samples analysed twice — necessary?  once on days 1 & 2; again on day 3 n In each Int2, 16 samples analyzed

8. 8 Processing order within Int1  Int2 Logical as similar to order obtained from field But confounding with systematic laboratory effects:  Preprocessing method effects  Depth effects Depths assigned to lowest level ─ sensible?

9. 9 Towards an analysis Dashed arrows indicate systematic assignment 2 Samples in B, P, D 4 Blocks 2 Plots in B 2 Depths 32 samples 2 Tillage 2 field treats 2 Methods 2 lab treats 64 analyses 2 Int1 2 Int3 in I1, I2 2 Int2 in I1 2 Int4 in I1, I2, I3 2 Int5 in I1, I2, I3, I4 2 Int6 in I1, I2, I3, I4, I5 2 B 1 2 B 2 in B 1 Int1Int2Block 111&2 123&4 211&2 223&4 64 analyses divided up hierarchically by 6 x 2-level factors Int1…Int6 of size 32, …, 2 analyses, respectively.

10. 10 Analysis of example for lab variability Variability for:  Int4 >Int5 > Int6  8 > 4 >2 Analyses  Int1, Int2, Int3 small (< Int4)

11. 11 Alternative blocking for the biodiversity example Want to assign the 32 samples to 64 analyses Consider with the experimenter: 1Uninteresting effects — Blocks 2Large effects — Depth? 3Some treatments best changed infrequently — Methods? 4Period over which analyses effectively homogeneous — 16 analyses? 4 analyses? 2 analyses?

12. 12 Alternative blocking for the biodiversity example For now, divide 64 analyses into 2 Occasions = Int1, 4 Times = Int2  Int3, 8 Analyses = Int4  Int5  Int6 Blocks of 8 would be best as 2 Plots x 2 Depths x 2 Methods, but Blocks = Times too variable.  Best if pairs of analyses in a block.  Also Times are similar  could take 4 Times x 2 Analyses. Many other possibilities: e.g. blocks of size 4 with Depths randomized to pairs of blocks.

13. 13 Proposed laboratory design Organise 64 analyses into blocks of 8: n Randomization of field units ignores treats n Two composed randomizations (Brien and Bailey, 2006) Ø Field treats to samples to analyses n Two independent randomizations (Brien and Bailey, 2006) Ø Field and lab treats to samples n Experiment with Ø hierarchical lab phase, composed randomizations, duplicates and treatments added at laboratory phase. 4 Blocks 2 Depths 2 Plots in B 2 Samples in B, P, D 32 samples 2 Methods 2 lab treats 2 Tillage 2 field treats 64 analyses 2 Occasions 4 Intervals in O 8 Analyses in O, I

14. 14 Decomposition table for proposed design Randomization-based mixed model (Brien & Bailey, 2006) :  Till*Meth*Dep : ((Blk/Plot)*Dep)/Sample – Dep + Occ/Int/Anl Each of the 15 lines is a separate subspace in the final decomp- osition Note Residual df determined by field phase Important for design: shows confounding and apportionment of variability  Or Till*Meth*Dep : ((Blk/Till)*Dep)/Meth – Dep + Occ/Int/Anl

15. 15 1b)When first-phase factors do not divide lab factors Need to use a nonorthogonal design and two randomized-inclusive randomizations (Brien and Bailey, 2006) Willow experiment (Peacock et al, 2003) Beetle damage inhibiting rust on willows? Glasshouse and lab phases Example here same problem but different details Will be an experiment with  hierarchical lab phase, randomized-inclusive randomizations, no duplicates and no treatments added at laboratory phase

16. 16 Glasshouse: 60 locations each with a plant  12 damages to assign to locations.  Only 6 locations per bench: — Damages does not divide no. locations or benches so IBD — Use RIBD with v = 12, k = 6, E = 0.893, bound = 0.898.  Randomize between Reps, Benches within Reps and Locations within Benches. Willow experiment (cont’d) 60 locations 5 Reps 6 Locations in R, B 2 Benches in R 12 treatments 12 Damages 

17. 17 Lab phase: disk/plant put onto 20 plates, 3 disks /plate  Plates divided into 5 groups for processing on an Occasion  Locations does not divide Cells — divide 6 Locations into 2 sets of 3: cannot do this ignoring Damages  RIBD related to 1 st -phase (v = 12, k =3, r = 5, E = 0.698, bound = 0.721) — In fact got this design using CycDesgN (Whittaker et al, 2002) and combined pairs of blocks to get 1 st -phase. — To include Locations, read numbers as Locations with these Damages. — Renumber Locations to L 1 and L 2 to identify those assigned same Plate. Willow experiment (cont’d) 5 Occasions 3 Cells in O, P 4 Plates in O 60 cells  Sometimes better design if allow for lab phase in designing 1 st 60 locations 5 Reps 6 Locations in R, B 2 Benches in R 12 treatments 12 Damages    2 L 1  3 L 2

18. 18 Decomposition table for proposed design Each of the 6 lines is a separate subspace in the final decomposition. Note Residual df for Locations from 1 st phase is 39 and has been reduced to 29 in lab phase.  s are strata variances or portions of E[MSq] from cells and  s from locations. Four estimable variance functions:  O +  R,  OP +  RB,  OP +  RBL,  OPC +  RBL, although 2 nd may be difficult. Randomization-based mixed model (Brien & Bailey, 2006) that corresponds to estimable quantities :  Damages : Rep/Benches/L 1 + Occasions  Plates  Cells.  Must have Locations in the form of L 1 in this model ─ i.e. cannot ignore unrandomized factors from 1 st phase.

19. 19 Willow experiment (cont’d) 10 L 1  150 leaves 10 treatments 5 Blocks 3 LeafPosns 10 Locations in B 10 Damages 5 Occasions 6 Cells in O, P 5 Plates in O 150 cells Glasshouse: 50 locations each with a plant  10 Damages assigned to 5 blocks using RCBD  Disks taken high, middle & low leafs Lab phase: 6 disks assigned put onto 25 plates, 2 locations x 3 leaves per plate  Plates divided into 5 groups for processing on an Occasion.  Locations assigned using a resolvable PBIB(2) for v = 10, k =2 and r = 5.  cannot assign Locations, ignoring Damages  use L 1 to explicitly identify which Damage assigned to a Location ─ connects them  Cannot ignore Locations as need it in the analysis. 

20. 20 Willow experiment ANOVA Mixed model (a model of convenience):  Damages*LeafPosn : Occasions/Plates + Blocks  (Locations+LeafPosn)

21. 21 2.Trend in the biodiversity example Trend can be a problem in laboratory phase. Is it here? Plot of Lab-only residuals in run order for 8 Analyses within Times Linear trend that varies evident Proposed design (4 x 2) is appropriate (  trend & low Times variability)  smallest Analysis Residual

22. 22 Trend adjustment for example REML analysis with vector of 1…8 for each Occasion Significant different linear trends (p < 0.001) Effect on fixed effects Trend adjustment reduced  Tillage effect from  0.99 to  0.07  Plot[Block] component from 13.25 to 0.001. Low Plot[Block] df makes this dubious.

23. 23 4. Taking trend into account in design Cox (1958, section 14.2) discusses trend elimination:  concludes that, where the estimation of trend not required, use of blocking preferred to trend adjustment; Yeh, Bradley and Notz (1985) combine blocking for trend and adjustment & provide trend-free and nearly trend-free designs with blocks  allow for common quadratic trends within blocks  minimize the effects of adjustment Look at design of laboratory phase  for field phase with RCBD, b = 3, v = 18  3 Occasions in lab phase to which 3 Blocks randomized  allow for different linear & cubic Trends within each Occasion

24. 24 Different designs for blocks of 18 analyses RCBD for this no. treats relatively efficient when adjusting for trend Blocks assigned to 3 Occasions × 6 Analyses (blocking perpendicular to trend?)  Use when Occasions variability low e.g. recalibration Nearly Trend-Free (using Yeh, Bradley and Notz, 1985) worse than RCBD for different trends:  optimal for common linear trend. Still to investigate designs that protect against different trends.

25. 25 Comparing RCBD with RIBDs for k = 6,9 Use Relative Efficiencies  = av. pairwise variance of RCBD to RIBD for sets of generated data Generate using random model:  Y = Occasion + Interval[Occasion] + Analyses[Occasions  Interval] + Plots[Blocks] Expect efficiency  k = 6 > k = 9  RIBD > RCBD provided   BP not dominant and   OI is non-zero. How much?   BP < 10   OI ≥ 0.5 (very little extra required, but after trend adjustment)

26. 26 Resolvable design with cols & latinized rows [using CycDesgN (Whittaker et al, 2002), Intra E = 0.49] Expect LRCD > RIBD if  IA ≠ 0 and  BP not dominant;  How much?  If  IA > 1 irrespective of  OI. Again only small  s. Expect LRCD > RCD if Occasions different. REs  as  BP   (LRCD/RCD < 2 if  BP ≥ 2.5).

27. 27 4.Duplicates Commonly used, but only need in two-phase experiments if Lab variation large compared to field. Possibilities:  Separated: analyze all & then reanalyze all in different random order  Nested: some analyzed & then these reanalyzed in a different random order  Crossed: some analyzed & then these reanalyzed in same order  Consecutive: duplicate immediately follows first analysis  Randomized: some analysed & everything randomized From ANOVAs and REs to randomized, when adjusting for different cubic trends, conclude  Separated duplicates superior, with nested duplicates 2 nd best; little gain in efficiency if  OI ≤ 0.5 and  BP is considerable;  Crossed and consecutive duplicates perform poorly with RE < 1 often

28. 28 5.Summary for lab phase design Two-phase: initial expt & lab phase  Leads to  2 randomizations: composed or r-inclusive related to whether 1 st – phase, unrandomized factors divide laboratory, unrandomized factors  Use of pseudofactors with r-inclusive does not ignore field terms and makes explicit what has occurred  Adding treatments in lab phase leads to more randomizations  Cannot improve on field design but can make worse Important to have some idea of likely laboratory variation:  Will there be recalibration or the like?  Are consistent differences between and/or across Occasions likely?  How does the magnitude of the field and laboratory variation compare?  Are trends probable: common vs different; linear vs cubic? Will laboratory duplicates be necessary and how will they be arranged?  If yes, separated duplicates best but other arrangements may be OK. RCBD will suffice if  field variation >> lab variation, in which case duplicates unnecessary.  after adjustment for trend, no extra laboratory variation, except Occasions  can block across occasions when no Occasion differences If Intervals differences, RIBD better than RCBD ─ not much needed. LRCD better than RIBD provided, after trend adjustment, moderate consistent differences between Analyses across Occasions.

29. 29 References Brien, C.J., and Bailey, R.A. (2006) Multiple randomizations (with discussion). J. Roy. Statist. Soc., Ser. B, 68, 571–609. Cox, D.R. (1958) Planning of Experiments. New York, Wiley. John, J.A. and Williams, E.R. (1995) Cyclic and Computer Generated Designs. Chapman & Hall, London. Harch, B.E., Correll, R.L., Meech, W., Kirkby, C.A. and Pankhurst, C.E. (1997) Using the Gini coefficient with BIOLOG substrate utilisation data to provide an alternative quantitative measure for comparing bacterial soil communities. Journal of Microbial Methods, 30, 91–101. McIntyre, G. (1955) Design and analysis of two phase experiments. Biometrics, 11, p.324–34. Peacock, L., Hunter, P., Yap, M. and Arnold, G. (2003) Indirect interactions between rust (Melampsora epitea) and leaf beetle (Phratora vulgatissima) damage on Salix. Phytoparasitica, 31, 226–35. Whitaker, D., Williams, E.R. and John, J.A. (2002) CycDesigN: A Package for the Computer Generation of Experimental Designs. (Version 2.0) CSIRO, Canberra, Australia. http://www.ffp.csiro.au/software http://www.ffp.csiro.au/software Yeh, C.-M., Bradely, R.A. and Notz, W.I. (1985) Nearly Trend-Free Block Designs. J. Amer. Statist. Assoc., 392, 985–92.

30. 30 Web address for link to Multitiered experiments site http://chris.brien.name/multitier