Some terms Parametric data assumptions(more rigorous, so can make a better judgment) – Randomly drawn samples from normally distributed population – Homogenous (at least roughly) variances in the samples Variance will be roughly the same – Data are interval or ratio in scale (continuous data) Non-parametric data – Data that don’t meet parametric assumptions

Some terms Power – The ability to find a difference, if one exists There is always a difference, just is it statistically sig. – Used a priori (how big of a sample size is needed) and post hoc (if the lack of difference due to a too small sample size) – Function of four factors (all go up as power goes up, direct relationship, except variance) Significance criterion Variance (within group variance changes opposite of power) Sample size Effect size Significance – The probability of committing a Type I error-acceptable risk of making a mistake Saying there is a difference when none exists – Also known as the alpha level

Some terms p value – Finding after your statistical analysis – Probability of finding that big a difference by chance – % that event occurred by chance Randomization – Selection Every member of group has equal chance of selection – Assignment Each member has an equal chance of being assigned to any of the groups

Experimental Design Sometimes called a Clinical Trial – Therapeutic – effect of treatment on disease – Preventive – effective at reducing development of disease Provides structure to evaluate causality Independent Variables – May be multiple – May each have multiple levels Dependent Variables – May be multiple Element of control – Improves argument for causality

Clinical Trial New therapies, drugs, procedures, devices Box 10.1 Distinct sequence – Preclinical Often animal model – Phase I Establish safety Small sample size – Phase II Still small sample Effectiveness

Clinical Trial – Phase III Usually randomized controlled, double blind Large sample Comparison to standard or placebo – Phase IV Other populations Risk factors/benefits Optimal use

Design Classifications True Experimental – RCT the “Gold Standard” of this design – This design is differentiated by assignment Between subjects (“completely randomized”) – Selected randomly, and divided randomly Randomized block (age, gender exclusion) Within subjects (subjects serve as their own control) – Sometimes described by the number of “Factors” Factors = Independent Variables (IV) in this context Single factor means one IV Multi-factor means more than one IV – Quasi- Experimental Lack random assignment &/or Lack comparison group

Selecting a Design What is your PICO? Can the IV be manipulated? Can you control extraneous factors? If experimental design is right then ask – How many IVs? – How many levels in each IV? – How many groups will be tested? – How will assignment be made? – How often will measurements be taken? – What is the time sequence?

Selecting a Design Pretest-posttest Control Group – Figures – Analysis Interval-scale data – Two groups – t-test (unpaired, also called independent) – Three or more groups – ANOVA (usually one-way) – Could be ANCOVA (pre-test score is the covariate) – Could be two way » Treatment as one factor » Other factor is the repeated factor of time (pre-test/post-test) Ordinal Data – Two groups – Mann-Whitney U-test – Three or more groups – Kruskal-Wallis analysis of variance by ranks

Selecting a Design Posttest-Only Control Group – Figures – Analyzed with Interval-scale data – Two groups – t-test (independent) – Three or more groups - One way ANOVA Ordinal Data – Two groups – Mann-Whitney U-test – Three or more groups – Kruskal-Wallis analysis of variance by ranks May also analyze with ANCOVA if extraneous relevant data are available Regression or discriminate analysis can be applied

Selecting a Design Multi-factorial – What are factors? – Nomenclature IV with number indicating the number of levels of that IV 3 X 4 multifactorial test – Two IVs – One with 3 levels, one with 4 levels 3x3x3 – Three Ivs – One with 3 levels, second with 3 levels, third with 3 levels – Analyzed with (most commonly) Two way ANOVA Three way ANOVA

Selecting a Design Multi-factorial – In Two way factorial design, three questions can be addressed (In this example, consider 2 x 2 design) Main effects (2) – Of each IV – The other IV “collapsed” across levels Interaction effect (1) – Between the two Ivs - – Every independent variable has a MAIN EFFECT: so 5 IV means 5 main effects

Selecting a Design Multi-factorial – In Three way factorial design, multiple questions can be addressed (In this example, consider 2 x 2 x 2 design) Main effects (3) – Of each IV – The other IV “collapsed” across levels Double interaction effects (3) – Between the three possibilities of IV pairings Triple interaction (8) – The possible interactions of all 6 levels – Figure 10.6 good to visualize this

Selecting a Design Randomized block – Homogeneous blocks – Then randomly assigned to one level of the IV (Fig 10.7) Can be thought of as two single factor randomized experiments – Analyzed with Two way ANOVA Multiple Regression or discriminate analysis can be applied

Selecting a Design Repeated Measures=type of analysis – What are factors=are IV’s – Can the control be any more equivalent? (Rhetorical ?) Serve as own control, so not really can’t get any more equivalent. – Disadvantages Carryover=irritation Practice effects=improved skill, comfort level with activity Outcome measure must return to baseline between interventions Single Factor Repeated – Analyzed with One way ANOVA

Selecting a Design Crossover Design – Counterbalance the treatment conditions – “Washout” period to return to baseline (like letting a drug leave the body) – May only have two levels of an independent variable – Analyzed with Interval-scale data – t-test for change scores by treatment condition – Two way ANOVA with two repeated measures » Pre-test post test » Across both conditions Ordinal Data – Wilcoxen signed ranks *IF it is named after someone than not continuous (exception Person’s Product.

Selecting a Design Two Way with Two Repeated Measures – 2 X 2 – Analyzed with – Two way ANOVA with two repeated measures Mixed Design – One factor is repeated (often time is the factor) – One Factor is randomly assigned – Analyzed with – Two way ANOVA with one repeated measures

Selecting a Design Sequential Clinical Trial – Special approach to the RCT – Data continually analyzed – Compares two treatments to find the preferred one – A series of “little experiments” – Preference subjective but clearly defined Those without a preference are excluded from analysis – Analyzed by charting – Three choices Stop and recommend one treatment Stop and state you found no difference Continue collecting data

Efficacy vs Effectiveness Efficacy is clinical – Under a controlled situation – The Lab result Effectiveness is “Real World” – When control cannot be maintained – The application in practice

Quasi Experimental One Group Pretest-posttest – Time is the IV – Treatment is not the IV (WHY? –Because they all get it) – Analyzed with Interval-scale data – Paired t-test – Why not ANOVA? Ordinal Data – Sign test – Wilcoxen signed-ranks test One-Way repeated Measures over Time – Analyzed with the ANOVA. WHY?

Quasi Experimental – Time Series Considered extension of the one-group pretest-posttest Multiple measurements – Before and after treatment – Serve as pseudo-control – Analyzed with Visual chart analysis Multivariate methodologies

Quasi Experimental – Multi-group Design Non-equivalent pretest – posttest Control Group Analyzed with – Multiple options here – Consider non-parametric tests!!! » Not ordinal/continuous data, not normally distributed, Non-equivalent posttest only Control Group Analyzed with – Regression approach » Looking for relationships, but not causality