Comparison of 2 Population Means Goal: To compare 2 populations/treatments wrt a numeric outcome Sampling Design: Independent Samples (Parallel Groups) vs Paired Samples (Crossover Design) Data Structure: Normal vs Non-normal Sample Sizes: Large (n 1,n 2 >20) vs Small

Independent Samples Units in the two samples are different Sample sizes may or may not be equal Large-sample inference based on Normal Distribution (Central Limit Theorem) Small-sample inference depends on distribution of individual outcomes (Normal vs non-Normal)

Parameters/Estimates (Independent Samples) Parameter: Estimator: Estimated standard error: Shape of sampling distribution: –Normal if data are normal –Approximately normal if n 1,n 2 >20 –Non-normal otherwise (typically)

Large-Sample Test of     Null hypothesis: The population means differ by  0 (which is typically 0): Alternative Hypotheses: –1-Sided: –2-Sided: Test Statistic:

Large-Sample Test of     Decision Rule: –1-sided alternative If z obs  z  ==> Conclude       If z obs Do not reject       –2-sided alternative If z obs  z  ==> Conclude       If z obs  -z  ==> Conclude       If -z  Do not reject      

Large-Sample Test of     Observed Significance Level (P-Value) –1-sided alternative P=P(z  z obs ) (From the std. Normal distribution) –2-sided alternative P=2P( z  |z obs | ) (From the std. Normal distribution) If P-Value   then reject the null hypothesis

Large-Sample (1-  100% Confidence Interval for     Confidence Coefficient (1-  ) refers to the proportion of times this rule would provide an interval that contains the true parameter value     if it were applied over all possible samples Rule:

Large-Sample (1-  100% Confidence Interval for     For 95% Confidence Intervals, z.025 =1.96 Confidence Intervals and 2-sided tests give identical conclusions at same  -level: –If entire interval is above  0, conclude       –If entire interval is below  0, conclude       –If interval contains  0, do not reject     ≠  

Example: Vitamin C for Common Cold Outcome: Number of Colds During Study Period for Each Student Group 1: Given Placebo Group 2: Given Ascorbic Acid (Vitamin C) Source: Pauling (1971)

2-Sided Test to Compare Groups H 0 :  1  2  No difference in trt effects) H A :  1  2 ≠  Difference in trt effects) Test Statistic: Decision Rule (  =0.05) –Conclude      > 0 since z obs = 25.3 > z.025 = 1.96

95% Confidence Interval for     Point Estimate: Estimated Std. Error: Critical Value: z.025 = % CI: 0.30 ± 1.96(0.0119)  0.30 ±  (0.277, 0.323) Entire interval > 0

Small-Sample Test for     Normal Populations Case 1: Common Variances (  1 2 =  2 2 =  2 ) Null Hypothesis : Alternative Hypotheses : –1-Sided: –2-Sided : Test Statistic: (where S p 2 is a “pooled” estimate of  2 )

Small-Sample Test for     Normal Populations Decision Rule: (Based on t-distribution with =n 1 +n 2 -2 df) –1-sided alternative If t obs  t , ==> Conclude       If t obs Do not reject       –2-sided alternative If t obs  t , ==> Conclude       If t obs  -t  ==> Conclude       If -t  Do not reject      

Small-Sample Test for     Normal Populations Observed Significance Level (P-Value) Special Tables Needed, Printed by Statistical Software Packages –1-sided alternative P=P(t  t obs ) (From the t distribution) –2-sided alternative P=2P( t  |t obs | ) (From the t distribution) If P-Value   then reject the null hypothesis

Small-Sample (1-  100% Confidence Interval for      Normal Populations Confidence Coefficient (1-  ) refers to the proportion of times this rule would provide an interval that contains the true parameter value     if it were applied over all possible samples Rule: Interpretations same as for large-sample CI’s

Small-Sample Inference for     Normal Populations Case 2:  1 2   2 2 Don’t pool variances: Use “adjusted” degrees of freedom (Satterthwaites’ Approximation) :

Example - Scalp Wound Closure Groups: Stapling (n 1 =15) / Suturing (n 2 =16) Outcome: Physician Reported VAS Score at 1-Year Conduct a 2-sided test of whether mean scores differ Construct a 95% Confidence Interval for true difference Source: Khan, et al (2002)

Example - Scalp Wound Closure H 0 :      H A :      0 (  = 0.05) No significant difference between 2 methods

Small Sample Test to Compare Two Medians - Nonnormal Populations Two Independent Samples (Parallel Groups) Procedure (Wilcoxon Rank-Sum Test): –Rank measurements across samples from smallest (1) to largest (n 1 +n 2 ). Ties take average ranks. –Obtain the rank sum for each group (T 1, T 2 ) –1-sided tests:Conclude H A : M 1 > M 2 if T 2  T 0 –2-sided tests:Conclude H A : M 1  M 2 if min(T 1, T 2 )  T 0 –Values of T 0 are given in many texts for various sample sizes and significance levels. P-values printed by statistical software packages.

Example - Levocabostine in Renal Patients 2 Groups: Non-Dialysis/Hemodialysis (n 1 = n 2 = 6) Outcome: Levocabastine AUC (1 Outlier/Group) 2-sided Test: Conclude Medians differ if min(T 1,T 2 )  26 Source: Zagornik, et al (1993)

Computer Output - SPSS

Inference Based on Paired Samples (Crossover Designs) Setting: Each treatment is applied to each subject or pair (preferably in random order) Data: d i is the difference in scores (Trt 1 -Trt 2 ) for subject (pair) i Parameter:  D - Population mean difference Sample Statistics:

Test Concerning  D Null Hypothesis : H 0 :  D =  0 (almost always 0) Alternative Hypotheses : –1-Sided: H A :  D >  0 –2-Sided : H A :  D   0 Test Statistic:

Test Concerning  D Decision Rule: (Based on t-distribution with =n-1 df) 1-sided alternative If t obs  t , ==> Conclude  D    If t obs Do not reject  D    2-sided alternative If t obs  t , ==> Conclude  D    If t obs  -t  ==> Conclude  D    If -t  Do not reject  D    Confidence Interval for  D

Example - Evaluation of Transdermal Contraceptive Patch In Adolescents Subjects: Adolescent Females on O.C. who then received Ortho Evra Patch Response: 5-point scores on ease of use for each type of contraception (1=Strongly Agree) Data: d i = difference (O.C.-EVRA) for subject i Summary Statistics: Source: Rubinstein, et al (2004)

Example - Evaluation of Transdermal Contraceptive Patch In Adolescents 2-sided test for differences in ease of use (  =0.05) H 0 :  D = 0 H A :  D  0 Conclude Mean Scores are higher for O.C., girls find the Patch easier to use (low scores are better)

Small-Sample Test For Nonnormal Data Paired Samples (Crossover Design) Procedure (Wilcoxon Signed-Rank Test) –Compute Differences d i (as in the paired t-test) and obtain their absolute values (ignoring 0 s ) –Rank the observations by |d i | (smallest=1), averaging ranks for ties –Compute T + and T -, the rank sums for the positive and negative differences, respectively –1-sided tests:Conclude H A : M 1 > M 2 if T -  T 0 –2-sided tests:Conclude H A : M 1  M 2 if min(T +, T - )  T 0 –Values of T 0 are given in many texts for various sample sizes and significance levels. P-values printed by statistical software packages.

Example - New MRI for 3D Coronary Angiography Previous vs new Magnetization Prep Schemes (n=7) Response: Blood/Myocardium Contrast-Noise-Ratio All Differences are negative, T - = 1+2+…+7 = 28, T + = 0 From tables for 2-sided tests, n=7,  =0.05, T 0 =2 Since min(0,28)  2, Conclude the scheme means differ Source: Nguyen, et al (2004)

Computer Output - SPSS Note that SPSS is taking NEW-PREVIOUS in top table