Session 8: Paired Samples (Zar, Chapter 9,24)

General: One population of subjects: x 1, x 2, …, x n, but a pair of data points on each. Examples: Before and after treatment Left and right Evaluator 1 and Evaluator 2 on the same subject Method 1 vs. Method 2

Paired t-test:

If Two-Sided Test: One-Sided Tests:

Example 9.1: H 0 : Hindleg length=Foreleg length H A : 

Sign Test: Chapter 24.7 H 0 : same # increases as decreases H A : increases ≠ decreases H A1 : increases < decreases H A2 : increases > decreases Form S + = # positive signs S - = # negative signs

For H 0 : increases = decreases H A : increases ≠ decreases Compare: Min{S +, S - } ≤ Table B.27[n*,  reject H 0 Note: n* = # pos + # neg = S + + S - Do not include zeros! Sign Test is the same as the test: where P + = true proportion of positives

One-Sided Tests: If S from table ≤ B.27[n*, , reject H o Note: n* = # pos + # neg = S + + S - Do not include zeros! Some statisticians would include zeros if one-sided as the zeros represent non-support for the alternative

Example The sign test for the paired-sample data of Examples 9.1 and 9.3

H 0 : No difference between hindleg and foreleg length. H A : Difference between hindleg and foreleg length. n* = 10; S + =8;S - =2; B.27[  =B.27[0.05(2),10]=1 Therefore, Accept H 0 Using Table B.26b for n=10 and p=0.5, Since the probability is greater than 0.05, do not reject H 0. 1-Sided Test:

Wilcoxon Signed-Rank Test: H 0 : Ranks decreases = Ranks increases H A : decreases ≠ increases H A1 : decreases < increases H A2 : decreases > increases

Rank the data (d i ’s) without regard to “sign”, from smallest to largest including ties as in the Mann-Whitney. Form T + = sum of + ranks. T - = sum of – ranks. If n*  100, Use Table B.12 (App 101): For n*= number of non-zero differences (n*=n + +n - ). For H A : decreases  increases: Min {T +,T - }≤B.12[  n*], reject H 0.

Example 9.3 The Wilcoxon paired –sample test applied to the data of Example 9.1. H 0 : Deer hindleg length is the same as foreleg length. H A : Deer hindleg length is not the same as foreleg length.

n = 10 T + = = 51 T - = = 4 Min {4,51}=4 From Table B.12: T 0,05(2), 10 = 8 Since T - < T 0.05(2), 10. H 0 is rejected. 0.01< P(T - or T + ≤ 4) < 0.02 H 0 : ranks + = ranks - HA: ranks + ≠ ranks - If Min{T +, T - } ≤ Table B.12[  n*]=T  n*, reject H 0

Note: and One-Sided Tests: If we use x 1 - x 2 = d For one-tailed testing we use one-tailed critical values from Table B.12 and either T + or T - as follows.

For the hypotheses H 0 : Measurements in reading 1≤ measurements in reading 2 and H A2 : Measurements in reading 1>measurements in reading 2 Decrease 1  2 For the opposite hypotheses: H 0 : Measurements in reading 1≥ measurements 2 and H A1 : Measurements in reading 1 < measurements in reading 2. Increase 1  2

If we use x 2 – x 1 : Normal Approximation: No Ties:

diff=x 1 -x 2 For H A : 1 ≠ 2 use either T - or T + for T. If x 2 - x 1, reverse the sides. If Z > K  (sides), reject H o (Table B.2)

For Ties: For Zero adjustment: (m = #Zeros)

McNemar’s Test: Analysis of Preference Tests or “which do you like better – Coke or Pepsi?” Many Product tests use this technique: Example 9.4 Comparison of Lotions: H 0 : The proportion of persons experiencing relief is the same with both lotions. H A : The proportion of persons experiencing relief is not the same with both lotions.

Principle: (Relief, Relief) and (No Relief, No Relief) give no information as to which is better! Under H o, f 12 and f 21 estimate the same quantity: Observedf 12 f 21 Total

Degrees of Freedom1 +1= 2-1 =1 estimated value Test: Chi-Square >   , Reject H 0.

Biomedical Applications 1) Examiner vs Examiner

Comparing Against Truth: (The diagnostic test) Ex: Test: X-ray, MR, CT, CEA, PSA, TGF, … Truth: Pathology (Biopsy, FNA, Surgical section), Time and observation, Panel of experts -- (The Gold Standard)

Other names and parameters: True Positive fraction = TPF = sensitivity True Negative fraction = TNF = specificity False Positive fraction = FPF = 1-TNF False Negative fraction = FNF = 1-TPF Positive Predictive Value = positive accuracy Negative Predictive Value = negative accuracy

Comparisons: Two sensitivities from two different studies: Two sensitivities from same study Compare to each other Select only True Positive or True Negative:

Higher Order Tables:

In Summary: 1) Individual McNemar chi-squares 2) Above versus Below -- 1 d.f. 3)Heterogeneity chi-square = Individual-above vs below (1-2) d.f.=#chisquares-1

Ex: Mildness Study Exam 1: after cleaning Exam 2: one month later

Hypothesis Chi-SquareD.F. H o : f 12 =f H o : f 13 =f H o : f 23 =f Total H 0 : below=above Heterogeneity Conclusions: f 12  f 21 Above versus Below not significantly different. No Heterogeneity -- Homogeneous.

5)Rating Scale Data Comparison of Rater to “truth” Examples: a)Diagnostic Radiology Systems 1)Diagnostic value of MR, Xerox, and screen film in detection of Breast Cancer b)Pathology 1)Comparison of staining systems to predict relapse (early vs. late) 2)Monoclonal stains or Micro-satellite probes to predict stage of cancer. c)Laboratory Medicine 1)Comparison of machine classification of cells d)Training 1) Comparison of novice to standard diagnosis

a)Often created from raters looking “blinded” at packets of cases. b)Easy to set up but requires “truth” from (1) another method, (2) gold standard (3) team of raters.

ROC (Receiver Operating Characteristic) Analysis (a)Calculate 2 x 2 Tables: 1: Make Cut point after “Very Likely”

Decide Abnormal Decide Normal

Decide Abnormal Decide Normal And so on to get:

(a)Plot the following points to create an ROC Curve: (0,0) 1:(FP 1,TP 1 ) 2:(FP 2,TP 2 ) 3:(FP 3,TP 3 ) 4:(FP 4,TP 4 ) (1,1)

Summary Chapter Paired Tests

2  2 and k  k tables McNemar tests on Likert scales: (1) Pairwise (2) Pooled (above vs below) (3) Heterogeneity chi-square