Biostatistics: Pre-test Primer Larry Liang, MD University of Texas Southwestern Medical Center

Question 1 Which of the following will increase the sensitivity of a test? A.Decrease Type I Error B.Increase false positive results C.Increase true negative results D.Decrease true negative results E.Decrease Type II Error

Question 2 A study is designed to evaluate the effect of diet on weight loss. 50 people were weighed, put on an experimental diet, and weighed again at 6 months. Which is the appropriate test to determine statistical significance? A.Chi squared test B.Kruskal-Wallis test C.Paired t-test D.ANOVA E.Mann-Whitney test

What You Need to Know How to calculate things –Specificity –Sensitivity –Standard error of the mean –Etc, etc, etc How to pick the right test –T-test –Chi-squared –Kruskal-Wallis –Etc, etc, etc

The Basics Condition (As determined by gold standard) PositiveNegative Test Outcome PositiveTrue Positive False Positive (Type I Error)  Positive Predictive Value Negative False Negative (Type II Error) True Negative  Negative Predictive Value  Sensitivity  Specificity

Sensitivity Measures of ability of a test to correctly identify actual positives Disease (As determined by gold standard) Has DiseaseNo Disease PositiveTrue Positive Negative False Negative (Type II Error)  Sensitivity A highly sensitive test, has a low Type II error rate A negative result on a sensitive test "rules out" SNOUT = SeNsitivity, a Negative test rules OUT

Specificity Measures ability of a test to correctly determine actual negatives Disease (As determined by gold standard) Has DiseaseNo Disease Test Outcome Positive False Positive (Type I Error) NegativeTrue Negative  Specificity Highly specific tests have a low Type I error rate A positive result on a specific test will "rule in" SPIN = SPecificity, a Positive test rules IN

High Specificity – Example Dr. BO will do almost any case…he hardly cancels anything Because of this, he will correctly identify almost all the cases that SHOULD be cancelled So, when he DOES cancel a case, that has real meaning. A positive test “rules In" Dr. BO is highly specific, so it is overwhelmingly likely that a true negative will test negative Should we cancel a case? Yes CancelNo, don't cancel Dr. BO's eval Yes Cancel 22 Don't Cancel 29991

High Sensitivity – Example In this example, we have anesthesia pre-op faculty evaluating patients This faculty is very conservative so most patients will be classified as "complex" This faculty will be highly sensitive for identifying truly complex patients So a positive test is not very meaningful. Negative test “rules out" If this faculty calls a patient an ASA I, easy patient, then you know it will be Patient is Complex and Difficult Actually ComplexNot Complex NG's Pre-Op Eval Complex Not Complex 14999

Positive/Negative Predictive Value Important because it gives the accuracy of a positive or negative result Colon Cancer (By biopsy results) PositiveNegative Occult blood test Positive218  Positive Predictive Value Negative1182  Negative Predictive Value 2 / 3 67% 182 / % PPV = 2 / (2 + 18) = 10% NPV = 182 / ( ) = 99.5%

Type I and Type II Error Type I Error (  -error) –False positives –Accepting a positive result when the true status is negative Type II Error (  -error) –False negatives –Accepting a negative result when the true status is positive

Hypothesis Testing Null Hypothesis: Typically that a condition does not exist (not guilty, no difference between groups, drug does not do anything) Null Hypothesis: Suspect is not guilty H 0 False: Guilty H 0 True: Not guilty Jury Decision Reject H 0 : (Guilty) Correct Innocent goes to jail (Type I Error) Accept H 0 : (Not guilty) Crook gets off (Type II Error) Correct  Type I Error: Rejecting a null hypothesis which is actually true Type II Error: Accepting a null hypothesis which is actually false Easier to remember false positive and false negative

Hypothesis Testing Null Hypothesis: Typically that a condition does not exist (not guilty, no difference between groups, drug does not do anything) Type I Error: Rejecting a null hypothesis which is actually true Type II Error: Accepting a null hypothesis which is actually false Easier to remember false positive and false negative Null Hypothesis: Patient does not have cancer H 0 False: Has Cancer H 0 True: No Cancer Test Results Reject H 0 : (Positive) Correct False Positive (Type I Error) Accept H 0 : (Negative) False Negative (Type II Error) Correct 

Statistical Power The probability that a person who has a condition will test negative The probability that a test will reject a false null hypothesis As power increases, the chance of a type II error decreases Condition (As determined by gold standard) Positive Negative False Negative (Power) Chance of a Type II error is the  of a test Is the same as the sensitivity of a test Power = 1 -   Same as sensitivity

Prevalence vs Incidence Prevalence – Total number of cases in a population in a given time –Example: Prevalence of obesity in the USA in 2003 is 20.9% Incidence – The number of new cases which develop over a given time –Example: 28 cases per 1000 persons per year

Bayes’ Theorem Shows the relationship between two conditional probabilities –Ex: The probability you have breast cancer given a positive mammogram P (A|B) = [P(B|A) x P(A)] / P(B) –P(A|B) = Probability you have breast cancer (A) given you have a positive mammogram (B) –P(B|A) = Probability that a positive mammogram (B) is truly breast cancer (A) –P(A) = Probability you have breast cancer overall –P(B) = Probability of a positive mammogram (probability of true positive + false positive) Let’s say mammograms are 99% specific AND 99% sensitive Let’s say prevalence of breast cancer is 1:200 or 0.5% Probability (Cancer|Positive Mammogram) = –P(Positive Mammogram|Cancer) x P(Cancer) / P(Positive Test) –= 0.99 x / (chance of picking up true positive + chance of getting a false positive) –= 0.99 x / (0.99 x x 0.995) –=

Bayes’ Theorem Bottom line is: When the prevalence is low… You are unlikely to have the disease even if the test is accurate

Measures of Central Tendency Mean: Average –(1, 2, 3, 4, 5, 6, 7) Mean = 4 Median: Middle value when arranged in order –(1, 2, 3, 4, 5, 6, 1000) Median = 4 –(1, 2, 3, 4, 5, 6) Median = 3.5 Mode: Most common observation –(1, 2, 2, 3, 4, 5, 6, 7, 8, 9, 10) Mode = 2

Measures of Dispersion Range: Difference between largest and smallest observations Variance: The sum of the square of the difference between an observation and the mean divided by the total observations. –Example: Two tests scores 90 and 100. Mean = 95. –Variance = ( ) 2 + ( ) 2 / 2 = 25 Standard Deviation: Square root of variance –From example above, SD = 5 Standard Error of the Mean: = SD / (square root n) –= 5 / 1.41 = 3.5

Types of Data Categorical: –What was your major? –What is your favorite color? Numerical (Discrete): –How many cars do you have? –How many siblings do you have? Numerical (Continuous): –How tall are you? –What is your blood pressure?

Types of Data Parametric –Assumes a normal distribution of continuous numerical data –Assumes when comparing two populations, they have the same variance. Examples: scores on a test, height of people, golf handicaps Nonparametric –Used when parametric assumptions do not apply Examples: 5 year survival, pass/fail, class rank, results of a dice throw

Parametric Tests One sample t-test: Compares one group to a hypothetical value –Compare blood pressure of Texans to 110/60 Unpaired t-test: Compare two unpaired groups –Compare blood pressure of Texans to Floridians Paired t-test: Compare two paired groups –Effect of a drug on an blood pressure of a group of people –Measure people, give them a drug, measure again. Paired data points ANOVA: Compare 3 or more groups –Compare blood pressure of Texans, Floridians, and Sooners Pearson correlation: Measure of correlation between two variables –Does blood pressure correlate with weight? Linear Regression: Measures relationship of two variables if you think one causes the other

Parametric Tests Use T test when sample number is < 100 Use Z test when sample number is > 100 Z is later in the alphabet than T, so Z is "bigger than" T

Nonparametric Tests Wilcoxon test: Compare one group to a hypothetical value –VAS pain score of a group to 5 Mann-Whitney Test: Compare two unpaired groups –VAS pain score of morphine group vs. Tylenol group Wilcoxon test: Compare two paired groups –VAS pain score of one group before and after fentanyl Kruskal Wallis test: Compare 3 or more groups –VAS pain score of morphine, acetaminophen, and fentanyl groups Spearman Correlation: Measures correlation between 2 variables –VAS pain score vs. type of drug used Nonparametric regression: Measures relationship of two variables if you think one causes the other –Type of surgery vs. VAS pain scores

Chi Squared and Fisher’s Exact Tests Chi-Squared: Used to compare a categorical variable to a set of known probabilities –We know a what the probabilities are for a normal casino die –Use chi-squared to test an experimental die vs. known odds Fisher's Exact Test: Used when only 2 groups and small numbers (less than 6)

Question 1 Which of the following will increase the sensitivity of a test? A.Decrease Type I Error B.Increase false positive results C.Increase true negative results D.Decrease true negative results E.Decrease Type II Error

Which of the following will increase the sensitivity of a test? A.Decrease Type I Error B.Increase false positive results C.Increase true negative results D.Decrease true negative results E.Decrease Type II Error Condition (As determined by gold standard) PositiveNegative Test Outcome PositiveTrue Positive False Positive (Type I Error)  Positive Predictive Value Negative False Negative (Type II Error) True Negative  Negative Predictive Value  Sensitivity  Specificity Answer: E

Question 2 A study is designed to evaluate the effect of diet on weight loss. 50 people were weighed, put on an experimental diet, and weighed again at 6 months. Which is the appropriate test to determine statistical significance? A.Chi squared test B.Kruskal-Wallis test C.Paired t-test D.ANOVA E.Mann-Whitney test

Type of data? Parametric or Nonparametric: Parametric Sample size: < 100 Number of groups: 1 group, two data points per person Answer: C 50 people were weighed, put on an experimental diet, and weighed again at 6 months. Which is the appropriate test to determine statistical significance? A.Chi squared test B.Kruskal-Wallis test C.Paired t-test D.ANOVA E.Mann-Whitney test