Introduction to Study Skills & Research Methods (HL10040) Hypothesis Testing Introduction to Study Skills & Research Methods (HL10040) Dr James Betts

Lecture Outline: What is Hypothesis Testing? Hypothesis Formulation Statistical Errors Effect of Study Design Test Procedures Test Selection.

Organising, summarising & describing data Statistics Descriptive Inferential Correlational Organising, summarising & describing data Generalising Relationships Significance

What is Hypothesis Testing? Null Hypothesis Alternative Hypothesis A B A B We also need to establish: 1) How …………………….. are these observations? Inevitably, especially with continuous data, score A and score B are never identical. So we need statistical procedures to assess ‘how’ different they are and whetehr we are happy to generalise this difference back to the larger (target) population from which we sampled. 2) Are these observations reflective of the ………………………….?

Example Hypotheses: Isometric Torque Is there any difference in the length of time that males and females can sustain an isometric muscular contraction? Alternative Hypothesis There is a significant difference in the DV between males and females. Null Hypothesis There is not a significant difference in the DV between males and females We start with what we think will be true and state this as a hypothesis, then we must state a null hypothesis. IMPROTANTLY, THIS MUST COVER ALL BASES. The result of the study must support one of these two hypotheses- the easiest way to do this is often just to insert the word not or no somewhere. PLEASE NOTE THAT THIS IS WHAT IS KNOWN AS A 2 TAILED OR NON-DIRECTIONAL HYPOTHESIS- i.e. we are just predicting a difference between groups, not that one is higher than the other.

Example Hypotheses: Isometric Torque Is there any difference in the length of time that males and females can sustain an isometric muscular contraction? N♀ N♂ n♀ n♂ POOR AND INSUFFICIENT ARE NOTED HERE. Clearly a random sample will increase the likelyhood that n will equal N. However, it would also follow that the larger n is the more accurate it will be… 16 17 18 19 20 Sustained Isometric Torque (seconds)

Statistical Errors Type 1 Errors -Rejecting H0 when it is actually true -Concluding a difference when one does not actually exist Type 2 Errors -Accepting H0 when it is actually false (e.g. previous slide) -Concluding no difference when one does exist Non-parametric tests are more likely to commit type 2 errors (i.e. less powerful so can miss true significant differences)

Independent t-test: Calculation 16 17 18 19 20 Sustained Isometric Torque (seconds) Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72 Now I know that many of you have been questioning the value of SPSS over excel but this manual calculation should demonstrate why SPSS can be so useful

Independent t-test: Calculation Step 1: Calculate the Standard Error for Each Mean SEM♀ = SD/√n = SEM♂ = SD/√n = Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72

Independent t-test: Calculation Step 2: Calculate the Standard Error for the difference in means SEMdiff = √ SEM♀2 + SEM♂2 = Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72

Independent t-test: Calculation Step 3: Calculate the t statistic t = (Mean♀ - Mean♂) / SEMdiff = Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72 So basically recruiting more subjects and controlling the experiment will make you more likely to find a significant difference. VIP TO NOTE FOR LATER THAT BIG VARIANCE IS THEREFORE A BAD THING BUT WE STILL NEED TO CONVERT OUR t VALUE INTO A P VALUE…

Independent t-test: Calculation Step 4: Calculate the degrees of freedom (df) df = (n♀ - 1) + (n♂ - 1) = Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72 We calculate df quite simply as follows and then only use it to determine a critical value for t.

Independent t-test: Calculation Step 5: Determine the critical value for t using a t-distribution table Degrees of Freedom Critical t-ratio 44 46 48 50 2.015 2.013 2.011 2.009 Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72 Note that you should always choose the column for 0.05 and for a 2 tailed test. Importantly, NOTE THE CONTINUED IMPORTANCE OF SAMPLE SIZE, THE MORE SUBJECTS WE HAVE THE LOWER THE CRITICAL VALUE GETS. THIS IS RELEVANT BECAUSE WE NEED OUR CALCULATED t TO EXCEED THE CRITICAL t TO CONCLUDE A STATISTICAL DIFFERENCE…

Independent t-test: Calculation Step 6 finished: Compare t calculated with t critical Calculated t = Critical t = Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72

Independent t-test: Calculation Evaluation: The wealth of available literature supports that females can sustain isometric contractions longer than males. This may suggest that the findings of the present study represent a type error Possible solution: Mean SD n ♀ 18.5 1.74 25 ♂ 17.5 1.72

Independent t-test: SPSS Output Swim Data from SPSS session 8 NOT THE SAME DATA AS WE JUST DID MANUALLY

Advantages of using Paired Data Data from independent samples is heavily influenced by variance between subjects i.e. SD is large because one individual performed higher than another but in both trials!!!

Paired t-test: Calculation …a paired t-test can use the specific differences between each pair rather than just subtracting mean A from mean B (see earlier step 3) Mean SD n Week 1 61.6 56.6 8 Week 2 65.5 57.5

Paired t-test: Calculation Subject Week 1 Week 2 Diff (D) Diff2 (D2) 1 10 12 2 50 52 3 20 25 4 8 5 115 120 6 75 80 7 45 170 175 Steps 1 & 2: Complete this table ∑D = ∑D2 =

Paired t-test: Calculation Step 3: Calculate the t statistic t = n x ∑D2 – (∑D)2 = √ (n - 1) ∑D

Paired t-test: Calculation Steps 4 & 5: Calculate the df and use a t-distribution table to find t critical Critical t-ratio (0.05 level) Critical t-ratio (0.01 level) Degrees of Freedom 1 2 3 4 5 6 7 8 9 12.71 4.303 3.182 2.776 2.571 2.447 2.365 2.306 2.262 63.657 9.925 5.841 4.604 4.032 3.707 3.499 3.355 3.250 Same table as before but now I am showing you an extra column

Paired t-test: Calculation Step 6 finished: Compare t calculated with t critical Calculated t = Critical t = Mean SD n Week 1 61.6 56.6 8 Week 2 65.5 57.5

Paired t-test: SPSS Output Push-up Data from lecture 3

Example Hypotheses: Isometric Torque Is there any difference in the length of time that males and females can sustain an isometric muscular contraction? t-test Mean A Mean B 16 17 18 19 20 Sustained Isometric Torque (seconds)

Example Hypotheses: Isometric Torque Is there any difference in the length of time that males and females can sustain an isometric muscular contraction? Mean A Mean B Comparing the means does not give a valid reflection of the group differences. 16 17 18 19 20 Sustained Isometric Torque (seconds)

…assumptions of parametric analyses All data or paired differences are ND (this is the main consideration) N acquired through random sampling Data must be of at least the interval LOM Data must be Continuous.

Non-Parametric Tests These tests use the median and do not assume anything about distribution, i.e. ‘distribution free’ Mathematically, value is ignored (i.e. the magnitude of differences are not compared) Instead, data is analysed simply according to rank. This also now shows why ordinal data should be analysed using these tests.

Non-Parametric Tests Independent Measures Repeated Measures Mann-Whitney Test Repeated Measures Wilcoxon Test This also now shows why ordinal data should be analysed using these tests.

Mann-Whitney U: Calculation Step 1: Rank all the data from both groups in one series, then total each School A School B Student Grade Rank Student Grade Rank J. S. L. D. H. L. M. J. T. M. T. S. P. H. B- B- A+ D- B+ A- F 9 9 14 3 11 12.5 1 T. J. M. M. K. S. P. S. R. M. P. W. A. F. D C+ C+ B- E C- A- 4 6.5 6.5 9 2 5 12.5 Median = ; ∑RA = Median = ; ∑RB =

Mann-Whitney U: Calculation Step 2: Calculate two versions of the U statistic using: U1 = (nA x nB) + (nA + 1) x nA - ∑RA 2 AND… U2 = (nA x nB) + (nB + 1) x nB - ∑RB 2

Mann-Whitney U: Calculation Step 3 finished: Select the smaller of the two U statistics (U1 = ………; U2 = ……..) …now consult a table of critical values for the Mann-Whitney test n 0.05 0.01 6 5 2 7 8 4 8 13 7 9 17 11 Conclusion Median A Median B Calculated U must be critical U to conclude a significant difference

Mann-Whitney U: SPSS Output

Wilcoxon Signed Ranks: Calculation Step 1: Rank all the diffs from in one series (ignoring signs), then total each Pre-training OBLA (kph) Post-training OBLA (kph) Athlete Diff. Rank Signed Ranks - + J. S. L. D. H. L. M. J. T. M. T. S. P. H. 15.6 17.2 17.7 16.5 15.9 16.7 17.0 16.1 17.5 16.7 16.8 16.0 16.5 17.1 0.5 0.3 -1 0.3 0.1 -0.2 0.1 6 4.5 -7 4.5 1.5 -3 1.5 6 4.5 4.5 1.5 1.5 -7 -3 Medians = ∑Signed Ranks =

Wilcoxon Signed Ranks: Calculation Step 2: The smaller of the T values is our test statistic (T+ = ….....; T- = ……) …now consult a table of critical values for the Wilcoxon test n 0.05 6 7 2 8 3 9 5 Conclusion Median A Median B Calculated T must be critical T to conclude a significant difference

Wilcoxon Signed Ranks: SPSS Output

So which stats test should you use? Q1. What is the …………? Q2. Is the data …….? Q3. Is the data …………….. or ……………..?