1. Introduction to Study Skills & Research Methods (HL10040)
Measurement Errors
Introduction to Study Skills & Research Methods (HL10040)
Dr James Betts

2. Lecture Outline: Measurement Errors Continued Types of Errors
Assessment of Error
Introduction to Inferential Statistics
Chi-Squared tests
Assessment Details.

3. Measured Score = ‘True’ Score  Error
Measurement Errors
Virtually all measurements have errors
i.e.
Measured Score = ‘True’ Score  Error
Therefore inherently linked to SD
Reliability and Measurement Error are not the same, rather Reliability infers an acceptable degree of Measurement Error.

4. This variability between methods is caused by both systematic and error factors
Direct Record
Retrospective Recall SD

5. Caused by systematic error
(SD2)
Total
Variance
Systematic
Variance
Caused by systematic error
This total variance can then be ‘partitioned’
Error
Variance
Caused by random error

6. Types of Errors Systematic Error Random Error
Any variable causing a consistent shift in the mean in a given direction
e.g. Retrospective diet records tend to omit the snacks between meals
Random Error
The fluctuation of scores due to chance
e.g. Innaccurate descriptions of the food consumed

7. Systematic Error 10 12 8 11 17 22 14 % Body-fat Skin-Fold Callipers
Subject 1
Subject 2
Subject 3
Subject 4 10 12 8 11 17 22 14
Skin-Fold Callipers
Hydrostatic Weighing

8. Random Error 14 18 10 9 11 15 21 17 % Body-fat Skin-Fold Callipers
Subject 1
Subject 2
Subject 3
Subject 4 14 18 10 9 11 15 21 17
Skin-Fold Callipers
Hydrostatic Weighing

9. Evidence of bias between means
Assessment of Error
Systematic Error
Evidence of bias between means

10. In general, good agreement requires r > 0.7
Assessment of Error
Random Error
r = 0 infers lots of error
r = 1 infers no error
In general, good agreement requires r > 0.7
r2 = 0.278

11. Assessment of Error Systematic & Random Error Callipers HydroStat.

12. Assessment of Error Systematic & Random Error
Callipers HydroStat. Difference Mean

13. The “Bland-Altman” Plot 3 points of visual assessment:
Assessment of Error
Systematic & Random Error
The “Bland-Altman” Plot points of visual assessment:
-Systematic Error: are points evenly distributed about the zero line?
-Random Error: do points deviate greatly from the mean line?
-Nature of error: is the error consistent left-right?

14. Examples of Bland-Altman Plots
Very Little Systematic Error
Very Little Random Error
Zero
Mean difference

15. Examples of Bland-Altman Plots
Some Systematic Error
Very Little Random Error
Mean difference
Zero

16. Examples of Bland-Altman Plots
Very Little Systematic Error
Some Random Error
Zero
Mean difference

17. Examples of Bland-Altman Plots
Mean difference
Some Systematic Error
Some Random Error
Zero

18. Examples of Bland-Altman Plots
Nature of Error:
Funnelling Effect?
Zero

19. Why is Error Important
Measurement Error is clearly of importance when evaluating the agreement between two measurement tools
A consideration of error is also relevant when attempting to establish intervention effects/treatment differences
i.e. where some of the variance between trials is due to the independent variable...

20. Dependent Variable
Independent
Variable
Total Variance
between trial 1
& trial 2
Systematic
Variance
Extraneous/
Confounding
(Error) Variables
Error
Variance

21. Dependent Variable
Independent
Variable
Total Variance
between trial 1
& trial 2
Primary
Variance
Systematic
Variance
Extraneous/
Confounding
(Error) Variables
Systematic
Variance
Error
Variance
So researchers strive to increase the proportion of variance due to IV.

22. Maximise effect (20 pints?)
Dependent Variable
Independent
Variable
Total Variance
between trial 1
& trial 2
Primary
Variance
Maximise effect (20 pints?)
Extraneous/
Confounding
(Error) Variables
Systematic
Variance
Increase control
Error Variance
So researchers strive to increase the proportion of variance due to IV.

23. Smallest Worthwhile Effect
It would appear that even a small amount of primary variance from an ergogenic aid would guarantee victory to either competitor…
…however, the error variance is such that a re-run could produce entirely different results…

24. Re-Run …for an effect to be considered ‘worthwhile’,
it would need to exceed the opponents time by more than his error variance.
Re-Run

25. Dependent Variable
Extraneous/
Confounding
(Error) Variables
Total Variance
between trial 1
& trial 2
Systematic
Variance
Error Variance
What is the probability of observing your variance if IV does nothing?

26. Scientific Reasoning (Logic)
I’m a
(talking)
swan
All swans
are white
Confirmation of a theory from your own observations
Deductive Reasoning
General Theory
Specific Observation
Inductive Reasoning
Formation of a theory grounded in your own observations
Stats tests give us the probability of seeing this evidence assuming this general rule is true
…NOT the probability that the general rule is true based on your observations

27. Introduction to Inferential Statistics
Before our next lecture you will be conducting some inferential statistics in your lab classes…
All you need to be aware of at this stage is that the ‘p-value’ represents the probability of the observed variance occurring if the null hypothesis is true
i.e. p = 0.01 infers a 1 % probability of making your observation if in fact the IV has no effect

28. Introduction to Inferential Statistics
Before our next lecture you will be conducting some inferential statistics in your lab classes…
All you need to be aware of at this stage is that the ‘p-value’ represents the probability of the observed variance occurring if the null hypothesis is true
i.e. p = 0.10 infers a 10 % probability of making your observation if in fact the IV has no effect

29. Introduction to Inferential Statistics
Before our next lecture you will be conducting some inferential statistics in your lab classes…
All you need to be aware of at this stage is that the ‘p-value’ represents the probability of the observed variance occurring if the null hypothesis is true
i.e. p = 0.05 infers a 5 % probability of making your observation if in fact the IV has no effect
n.b. this DOES NOT mean that you will find this result in 95/100 test-retests or that your false positive rate is 5 %

30. https://aeon.co/essays/it-s-time-for-science-to-abandon-the-term-statistically-significant

31. Quantitative Analysis of Nominal Data
Recall that nominal data infers that variables are dichotomous, i.e. belong to distinct categories
e.g. Athlete/Non-Athlete, Male/Female, etc.
We know that such qualitative data can be coded quantitatively to allow a more objective analysis
Nominal data does not require any consideration of normality and is analysed used a Chi2 test.

32. The Chi-Squared Test Goodness of fit χ2 test Contingency χ2 test
A comparison of your observed frequency counts against what would be expected according to the null hypothesis
i.e. null hypothesis infers equal dispersion (50:50)
Contingency χ2 test
A comparison of two observed frequency counts

33. Goodness of fit χ2 test
Is a leisure centre used more by males than by females?
n =150
Observed Frequency
Expected
Frequency
Male 62 75
Female 88

34. P-value AKA significance level
Goodness of fit χ2 test
SPSS Output
i.e. significant difference in the proportion of users according to gender
P-value AKA significance level

35. Do not take supplements
Contingency χ2 test
Are elite athletes more likely to take nutritional supplements than non-athletes
n =60
Do take supplements
Do not take supplements
Athletes 18 12
Non-athletes 11 19

36. This is the test of interest
Contingency χ2 test
SPSS Output
This is the test of interest
i.e. no significant difference in the proportion of users according to group

37. Assumptions for Chi-Squared
Although ND not required…
Cells in the table should all be independent
i.e. one person could have visited the leisure centre twice
80 % of the cells must have expected frequencies greater than 5 and all must be above 1
i.e. the more categories available, the more subjects needed
Cannot use percentages
i.e. a 15:45 split cannot be expressed as 25%:75%

38. Selected Reading
I know error and variance can be confusing topics, try these:
Atkinson, G. and A. M. Nevill. Statistical methods for assessing measurement error (Reliability) in variables relevant to sports medicine. Sports Medicine. 26: , 1998.
Hopkins, W. G. et al. Design and analysis of research on sport performance enhancement. Med. Sci. Sport and Exerc. 31: , 1999.
Hopkins, W. G. et al. Reliability of power in physical performance tests. Sports Medicine. 31: , 2001.
Atkinson, G., ''What is this thing called measurement error?'' , in Kinanthropometry VIII: Proceedings of the 8th International Conference of the International Society for the Advancement of Kinanthropometry (ISAK) , Reilly, T. and Marfell-Jones, M. (Eds.), Taylor and Francis, London , 2003.

39. Coursework (60% overall grade)
Your coursework will require you to address 2 of the following research scenarios:
1) Effect of Plyometric Training on Vertical Jump
2) Effect of Ice Baths on Recovery of Strength
3) Effect of Diet on the Incidence of Muscle Injury
4) Effect of Footwear on Sprint Acceleration
5) Effect of PMR on Competitive Anxiety.

40. Coursework Outline For each of the 2 scenarios you will need to:
Perform a literature search in order to provide a comprehensive introduction to the research area
Identify the variables of interest and evaluate the research design which was adopted
Formulate and state appropriate hypotheses
Summarise descriptive statistics in an appropriate and well presented manner…

41. Coursework Outline Cont’d…
Select the most appropriate statistical test with justification for your decision
Transfer the output of your inferential statistics into your word document
Interpret your results and discuss the validity and reliability of the study
Draw a meaningful conclusion (state whether hypotheses are accepted or rejected).

42. Coursework Details (see unit outline)
2000 words maximum (i.e. 1000 for each)
Any supporting SPSS data/outputs to be appended
Assessment Weighting
Evaluation & Analysis (30 %)
Reading & Research (20 %)
Communication & Presentation (20 %)
Knowledge (30 %)

43. Web address also referenced on shared area
Coursework Details
All information relating to your coursework (including the relevant data files) are accessible via the unit web page:
Web address also referenced on shared area

44. How far does the bird fly between the trains before they collide?
120 mph
100 miles
30 mph
20 mph
Daniel Dennett (2013) Intuition Pumps and Other tools for Critical Thinking

45. How far does the bird fly between the trains before they collide?
120 mph
86 miles
65 miles
30 mph
20 mph
120+20=140
100/140 =
*120 = 86
*20 = 14
86+14=100
So the bird will have travelled 86 miles before getting to the slower train and turning around,
That will have taken 42.8 minutes, so train 1 will have travelled 21.4 miles and so now there remains: 100 – ( ) = 64.6 miles between the trains, and you can start over, etc etc etc (i.e. summing an infinite series)
Or just ignore the bird and know that the two trains’ combined speed is 50 mph, so they will collide in 2 h – thus bird will travel 240 miles
Daniel Dennett (2013) Intuition Pumps and Other tools for Critical Thinking