Introduction to Study Skills & Research Methods (HL10040) Measurement Errors Introduction to Study Skills & Research Methods (HL10040) Dr James Betts

Lecture Outline: Measurement Errors Continued Types of Errors Assessment of Error Introduction to Inferential Statistics Chi-Squared tests Assessment Details.

Measured Score = ‘True’ Score  Error Measurement Errors Virtually all measurements have errors i.e. Measured Score = ‘True’ Score  Error Therefore inherently linked to SD Reliability and Measurement Error are not the same, rather Reliability infers an acceptable degree of Measurement Error.

This variability between methods is caused by both systematic and error factors Direct Record Retrospective Recall SD

Caused by systematic error (SD2) Total Variance Systematic Variance Caused by systematic error This total variance can then be ‘partitioned’ Error Variance Caused by random error

Types of Errors Systematic Error Random Error Any variable causing a consistent shift in the mean in a given direction e.g. Retrospective diet records tend to omit the snacks between meals Random Error The fluctuation of scores due to chance e.g. Innaccurate descriptions of the food consumed

Systematic Error 10 12 8 11 17 22 14 % Body-fat Skin-Fold Callipers Subject 1 Subject 2 Subject 3 Subject 4 10 12 8 11 17 22 14 Skin-Fold Callipers Hydrostatic Weighing

Random Error 14 18 10 9 11 15 21 17 % Body-fat Skin-Fold Callipers Subject 1 Subject 2 Subject 3 Subject 4 14 18 10 9 11 15 21 17 Skin-Fold Callipers Hydrostatic Weighing

Evidence of bias between means Assessment of Error Systematic Error Evidence of bias between means

In general, good agreement requires r > 0.7 Assessment of Error Random Error r = 0 infers lots of error r = 1 infers no error In general, good agreement requires r > 0.7 r2 = 0.278

Assessment of Error Systematic & Random Error Callipers HydroStat. 10.00 17.00 12.00 22.00 8.00 14.00 11.00 12.00 14.00 11.00 18.00 15.00 10.00 21.00 9.00 17.00

Assessment of Error Systematic & Random Error Callipers HydroStat. Difference Mean 10.00 17.00 7.00 13.50 12.00 22.00 10.00 17.00 8.00 14.00 6.00 11.00 11.00 12.00 1.00 11.50 14.00 11.00 -3.00 12.50 18.00 15.00 -3.00 16.50 10.00 21.00 11.00 15.50 9.00 17.00 8.00 13.00

The “Bland-Altman” Plot 3 points of visual assessment: Assessment of Error Systematic & Random Error The “Bland-Altman” Plot 3 points of visual assessment: -Systematic Error: are points evenly distributed about the zero line? -Random Error: do points deviate greatly from the mean line? -Nature of error: is the error consistent left-right?

Examples of Bland-Altman Plots Very Little Systematic Error Very Little Random Error Zero Mean difference

Examples of Bland-Altman Plots Some Systematic Error Very Little Random Error Mean difference Zero

Examples of Bland-Altman Plots Very Little Systematic Error Some Random Error Zero Mean difference

Examples of Bland-Altman Plots Mean difference Some Systematic Error Some Random Error Zero

Examples of Bland-Altman Plots Nature of Error: Funnelling Effect? Zero

Why is Error Important Measurement Error is clearly of importance when evaluating the agreement between two measurement tools A consideration of error is also relevant when attempting to establish intervention effects/treatment differences i.e. where some of the variance between trials is due to the independent variable...

Dependent Variable Independent Variable Total Variance between trial 1 & trial 2 Systematic Variance Extraneous/ Confounding (Error) Variables Error Variance

Dependent Variable Independent Variable Total Variance between trial 1 & trial 2 Primary Variance Systematic Variance Extraneous/ Confounding (Error) Variables Systematic Variance Error Variance So researchers strive to increase the proportion of variance due to IV.

Maximise effect (20 pints?) Dependent Variable Independent Variable Total Variance between trial 1 & trial 2 Primary Variance Maximise effect (20 pints?) Extraneous/ Confounding (Error) Variables Systematic Variance Increase control Error Variance So researchers strive to increase the proportion of variance due to IV.

Smallest Worthwhile Effect It would appear that even a small amount of primary variance from an ergogenic aid would guarantee victory to either competitor… …however, the error variance is such that a re-run could produce entirely different results…

Re-Run …for an effect to be considered ‘worthwhile’, it would need to exceed the opponents time by more than his error variance. Re-Run

Dependent Variable Extraneous/ Confounding (Error) Variables Total Variance between trial 1 & trial 2 Systematic Variance Error Variance What is the probability of observing your variance if IV does nothing?

Scientific Reasoning (Logic) I’m a (talking) swan All swans are white Confirmation of a theory from your own observations Deductive Reasoning General Theory Specific Observation Inductive Reasoning Formation of a theory grounded in your own observations Stats tests give us the probability of seeing this evidence assuming this general rule is true …NOT the probability that the general rule is true based on your observations

Introduction to Inferential Statistics Before our next lecture you will be conducting some inferential statistics in your lab classes… All you need to be aware of at this stage is that the ‘p-value’ represents the probability of the observed variance occurring if the null hypothesis is true i.e. p = 0.01 infers a 1 % probability of making your observation if in fact the IV has no effect

Introduction to Inferential Statistics Before our next lecture you will be conducting some inferential statistics in your lab classes… All you need to be aware of at this stage is that the ‘p-value’ represents the probability of the observed variance occurring if the null hypothesis is true i.e. p = 0.10 infers a 10 % probability of making your observation if in fact the IV has no effect

Introduction to Inferential Statistics Before our next lecture you will be conducting some inferential statistics in your lab classes… All you need to be aware of at this stage is that the ‘p-value’ represents the probability of the observed variance occurring if the null hypothesis is true i.e. p = 0.05 infers a 5 % probability of making your observation if in fact the IV has no effect n.b. this DOES NOT mean that you will find this result in 95/100 test-retests or that your false positive rate is 5 %

https://aeon.co/essays/it-s-time-for-science-to-abandon-the-term-statistically-significant

Quantitative Analysis of Nominal Data Recall that nominal data infers that variables are dichotomous, i.e. belong to distinct categories e.g. Athlete/Non-Athlete, Male/Female, etc. We know that such qualitative data can be coded quantitatively to allow a more objective analysis Nominal data does not require any consideration of normality and is analysed used a Chi2 test.

The Chi-Squared Test Goodness of fit χ2 test Contingency χ2 test A comparison of your observed frequency counts against what would be expected according to the null hypothesis i.e. null hypothesis infers equal dispersion (50:50) Contingency χ2 test A comparison of two observed frequency counts

Goodness of fit χ2 test Is a leisure centre used more by males than by females? n =150 Observed Frequency Expected Frequency Male 62 75 Female 88

P-value AKA significance level Goodness of fit χ2 test SPSS Output i.e. significant difference in the proportion of users according to gender P-value AKA significance level

Do not take supplements Contingency χ2 test Are elite athletes more likely to take nutritional supplements than non-athletes n =60 Do take supplements Do not take supplements Athletes 18 12 Non-athletes 11 19

This is the test of interest Contingency χ2 test SPSS Output This is the test of interest i.e. no significant difference in the proportion of users according to group

Assumptions for Chi-Squared Although ND not required… Cells in the table should all be independent i.e. one person could have visited the leisure centre twice 80 % of the cells must have expected frequencies greater than 5 and all must be above 1 i.e. the more categories available, the more subjects needed Cannot use percentages i.e. a 15:45 split cannot be expressed as 25%:75%

Selected Reading I know error and variance can be confusing topics, try these: Atkinson, G. and A. M. Nevill. Statistical methods for assessing measurement error (Reliability) in variables relevant to sports medicine. Sports Medicine. 26:217-238, 1998. Hopkins, W. G. et al. Design and analysis of research on sport performance enhancement. Med. Sci. Sport and Exerc. 31:472-485, 1999. Hopkins, W. G. et al. Reliability of power in physical performance tests. Sports Medicine. 31:211-234, 2001. Atkinson, G., ''What is this thing called measurement error?'' , in Kinanthropometry VIII: Proceedings of the 8th International Conference of the International Society for the Advancement of Kinanthropometry (ISAK) , Reilly, T. and Marfell-Jones, M. (Eds.), Taylor and Francis, London , 2003.

Coursework (60% overall grade) Your coursework will require you to address 2 of the following research scenarios: 1) Effect of Plyometric Training on Vertical Jump 2) Effect of Ice Baths on Recovery of Strength 3) Effect of Diet on the Incidence of Muscle Injury 4) Effect of Footwear on Sprint Acceleration 5) Effect of PMR on Competitive Anxiety.

Coursework Outline For each of the 2 scenarios you will need to: Perform a literature search in order to provide a comprehensive introduction to the research area Identify the variables of interest and evaluate the research design which was adopted Formulate and state appropriate hypotheses Summarise descriptive statistics in an appropriate and well presented manner…

Coursework Outline Cont’d… Select the most appropriate statistical test with justification for your decision Transfer the output of your inferential statistics into your word document Interpret your results and discuss the validity and reliability of the study Draw a meaningful conclusion (state whether hypotheses are accepted or rejected).

Coursework Details (see unit outline) 2000 words maximum (i.e. 1000 for each) Any supporting SPSS data/outputs to be appended Assessment Weighting Evaluation & Analysis (30 %) Reading & Research (20 %) Communication & Presentation (20 %) Knowledge (30 %)

Web address also referenced on shared area Coursework Details All information relating to your coursework (including the relevant data files) are accessible via the unit web page: www.bath.ac.uk/~jb335/Y1%20Research%20Skills%20(FH10040).html Web address also referenced on shared area

How far does the bird fly between the trains before they collide? 120 mph 100 miles 30 mph 20 mph Daniel Dennett (2013) Intuition Pumps and Other tools for Critical Thinking

How far does the bird fly between the trains before they collide? 120 mph 86 miles 65 miles 30 mph 20 mph 120+20=140 100/140 = 0.71428 0.71428*120 = 86 0.71428*20 = 14 86+14=100 So the bird will have travelled 86 miles before getting to the slower train and turning around, That will have taken 42.8 minutes, so train 1 will have travelled 21.4 miles and so now there remains: 100 – (21.4+14) = 64.6 miles between the trains, and you can start over, etc etc etc (i.e. summing an infinite series) Or just ignore the bird and know that the two trains’ combined speed is 50 mph, so they will collide in 2 h – thus bird will travel 240 miles Daniel Dennett (2013) Intuition Pumps and Other tools for Critical Thinking

J.Betts@bath.ac.uk