KNR 405 Intro & Validity Slide 1 KNR 405 Applied Motor Learning

KNR 405 Critiquing Research Slide 2 Applied Motor Learning  What’s it about  The web site   The general structure...  Get a syllabus and read it  Look particularly at the course schedule  3 or 4 bits to it

KNR 405 Critiquing Research Slide 3 Applied Motor Learning  The general structure...  3 or 4 bits to it  Research critiquing  Introduction (motor control)  (mostly) Motor Learning & (some) Sport Psychology research readings  Summary – overall message

KNR 405 Critiquing Research Slide 4 Applied Motor Learning  So, now the first bit...  Research critiquing

KNR 405 Critiquing Research Slide 5 Elements of research design  Operationalization  What do you want to measure?  Whatever it is, you’ve got to choose a way to measure it  When you do, you will operationalize it  Whether or not you’ve made a good choice is determined by measurement validity  If you operationalize well, you should have good construct validity

KNR 405 Critiquing Research Slide 6 Elements of research design  Independent variable  What you or nature manipulates in some way  E.g. 1: What happens when you get older?  Age is the independent variable (nature is the manipulator)  E.g. 2: What happens when you drink?  Blood alcohol level is the IV (you are the manipulator)  Critiquing IVs: Exhaustive? Mutually exclusive attributes? See also construct validity…

KNR 405 Critiquing Research Slide 7 Elements of research design  Dependent variable  The thing that is influenced (changed) by your independent variable  E.g. 1 (IV = Age): Skin sag, baldness, frequency of urine expulsion, memory strength  E.g. 2 (IV = Alcohol consumption): Balance, inhibition, frequency of urine expulsion  Can you think of any others?  Critiquing DV’s: see operationalization, reliability, measurement validity (all later)

KNR 405 Critiquing Research Slide 8 Elements of research design  Hypothesis  A specific statement of prediction  Inductive vs. deductive research  Deductive has ‘em, inductive often doesn’t  Types  One-tailed vs. two-tailed  Directional vs. non-directional  Association vs. difference  Hypothetical-deductive model

KNR 405 Critiquing Research Slide 9 Validity as ‘aspects of truth’  Validity: the best available approximation to the truth* of a given proposition, inference, or conclusion *This allows for criticism – which is where we come in Conclusion Internal External Construct

KNR 405 Critiquing Research Slide 10 Validity - General  Principles of validity

KNR 405 Critiquing Research Slide 11 Conclusion Validity  Principles:  Conclusion validity  “Conclusion validity is the degree to which conclusions we reach about relationships in our data are reasonable”  Two possible problems:  conclude that there is no relationship when in fact there is (you missed the relationship or didn't see it)  conclude that there is a relationship when in fact there is not (you're seeing things that aren't there!)

KNR 405 Critiquing Research Slide 12 Conclusion Validity  Principles:  Conclusion validity  conclude that there is no relationship when in fact there is (you missed the relationship or didn't see it)  Low reliability  poor reliability of treatment implementation  random irrelevancies in the setting  random heterogeneity of respondents  Low statistical power  Sample size, effect size, alpha level, power

KNR 405 Critiquing Research Slide 13 Conclusion Validity  Principles:  Conclusion validity  conclude that there is a relationship when in fact there is not (you're seeing things that aren't there!)  fishing and the error rate problem  Too many analyses conducted at an inappropriate alpha level

KNR 405 Critiquing Research Slide 14 Conclusion Validity  Principles:  Conclusion validity  Using stats the wrong way can lead to either problem – violate statistical assumptions and the tests don’t work properly (so you can’t have faith in your findings)

KNR 405 Critiquing Research Slide 15 Conclusion Validity  Principles:  Improving Conclusion validity  Good statistical power  Good reliability  Good implementation

KNR 405 Critiquing Research Slide 16 Internal Validity

KNR 405 Critiquing Research Slide 17 Internal Validity Single-group threats – taken care of by adding control group Multiple-group threats – taken care of by random assignment Social interaction threats

KNR 405 Critiquing Research Slide 18 Internal Validity  Principles:  Internal validity  First the design – what type of threats should we be looking for?  See handout  Use the internal validity of the design to guide your discussion of the likelihood of alternative plausible explanations of the relationship examined in the study

KNR 405 Critiquing Research Slide 19 Internal Validity  Principles:  Internal validity  Use the internal validity of the design to guide your discussion of the likelihood of alternative plausible explanations of the relationship examined in the study  Note – the key word is plausible  Also, note that you are trying to suggest an alternative reason why the relationship being studied might come about

KNR 405 Critiquing Research Slide 20 Construct Validity: Critiquing  Determined by Operationalization

KNR 405 Critiquing Research Slide 21 Construct Validity  Principles:  Construct validity  Here the dependent and independent variables must be considered  State what they are first, and what they are purporting to measure, then proceed to critique whether they do the job

KNR 405 Critiquing Research Slide 22 External Validity  Principles:  External validity  Think of the goals for generalization of the study, and try to evaluate whether there are exceptions (important instances in which the expected relationship might not be found)

KNR 405 Critiquing Research Slide 23 Validity  Proximal similarity model (Campbell, 1963)