Test-Retest Reliability (ICC) and Day to Day Variation.

Slides:

Advertisements

Similar presentations

Descriptive Statistics

Advertisements

Measurement Concepts Operational Definition: is the definition of a variable in terms of the actual procedures used by the researcher to measure and/or.

Correlation, Reliability and Regression Chapter 7.

RELIABILITY Reliability refers to the consistency of a test or measurement. Reliability studies Test-retest reliability Equipment and/or procedures Intra-

Correlation Chapter 6. Assumptions for Pearson r X and Y should be interval or ratio. X and Y should be normally distributed. Each X should be independent.

© 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Validity and Reliability Chapter Eight.

Assessment Procedures for Counselors and Helping Professionals, 7e © 2010 Pearson Education, Inc. All rights reserved. Chapter 5 Reliability.

VALIDITY AND RELIABILITY

Aim: How do we find confidence interval using SPSS? SPSS Assignment 3 due Thursday.

2005 All Hands Meeting Measuring Reliability: The Intraclass Correlation Coefficient Lee Friedman, Ph.D.

Measuring Research Variables KNES 510 Research Methods in Kinesiology 1.

Dependent t-Test CJ 526 Statistical Analysis in Criminal Justice.

RELIABILITY consistency or reproducibility of a test score (or measurement)

AEB 37 / AE 802 Marketing Research Methods Week 5

Matching level of measurement to statistical procedures

Correlations and T-tests

Analyzing quantitative data – section III Week 10 Lecture 1.

Correlation Analysis 5th - 9th December 2011, Rome.

Week 14 Chapter 16 – Partial Correlation and Multiple Regression and Correlation.

Repeated Measures ANOVA Used when the research design contains one factor on which participants are measured more than twice (dependent, or within- groups.

Social Science Research Design and Statistics, 2/e Alfred P. Rovai, Jason D. Baker, and Michael K. Ponton Internal Consistency Reliability Analysis PowerPoint.

Analysis of Variance with Repeated Measures. Repeated Measurements: Within Subjects Factors Repeated measurements on a subject are called within subjects.

Dr Sally Boa and Dr Joan Murphy Professor Pam Enderby Funded by NHS Education Scotland Conducted by Talking Mats Limited © Talking Mats Ltd 2014.

USING DATA MANAGEMENT PLANS as a RESEARCH TOOL for IMPROVING DATA SERVICES in ACADEMIC LIBRARIES Jake Carlson, Patricia Hswe & Susan Wells Parham Amanda.

Statistical Methods for Multicenter Inter-rater Reliability Study

Inference for regression - Simple linear regression

1 Cronbach’s Alpha It is very common in psychological research to collect multiple measures of the same construct. For example, in a questionnaire designed.

1 rules of engagement no computer or no power → no lesson no SPSS → no lesson no homework done → no lesson GE 5 Tutorial 5.

Rater Reliability How Good is Your Coding?. Why Estimate Reliability? Quality of your data Number of coders or raters needed Reviewers/Grant Applications.

Assessing Learners with Special Needs: An Applied Approach, 6e © 2009 Pearson Education, Inc. All rights reserved. Chapter 4:Reliability and Validity.

Questionnaire Development: SPSS and Reliability Personality Lab October 8, 2010.

Then click the box for Normal probability plot. In the box labeled Standardized Residual Plots, first click the checkbox for Histogram, Multiple Linear.

Reliability & Agreement DeShon Internal Consistency Reliability Parallel forms reliability Parallel forms reliability Split-Half reliability Split-Half.

Reliability vs. Validity.  Reliability  the consistency of your measurement, or the degree to which an instrument measures the same way each time it.

What is SPSS  SPSS is a program software used for statistical analysis.  Statistical Package for Social Sciences.

Perform Descriptive Statistics Section 6. Descriptive Statistics Descriptive statistics describe the status of variables. How you describe the status.

Inter-rater reliability in the KPG exams The Writing Production and Mediation Module.

Measures of Reliability in Sports Medicine and Science Will G. Hopkins Sports Medicine 30(4): 1-25, 2000.

Examining Rubric Design and Inter-rater Reliability: a Fun Grading Project Presented at the Third Annual Association for the Assessment of Learning in.

Slide 1 Mixed ANOVA (GLM 5) Chapter 15. Slide 2 Mixed ANOVA Mixed: – 1 or more Independent variable uses the same participants – 1 or more Independent.

Correlation The apparent relation between two variables.

Calculating Inter-coder Reliability

Review of Factorial ANOVA, correlations and reliability tests COMM Fall, 2007 Nan Yu.

Conduct Simple Correlations Section 7. Correlation –A Pearson correlation analyzes relationships between parametric, linear (interval or ratio which are.

Reliability and Validity Themes in Psychology. Reliability Reliability of measurement instrument: the extent to which it gives consistent measurements.

Dynamic Connectivity: Pitfalls and Promises

Using SPSS Note: The use of another statistical package such as Minitab is similar to using SPSS.

Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 12 Analyzing the Association Between Quantitative Variables: Regression Analysis Section.

Chapter 6 Norm-Referenced Reliability and Validity.

Analyzing Data. Learning Objectives You will learn to: – Import from excel – Add, move, recode, label, and compute variables – Perform descriptive analyses.

Lab 4: Alpha and Standard Error of Measurement. Reliability Reliability refers to consistency Types of reliability estimates – Test-retest reliability.

1 Measuring Agreement. 2 Introduction Different types of agreement Diagnosis by different methods  Do both methods give the same results? Disease absent.

SPSS Statistical Package for Social Sciences Bivariate Correlations Department of Psychology California State University Northridge

Reliability Analysis.

Measures of Agreement Dundee Epidemiology and Biostatistics Unit

Coefficient of Determination

Reliability and Validity in Research

Using SPSS Note: The use of another statistical package such as Minitab is similar to using SPSS.

Week 14 Chapter 16 – Partial Correlation and Multiple Regression and Correlation.

Quick and Easy Alpha Reliability

CJ 526 Statistical Analysis in Criminal Justice

Descriptive Statistics

Reasoning in Psychology Using Statistics

Natalie Robinson Centre for Evidence-based Veterinary Medicine

Reliability Analysis.

The first test of validity

Reasoning in Psychology Using Statistics

Using SPSS Note: The use of another statistical package such as Minitab is similar to using SPSS.

Qualities of a good data gathering procedures

Presentation transcript:

Test-Retest Reliability (ICC) and Day to Day Variation

Consistency of Measurements Reliability refers to the consistency of a test or measurement. A test cannot be considered valid if it is not reliable. You should know the day to day variation in your dependent variable. How much does 1RM bench press change from day to day? How much does VO2 max change from day to day?

Test-Retest Reliability The 1RM squat strength was measured on 8 subjects on two separate days. In this experiment we want to measure the reliability of our measurements. Here is the day to day variation in 1RM bench press strength. SubjectTrial 1Trial 2Difference Mean SD In this experiment, we plan to average trials, so we will use Averaged Measures ICC.

Enter the between day variables.

Click Statistics

Click OK to run

SPSS Output

There is no difference between trials F(1,7) =.530, p =.49 If there is a difference between trials you may have a learning effect or a fatigue effect and you must modify your methods to control for learning and/or fatigue.

ICC =.973 [High Degree of Reliability] Using Mixed Model, Type Consistency, Averaged Measures.

If you train subjects for 8 weeks and they improve their strength by 2.8 Kg have you actually done anything? Is the program effective? Here is the day to day variation in 1RM bench press strength. SubjectTrial 1Trial 2Difference Mean SD Day to Day Variation

Manuscript Methods & Results Sections METHODS Measurement of Reliability Prior to initiating the study, 8 subjects (not in the experiment), participated in a test-retest assessment of measurement reliability. One RM bench press strength was measured on two separate days for subjects in the reliability study. Reproducibility of bench press strength was analyzed using SPSS (16.0 for Windows) to compute the intraclass correlation coefficient (ICC) using a two factor mixed effects model and type consistency (McGraw and Wong, 1996; Shrout and Fleiss, 1979). RESULTS A high degree of reliability was found between bench press strength measurements the average measure ICC was.973 with a 95% confidence interval from The mean between day variation for 1RM strength was 2.75±10.7 Kg. REFERENCES McGraw KO and Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods 1: 30-46, Shrout PE and Fleiss JL. Intraclass correlations: Uses in assessing reliability. Psychol Bull 86: , 1979.