Data and the Nature of Measurement Copyright © Allyn & Bacon (2007) Data and the Nature of Measurement Graziano and Raulin Research Methods: Chapter 4 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program. Graziano & Raulin (1997)

Copyright © Allyn & Bacon (2007) Research Variables Variable: Any characteristic that can take on more than one value Examples: speed, level of hostility, accuracy of feedback, reaction time Research is the study of the relationship among variables Therefore, there must be at least two variables in a research study (or there is no relationship to study)

Copyright © Allyn & Bacon (2007) Measuring Variables Measurement: Assigning numbers to indicate the level of a variable Sometimes the number assignment is easy to understand (e.g., time measured in seconds) Sometimes it is more arbitrary (e.g., 1 for male and 2 for female)

Copyright © Allyn & Bacon (2007) Scales of Measurement Based on how closely the measurement scale matches the real number system Scales of Measurement (Stevens, 1946, 1957) Nominal Ordinal Interval Ratio

Copyright © Allyn & Bacon (2007) Nominal Scales Naming scale Each number reflects a category Examples: diagnostic categories, political affiliations Produces nominal or categorical data Mathematical properties Identity

Copyright © Allyn & Bacon (2007) Ordinal Scales Scale indicating rank order Reflects the order, but not the amount Example: order of finish in a race, class rankings Produces ordered data Mathematical properties Identity Magnitude

Copyright © Allyn & Bacon (2007) Interval Scales Scale with equal intervals The scale indicates amount, but with no zero point Examples: temperature on the Celsius scale, most psychological tests Produces score data Mathematical properties Identity Magnitude Equal intervals

Copyright © Allyn & Bacon (2007) Ratio Scales Scale that fits the number system well Includes equal intervals and a true zero Examples: time, distance, frequency Produces score data Mathematical properties Identity Magnitude Equal intervals True zero

Copyright © Allyn & Bacon (2007) Psychological Tests Most psychological test fall somewhere between an ordinal and a ratio scale Ordinal in that the distance between scores may not be equal Ratio in that one can view the test score as the number of correct items Norm is to assume such tests represent an interval scale

Copyright © Allyn & Bacon (2007) Measurement Error Decreases the accuracy of measurement Possible sources of measurement error Response set biases Inconsistent measurement procedures Sloppy procedures Unreliable measures

Operational Definitions Copyright © Allyn & Bacon (2007) Operational Definitions Specific procedures for measuring and/or manipulating a variable Every variable should be operationally defined The more careful and complete the operational definition, the more precise the measurement of the variable

Copyright © Allyn & Bacon (2007) Reliability The consistency of measurement Consistency can be conceptualized in different ways Therefore, there are different types of reliability Usually measured with a correlation Covered in Chapter 5 Sensitive to the consistency of rank orderings of participants

Copyright © Allyn & Bacon (2007) Types of Reliability Interrater reliability: degree of agreement between two independent raters Test-retest reliability: degree of consistency over time Internal consistency reliability: degree to which the items of a measure all measure the same thing

Copyright © Allyn & Bacon (2007) Perfect Reliability Reliability is a measure of consistency. Perfect reliability means that the scores are perfectly consistent. Copyright © Allyn & Bacon (2007)

Copyright © Allyn & Bacon (2007) Good Reliability Reliability deteriorates when the consistency is lost. Here the rank orderings are still reasonably consistent. Copyright © Allyn & Bacon (2007)

Copyright © Allyn & Bacon (2007) Fair Reliability As reliability deteriorates further, the rank orderings from the two testings are less similar. Copyright © Allyn & Bacon (2007)

Copyright © Allyn & Bacon (2007) Poor Reliability When you reach the point where the rank orderings have no relationship to one another, your reliability (i.e., consistency) is poor. Copyright © Allyn & Bacon (2007)

Copyright © Allyn & Bacon (2007) Effective Range The range in which a measure gives an accurate indication of the level of the variable Critical to select measures with an effective range appropriate for Your sample Your study Example: Using a calculus test to measure math skills in second graders would NOT work

Scale Attenuation Effects Copyright © Allyn & Bacon (2007) Scale Attenuation Effects Results from a restriction of the range of the measure Therefore, people above or below the effective range are not measured accurately Two types of scale attenuation effects Floor effects Ceiling effects

Illustrating These Effects Copyright © Allyn & Bacon (2007) Illustrating These Effects Situation Assume that we are measuring the weight of 9 men using a standard bathroom scale Floor Effect The needle is stuck so that it never reads below 175 Ceiling Effect The needle is stuck so that it never reads above 200

Copyright © Allyn & Bacon (2007) Weights for 9 Men

Copyright © Allyn & Bacon (2007) Floor Effect Range on the bottom of the scale is insufficient to measure people who score near the bottom If the scale needle was stuck so that it never read below 175 pounds There would be a floor at 175 Error due to this floor effect is shown in dark blue in the next slide Weights recorded for these people are too high

Copyright © Allyn & Bacon (2007) Floor Effect

Copyright © Allyn & Bacon (2007) Ceiling Effect The range on the top of the scale is insufficient to measure people who score near the top If the scale could not read above 200 pounds There would be a ceiling at 200 (everyone above 200 would be reported as weighing 200) Error due to this ceiling effect is shown in dark blue in the next slide

Copyright © Allyn & Bacon (2007) Ceiling Effect

Source of These Effects Copyright © Allyn & Bacon (2007) Source of These Effects Result of psychological scale with an insufficient range to measure the range of performance in the sample Example: a measure of memory that is either too easy or too hard

Copyright © Allyn & Bacon (2007) Validity A scale is valid if it measures what it is supposed to measure Validity also refers to how well a scale predicts other variables Example: An IQ test is likely to be a valid predictor of grades in school. When used this way, the scale is called the predictor measure and the measure predicted is called the criterion

Copyright © Allyn & Bacon (2007) Measuring Validity Validity is usually measured with a correlation The correlation is between The measure and A SPECIFIED criterion measure Always list the criterion when reporting the level of validity For example, the IQ test is a valid predictor of school grades, but not a valid predictor of athletic ability.

Copyright © Allyn & Bacon (2007) Perfect Validity Validity is the degree to which one measure predicts another. With perfect validity, the rank orderings on the predictor and criterion measures are identical. Copyright © Allyn & Bacon (2007)

Copyright © Allyn & Bacon (2007) Good Validity If the rank orderings from the predictor and criterion measures are similar, you have good validity. Copyright © Allyn & Bacon (2007)

Copyright © Allyn & Bacon (2007) Fair Validity But the less the rank orderings from the predictor and criterion measures agree, the lower the validity. Copyright © Allyn & Bacon (2007)

Copyright © Allyn & Bacon (2007) Poor Validity When you reach the point that the rank orderings of predictor and criterion measures are unrelated, you have essentially zero validity. Copyright © Allyn & Bacon (2007)

Objective Measurement Copyright © Allyn & Bacon (2007) Objective Measurement The hallmark of science Objective measures produce the same result no matter who does the measuring Therefore, scientific principles will apply no matter who tests these principles Objective measures reduce biases that could distort results (see Chapters 8 and 9)

Copyright © Allyn & Bacon (2007) Summary Measuring variables is central to research Several scales of measurement exist Reduce measurement error with carefully developed operational definitions Enhance reliability and validity of your measures The goal is to produce objective, accurate measures of your variables