Data and the Nature of Measurement Graziano and Raulin Research Methods: Chapter 4 Graziano & Raulin (2000) Graziano & Raulin (1997)

Research Variables Variable: Any characteristic that can take on more than one value Examples: speed, level of hostility, accuracy of feedback, reaction time Research is the study of the relationship between variables Therefore, there must be at least two variables in a research study (or there is no relationship to study) Graziano & Raulin (2000)

Measuring Variables in Research Measurement: A process by which we assign numbers to indicate the amount of some variable present Sometimes the number assignment is easy to understand (e.g., speed measured in number of seconds) Sometimes it is more arbitrary (e.g., 1 for male and 2 for female) Graziano & Raulin (2000)

Scales of Measurement Based on how closely the scale matches the real number system Scales of Measurement (as proposed by Stevens) Nominal Ordinal Interval Ratio Graziano & Raulin (2000)

Nominal Scales A naming scale Produces nominal or categorical data Each number reflects an arbitrary category label rather than an amount of a variable Examples: diagnostic categories, political affiliations, preference for consumer products Produces nominal or categorical data Has mathematical property of identity Graziano & Raulin (2000)

Ordinal Scales A scale that indicates rank ordering Reflects the order, but not the amount of a variable Examples: order of finish in a race, class rankings Produces ordered data Has mathematical properties of identity and magnitude Graziano & Raulin (2000)

Interval Scales A scale that has equal intervals Produces score data The scale indicates amount, but there is no zero point on the scale Examples: temperature on the Celsius scale, most psychological tests Produces score data Has the mathematical properties of identity, magnitude, and equal intervals Graziano & Raulin (2000)

Ratio Scales A scale that fits the number system well The scale has a true zero and equal intervals, just like the real number system Examples: time, distance, number correct, weight, frequency of behavior Produces score data Has the mathematical properties of identity, magnitude, equal intervals, and a true zero Graziano & Raulin (2000)

Controlling Measurement Error Measurement error decreases the accuracy of measurement Anything that lead to inconsistency in the measurement process can produce measurement error Response set biases (e.g., social desirability) will add measurement error to any self-report measure Graziano & Raulin (2000)

Operational Definitions The specific procedures by which the researcher measures and/or manipulates a variable In research, every variable should be operationally defined The more careful and complete the operational definition, the more precise the measurement of the variable will be Graziano & Raulin (2000)

Reliability Reliability refers to the consistency of measurement Types of reliability Interrater reliability: degree of agreement between two independent raters Test-retest reliability: degree of consistency over time Internal consistency reliability: degree to which the items of a measure are in agreement Graziano & Raulin (2000)

High Reliability High reliability occurs when the rank ordering of individuals is very similar in the separate measures High test-retest reliability is illustrated in the figure at the right Graziano & Raulin (2000)

Low Reliability Low reliability occurs when the rank ordering of individuals is very different in the separate measures Low test-retest reliability is illustrated in the figure at the right Graziano & Raulin (2000)

Effective Range The effective range of the measure is the range in which it can give an accurate indication of the level of the variable All measures have a limit to their effective range, so it is critical to select measures whose effective range will pick up the variability expected in your study Example: Using a calculus test to measure math skills in second graders would NOT work Graziano & Raulin (2000)

Validity A scale is valid if it measures what it is supposed to measure Validity also refers to how well a scale predicts other variables (e.g., an IQ test is likely to be a reasonably valid predictor of grades in school) When used this way, the scale is called the predictor measure and the measure predicted is called the criterion Graziano & Raulin (2000)

High Validity High validity is indicated when the rank ordering of test scores is very similar to the rank ordering on the criterion measure High validity in indicated in the figure on the right Graziano & Raulin (2000)

Low Validity Low validity is indicated when the rank ordering of test scores is very different from the rank ordering on the criterion measure Low validity in indicated in the figure on the right Graziano & Raulin (2000)

Scale Attenuation Effects If the effective range is insufficient, scores will will cluster at the top or bottom of the scale (termed scale attenuation effects) Floor effect: insufficient range at the bottom of the scale, so most low scorers are bunched together Ceiling effect: insufficient range at the top of the scale, so most high scorers are bunched together Scale attenuation effects distort score, thus reducing both reliability and validity Graziano & Raulin (2000)

Objective Measurement Objective measures are the hallmark of science Produce the same result no matter who does the measurement Therefore, scientific principles will apply no matter who tests these principles Objective measures reduce biases that could distort results (see Chapters 8 and 9) Graziano & Raulin (2000)

Summary Measuring variables is central to research Several scales of measurement exist Reduce measurement error with carefully developed operational definitions Want to enhance reliability and validity of your measures The goal is to produce objective, accurate measures of your variables Graziano & Raulin (2000)