Response Processes Psych DeShon

Response Elicitation Completion Completion Requires production Requires production Allows for creative responses Allows for creative responses Some ambiguity in scoring process Some ambiguity in scoring process Guessing is largely irrelevant Guessing is largely irrelevant Scored right/wrong Scored right/wrong

Response Elicitation Multiple Choice Multiple Choice 3 or more response options 3 or more response options 1 correct answer 1 correct answer Remaining alternatives are called “distractors” Remaining alternatives are called “distractors” Distractors should be plausible but demonstrably incorrect Distractors should be plausible but demonstrably incorrect Situational judgment items Situational judgment items Select best and worst options Select best and worst options Requires a recognition or a matching process from the test taker Requires a recognition or a matching process from the test taker Guessing is a problem Guessing is a problem Scored right/wrong Scored right/wrong

Response Elicitation Two-Choice Two-Choice Choose one of two response options Choose one of two response options True/False True/False Agree/Disagree Agree/Disagree Scored right/wrong; more/less Scored right/wrong; more/less High probability of guessing if there is a right answer High probability of guessing if there is a right answer Often asked in checklist form Often asked in checklist form

Response Elicitation Matching Matching Match concepts in one list to concepts in another list Match concepts in one list to concepts in another list Scored as number of correct matches Scored as number of correct matches There may be more than one way to match concepts leading to ambiguity or multidimensionality There may be more than one way to match concepts leading to ambiguity or multidimensionality

Response Elicitation Ordered Category or Graded Response Ordered Category or Graded Response Likert method Likert method At least three response options that have a natural order At least three response options that have a natural order Graded Response options Graded Response options

Response Elicitation Forced Choice Forced Choice Individual must choose one of 2 or more options or preferences Individual must choose one of 2 or more options or preferences Results in ipsative measures Results in ipsative measures

Ipsative Measures Ipsative responses yield the same total score for all individuals Ipsative responses yield the same total score for all individuals This forces a covariance structure on the item responses This forces a covariance structure on the item responses Average intercorrelation = -1/(k-1) Average intercorrelation = -1/(k-1) Also imposes restrictions on the pattern of item correlations with an external criterion Also imposes restrictions on the pattern of item correlations with an external criterion Sum of covariances between the items and the criterion must equal zero Sum of covariances between the items and the criterion must equal zero Reliability is decreased by ipsative responses Reliability is decreased by ipsative responses Factor analysis of ipsative data is challenging at best Factor analysis of ipsative data is challenging at best Artificial patterns of covariances among the factors Artificial patterns of covariances among the factors

Response Elicitation Ranking Ranking Hard to assign scales scores Hard to assign scales scores Ipsative problems Ipsative problems Unordered Categories Unordered Categories 3 or more options without a natural ordering 3 or more options without a natural ordering Not common Not common hard to impossible to score hard to impossible to score

Measurement or Scaling Once you’ve elicited responses to your probes (e.g., items) you need to assign numbers Once you’ve elicited responses to your probes (e.g., items) you need to assign numbers Measurement and scaling are often hard to distinguish Measurement and scaling are often hard to distinguish Scaling often involves transformations to assigned numbers Scaling often involves transformations to assigned numbers Let’s talk about scaling… Let’s talk about scaling…

Scaling Physical Stimuli Two questions: Two questions: What is the connection between variation in a physical stimulus and variation in human perception of the stimulus? What is the connection between variation in a physical stimulus and variation in human perception of the stimulus? Can humans be used as measurement instruments for physical quantities Can humans be used as measurement instruments for physical quantities Must be able to do this to justify using humans as measurement instruments for mental quantities Must be able to do this to justify using humans as measurement instruments for mental quantities

Scaling History Psychophysics Psychophysics Psychophysics is the study of the functional relationships between the physical properties of stimuli and the psychological responses to them Psychophysics is the study of the functional relationships between the physical properties of stimuli and the psychological responses to them

Psychophysics 2 central concepts 2 central concepts Absolute threshold Absolute threshold The smallest amount of stimulus energy to produce a sensation or that can be reliably detected The smallest amount of stimulus energy to produce a sensation or that can be reliably detected Minimum stimulus needed to detect 50% ot time Minimum stimulus needed to detect 50% ot time Addresses sensitivity of the perceiving system Addresses sensitivity of the perceiving system Difference threshold Difference threshold The smallest amount of stimulus energy change to produce a sensation or that can be reliably discriminated The smallest amount of stimulus energy change to produce a sensation or that can be reliably discriminated Minimum difference needed to detect difference 50% of time Minimum difference needed to detect difference 50% of time jnd - just noticeable difference jnd - just noticeable difference

Psychophysics Absolute threshold Absolute threshold

Psychophysics Difference Threshold Difference Threshold

Psychophysics Webers Law Webers Law subjects were asked to hold a weight in each hand and to state which weight was heaviest subjects were asked to hold a weight in each hand and to state which weight was heaviest Over many pairs of weights he found that the jnd was not a fixed quantity, but was proportional to the stimulus intensity Over many pairs of weights he found that the jnd was not a fixed quantity, but was proportional to the stimulus intensity I is the stimulus intensity k is the Weber constant that depends upon the nature of the stimulus

Psychophysics Weber's Law Weber's Law k measures the sensitivity of human perception to the specific stimulus. Lower values k, mean greater capacity of human sensory systems to discriminate small differences in the stimulus.

Psychophysics Weber constants Weber constants

Psychophysics Weber's Law Weber's Law Fits well except at extremes

Psychophysics Fechner’s Law Fechner’s Law Fechner's recognized that the jnd could be made the basic unit of measurement of the intensity of sensation Fechner's recognized that the jnd could be made the basic unit of measurement of the intensity of sensation Assuming that sensation is zero at the absolute threshold and that all jnd's are equal regardless of where on the scale of physical intensity they fall, a given sensation can be said to be some number of just noticeable differences above zero or above or below another sensation Assuming that sensation is zero at the absolute threshold and that all jnd's are equal regardless of where on the scale of physical intensity they fall, a given sensation can be said to be some number of just noticeable differences above zero or above or below another sensation The magnitude of sensation can be scaled in relation to the scale of physical intensity The magnitude of sensation can be scaled in relation to the scale of physical intensity The relative increase in mental intensity can be measured in terms of the relative increase in physical energy required to bring it about The relative increase in mental intensity can be measured in terms of the relative increase in physical energy required to bring it about

Psychophysics Graphic example of Fechner’s Law Graphic example of Fechner’s Law

Psychophysics Another view of Fechner's Law Another view of Fechner's Law

Psychophysics Fechner's Law Fechner's Law This curve shows that a fixed percentage change in one axis is equal to a fixed linear increment in the other This curve shows that a fixed percentage change in one axis is equal to a fixed linear increment in the other Best expressed as a log function Best expressed as a log function S = subjective intensity I = stimulus intensity I 0 = stimulus intensity at threshold

Psychophysics Fechner's Law Fechner's Law example... example... Rate the intensity of a series of acoustic signals Rate the intensity of a series of acoustic signals Plot sound intensity against subjective intensity Plot sound intensity against subjective intensity

Psychophysics Steven's Power Law Steven's Power Law Argued that it was nonsensical to use error (jnd) as the basic measurement unit Argued that it was nonsensical to use error (jnd) as the basic measurement unit Instead, wanted to find a way to use direct ratings of intensity using a rating scale Instead, wanted to find a way to use direct ratings of intensity using a rating scale Direct scaling Direct scaling Data from direct ratings were not well described by Fechner's function Data from direct ratings were not well described by Fechner's function Developed a power function to describe the observed relationships Developed a power function to describe the observed relationships

Psychophysics Steven's Power Law Steven's Power Law S = subjective intensity I = stimulus intensity α = constant reflecting measurement units m = exponent that changes depending upon the nature of the stimuli

Psychophysics Steven's Power Law Steven's Power Law

Steven's Power Law

Psychophysics Steven’s Power Law (log-log plot) Steven’s Power Law (log-log plot)

Fechner Fechner Method of limits Method of limits Method of constant stimuli Method of constant stimuli Staircase method Staircase method Method of adjustment Method of adjustment subject controls the level of the stimulus and alters it until it is just barely detectable against the background noise or is the same as the level of another stimulus subject controls the level of the stimulus and alters it until it is just barely detectable against the background noise or is the same as the level of another stimulus

Likert Scaling Rensis Likert (1932) Rensis Likert (1932) Developed a method for scaling attitudes Developed a method for scaling attitudes Steps Steps Construct a large number of items thought to reflect the construct that will yield response variance from those differing in the level of the construct Construct a large number of items thought to reflect the construct that will yield response variance from those differing in the level of the construct Items should be clear and consice Items should be clear and consice

Likert Scaling Steps Steps Items should be worded such that the modal response is in the middle of the possible responses Items should be worded such that the modal response is in the middle of the possible responses To avoid stereotypic responses, half the items should be phrased positively and half negatively To avoid stereotypic responses, half the items should be phrased positively and half negatively Give the large set of items to a large group of pilot responders Give the large set of items to a large group of pilot responders

Likert Scaling Steps Steps Provide “strongly approve” to “strongly disapprove” response options Provide “strongly approve” to “strongly disapprove” response options Assign contiguous numbers from 1-5 to the response options Assign contiguous numbers from 1-5 to the response options Reverse score items that reflect the construct in an opposite direction Reverse score items that reflect the construct in an opposite direction Make sure item responses show adequate variance Make sure item responses show adequate variance Compute the split half reliability of the item responses Compute the split half reliability of the item responses Compute item-total correlations Compute item-total correlations Drop items with low item-total correlations Drop items with low item-total correlations

Scale Coarseness

Scaling Methods Unfolding Unfolding Ideal point model vs. dominance Ideal point model vs. dominance