1. Standardized Scales

2. Standardization
Use of identical procedures to collect, score, interpret, and report results of a measure
Assures that differences over time or among different people are due to the variable being measured and not to different measurement procedures

3. What are Standardized Scales?
Set of uniform procedures to collect, score, interpret, and report numerical results
Usually have norms and empirical evidence of reliability and validity
Typically include multiple items aggregated into one or more composite scores
Frequently used to measure constructs

4. Construct Complex concept (e.g., intelligence, well-being, depression)
Inferred or derived from a set of interrelated attributes (e.g., behaviors, experiences, subjective states, attitudes) of people, objects, or events
Typically embedded in a theory
Oftentimes not directly observable but measured using multiple indicators

5. Evaluating and Selecting Standardized Scales
Purpose
Reference populations and normative groups
Reliability
Validity
Practical considerations

6. Purpose Identify whether or not a client has a significant problem
Measure and monitor your client’s outcomes to determine if your client is making satisfactory progress

7. Reference Population
Population of people for which a measure is intended and from which a normative group is sampled and norms are created

8. Normative Group
Representative sample of a reference population, used to estimate norms for that population and, more generally, used to develop and test standardized measures
Also known as a “standardization group” or “standardization sample”
Population
Sample

9. Reliability
Internal consistency reliability (coefficient alpha) (most important)
Interrater rater reliability (sometimes)
Test-retest reliability

10. Validity Face Content Criterion Construct
Sensitivity to change especially important

11. Practical Considerations
Time
Effort
Training
Cost
Availability
Acceptability (e.g., clients, practitioners, etc.)

12. Decisions, Decisions…
Who
Where
When
How often to
collect outcome
data

13. Who
Client
Practitioner
Relevant others
Independent evaluators

14. Where and When Private, quiet, physically comfortable location
Complete at about the same time and under the same conditions on a regular basis

15. How Often Regular, frequent, pre-designated intervals
Often enough to detect significant changes in the problem, but not so often that it becomes problematic
In general about once per week

16. Engage and Prepare Clients
Be certain the client understands and accepts the value and purpose of monitoring progress
Discuss confidentiality
Present measures with confidence
Don’t ask for info the client
can’t provide

17. Engage and Prepare Clients (cont’d)
Be sure the client is prepared
Be careful how you respond to information
Use the information that is collected

18. Administering, Scoring, and Interpreting Standardized Scales
Score, scoring formula, composite score
Unidimensional and multidimensional scales
Cut scores
Reverse-worded items
Reliable change, reliable improvement, reliable deterioration
Clinically significant improvement
Expected treatment response

19. Score
Generic term for a number derived from a measure that represents the quantity or amount of an attribute or observation (e.g., number of times a behavior is observed, value obtained from a standardized scale)
Interpret in context of all available quantitative and qualitative information

20. Scoring
Procedure by which data from a measure are used to produce a score (e.g., number of times a behavior occurs or value on a standardized scale) or category (e.g., diagnostic category)

21. Scoring Formula
A mathematical rule by which data from a measure are used to produce a score (e.g., sum or average of responses to items on a multi-item standardized scale)
Item 1
Item 2
Item 3
Score

22. Composite Score
Score that combines results from two or more related items or other measures using a specified formula (e.g. percentage of items answered correctly on a statistics test)
Score
Item 3
Item 2
Item 1

23. Unidimensional Scale
Scale that measures a single attribute or construct (e.g., depression). (Contrast with multidimensional scale.)

24. Multidimensional Scale
Scale that measures two or more distinct but related attributes or constructs, and measures of the different attributes or constructs are referred to as “subscales”
Global Distress
Subjective
Well-Being
Problems & Symptoms
Social Functioning

25. Cut Scores
Specific predetermined numerical values along a continuum of scores
Used to separate people into categories with distinct substantive interpretations (e.g., clinically depressed or not)
Used to make decisions (provide treatment for depression or not)
Only as good as the normative sample(s) on which it is derived
Interpret in context of all available quantitative and qualitative information

26. Reverse-Worded Item
Item for which smaller numbers indicate a higher score on the measured variable because the item is worded to mean the opposite of the measured variable

27. Reliable Change
Change in a score from one time to another that is more than expected just from random measurement error
Clinical significance.xls

28. Reliable Improvement
Improvement in a score from one time to another that is more than expected just from random measurement error

29. Reliable Deterioration
Deterioration in a score from one time to another that is more than expected just from random measurement error

30. Clinically Significant Improvement
Change that occurs when a client’s measured functioning on a standardized scale is:
In the dysfunctional range before intervention (e.g., greater than 5 on the QIDS-SR)
In the functional range after intervention (e.g., 5 or below on the QIDS-SR)
Change is reliable

31. Clinically Significant Improvement (cont’d)
Interpret in context of all available quantitative and qualitative information
Does not guarantee a meaningful change in a client’s real-world functioning or quality of life
Only as good as the normative sample(s) on which it is derived
Does not speak to the question of whether it was your intervention or something else that caused the change

32. Expected Treatment Response
Session-by-session progress is determined in comparison to normative data from ongoing responses to treatment of thousands of clients
Feedback used in real time to monitor client progress and modify services as needed to reduce treatment failures and increase overall effectiveness

33. Global Rating
Single rating based on a rater’s integration of information about numerous factors (e.g., global rating of change, improvement, or social functioning)

34. Single-Item Global Standardized Scales
Global Assessment of Functioning (GAF)
Children’s Global Assessment Schedule (CGAS)
Social and Occupational Functioning Assessment Scale (SOFAS)
Global Assessment of Relational Functioning (GARF)

35. Potential Advantages of Standardized Scales
Pretested for reliability and validity
Structured, so information less likely to be missed
Can be used to compare individual functioning to normative group functioning
Can be efficient and simple to use

36. Cautions in the Use of Standardized Scales
May not measure concept suggested by scale name
Different measures of the same concept may not be equivalent
Sometimes limited information about reliability and validity
Concepts as measured
may not be completely
relevant to individual
clients

37. Resources Compendiums of measures Web measurement resources
See Appendix B
Web measurement resources