1. Clinical Outcome Assessments and epistemic Risk
Leah McClimans

2. Outcome Measure Ambiguity
What is a health outcome?
There are three major health outcomes: mortality, morbidity and quality of life
What is an outcome measure?
Clinical outcome assessments, e.g. patient-reported outcome measures (PROMs)
Refer to measuring instruments or tools, usually questionnaires, that measure patients’ health status by asking them questions about it
Odds Ratio, number needed to treat, relative risk, etc.
Refer to formal statements that aim to quantify causality of e.g. exposure to an outcome
Health outcome refer to changes in health from the result of health care interventions and investments

3. Measuring what is hard to quantify
Patient reported outcome measures
Usually take the form of a questionnaire, answers to questions come from patient without “amendment or interpretation…by a clinician or anyone else” (FDA)
Attempts to quantify “symptoms or unobservable concepts known only to the patient, e.g. pain severity or nausea” or functionings or activities that are observable to others but which the patient’s perspective is considered important (FDA)

4. How Do PROMs quantify their phenomena?

5. Two Measurement Theories
Classical test theory
Latent trait theory
Most popular measurement theory in psychology
Utilizes a simple model: O=T+E
Unfalsifiable
Ontologically profligate
Popular in psychometrics and educational testing
Utilizes a variety of models that relate observed scores to a latent variable, e.g. ability.
The position on the latent variable “causes” the observed scores
Adequacy of the model is determined by the goodness of fit between the observed scores and the predictions of the model
Falsifiable
Ontologically parsimonious

6. Which Theory is better? Better how? Interpretable? Simple?
Useful? (to whom?)
Ethical/value adequate?
Reliable?
Valid?
Ensure quantitative variable?

7. Making choices, accepting risk
Radical suggestion: No single measurement theory or model is better on all dimensions
Classical test theory
Simple, but difficult to interpret
Latent trait theory
Complex, but interpretable
Both have found uses, yet have validity and ethical/ value adequacy problems
QoL is difficult to quantify because it’s a “Ballung” concept
Operationalizing it, determining what model to use involves emphasizing certain qualities over others
Simplicity, interpretability, reliability, etc.
This emphasis entails accepting a level of epistemic risk

8. Risky Choices or metaphysical commitments?
“Coherent interpretation of many claims about measurement in the psychological sciences depend on philosophically realist commitments regarding psychological attributes” (Andrew Maul, “on the ontology of psychological attributes)
To summarize the realist position: understanding measurements under the umbrella of the realist concept of truth, commits us not just to the logically independent existence of things in space and time, but also to the existence of quantitatively structured properties and relations, and to the existence of real numbers, understood as relations of ratio between specific levels of such attributes. (Joel Michell, “The logic of measurement: a realist overview)
“Latent variable theory adheres to entity realism, because this form of realism is needed to motivate the choice of model in psychological measurement” (Denny Borsboom, measuring the Mind)
Classical test theory vs. Latent Trait theory redux
Metaphysics in psychological measurement
A radical suggestion
If Risks are adjudicated in light of values and interests, then values and interests should play a more explicit role than they do in qoL measurement