1. Research Curriculum Session II –Study Subjects, Variables and Outcome Measures Jim Quinn MD MS Research Director, Division of Emergency Medicine Stanford University

2. Overview Study Subjects -Sampling -Recruitment Variables -Types of outcome measures -Precision, accuracy, validity, reliability

3. Study Subjects Generalizing the Results “Research is only interesting to others if they can apply it to their practice”

5. Study Subjects Subjects in the study sample should be representative of the population of interest Depending on study different populations may yield different results. -Examples: General population, ED patients, Clinic Patients, Attitudes of patients -Laceration studies, syncope study

6. Study Subjects Specify the best clinical and demographic characteristics of the study population to best answer question Appropriate sampling from that target population Results = truth in the study Best possible chance to have the results generalizable.

7. Selection Criteria Defining the Target Population Inclusion Criteria -defines the main characteristics of the target population – be specific

8. Selection Criteria Defining the Target Population Exclusion Criteria -Individuals whose characteristics may interfere with the quality of the results E.g. – rare events, poor follow-up - May compromise generalizability of the study

9. Sampling Convenience Sample Consecutive Sample Probability Samples - Simple Random Sample -Stratified Random Sample -Cluster Samples

10. Recruitment Goals A sample that represents the target population - Non responders, lost follow-up Enough subjects to meet sample size requirements - Play it safe, overestimate - There is always fewer patients than you think!

11. “The best way to eliminate disease is to study it!”

12. Outcome Measures Selection of Variables and Scales

13. Selection of Variables Practical Points/Precision/Accuracy Continuous Variables -“discrete” variables -rich in information -Potential sample size “relief” Categorical -Dichotomous -Nominal -Ordinal

14. Measurement Scales Categorical Variables -Phenomena often not suited for measurement (e.g. Death) -Dichotomous -Nominal -Ordinal – categories have order but no specific numerical or uniform difference

15. Measurement Scales Continuous (infinite values) Ordered discrete (ordinal with numerical meaning) - Statistically handled very similarly

16. Measurement Scales Summary Categorical -Scales may have more meaning and make more sense. -Less information, need large numbers Continuous -some times hard to determine meaningful differences -sample size friendly

17. Attributes of Outcome Measures Precision Is the measure “reproducible, reliable and consistent” Subject to random error and variability -Observer variability -Instrument variability -Subject variability

18. Assessing Precision Inter and Intraobserver reproducibility Within and between instrument reproducibility -Continuous variables – Coefficient of variation -Categorical – kappa statistic

19. Enhancing Precision Standardize measurement methods Train and certify observers Refining the instruments Automating the instrument Repetition (reduces random error)

20. Accuracy “Does the variable actually measure or represent what it intends to” Assessed by comparison to a “Gold Standard” Different than precision, but many things that improve precision improve accuracy A function of systematic error -Observer bias -Subject Bias -Instrument Bias

21. Enhancing Accuracy Standardized measurement methods Training observers Refining instruments Automating instruments Making Unobtrusive measures Blinding Calibration of Instruments

23. Validity Accuracy when there is no “Gold Standard” -Measuring an abstract or subjective phenomena (e.g. – pain, quality of life) -Content Validity (Face, Inherent or sampling validity) -Construct Validity -Criterion Related Validity (Predictive Validity)

24. Final Thoughts An outcome measure should be sensitive enough to determine important clinical differences It should be associated with only the characteristic of interest Measurements should involve data collection that is efficient in time and cost Efficiency is improved by increasing the quality of each item measured