1 11/17/2015 Psychometric Modeling and Calibration Ron D. Hays, Ph.D. September 11, 2006 PROMIS development process session 10:45am-12:30 pm.

Slides:

Advertisements

Similar presentations

Item Response Theory in Health Measurement

Advertisements

How does PROMIS compare to other HRQOL measures? Ron D. Hays, Ph.D. UCLA, Los Angeles, CA Symposium 1077: Measurement Tools to Enhance Health- Related.

PROMIS DEVELOPMENT METHODS, ANALYSES AND APPLICATIONS Presented at the Patient-Reported Outcomes Measurement Information System (PROMIS): A Resource for.

PROMIS: The Right Place at the Right Time? David Cella, Ph.D. Department of Medical Social Sciences Northwestern University Chair, PROMIS Steering Committee.

Remaining Challenges and What to Do Next: Undiscovered Areas Ron D. Hays UCLA Division of General Internal Medicine & Health Services Research

Introduction to Item Response Theory

15-minute Introduction to PROMIS Ron D. Hays, Ph.D UCLA Division of General Internal Medicine & Health Services Research Roundtable Meeting on Measuring.

Why Patient-Reported Outcomes Are Important: Growing Implications and Applications for Rheumatologists Ron D. Hays, Ph.D. UCLA Department of Medicine RAND.

1 Evaluating Multi-Item Scales Using Item-Scale Correlations and Confirmatory Factor Analysis Ron D. Hays, Ph.D. RCMAR.

Item Response Theory in Patient-Reported Outcomes Research Ron D. Hays, Ph.D., UCLA Society of General Internal Medicine California-Hawaii Regional Meeting.

Overview of field trial analysis procedures National Research Coordinators Meeting Windsor, June 2008.

Item Response Theory. Shortcomings of Classical True Score Model Sample dependence Limitation to the specific test situation. Dependence on the parallel.

1 Health-Related Quality of Life Ron D. Hays, Ph.D. - UCLA Department of Medicine: Division of General Internal Medicine.

Rare Diseases and PROMIS : Opportunities Natcher Conference Center March 1, 2013 James Witter MD, PhD FACR CSO PROMIS Medical Officer: Rheumatic Diseases.

Advising on Test Validity: Comments on Denny Borsboom Neil K. Aaronson The Netherlands Cancer Institute KNAW Colloquium on Advising on Research Methods.

Construction and Evaluation of Multi-item Scales Ron D. Hays, Ph.D. RCMAR/EXPORT September 15, 2008, 3-4pm

The emotional distress of children with cancer in China: An Item Response Analysis of C-Ped-PROMIS Anxiety and Depression Short Forms Yanyan Liu 1, Changrong.

The Life Story of Ron Hays UCLA GIM/HSR Introduction to Fellows August 7, 2009 ~ 12:42 pm.

® From Bad to Worse: Comorbidities and Chronic Lower Back Pain Margaret Cecere JD, Richard Young MD, Sandra Burge PhD The University of Texas Health Science.

1 9/23/2015 Patient Reported Outcomes Measurement Information System (PROMIS) Patient Reported Outcomes Measurement Information System (PROMIS) National.

Introduction Neuropsychological Symptoms Scale The Neuropsychological Symptoms Scale (NSS; Dean, 2010) was designed for use in the clinical interview to.

Measurement Invariance of the Pediatric Version of PROMIS Symptom Scales between American and Chinese Children with Cancer Changrong Yuan 1,* ， Pamela.

Use of Health-Related Quality of Life Measures to Assess Individual Patients July 24, 2014 (1:00 – 2:00 PDT) Kaiser Permanente Methods Webinar Series Ron.

SAS PROC IRT July 20, 2015 RCMAR/EXPORT Methods Seminar 3-4pm Acknowledgements: - Karen L. Spritzer - NCI (1U2-CCA )

PROMIS ® : Advancing the Science of PRO measurement Common Data Elements NIH CDE Webinar September 8, 2015 Ashley Wilder Smith, PhD, MPH Chief, Outcomes.

1 10/19/2015  Course materials copyrighted 2003 by Ron D. Hays A Comprehensive Approach to the Measurement of Health Outcomes Ron D. Hays, Ph.D. UCLA.

Another Perspective on PRO Content in Clinical Practice Ron D. Hays, Ph.D. University of California, Los Angeles June 25, 2007.

Development of Physical and Mental Health Summary Scores from PROMIS Global Items Ron D. Hays ( ) UCLA Department of Medicine

Introduction to the Patient-Reported Outcomes Measurement Information System (PROMIS) UCLA Center for East-West Medicine 2428 Santa Monica Blvd., Suite.

Survey of alternative uses of ACTIVE data, review of completed work and lessons learned Friday Harbor Advanced Psychometrics Workshop June 9-13, 2014 Presenter:

Overview of Health-Related Quality of Life Measures May 22, 2014 (1:00 – 2:00 PDT) Kaiser Methods Webinar Series 1 Ron D.Hays, Ph.D.

Measurement Issues by Ron D. Hays UCLA Division of General Internal Medicine & Health Services Research (June 30, 2008, 4:55-5:35pm) Diabetes & Obesity.

Impact of using a next button in a web-based health survey on time to complete and reliability of measurement Ron D. Hays (Rita Bode, Nan Rothrock, William.

Presented By Ron D. Hays, Ph.D. April 8, 2010 (MNRS Pre-Conference Workshop) Domains of PROMIS and how they were developed Dynamic Tools to Measure Health.

Item Response Theory (IRT) Models for Questionnaire Evaluation: Response to Reeve Ron D. Hays October 22, 2009, ~3:45-4:05pm

Multitrait Scaling and IRT: Part I Ron D. Hays, Ph.D. Questionnaire Design and Testing.

Item Factor Analysis Item Response Theory Beaujean Chapter 6.

Reliability and validity of the adapted Spanish version of the Early Onset Scoliosis-24 questionnaire María del Mar Pozo-Balado, PhD Hiroko Matsumoto PhD.

Psychometric Evaluation of Questionnaire Design and Testing Workshop December , 10:00-11:30 am Wilshire Suite 710 DATA.

Item Response Theory in Health Measurement

Ron D. Hays, Ph.D. Joshua S. Malett, Sarah Gaillot, and Marc N. Elliott Physical Functioning Among Medicare Beneficiaries International Society for Quality.

Overlap between Subjective Well-being and Health-related Quality of Life. 3 Ron D. Hays, Ph.D. (Alina Palimaru) November 18, 2015 (11:30-12:00 noon) Geriatric.

Item Response Theory Dan Mungas, Ph.D. Department of Neurology

Considerations in Comparing Groups of People with PROs Ron D. Hays, Ph.D. UCLA Department of Medicine May 6, 2008, 3:45-5:00pm ISPOR, Toronto, Canada.

Item Response Theory Dan Mungas, Ph.D. Department of Neurology University of California, Davis.

Patient-Reported Physical Functioning Ron D. Hays November 27, 2012 (11:15-11:30) UCLA Department of Medicine MCID for Orthopaedic Devices Silver Springs,

Multitrait Scaling and IRT: Part I Ron D. Hays, Ph.D. Questionnaire.

Impact of using a next button in a web-based health survey on time to complete and reliability of measurement Ron D. Hays (Rita Bode, Nan Rothrock, William.

Overview of Item Response Theory Ron D. Hays November 14, 2012 (8:10-8:30am) Geriatrics Society of America (GSA) Pre-Conference Workshop on Patient- Reported.

Evaluating Multi-item Scales Ron D. Hays, Ph.D. UCLA Division of General Internal Medicine/Health Services Research HS237A 11/10/09, 2-4pm,

Health-Related Quality of Life in Outcome Studies Ron D. Hays, Ph.D UCLA Division of General Internal Medicine & Health Services Research GCRC Summer Session.

Lesson 2 Main Test Theories: The Classical Test Theory (CTT)

REGRESSION MODEL FITTING & IDENTIFICATION OF PROGNOSTIC FACTORS BISMA FAROOQI.

Reducing Burden on Patient- Reported Outcomes Using Multidimensional Computer Adaptive Testing Scott B. MorrisMichael Bass Mirinae LeeRichard E. Neapolitan.

Instrument Development and Psychometric Evaluation: Scientific Standards May 2012 Dynamic Tools to Measure Health Outcomes from the Patient Perspective.

Evaluating Patient-Reports about Health

Ron D. Hays September 12, 2006 Network Testing ~ 5 or 6pm

Psychometric Evaluation of Items Ron D. Hays

Mild, moderate, and severe intensity cut-points for the Respiratory Distress Observation Scale©: A receiver operating characteristic curve analysis Margaret.

UCLA Department of Medicine

Evaluating Patient-Reports about Health

UCLA Department of Medicine

Evaluating Multi-Item Scales

PROMIS-29 V2.0 Physical and Mental Health Summary Scores Ron D. Hays

Evaluating IRT Assumptions

Multitrait Scaling and IRT: Part I

Res.Asst. Nurten TERKES, Prof. Hicran BEKTAS

Item Response Theory Applications in Health Ron D. Hays, Discussant

Evaluating the Significance of Individual Change

GIM & HSR Research Seminar: October 5, 2018

Presentation transcript:

1 11/17/2015 Psychometric Modeling and Calibration Ron D. Hays, Ph.D. September 11, 2006 PROMIS development process session 10:45am-12:30 pm

2 11/17/2015 PROMIS Domains Physical functioning (Hays/Bjorner) Pain (Revicki) Fatigue (Lai) Emotional distress (Choi/Reise) Social/role participation (Bode/Hahn)

3 11/17/2015 Datasets Cancer Item Banks (Northwestern) Digitalis Investigation Group Study--randomized double- blind placebo-controlled trial evaluating effect of digoxin on mortality in 581 patients with heart failure and sinus rhythm. IMMPACT--internet-based survey of individuals with chronic pain from the American Chronic Pain Association website Medical Outcomes Study--observational study of persons with hypertension, diabetes, heart disease, and/or depression in Boston, Chicago, and Los Angeles WHOQOL-100 data (n = 442 from U.S. field center)

4 11/17/2015 Types of Analyses Classical Test Theory StatisticsClassical Test Theory Statistics IRT Model AssumptionsIRT Model Assumptions Model FitModel Fit Differential Item FunctioningDifferential Item Functioning Item CalibrationItem Calibration

5 11/17/2015 Classical Test Theory Statistics Out of rangeOut of range Item frequencies and distributionsItem frequencies and distributions Inter-item correlationsInter-item correlations Item-scale correlationsItem-scale correlations Internal consistency reliabilityInternal consistency reliability

6 11/17/2015 IRT Model Assumptions (Uni)dimensionality(Uni)dimensionality Local independenceLocal independence MonotonicityMonotonicity

7 11/17/2015 Sufficient Unidimensionality Confirmatory factor modelsConfirmatory factor models  One factor  Bifactor (general and group factors)

8 11/17/2015 Local Independence After controlling for dominant factor(s), item pairs should not be associated.After controlling for dominant factor(s), item pairs should not be associated.  Look at residual correlations (> 0.20)

9 11/17/2015 Monotonicity Probability of selecting a response category indicative of better health should increase as underlying health increases.Probability of selecting a response category indicative of better health should increase as underlying health increases. Item response function graphs withItem response function graphs with  y-axis: proportion positive for item step  x-axis: raw scale score minus item score

10 11/17/2015

11 11/17/2015 Category Response Curves for Samejima’s Graded Response Model

12 11/17/2015 Model Fit Compare observed and expected response frequencies by item and response categoryCompare observed and expected response frequencies by item and response category Items that do not fit and less discriminating items identified and reviewed by content expertsItems that do not fit and less discriminating items identified and reviewed by content experts

13 11/17/2015 Differential Item Functioning Uniform DIFUniform DIF  Threshold parameter Non-uniform DIFNon-uniform DIF  Discrimination parameter Gender, race/ethnicity, age, diseaseGender, race/ethnicity, age, disease

14 11/17/2015 Item Calibration Item parameters (threshold, discrimination)Item parameters (threshold, discrimination) Mean differences for studied disease groupsMean differences for studied disease groups

15 11/17/2015 Example of Lessons Learned in Secondary Analyses Emotional distress Cannot be adequately modeled as a unidimensional construct. Limited representation of positive end of construct Several items having some response options that provide little information.

16 11/17/2015 Documentation eRoom Public website: Peer-reviewed manuscripts, e.g.: Hays, R. D. et al. (submitted). Item response theory analyses of physical functioning items in the Medical Outcomes Study. Reeve, B. B. (submitted). Psychometric evaluation and calibration of health-related quality of life items banks: Plans for the Patient-Reported Outcome Measurement Information System (PROMIS) Presentations: Tomorrow 4:15-6:15

17 11/17/2015 Questions?

18 11/17/2015 Datasets Subjected to Psychometric Analysis Cancer Fatigue: Cancer Item Banking Project at NWU Cancer Pain: Cancer Item Banking Project at NWU Cancer Social: Cancer Item Banking Project at NWU CSSCD: Cooperative Study of Sickle Cell Disease (pediatric) CHC: Chronic Hepatitis C Study CHS: Cardiovascular Health Study DIG: Digitalis Investigation Group Quality of Life Sub-study IMMPACT: Multiple Pain Projects MOS: Medical Outcomes Study NGHS: National Growth and Health Study (peds.) Q-Score: Cancer Quality of Life Project at NWU WHOQOL: World Health Organization Quality of Life Project

19 11/17/2015 Datasets Subjected to Psychometric Analysis PROMIS Domains Emotional DistressFatiguePainPhysical Function Social Role Participation CHS (NWU/Cook) DIG (UCLA/Hays) Q-Score (NWU/Bode) CHC (Medtap/Revicki & Chen) Cancer Pain (NWU/Lai) IMMPACT (Medtap/Revicki & Chen) CHS (NWU/Cook) DIG (UCLA/Hays) CHS (NWU/Cook) Cancer Social (NWU/Bode) CHS (NWU/Cook) Cancer Fatigue (NWU/Lai) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) MOS (UCLA/Hays & Spritzer) Q-Score (NWU/Lai) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) CSSCD (NWU) NGHS (NWU) NGHS (NWU)