1. Standardizing Assessment Data: Continuity Assessment Record and Evaluation (CARE) Item Set
Presented by:
Barbara Gage, PhD
Engelberg Center for HealthCare Reform/The Brookings Institution
Stella Mandl, BSW, BSN, PHN, RN
Center for Clinical Standards & Quality/CMS
Presented to:
The Long Term Care Discussion Group
Wednesday, November 6, 2103

2. Thinking Ahead: Data Element Standardization
Stella Mandl, RN
Technical Advisor
Division of Chronic and Post Acute Care
Center for Clinical Standards and Quality
Center for Medicare & Medicaid Services

3. Data Assessment Elements Goal
When we keep in mind the ultimate goal of and step back to look at the big picture of what’s been done to prepare, it becomes clearer where the work converges; how much of the work is connected and has already been done to achieve
Achieving Uniformity to Facilitate Effective Communication for
Better Care of Individuals and Communities

4. CARE: Concepts Guiding Principles and Goals: Assessment Data is:
Standardized
Reusable
Informative
Communicates in the same information across settings
Ensures data transferability forward and backward allowing for interoperability
Standardization:
Reduces provider burden
Increases reliability and validity
Offers meaningful application to providers
Facilitates patient centered care, care coordination, improved outcomes, and efficiency
Fosters seamless care transitions
Evaluates outcomes for patients that traverse settings
Allows for measures to follow the patient
Assesses quality across settings, and Inform payment modeling

5. To Be: Uniform Assessment Data Elements
As Is To Be
Transition
As Is: Multiple Incompatible Data Sources
Outpatient Settings
No Standard
Data Set
Nursing
Homes
MDS
LTCHS
LTCH CARE
Data Set
Physicians
No Standard
Data Set
Inpatient
Rehab Facilities
IRF-PAI
Hospitals
No Standard
Data Set
Home Health
Agencies
OASIS
GOAL:
Uniform Data Elements
Across Providers
Standardized
Nationally Vetted
2008
To Be: Uniform Assessment Data Elements
Enable Use/re-use of Data
Exchange Patient-Centered Health Info
Promote High Quality Care
Support Care Transitions
Reduce Burden
Expand QM Automation
Support Survey & Certification Process
Generate CMS Payment

6. CMS Vision for Quality Measurement
Align measures with the National Quality Strategy and Six Measure Domains
Implement measures that fill critical gaps within the six domains
Develop parsimonious sets of measures - core sets of measures
Remove measures that are no longer appropriate (e.g., topped out)
Align measures with external stakeholders, including private payers and boards and specialty societies
Continuously improve quality measurement over time
Align measures across CMS programs whenever and wherever possible

7. CMS Framework for Measurement
Care Coordination
Patient and family activation
Infrastructure and processes for care coordination
Impact of care coordination
Clinical Quality
of Care
Measures should be patient-centered and outcome-oriented whenever possible
Measure concepts in each of the six domains that are common across providers and settings can form a core set of measures
Population/
Community Health
Care type (preventive, acute, post-acute, chronic)
Conditions
Subpopulations
Health Behaviors
Access
Physical and Social environment
Health Status
Function
Person- and
Caregiver- Centered Experience and Outcomes
Efficiency and
Cost Reduction
Patient experience
Caregiver experience
Preference- and goal-oriented care
Safety
Cost
Efficiency
Appropriateness
All-cause harm
HACs
HAIs
Unnecessary care
Medication safety 7

8. Building the Future State
Assessment Instrument/Data Sets use uniform and standardized items
Measures are harmonized at the Data Element level
Providers/vendors have public access to standards
Data Elements are easily available with national standards to support PAC health information technology (IT) and care communication
Transfer of Care Documents are able to incorporate uniform Data Elements used in PAC settings, if desired
Measures can evaluate quality across settings and be used for setting comparisons

9. Keeping in Mind, the Ideal State
Facilities are able to transmit electronic and interoperable Documents and Data Elements
Provides convergence in language/terminology
Data Elements used are clinically relevant
Care is coordinated using meaningful information that is spoken and understood by all
Measures can evaluate quality across settings and evaluate intermittent and long term outcomes
Incorporates needs beyond healthcare system

10. Standardizing Assessment Data: CARE
Presented by Barbara Gage, PhD
Phone

11. State of the Art in Measuring Patients’ Health Status in Medicare
Acute Hospitals  no standard assessment tool to admit and monitor patients
Long-Term Care Hospitals  newly standardized items for quality reporting
Inpatient Rehabilitation Facilities  IRFPAI required
Skilled Nursing Facilities  MDS required
Home Health Agencies  OASIS required

12. DRA of 2005 called for one uniform tool that could be used to measure patient health status at:
Acute hospital discharge
Admission/discharge/interim times for cases using:
Long Term Care Hospitals
Inpatient Rehabilitation Facilities
Skilled Nursing Facilities
Home Health Agencies

13. Standard Language important for:
Improving coordination of care – one set of terms to define pressure ulcer severity, functional impairment, cognitive impairment across providers
Improving data exchangeability – can’t merge inconsistent items; need standard language to transfer information between providers treating the case

14. Continuity Assessment Record & Evaluation (CARE) Development
Sponsored by CMS, Office of Clinical Standards and Quality
Principal Investigator/RTI Team: Barbara Gage, Shula Bernard, Roberta Constantine, Melissa Morley, Mel Ingber
Co- Principal Investigators: Rehabilitation Institute of Chicago, Northwestern University
Consultants: Visiting Nurse Services of NY, University of Pennsylvania, RAND, Case Western University
Input by pilot test participants, including participating acute hospitals, LTCHs, IRFs, SNFs, and HHAs

15. Consensus Development
Year 1 of CARE development: Gain input from the providers/research community
Review existing assessment tools (MDS, IRFPAI, OASIS, LTCH tools, acute items)
Technical Expert Panels
Clinical communities from 25 associations, including AHA, AMRPA, NALTH, ALTHA, NAHC, VNAA, AHCA, AAHSA, APTA, AOTA, ASHA, ANA,ARN, CMA, Discharge Planners, Joint Commission, to name a few
Research/case-mix communities, including DRG, FRG, HHRG, RUG

16. Standardized Assessment Items Should:
Build on current measurement science but also add new instrument development methodologies, and
Modify existing assessment instruments to develop a standard assessment instrument that will:
Measure health and functional status
Assess service needs
Evaluate treatment outcomes
Guide payment policy
Improve seamless transitions

17. Common Domains in Existing Assessment Tools
Administrative Information
Social Support Information
Medical Diagnosis/Conditions
Functional Limitations
Physical
Cognitive

18. Differences Across Tools
Individual items that measure each concept
Rating scales used to measure items
Look-back or assessment periods
Unidimensionality of individual items

19. Functional Item Comparisons
Tools
No. of Functional Items
Scale Levels
Assessment Periods
IRFPAI 18 7
Past 3 days
MDS 3.0 12 8
Past 5 days
OASIS
varies
Assessment day

20. Differences in Item Details
Bathing:
IRFPAI and OASIS – bathing only
MDS – bathing and transferring in/out tub/shower
Dressing:
IRFPAI and OASIS – 2 items (Upper/Lower)
MDS – 1 item
Toileting:
IRFPAI – level of independence
OASIS – ability to get to/from
MDS – ability to use toilet, transfer, change pads
Source: Gage and Green, Chapter 2. The State of the Art: Current CMS PAC Instruments in Uniform Patient Assessment for Post-Acute Care, CMS Report, Contract #IFMC IA03.

21. Incomparable Functional Scales
IRF-PAI
MDS
OASIS
7= Complete independence
0= Independent
0= bathe independent tub/shower
6=Modified (device)
1= Supervision
1= with devices, independent
5=Supervision
2= Limited Asst. (guided maneuvering)
2= with person (reminders, access, reach difficult areas
4=Minimal Assistance 25%
3= Extensive Asst (3+ times/week)
3= participates but req. other person
3= Moderate Assistance 50%
4= Total Dependence
4= unable, bathes in bed/chair
2=Maximal Asst. 25%
8= Activity NA
5= totally bathed by other
1= Total Asst.
0= Activity NA
Unknown

22. CARE Item Development Formed 4 Workgroups
Medical acuity/continuity of care
Functional impairment
Cognitive impairment
Social/Environmental support

23. Workgroup Charge:
Identify critical areas/domains for measuring case-mix acuity, resource use, or outcomes
Review existing legacy tools (MDS, IRFPAI, OASIS), other leading measurement tools (PROMIS, COCOA-B, VA)
Propose core set that can be used at hospital discharge and across all PAC settings

24. Issues in Selecting Items
Identify Standard –
Measures that applied across severity groups but capture the range of severity
Scales that do not lead to ceiling or floor effects when measuring severity
Assessment windows that would allow severity comparisons across settings
Self-report/performance-based items
Current Medicare payment methods
Minimal burden on providers
Varying technology options across providers

25. Medical Items Primary Acute Care Diagnosis PAC Diagnosis
Comorbidities/Complicating Conditions
Physiologic Factors
Treatments
Prognosis/Life Expectancy/Frailty

26. Social/Environmental Items
Physical Living Environment
Prior residence
Structural barriers
Social Support and Assistance
Prior lives with
Lives with after discharge
Type of caregiver
Frequency of Assistance

27. Function Items Core Function Items All patients, all settings
Supplemental Function Items
Based on patient’s functional status (e.g., bedfast, self-care, basic mobility, IADLs)
Maximize range of patient ability captured (i.e., avoiding floor and ceiling effects)
Provide sufficient variation to capture improvement in function
This project was charged with the development of an instrument that did not differ by site of care, yet recognized the wide range of patient abilities that would need to be captured. To this end, we have identified a core set of items that would be used to evaluate all patients, regardless of functional level. Additional items would be administered based on patients’ functional level e.g. bedfast, self-care, basic mobility, IADLs. It is intended that this approach minimizes burden while maximizing the range of patient ability captured (i.e., avoiding floor and ceiling effects). The core items make up a range of ability, have been shown to work well and be easily scored in existing instruments, and play a role clinically in discharge planning decisions.

28. Core Function Domains Prior Functional Status Need for Assistance
Eating
Bed Mobility
Oral Hygiene
Toilet Hygiene
Transfer
Dressing - Upper Body
Locomotion
Function Modifiers
Weight-bearing
Sitting Unassisted
Swallowing

29. Supplemental Function Items
Bedfast
Sit to lying, roll left or right, sponge bath
Self Care
Lower body dressing, shower/bathe, get in/out of car, curb/1 step, short ramp
Basic Mobility
4 steps-exterior, long ramp exterior, walk longer distance-interior, wheel longer distance- interior, 12 steps-interior

30. Supplemental Function Items
IADLs
Laundry, light shopping, make light meal, dishwashing-by hand, dishwashing-machine, telephone-answering, telephone-placing call, medication management-oral meds, medication management-inhalers, medication management injectables

31. Cognitive/Self Report Items
Mental status
Orientation
Memory
Screening for delirium
Mood/Depression
Behavioral symptoms
Pain
Sensory input
Vision
Hearing

32. Cognitive Items (cont’d)
Communication
Comprehension
Expression
Fatigue

33. Reliability of the Standardized CARE Items
Most CARE items based on existing validated items currently used in the Medicare program; but few items had been used in multiple settings or across different levels of care.
Two types of reliability tests were conducted to examine whether the items performed consistently across settings and across disciplines
Traditional Inter-rater Reliability (pairs of assessors rate the same patient similarly)
Video Reliability (cross disciplinary rating of standard video patients)

34. Traditional Inter-rater Reliability Methods
Paired staff, matched on credential, assessed the same patient
Tested in subsample of participating PACPRD providers (n = 34)
Each site collected data on patients
455 pairs of assessments collected

35. IRR Methods: Item Selection and Analysis
Analyses followed methods used to evaluate existing CMS tools (MDS, OASIS, IRFPAI)
Reliability scores at least equal to existing tools
Categorical items: Kappa (for 2 levels), Weighted Kappa (for > 2 levels, Fleiss-Cohen weights)
Range: 0 poor, 0.01–0.20 slight, 0.21–0.40 fair, 0.41– 0.60 moderate, 0.61–0.80 substantial, and 0.81–1 almost perfect
Continuous items: Pearson Correlation

36. IRR: Results
Overall, the vast majority of Kappas (weighted and unweighted) found were above 0.60
Prior functioning and history of falls:
Skin integrity:
Pressure ulcers – 7 of 8 categorical items 0.67 or higher
Length and width correlations – approx 0.6
Major wounds – 0.64 and higher
Turning Surfaces – 3 of 5 above 0.6, 4 of 5 above 0.5

37. IRR: Results Cognitive status and Mood: 26 of 29 above 0.6 Pain:
Interview items – all 5 above 0.6, 4 above 0.79
Observational assessment – 4 of 5 above 0.6

38. IRR: Results Impairments: Bladder and bowel:
Current status items higher than 0.7
Prior status items higher than 0.65
Swallowing signs and symptoms: NPO (0.97), None (0.84), coughing (0.68), loss of liquids and holding food in mouth all low prevalence ranged
All hearing, weight bearing, respiratory and endurance items had kappas above 0.6

39. IRR: Results Functional status:
(calculated with and without letter codes)
Core (self-care and mobility including walk and wheel): all above 0.6 except ‘tube feeding’ and ‘walk 150 feet’
Supplemental Self Care: all above 0.63 except ‘roll left and right’
Supplemental Mobility: all above 0.63 except ‘walk 50 feet with two turns’ ‘walk 10 feet on uneven surface’ and the wheel long and short ramps

40. IRR: Results Functional status (cont’d):
(calculated with and without letter codes)
IADLs: all above 0.7 excluding letter codes except ‘laundry’ and ‘light shopping’; all above 0.6 when including letter code, except ‘use public transportation’
Overall plan of care: above 0.6

41. IRR: Summary
IRR results indicate substantial to almost perfect agreement for the majority of items evaluated
The few lower kappa scores tend to be for low prevalence items
IRR results for CARE items are in line with the majority of IRR results available for equivalent items on MDS, OASIS, and FIM

42. CARE Item Reports 2 website URLs
Quality-Initiatives/CARE-Item-Set-and-B-CARE.html
  Quality-Initiatives/Functional-Measures-.html
The Development and Testing of the Continuity Assessment Record and Evaluation (CARE) Item Set: Final Report on the Development of the CARE Item Set. Volume 1 of 3 [PDF, 8MB]
The Development and Testing of the Continuity Assessment Record and Evaluation (CARE) Item Set: Final Report on Reliability Testing. Volume 2 of 3 [PDF, 2MB]
The Development and Testing of the Continuity Assessment Record and Evaluation (CARE) Item Set: Final Report on the Development of the CARE Item Set and Current Assessment Comparisons. Volume 3 of 3 [PDF, 2MB]
Continuity Assessment Record and Evaluation (CARE) Item Set: Additional Provider-Type Specific Interrater Reliability Analyses [PDF, 902KB]
Continuity Assessment Record and Evaluation (CARE) Item Set: Video Reliability Testing. [PDF, 348KB]
ASPE Report: Analysis of Crosscutting Medicare Functional Status Quality Metrics Using the Continuity and Assessment Record and Evaluation (CARE) Item Set. Final Report [PDF, 2MB]

43. On-Going Efforts with standardized CARE Item Set
Developing setting-agnostic quality measures
Quality Reporting Programs for IRF, LTCH, and hospice
E-specification of “Best in Class” by CMS/ONC standards &interoperability groups

44. On-Going Efforts with Standardized CARE Item Set
Developing Outpatient Therapy Payment Alternatives
CARE-C: community therapy
CARE-F: NF therapy
B-Care (subset of standardized items for Bundled Payment Initiatives)
LTSS Care Items (add items for long term social support programs)

45. For More Information: Barbara Gage Bgage@Brookings.edu or Stella Mandl