Data rich but information poor: Leveraging performance measurement and evaluation to support person-centered care Presented by April Furlong and Chelsea Kirkby The Jean Tweed Centre

Learning Objectives Strategies to improve the quality of performance measurement and evaluation (PM&E) data Processes to integrate PM&E with service delivery Mechanisms to rapidly analyze and share PM&E data with funders, clients, staff and external stakeholders

The Jean Tweed Centre Mission We provide excellent care to support women and their families in the pursuit of health and well-being. Vision To be leaders in the delivery of innovative, accessible, and effective services that address substance use, mental health, and/or problem gambling experienced by women and their families.

Why is Performance Measurement and Evaluation (PME) important? Accountability (funders, board, clients) Monitoring need Identifying gaps – Programs – Staffing – Infrastructure Measuring impacts Assessing client satisfaction/perceptions of care Knowledge exchange

Why is Performance Measurement and Evaluation (PME) challenging? Limited resources Reporting burden Data quality Buy-in Relevance Measurement

The Jean Tweed Centre’s PME Framework Population Dimensions Temporal Dimensions InputsProcessesOutcomes System Program Client Values & Principles Evidence- informed Collaborative Welcoming Culture Holistic Inclusive Accessible Accountable Framework based on Rush, Martin, Korea and Khobzi Rotondi (2012) matrix model framework for outcome monitoring

The Jean Tweed Centre’s PME Implementation Cycle Program Logic Model/ Evaluation Matrix Data Collection Analysis Application Knowledge Exchange Data quality Integration with service delivery Rapid analysis and sharing

Develop Program Logic Model/Evaluation Matrix Based on PME framework – Agency mandate, values and principles Identification of indicators (Health Quality Ontario): – Important/Relevant – Measureable – Actionable – Evidence-based – Feasible – Interpretable – Data quality Population Dimensions Temporal Dimensions InputsProcessesOutcomes System Program Client

Data Collection Goals: High quality and quantity (e.g. response rate) Efficient (minimize resources and staff/client time for collection) Accessible and useable data

Common Data Quality Challenges Collecting standard data – Processes – Measures – Interpretation Collecting accurate data – Staff/client buy-in – Data entry Collecting representative data – Resources – Hard to reach populations

Data Quality Strategies Options for tool administration/data collection – Electronic versus hard copy Data quality reports Staff engagement and reminders Process flowcharts/resources

Example of Process Flowchart (and why it’s needed!)

Data Analysis Issues: Interpretation Resources Timeliness

Application PME Data Program/ Service Improvement Agency-wide comparison Performance Measurement

Integrating PME and Service Delivery Staff/client engagement – Begins with development of logic model/evaluation matrix – Tools inform clinical process – Timely access to results – Mechanisms to discuss results – Mechanisms to apply results Trauma-Informed PME

Why Trauma-Informed PME? Without trauma- informed evaluation: 16 Evaluation may be less effective Individuals may not participate May trigger or retraumatize Welcome?

Principles of a Trauma-Informed Approach Acknowledgement Safety Trustworthiness Choice and control Relational and collaborative approaches Strength-based empowerment modalities 17

A Trauma-Informed Evaluator  Asks permission  Checks in during the conversation as needed  Is sensitive to signs of being overwhelmed  Is prepared to step back  Supports trigger identification  Always mindful of safety  Is self-aware 18  Uses encouraging language  Uses task focussed inquiry  Pays attention to the physical environment  Avoids going into the trauma story other than to validate  Expects some unease  Can teach/use safety and regulation skills such as grounding

Knowledge Exchange (KE) Value of a KE planning Barwick, M. (2011). Knowledge Translation Planning Template-R (TM). Toronto, ON: Hospital for Sick Children Toronto.

Coming up…! (and why PME need to be adaptive) Enhanced client engagement Ontario Perceptions of Care (OPOC) – Mental Health and Addiction systems Staged Screening and Assessment Process – Addictions system Data and Performance Measurement Framework for Mental Health and Addictions in Ontario Health Equity Impact Assessment (Ministry of Health and Long- term Care) System integration