Measurement for Improvement

Turn to your neighbor What have been your biggest learnings or challenges regarding data gathering and measurement for your Early Years work? – Definitions – Collection – Reporting – Frequency – Analysis – Other?

Components of a Learning System 1.System level measures 2.Explicit theory or rationale for system changes 3.Segmentation of the population 4.Learn by testing changes sequentially 5.Use informative cases: “Act for the individual learn for the population” 6.Learning during scale-up and spread with a production plan to go to scale 7.Periodic review 8.People to manage and oversee the learning system From Tom Nolan PhD, IHI

What are we trying to Accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? The Model for Improvement ActPlan StudyDo Source: Langley, et al. The Improvement Guide, Our focus today The three questions provide the strategy The PDSA cycle provides the tactical approach to work

AIM (How good? By when?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA The Quality Measurement Journey Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004.

Case Study Promoting Bedtime Reading Lochrin and Grassmarket Nursery Schools Edinburgh Lead: Donna Murray

AIM – Improve bedtime routines, attachment, literacy Concept – Increase bedtime reading Measures – Percentage of children receiving a bedtime story Operational Definitions – Number read story/Total Children Data Collection Plan – daily; no sampling Data Collection – teacher asks parents at drop off, spreadsheet Analysis – run chart PDSA The Quality Measurement Journey

Children have all the developmental skills and abilities expected at the start of primary school Societal Issues Child’s physical & mental health and emotional development Improved sharing of information Improved management, planning and quality of services Improved joint working Improved identification Detailed Aim: 90% of all children within each CPP have reached all of the expected developmental milestones at the time the child starts primary school, by end-2017 Health Improved child’s dental health Improving child nutrition Attachment Improved stability / permanence for LAC Improved uptake of benefits Improved family centred response Improved leadership, culture & planning` Improved teamwork, communication and collaboration Improving brain development and physical play Early Learning & Play WORKSTREAM 3 (30 months to start of primary school) Identification & reasons for current resilience Aim 1⁰1⁰ 2⁰2⁰ Theory of what drives developmental milestones Theory of what actions will ensure developmental milestones are reached at the start of primary school Poverty Quality Of Home Environment Domestic Abuse & Violence Workforce Issues Transport, Community Capacity & Cultures Access To Services Employment Carer’s physical & mental health and skills Level of education Misuse of alcohol & drugs Nutrition Disabilities & Mental health Parenting skills & knowledge Additional Support Version: 06/03/2013

AIM (How good? By when?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA The Quality Measurement Journey Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004.

Project Aim Statements What, by when, by how much. – “By the end of June, 90% of children at Grassmarket Nursery will receive a bedtime story at least 3 times a week.”

AIM (How good? By when?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, PDSA The Quality Measurement Journey

Children have all the developmental skills and abilities expected at the start of primary school Societal Issues Child’s physical & mental health and emotional development Improved sharing of information Improved management, planning and quality of services Improved joint working Improved identification Detailed Aim: 90% of all children within each CPP have reached all of the expected developmental milestones at the time the child starts primary school, by end-2017 Health Improved child’s dental health Improving child nutrition Attachment Improved stability / permanence for LAC Improved uptake of benefits Improved family centred response Improved leadership, culture & planning` Improved teamwork, communication and collaboration Improving brain development and physical play Early Learning & Play WORKSTREAM 3 (30 months to start of primary school) Identification & reasons for current resilience Aim 1⁰1⁰ 2⁰2⁰ Theory of what drives developmental milestones Theory of what actions will ensure developmental milestones are reached at the start of primary school Poverty Quality Of Home Environment Domestic Abuse & Violence Workforce Issues Transport, Community Capacity & Cultures Access To Services Employment Carer’s physical & mental health and skills Level of education Misuse of alcohol & drugs Nutrition Disabilities & Mental health Parenting skills & knowledge Additional Support Version: 06/03/2013 Drivers Concept: Bedtime Reading Concept: Bedtime Reading

Driver Concept and Measure Measures to provide feedback on the concept – % receiving a story – % of stories read at bedtime – % of parents reporting improved bedtime routine – % enjoying the bedtime story – % reporting increase in bedtime story reading

AIM (Why are you measuring?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA The Quality Measurement Journey Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004.

Operational Definitions What does reading a story mean? When is bedtime versus other time? What is improved bedtime routine? What is the definition of enjoyed?

AIM (Why are you measuring?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, The Quality Measurement Journey

Data Collection Planning & Doing All children are included, no sampling. 7 days a week. Donna will inquire with each parent at drop off Monday through Friday. Data will be captured on a paper form. Donna inputs daily data into spreadsheet following drop off.

AIM (Why are you measuring?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, The Quality Measurement Journey

Analysis – Run Charts Grassmarket changes introduced. Books available at collection time. Research information handed to parents. weekly average displayed for parents

AIM (Why are you measuring?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, The Quality Measurement Journey

Action – Testing Change Ideas

NHS Lanarkshire – Our Month Review Data Journey Stacey Rooney

Background NHS Lanarkshire Clinical Quality Service, supporting both North and South Lanarkshire CPPs from a health perspective month reviews commenced June 2013 in Lanarkshire – measurement data for stretch aim 2 But we don’t have baseline data……. ….so we need early access to our review data to develop a baseline……. ISD Proposed quarterly reports…….November 2013 before we see our first set of results

Accessing our data – lots of questions.... Where do the forms go? Does anyone in Lanarkshire have access to the CHSP system? Can we access the system? Technical challenges accessing the system....patience patience we’re in!! We’re in but what do all these codes mean? Expertise to interrogate the data

How did we access CHSP Business Objects Universe (AKA – our data)? Contact the ATOS Helpdesk ITServiceDesk.NHSS- Receive a form to complete via , needs authorising by your Child Health Administrator Receive complex instructions from ATOS on how to set-up your PC/laptop – stick with it, you’re getting there!! Receive your Business Objects Universe login and password from your local Child Health Administrator

CHSP Business Objects

Benefits of accessing our data Earlier access to our data – some delays come from the time to return the forms for data entry and data entry itself (approx 4 weeks) Early identification of quality issues with form completion enabled us to feed this back and engage with staff We have early baseline data Stratify the data as we like (unit, locality, team) Results are helping us identify early tests of change

Early data focus Meaningful reviews (9 developmental milestones reviewed and recorded) Stretch aim 2 (meeting all developmental milestones) Future actions (request for assistance/referral) Looked After Status Breeches Health Plan Indicator (HPI) – status change Attendance / DNA – obtained locally

Coverage – A work in progress: Number of Eligible Children Number Invited for Review Number Attending for Review Number of Meaningful Reviews A national definition of ‘eligible children’ in a reporting period is still in development. Eligible = denominator/starting point Invited:coverage measure 1 Attended:coverage measure 2 Meaningful:coverage measure 3

Coverage: An early example

Examples of our early data (p) Provisional data - incomplete

Limitations of our early data We’re still developing a robust process to measure uptake - so results are only representative of those attending Non attenders are potentially our more vulnerable children and this may be skewing our results – particularly stretch aim 2 Non-meaningful reviews are being excluded, we need to reduce this (approx 25%)

Stage 2 data focus What other data is available from the review? –Age of child at review –SIMD quintiles –Ethnicity –Place of review (home, GP practice, clinic) –Exposure to smoke –Registered with a dentist –Bi/Multi-lingual

Reporting

Monthly reports Monthly summary and locality level reports are in development

Next steps Continue to work with colleagues to develop reporting that is meaningful and facilitates/guides improvement Continue to feedback real time issues with data completeness and quality Further work to link other reviews and develop data intelligence over time for each child e.g. HPI Further work to consider how best to share this data with partners out with health to benefit children and families

Thank you. I hope that didn’t leave you feeling like this or this

I hope it did make you feel like this..... Thank you to Scott Purdie, Clinical Quality Co-ordinator, NHS Lanarkshire for his work in this area. And My niece Emily, age 9 months for brightening up the data talk!