Star Rating System & Findings from Assessments 2015-2016 Presentation to DPG Health 1st March 2017

Star Rating Presentation Outline Background – BRN key results areas SR system development Milestones – SR implementation Assessment tools and scoring scheme Assessments – illustrative findings Re-assessment model - post Shinyanga Issues for consideration 1

Background - Big Results Now BRN national key result areas (4 NKRAs): Reproductive, Maternal, Neonatal & Child Health Human Resources for Health (HRH); Health Commodities Performance Management - Health Facilities 1) Star rating of PHCF; 2) Fiscal autonomy; 3) Staff performance contracts; 4) Social accountability. 2

Star Rating System Development DHIS DATABASE 12 data collection forms EXCEL score-sheet in dropbox PAPER TOOLS DISP, HC, Hosp. 300+ Questions, 100+ indicators; 12 service areas 4 domains On-site scoring using excel Final star rating result Ongoing: Data cleaning and verification, reports. Off-line analytics on android OS tablet (DHIS scoring & rating). Electronic QI Plans & progress monitoring. Scorecards compiled by facility type, council, region. Dashboard / visualisation.

Assessment Methodology Each day, 1 or 2 health facilities are assessed by a team of two certified assessors plus CHMT/RHMT representative BRN star rating tool (SRT) is used Assessors use a variety of methods: interview of staff; observation; exit interview of clients; review of documents/registers; visit to village/ward office Scoring is done at the facility, and a scorecard plus copy of the completed SRA tool left there Feedback is shared with facility staff including star rating result and areas for improvement 4

Star rating scoring system Four domains (A-D) Facility Management and Staff Performance B Service Charters and Accountability C Safe and Conducive Facilities D Quality of Care and Services

Star rating scoring system 12 Service areas in 4 Domains Health Facility Management (12 indicators) Use of facility data for planning and service improvements (6 indicators) Staff Performance Management (5 indicators) A 2 3 4 Organisation of services (8 indicators) Handling of emergency cases and referral system (7 indicators) Client Focus (4 indicators) Social accountability of the health facility (7 indicators) B 5 6 7 8 C Facility infrastructure (14 indicators) Infection Prevention and Control (11 indicators) 9 10 D Clinical Services (13 indicators) Clinical Support Services (20 indicators) 11 12

ILLUSTRATIVE FINDINGS Dashboards / scorecards for a selection of facilities at baseline and repeat assessment, illustrating improvement from 0-Star facilities through 3-Star. ILLUSTRATIVE FINDINGS Star rating baseline results, comparing council & region performance, with the overall picture at the national level. ‘League tables’ based on proportion of facilities rated 1-Star and above.

Timeline 2015 SR Tools development JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC SR Tools development Excel scoring and dashboard development DHIS database development: forms, scoring and reports, dashboard… Ready for SRA implementation SRA Regions 1-3 (MWA, GEI, SHI) Excel database and data management setup in dropbox SRA (Dar)

Timeline 2016 SRA Regions 7-12 Completed SRA 12 BRN Regions JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC SRA Regions 7-12 Completed SRA 12 BRN Regions Test DHIS mobile App for repeat SRA DHIS mobile App development (offline use) SRA Regions 13-24 Develop DHIS reports for repeat SRA Repeat SRA Pilot (SHI) SRA Regions 25-26 (KIL, ARU) Analysis & Report SRA 20 Regions Completed SRA 26 Region Baseline

Dashboard for Kambarage HC Shinyanga Municipal, 2015 Kambaraga HC is a “quick win” because only one domain has a low score, and three domains already justify a higher star rating

Domain Performance Kambarage HC, Shinyanga Municipal Domain A: Facility Management and Staff Performance Domain B: Service Charters and Accountability Domain D: Quality of Care and Services Domain C: Safe and Conducive Facilities

Scorecard for Dispensaries (n=11) SHINYANGA DC, 2015 Yellow highlighted domains indicate ‘QUICK WINS”

Repeat Assessment 2015-2016 Dispensaries, n=11, SHINYANGA DC Substantial improvement among zero-star facilities, n=9 (1 rated 1-star, 6 rated 2-star, 2 rated 3-star). Moderate improvement among one-star facilities, n=2 (1 rated 2-star). RBF eligibility requirement of at least 1-Star is a driver for improvement of 0-Star facilities 13

KISHAPU DC Findings Dramatic improvement at Maganzo HC: quick win 1-star in 2015 is now 3-star All zero-star facilities raised to 1-star 14

MAGANZO DISP Domains 2015-2016 15

Scorecard for MAGANZO DISP Service Areas 2015-2016

MAGANZO DISP Service Areas 2015-2016 17

Summary of Star Rating Results Shinyanga Region: Re-assessment New facility assessments (n=5) not reported in this tabulation 18

Star Rating Baseline

Star Rating Baseline

Star Rating Baseline

Star Rating Baseline

League Table - Star Rating High and Low Performing Regions

League Table - Star Rating High and Low Performing Councils

Quick Wins Analysis and Readiness for Repeat Star Rating Analysis of 2015 domain scores gives an indication of ‘quick wins’ Quick wins are facilities that are low scoring in a single domain, while high scoring in the other three domains of star rating With focused quality improvement, quick win facilities are likely to be ready for repeat assessment sooner than others

Quick Wins Analysis: Shinyanga Region 2015 Overall, 32% facilities are quick wins Distribution of quick wins (QW) by star-rating for Shinyanga Region: 0-Star: 26 QW / 41 facilities (63%) 1-Star: 35 QW / 144 facilities (24%) 2-Star: 3 QW / 18 facilities (17%) 3-Star: 0 QW / 3 facilities (0%)

Summary of Findings: Shinyanga Region Re-assessments showed 4/41 zero-star health facilities remain at zero star (two in Ushetu DC and two in Shinyanga MC) Shinyanga DC shows dramatic improvement among nine zero-star HF: two are now 3-star, six are 2-star, and one is 1-Star – secret of success? Kishapu DC suggests that RBF can accelerate quality improvement within 2-3 cycles, with verification visits improving documentation 27

Readiness Criteria for Re-assessment (Shinyanga Region Experience) Include all 0-Star (next year all 1-Star) HF Add ‘quick wins’ among 1-Star to 3-Star HF HF showing evidence of improvement (RBF verification, or other visits/ assessments) HF with known active QI teams / investments Strategic location or services (BEmONC) CHMT justified selection of facilities 28

Model for continuing SR assessment? Re-assess 40-60% of HFs annually Rating valid for two years, maximum Assessors – certified? Assigned to zone? Accredited training institutions for SR/QI? Scheduling – demand driven by region? Funding assessments – purchaser? Validation & approval HQA – MOHGDEC? 29

ISSUES Further Consideration Drivers of quality improvement? Synergies - RBF priming, QI implementers Financing QI – NHIF, public & private HF Sustaining quality - pricing (capitation, FFS Repeat assessment - by whom, when? Mandatory certification and accreditation Institutionalisation and sustainability

Welcoming questions, feedback and further discussion Thank you for listening! Welcoming questions, feedback and further discussion 31