1. Star Rating System & Findings from Assessments 2015-2016
Presentation to DPG Health
1st March 2017

2. Star Rating Presentation Outline
Background – BRN key results areas
SR system development
Milestones – SR implementation
Assessment tools and scoring scheme
Assessments – illustrative findings
Re-assessment model - post Shinyanga
Issues for consideration 1

3. Background - Big Results Now
BRN national key result areas (4 NKRAs):
Reproductive, Maternal, Neonatal & Child Health
Human Resources for Health (HRH);
Health Commodities
Performance Management - Health Facilities 1) Star rating of PHCF; 2) Fiscal autonomy; 3) Staff performance contracts; 4) Social accountability. 2

4. Star Rating System Development
DHIS DATABASE 12 data collection forms
EXCEL score-sheet in dropbox
PAPER TOOLS DISP, HC, Hosp.
300+ Questions, indicators; 12 service areas
4 domains
On-site scoring using excel Final star rating result
Ongoing: Data cleaning and verification, reports.
Off-line analytics on android OS tablet (DHIS scoring & rating). Electronic QI Plans & progress monitoring.
Scorecards compiled by facility type, council, region. Dashboard / visualisation.

5. Assessment Methodology
Each day, 1 or 2 health facilities are assessed by a team of two certified assessors plus CHMT/RHMT representative
BRN star rating tool (SRT) is used
Assessors use a variety of methods: interview of staff; observation; exit interview of clients; review of documents/registers; visit to village/ward office
Scoring is done at the facility, and a scorecard plus copy of the completed SRA tool left there
Feedback is shared with facility staff including star rating result and areas for improvement 4

6. Star rating scoring system Four domains (A-D)
Facility Management and Staff Performance B
Service Charters and Accountability C
Safe and Conducive Facilities D
Quality of Care and Services

7. Star rating scoring system 12 Service areas in 4 Domains
Health Facility
Management
(12 indicators)
Use of facility data for planning and service improvements
(6 indicators)
Staff Performance Management
(5 indicators) A 2 3 4
Organisation of services
(8 indicators)
Handling of emergency cases and referral system
(7 indicators)
Client Focus
(4 indicators)
Social accountability of the health facility (7 indicators) B 5 6 7 8 C
Facility infrastructure
(14 indicators)
Infection Prevention
and Control
(11 indicators) 9 10 D
Clinical Services
(13 indicators)
Clinical Support Services
(20 indicators) 11 12

8. ILLUSTRATIVE FINDINGS
Dashboards / scorecards for a selection of facilities at baseline and repeat assessment, illustrating improvement from 0-Star facilities through 3-Star.
ILLUSTRATIVE FINDINGS
Star rating baseline results, comparing council & region performance, with the overall picture at the national level. ‘League tables’ based on proportion of facilities rated 1-Star and above.

9. Timeline 2015 SR Tools development
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
DEC
SR Tools development
Excel scoring and dashboard development
DHIS database development:
forms, scoring and reports, dashboard…
Ready for SRA implementation
SRA Regions 1-3 (MWA, GEI, SHI)
Excel database and data management setup in dropbox
SRA (Dar)

10. Timeline 2016 SRA Regions 7-12 Completed SRA 12 BRN Regions
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
DEC
SRA Regions 7-12
Completed SRA 12 BRN Regions
Test DHIS mobile App for repeat SRA
DHIS mobile App development (offline use)
SRA Regions 13-24
Develop DHIS reports for repeat SRA
Repeat SRA Pilot (SHI)
SRA Regions (KIL, ARU)
Analysis & Report SRA 20 Regions
Completed SRA 26 Region Baseline

11. Dashboard for Kambarage HC Shinyanga Municipal, 2015
Kambaraga HC is a “quick win” because only one domain has a low score, and three domains already justify a higher star rating

12. Domain Performance Kambarage HC, Shinyanga Municipal
Domain A: Facility Management and Staff Performance
Domain B: Service Charters and Accountability
Domain D: Quality of Care and Services
Domain C: Safe and Conducive Facilities

13. Scorecard for Dispensaries (n=11) SHINYANGA DC, 2015
Yellow highlighted domains indicate ‘QUICK WINS”

14. Repeat Assessment 2015-2016 Dispensaries, n=11, SHINYANGA DC
Substantial improvement among zero-star facilities, n=9 (1 rated 1-star, 6 rated 2-star, 2 rated 3-star).
Moderate improvement among one-star facilities, n=2 (1 rated 2-star).
RBF eligibility requirement of at least 1-Star is a driver for improvement of 0-Star facilities 13

15. KISHAPU DC Findings
Dramatic improvement at Maganzo HC: quick win 1-star in 2015 is now 3-star
All zero-star facilities raised to 1-star 14

16. MAGANZO DISP
Domains 15

17. Scorecard for MAGANZO DISP Service Areas 2015-2016

18. MAGANZO DISP Service Areas 2015-201617

19. Summary of Star Rating Results Shinyanga Region: Re-assessment
New facility assessments (n=5) not reported in this tabulation 18

20. Star Rating Baseline

21. Star Rating Baseline

22. Star Rating Baseline

23. Star Rating Baseline

24. League Table - Star Rating High and Low Performing Regions

25. League Table - Star Rating High and Low Performing Councils

26. Quick Wins Analysis and Readiness for Repeat Star Rating
Analysis of 2015 domain scores gives an indication of ‘quick wins’
Quick wins are facilities that are low scoring in a single domain, while high scoring in the other three domains of star rating
With focused quality improvement, quick win facilities are likely to be ready for repeat assessment sooner than others

27. Quick Wins Analysis: Shinyanga Region 2015
Overall, 32% facilities are quick wins
Distribution of quick wins (QW) by star-rating for Shinyanga Region:
0-Star: 26 QW / 41 facilities (63%)
1-Star: 35 QW / 144 facilities (24%)
2-Star: 3 QW / 18 facilities (17%)
3-Star: 0 QW / 3 facilities (0%)

28. Summary of Findings: Shinyanga Region
Re-assessments showed 4/41 zero-star health facilities remain at zero star (two in Ushetu DC and two in Shinyanga MC)
Shinyanga DC shows dramatic improvement among nine zero-star HF: two are now 3-star, six are 2-star, and one is 1-Star – secret of success?
Kishapu DC suggests that RBF can accelerate quality improvement within 2-3 cycles, with verification visits improving documentation 27

29. Readiness Criteria for Re-assessment (Shinyanga Region Experience)
Include all 0-Star (next year all 1-Star) HF
Add ‘quick wins’ among 1-Star to 3-Star HF
HF showing evidence of improvement (RBF verification, or other visits/ assessments)
HF with known active QI teams / investments
Strategic location or services (BEmONC)
CHMT justified selection of facilities 28

30. Model for continuing SR assessment?
Re-assess 40-60% of HFs annually
Rating valid for two years, maximum
Assessors – certified? Assigned to zone?
Accredited training institutions for SR/QI?
Scheduling – demand driven by region?
Funding assessments – purchaser?
Validation & approval HQA – MOHGDEC? 29

31. ISSUES Further Consideration
Drivers of quality improvement?
Synergies - RBF priming, QI implementers
Financing QI – NHIF, public & private HF
Sustaining quality - pricing (capitation, FFS
Repeat assessment - by whom, when?
Mandatory certification and accreditation
Institutionalisation and sustainability

32. Welcoming questions, feedback and further discussion
Thank you for listening!
Welcoming questions, feedback and further discussion 31