1. Evaluating and improving a clinical practice guideline in the Western Cape,
South Africa
AIM STATEMENT: To design and use an appropriate evaluation tool to understand the effectiveness of the PACK
(Practical Approach to Care Kit) programme in the Western Cape, allowing for further improvements in the programme
BACKGROUND
METHODS – EVALUATION TOOL
PACK: A clinical practice guideline and associated training programme aiming to aid clinicians in primary care to diagnose and manage common conditions.
PACK was first implemented in the Western Cape, South Africa in 2014 by the Knowledge Translation Unit at the University of Cape Town. There was no formal evaluation of strategy of the programme in place, so there was limited understanding of how effective the programme was.
It was proposed that an increased understanding of how effective the programme was, and therefore of any problems associated with the programme, would allow for evidence-based, appropriate improvements.
The evaluation tool was implemented in a sample of facilities across the province and comprised multiple techniques:
Questionnaires from clinical staff e.g. how much do they use PACK/how useful do they find it?
Interviews – qualitative detail on effectiveness of PACK to those that use it
Room audits – proportion of clinical room with PACK present
Observed cases – clinical staff observed using PACK
Review of training records – how many staff had been trained in using PACK
Analysis of provincial existing data was analysed to assess current recorded coverage of the PACK programme
PROJECT DESIGN
RESULTS & IMPACT
Plan
Understanding of PACK (implementation process, structure, design)
Stakeholder analysis – current problems, involvement, aims
Design of appropriate evaluation tool to use in primary health care facilities Do
Implementation of evaluation tool in selection of facilities to understand PACK effectiveness
Evaluation of existing provincial PACK data
Study
Analysis of data from evaluation tool and provincial analysis
Understanding of current effectiveness of PACK and associated issues
Act
Written and verbal presentation of results to stakeholders/PACK team
Suggestion and agreement of improvements to PACK
Plan for repeated evaluation in 6 months time
Multiple areas within the programme were highlighted for quality improvement, a selection of which are summarised below. Sustainable, achievable strategies were proposed to address specific identified problems. These were discussed with the PACK committee and implementation strategies for the various solutions were agreed.
Team members
Naina McCann, IGH Fellow
Knowledge Translation Unit (KTU)
Health Programmes Directorate, WC DoH
Pharmacy Directorate, WC DoH
Nursing Directorate, WC DoH
Health Impact Assessment Direcctorate, WC DoH
City of Cape Town Municipality
Highlighted Problem
Evidence
Outcome / Improvement Proposal
Guideline coverage
Only 24% of adult rooms evaluated contained the most up-to date PACK guideline
Designated responsible person per sub-district for new guideline distribution and removal of old guidelines
Annual provincial monitoring of guideline coverage by inclusion in existing provincial audit
Training coverage
55% of clinical staff attended some training
Only 20% of staff able to complete a training module
Quarterly reporting of training coverage and realistic targets set for each sub-district to ensure on-going improvement
Training structure
Feedback from staff that suggested on-site training is difficult as distracted by clinical duties
Each facility to design own training structure according to own needs including off-site, sub-district training if required
Guideline content
Staff wishes for guideline to include a comprehensive index
- Comprehensive index to be considered as addition to PACK 2016 after discussion with PACK committee
M&E
0% of training modules completed currently recorded on database
No feedback to sub0districts on performance
Provincial database to be updated with new data entry system
Quarterly reports to be sent to sub-districts to allow targeted improvement by service users
LESSONS LEARNT
Stakeholder involvement and agreement is vital for any change to occur
Any QI project should have M&E strategies to assess their effectiveness as they are implemented
Change can occur from a variety of different ways (e.g. improved understanding)