1. Inf-Qual November 11. 2004 Elisabeth Jakobsen

2. “All I want is a system that works” Evaluation of the health information system in Cape Town, South Africa – using an information audit to capture views from the grass root level

3. Theme for the thesis: Evaluation The thesis covers two different kinds of evaluation: 1. Evaluation of the health information system in Cape Town. 2. Evaluation of a tool used to investigate the information system; an information audit.

4. Background for the work HISP – Health Information System Program. DHIS – District Health Information Program. Ongoing project in Cape Town due to limited use of health data at district level.

5. Health information Different levels of information Figure 1. Information pyramid

6. Kinds of health information systems Two main groups:  Clinic health information systems  Routine health information systems (our focus) Health information systems in developing countries  Poor infrastructure  Lack of qualified personnel, ”brain drain”  Epidemics (HIV/ADIS, TB)  Unstable political structures

7. Evaluation methods Qualitative and quantitative choice Qualitative methods in evaluation:  In depth interviews  Direct observation  Studies of written documentation “The challenge in evaluation is getting the best possible information to the people who need it – and then getting those people to actually use the information in decision making”. (Source: Patton, 1987)

8. Evaluation methods (2) Evaluation of health information systems:  Often complex systems  Need for a multi method approach An evaluation methodology should consider different aspects of an information system, not only the technical or economic.

9. Method for collecting data Action research Questionnaire, interviews, observation, tape recordings, note taking. Fig 3. Action research – basic cycle

10. Health services and health information systems in Cape Town (1) Facts about Cape Town:  Population: 3,29 million.  11 health districts.  Health services administrated by Local Authority (LA) and Provincial Administration of Western Cape (PAWC).  LA runs 105 primary health clinics (PHC), TB and HIV services. Preventive branch.  PAWC runs birth clinics, Community Health Service Organisations, secondary and tertiary hospitals (4 district hospitals and 52 clinics). Curative branch.

11. Health services and health information systems in Cape Town (2) Restructuring. Separate curative and preventive services Competition between different software- programs; think globally or develop locally?

12. Our approach Focus on clinic level in curative branch  Overview over existing equipment  Views from the grass root level Investigated:  4 district hospitals  17 clinics

13. Future evaluations Yearly basis Three parts Paper based questionnaires Summaries for each district Make the results available for decision makers, employees and others!!

14. Findings

15. Critical work stations A point where much of the information work is performed, and where the work load is large  Reception  Managers office  Information officer  Pharmacy

16. Perceptions of the information system

17. Need for resources Agreement on what the most needed resources are:  Training  Computers  Personnel

18. Application of training Many reported they had not made any use of training received. Mainly due to lack of computers in the clinics where they work.

19. Data accuracy Large confidence in data accuracy  47% believes collected data is more than 90% accurate Most, 85%, check the accuracy. This is done manually.

20. Information flow

21. Use of data

22. Star example Oceans View More staff due to increased number of psychiatric patients. Project due to high number of teenage pregnancies. Detected outbreak of meningitis. Increased focus on TB. Increased number of HIV-tests. Report need for extra dentist and dental assistant.

23. Recommended focus Transfer power to lower levels. More training. Better equipment. Useful feedback that makes sense. More communication, both vertically and horizontally in the organisation.

24. Similarities Lack of equipment (PCs) Too little training Lack of personnel Varying view of accuracy Lack of power as argument for limited use of information Recommendations of patient based systems

25. Differences and changes How to recruit information workers. The view on including the ground levels. Better information flow. Better tools for validation and accuracy checks.

26. Conclusion Large need for:  Equipment  Training  Personnel Limited use of collected information at all levels – communication is the key!