Methods of evaluation of the effects of care Course for Young Psychiatrists Addis Ababa, 28 th April 2006 David Goldberg

What are the real aims of your care? Acute illnesses – include measures of symptom severity Chronic illnesses – spend less time on symptom severity, use a brief measure, remember Quality of Life ALWAYS include disability measures Usually include satisfaction measures

What are the real aims of your care? Acute illnesses – include measures of symptom severity Chronic illnesses – spend less time on symptom severity, use a brief measure, remember Quality of Life ALWAYS include disability measures Usually include satisfaction measures

Other indicators of care Unmet Need – measures include the “CAN”, Camberwell Assessment of Need Economic measures – if you can reduce the cost of care, politicians in your country will be greatly interested!

Evaluation design Before/after evaluations Controlled studies Short/long term outcomes

Design of the Evaluation There are two sorts of control groups: 1) Random assignment of individuals to the new treatment or the “usual treatment” (called “TAU”) 2)Choosing two different places with comparable patients, and introducing the new treatment in only one of them Evaluations without a control group are a waste of everyone’s time!

Methods of evaluation of success of training courses Course for Young Psychiatrists Addis Ababa, 28 th April 2006 David Goldberg

Evaluation: impact on clinicians Is it possible to bring about a change in clinician knowledge, attitudes or skills? Does the intervention improve morale or confidence? How satisfied are the clinicians with the intervention?

Assessing knowledge Self-evaluation Instructor’s opinions objective measures

Assessing Attitudes Questionnaires (e.g. Depression Attitude Questionnaire; Attitudes to Psychiatry) Interviews –structured –semistructured

Skill acquisition Observed Structured Clinical Examinations –rate against predefined scorecard. Both of these are “before and after”: Videotaped interviews with real or role- played patients –rated ‘blind’ using structured rating scales Changes in ability to identify or assess accurately emotional disorder using comparison of patient GHQ rating and PCP rating; numbers of patients identified; prescriptions issued

Improvement in morale/confidence Self-rated linear analogue scales Interviews Post-training assessment rated against pre-training self-assessment of needs/objectives

Evaluation: Impact on process and outcome of care Process of care: –frequency and length of visits –prescription of medication –referrals –use of investigations Outcome of care: –hospitalisations –numbers improved, symptom free

Evaluation: Impact on process and outcome of care Patient satisfaction Compliance Clinical outcome –symptoms –disability Economic outcome - how many return to work?