Evaluating community-based peer support programs Christine Walker Chronic Illness Alliance 1

Why evaluate? Previous systematic reviews found that peer support programs were often poorly and inconsistently evaluated if evaluated at all. Most reviewers considered that peer support was effective though the evidence was poor. CIA set out to develop a tool based on the most recent evidence of effectiveness 2

Why evaluate? Overall aim to have peer support recognised as effective adjunct in self-management and give GPs etc confidence to refer their patients. Aim of the tool is to develop easy to use tool that can be applied across all kinds of support groups Tool will provide a database of consistent information Feedback to the individual support groups Database of evidence on peer support effectiveness Undertaken with HCSG funding from DHHS Victoria 3

The project First step- a lit review on most recent evidence of effectiveness conditions-2/ Consultations with peer support group leaders and members Consultations with GPs and PNs on what constitutes quality in delivery of peer support Development and piloting of tool 4

Literature review Undertaken in partnership with university There were 162 relevant articles identified, and synthesized based on three levels: (1) organisational level (governance structure and evaluation framework); (2) program level (best practice to evaluate elements and delivery of peer support); (3) participant level (outcomes of peer support programs). This scoping review identified several examples of peer support programs with effective program evaluation, as well as evidence supporting the benefit of peer support for people with chronic conditions, particularly for improving psychological wellbeing. Yet there is a lack of an evaluation framework used by peer support leaders to ensure their program meets best-practice standards. 5

Consulting peer support leaders and members Took place in a small workshop in Bendigo where there is an active hub of support groups Attendees from support groups represented heart conditions, cancer, arthritis, epilepsy, cystic fibrosis and diabetes Discussion centred on the topic of ‘how do you know your group is successful’ Most groups evaluate at an informal level and are concerned with keeping attendance up, building community relations, having a program that keeps people engaged Most want to build better relationships with primary care Varying levels of IT literacy 6

Consulting with GPs and PNs Many PNs involved in support groups; some as patients All considered governance was important as this underpinned a well functioning group Facilitation important Value of groups was reducing social isolation, education, building confidence and well-being Patients talk about the groups to their GP; not necessarily to the PN Some groups (Bendigo) require a doctor’s referral Willingness by GPs and PNs to refer to support groups Some suggestion of need for measurable outcomes e.g. blood glucose, BP, weight. 7

Components of an evaluation framework based on lit review and GP/PN consults Governance: constitution, strategic planning, executive structure Quality assurance: training, recruitment strategy, policies, risk management Finances: source of ongoing funding or substitute (volunteers) budget and financial management Referrals: recruitment and promotion strategy Benchmarking: tools to identify how well group is going Evaluation strategy 8

Designing an evaluation tool Based on the responses of support groups the tool needs to be: simple to use (cater for all IT skill levels) needs to be available as a document cover all aspects of the domains important in a framework provide evidence of outcomes (well-being) respect support groups privacy and autonomy 9

The evaluation tool so far Has been designed as a survey covering all the domains Short questions, largely yes, no answers; opportunities to explain or give more details Ryff well-being scale modified and provided as a separate survey that leaders distribute Answers may be collated and scores gained Explanations accompany the surveys Small pilot conducted 10

Ryff scale for well-being and survey tool Modified version (see the full version) Simple to use and calculate We collect de-identified data on people’s sense of control in their lives and on their relationships Use the results with a survey covering domains identified as important by GPs/PNs and support groups 11

Data collection Data collected on a 21 question survey covering – governance, – program structure, – benefits of group to members, – sustainability issues We used plain English rather than these terms. 12

Data collection Well-being survey is distributed to group members to complete anonymously. The results are collated and the peer support leader can judge the contribution of the support program to people’s well-being. Survey on group is completed by the support group leader. We undertake a brief analysis of the relationship between the running of the group and people’s well- being. Make recommendations on any improvements. 13

Where are we up to? Surveys are on-line under PLOT A piece of software being developed to assist analysis. ID to be assigned to each p s group to maintain anonymity. Each group will receive its own report and be able to see how it performs against the average/overall groups. Receive recommendations for any improvements 14

15

16

17