2. 1 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Who am I ? Internal evaluator employed by North Stoke PCT covering 3 Wave V programmes Started August 2002 Abbey/Bucknall Longton South Shelton/Cobridge/Hanley pam.carter@nsch-tr.wmids.nhs.uk

3. 2 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent

4. 3 Sewing Group Evaluation Programme Context Shelton/Cobridge/Hanley Diverse communities Language barriers Consultation fatigue (Data drawn from Delivery Plan )

5. 4 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Project Context The sewing group has been operating for around 12 months The co-ordinator is proud of her work Around 8 women attend the group Experience of being an internal evaluator – ethics of accidentally observing an upset woman in the group versus “formal” evaluation role

6. 5 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Methodology = Case Study approach (Yin 2003) Gaining informed consent & access – verbal permission – right to refuse Key informant – “gatekeeper” – what’s in it for them ? Distinguish between evaluation and Public Relations Data protection – access to programme records but anonymise individuals

7. 6 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Who is the evaluation for ? Local families Programme Board – useful evaluation Partner organisations Staff NESS Broad community of practitioners – extending knowledge Researcher’s own interests

8. 7 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Researcher relationship to participants Not counselling Detached observer ? (non-participant observer) Critical friend ? Need for sensitivity, rapport, empathy but not sympathy The ethics of field work (Fox 1998) How to Use Observations in a Research Project Trent Focus for Research and Development in Primary Health Care

9. 8 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Ethical principle of no harm – consider human rights Privacy / confidentiality – complete anonymity not always possible Secure all tapes, transcripts, field notes Data protection – access to programme records but anonymise individuals

10. 9 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Observation technique Strengths : Naturalistic setting – use of senses, ethnography Weaknesses : Can’t eliminate bias Ethical issues – honesty & transparency Honesty and openness enhances credibility

11. 10 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Consider dissemination at the outset Who will read the report ? What might their response be ? Some sensitive issues in this study around women’s permission to access the group – evaluation shouldn’t jeopardise women's’ freedom

12. 11 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Power & Prejudice : Race & representation – can a white woman speak for bme women ? Gender – sensitivities around the acceptability of womens’ groups Communicating across cultural boundaries ( Peberdy 1997) Discuss findings with participants

13. 12 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent LREC – appears daunting & medically biased ! Establish relationship with Research governance manager Enabling not constraining Evaluation on continuum from audit to research so not all evaluation requires ethical approval

14. 13 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Evaluation for the learning organisation : Celebrate success Learn from failure Ethical conduct at all times