1 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Who am I ? Internal evaluator employed by North Stoke PCT covering 3 Wave V programmes Started August 2002 Abbey/Bucknall Longton South Shelton/Cobridge/Hanley

2 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent

3 Sewing Group Evaluation Programme Context Shelton/Cobridge/Hanley Diverse communities Language barriers Consultation fatigue (Data drawn from Delivery Plan )

4 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Project Context The sewing group has been operating for around 12 months The co-ordinator is proud of her work Around 8 women attend the group Experience of being an internal evaluator – ethics of accidentally observing an upset woman in the group versus “formal” evaluation role

5 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Methodology = Case Study approach (Yin 2003) Gaining informed consent & access – verbal permission – right to refuse Key informant – “gatekeeper” – what’s in it for them ? Distinguish between evaluation and Public Relations Data protection – access to programme records but anonymise individuals

6 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Who is the evaluation for ? Local families Programme Board – useful evaluation Partner organisations Staff NESS Broad community of practitioners – extending knowledge Researcher’s own interests

7 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Researcher relationship to participants Not counselling Detached observer ? (non-participant observer) Critical friend ? Need for sensitivity, rapport, empathy but not sympathy The ethics of field work (Fox 1998) How to Use Observations in a Research Project Trent Focus for Research and Development in Primary Health Care

8 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Ethical principle of no harm – consider human rights Privacy / confidentiality – complete anonymity not always possible Secure all tapes, transcripts, field notes Data protection – access to programme records but anonymise individuals

9 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Observation technique Strengths : Naturalistic setting – use of senses, ethnography Weaknesses : Can’t eliminate bias Ethical issues – honesty & transparency Honesty and openness enhances credibility

10 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Consider dissemination at the outset Who will read the report ? What might their response be ? Some sensitive issues in this study around women’s permission to access the group – evaluation shouldn’t jeopardise women's’ freedom

11 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Power & Prejudice : Race & representation – can a white woman speak for bme women ? Gender – sensitivities around the acceptability of womens’ groups Communicating across cultural boundaries ( Peberdy 1997) Discuss findings with participants

12 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent LREC – appears daunting & medically biased ! Establish relationship with Research governance manager Enabling not constraining Evaluation on continuum from audit to research so not all evaluation requires ethical approval

13 Pam Carter Senior Evaluation Officer Wave V SureStart Programmes Stoke-on-Trent Evaluation for the learning organisation : Celebrate success Learn from failure Ethical conduct at all times