1. Seattle Partners Process Evaluation: Participation
Noel Chrisman, Interview Participants, Executive Committee
Draft for the Community Board
December 15, 2003

2. Seattle Partners Process Evaluation: Participation
From the beginning, Seattle Partners has focused on partnerships and participation.
This most recent edition of the process evaluation examines these ideas in the context of our partnerships.

3. Evaluation
Is a research process to assess and monitor the degree to which stated goals are reached

4. Three Types of Evaluation
Outcome. Did the project have the effects that it projected?
Intermediate (“impact”). Were the correct steps taken to fulfill the theory/method of the project?
Process. How was the project carried out?

5. Participation
Involvement of people to define their own needs, set their own priorities, control their own solutions, and evaluate their own progress.
Green 1996, p. 212

6. Participation To be part of the project.
Have some part of the benefits.
Work in the project.
Decision-making role in the project.
Work on monitoring and evaluation.
Develop and implement the project.
Stone 1992

7. Aspects of Participation: From the Literature
Relationships between researchers and CBOs & communities:
Consultation
Planning
implementation
Within an organization:
degree to which the community has an influence;
degree to which staff members affect direction.

8. Aspects of Participation: Community Collaboration Principles
Early involvement
Real influence
Involved with projects
Respect of values, confidentiality
Benefit the community

9. Methods Descriptive design (what has happened?)
Open-ended interviews to maximize hearing community voices
Sample included nine agencies (including two governmental units) and one grass roots committee carrying out seven projects.

10. Methods Nineteen interviews; nineteen interview participants.
One person interviewed twice for two different projects. Two people from one project interviewed jointly.
Agency directors, project staff, and researchers.
Content analysis to highlight interview participant perspectives

11. Community Collaboration Principles
Community involved in plans and development from the beginning. Benefits. “In the early meetings, they took time to see what we want from the research project. How do we see their participation” “It was almost dreamlike” “We say how we do research and what to look at and they have to learn how to do it.”

12. Community Collaboration Principles
Community involved in plans and development from the beginning. Challenges. “Our staff was exhausted, stretched too far.” “We need to focus on the service. The project was not supposed to pull our staff so heavily.”

13. Community Collaboration Principles
Community partners have real influence on SPHC directions and activities. Benefits. “This took a long time [planning and starting]. We were supposed to start. It held us back. There were lots of meetings. Like the planning, we had to iron out our differences, to understand more. It was very useful.”

14. Community Collaboration Principles
Community partners have real influence on SPHC directions and activities. Challenges. “[working in CBPR] is moving into new territory for CBOs. They need pampering and guidance.” “Since the agency had not done research before, how can they be expected to know how to do it.” >Little interaction between the CB and agencies/projects.

15. Community Collaboration Principles
Community involved with specific projects through all aspects of research. Benefits. “Now we know how to do a project!” “to pull off the research, show the community, and we had a good outcome.” “to work with the health department.”

16. Community Collaboration Principles
Community involved with specific projects through all aspects of research. Challenges. “Agencies feel they were burdened by time consuming relations with researchers trying to get things done in a small amount of time. Line workers abandoned their own work and had to do the research. The time demand was not reasonable at the time.”

17. Community Collaboration Principles
The values, perspectives, contributions, and confidentiality of community members are respected. Benefits. “Still the advocates have a good relationship with [the researchers]. They are comfortable with them. It’s hard to get to trust. Not business only. We want to see your human side. Relationships are very important.”

18. Community Collaboration Principles
The values, perspectives, contributions, and confidentiality of community members are respected. Challenges. “it’s the whole way you view the world, approach the research and the community. That attitude seeps through and the community sees it.”

19. Community Collaboration Principles
Research will serve the community through sustaining long-term, beneficial projects and developing community capacity. Benefits. “Yes, there was capacity building. [The staffer] had encouragement, ideas, located resources, grants, a location, grant writing….Resources: where to go, what to do, who to seek, training, work on grants.”

20. Community Collaboration Principles
Research will serve the community through sustaining long-term, beneficial projects and developing community capacity. Challenges. “Public health and the UW are in a different world from community agencies.” “The research drives Seattle Partners behavior: timelines, quantitative focus of the work, academic pressures to produce papers, to produce a piece of research to affect public health specifically.”

21. Organizational Partners
Degree to which staff members affect direction.
member participation is well structured into the life of the agency in all the agencies.
Director, coordinators, program staff.
Regular meetings in which all had a voice.

22. Organizational Partners
Degree to which the community has an influence.
personal contact with community, on the street and as clients.
surveys, town meetings, focus groups, regular reports from project coordinators.
community advisory board.

23. Organizational Partners
CBOs are organized in these ways and are closely related to communities because they have a mission of service to their communities and clienteles. In addition, many people from these organizations stressed that powerful relationships with communities is a central goal of their work.

24. Organizational Partners
Between researchers and CBOs
Consultation
Planning
implementation
It is clear from the interviews that all three levels of participation are common among the partners of Seattle Partners.

25. Themes
Positive Reaction “When Seattle Partners first started, I was really happy because the community does amazing work, but we need to document and publish. Here was a group who could do that.” They did some great evaluations for us.” “Having UW and health department cachet is helpful for later funding. It lends credence.”

26. Themes
Positive Reaction “Without SPHC we probably would not have started.” “lots of funders do not understand the community. Seattle Partners does; starting a step ahead in terms of other funders.

27. Themes Expenditure of staff time on research goals.
Contrasts in who had to hurry. “There was pressure to conform to quick timelines of the UW, PH-SKC, APHA, CDC, but not the same willingness for those institutions to conform.” “It took two or three months to get supplies because of red tape and bureaucracy.”

28. Themes Community-Researcher Relationships Get to know the community
Closer relationship between the CB and agencies
Greater understanding of agency and health department goals and pressures
“I don’t think the consortium had an understanding of the impact of the project. I don’t know how we set the agenda at the table. Often it was the opposite. I feel like public health set the agenda and we spent energy and resources for the evaluation.”

29. Themes Community-Researcher Relationships
Seattle Partners was frequently referred to as the University and the Health Department. “The struggle involved internalizing the power of public health and the university coming to communities.”

30. Themes
Capacity building “CBOs need pampering and guidance” “What it did for us was to help us understand the value of community-based research: to change policy, organize, provide information, these were the major things we learned. Now we have a full time staff member to do the organizing [presumably because of the success of research].”

31. Themes
Scientific and personal styles of researchers varied. Researchers “did not understand the community, but they did seek out ways to understand. Academicians who deal with numbers…with very structured complicated issues. The community was disconnected from them.” “For example, a public health staff person referred to the asset based approach as just semantics.”

32. Themes
Contrasts between research and community work. “They are so driven to report to the CDC they did not see what was going on.” “If researchers want community-based research, their heart has to be in the community.”

33. Themes
Contrasts between research and community work. “the staff really struggled with the new system, on a number of fronts. They had epidemiology: a cut and dried evaluation, standards, rigor, IRB approval. At the same time, they wanted to be sensitive to working in communities.” “The struggle was not so much about design as about implementation.”

34. Recommendations Relationships with Partners
Reciprocal relationships between agencies and a community board; active involvement.
Equal contributions to agenda setting.
Clarity in the relationships of mission and goals for the community board and agencies.
Assess strengths and weaknesses of agency capacity for research.

35. Recommendations Relationships with Partners
Mutual discussion of agency and board expectations in which compromises are expected.
Balance the science and service goals of researchers and agencies.
Opportunities for agencies to work on their interrelationships.

36. Recommendations
Maintain an environment in which research planning and implementation are shared with partners.
How the research is structured (e.g., design; sampling; time frames; role of the IRB).
Accurate prediction of agency workload and agreed upon reimbursement.
Agency and community culture must be accounted for in study design and process.

37. Recommendations
Community Board projects should be lodged in community coalitions or agencies to promote sustainability.
Staff preparation/discussion should be transparent to the board and agencies.

38. Recommendations
The coalition should overtly value interpersonal and interorganizational relationships.
Personal relationships are crucial in community work. Participation and decision-making are facilitated through relationships.
Personal relationships promote the community sense that the coalition is committed to the community.

39. Recommendations
Researchers should receive explicit training in CBPR processes.
Cultural competence is necessary.
Foster a continuing dialog between traditional public health and CBPR to allow for the growth of both.