1. Participant-Oriented Evaluation
No Script on Intro Slide
Prepared by:
Daniel Wagner
Jahmih Aglahmie
Kathleen Samulski
Joshua Rychlicki

2. Participant-Oriented Evaluation
Evaluation approach focusing on the participants
Participants have interest in the results
Developed in response to the needs and interests of those associated with the evaluation
Script:
What is participant-oriented evaluation?
Participant-oriented evaluation is an evaluation approach focused primarily on serving the needs, interests and values of those participating in the program. These approaches emphasize the fact that evaluations are done for particular participants whose values vary and must be addressed in fair and systematic ways if justice is to be met and the participants are to have sufficient interest in using the evaluation results. Indeed, over time, evaluation has become increasingly attentive to the needs and interests of wider and more diverse groups of people associated with the things being evaluated.

3. Characteristics of a Participant-Oriented Approach
The evaluation is dependent on inductive reasoning
The evaluation utilizes data from multiple sources
The evaluation does not follow a standard plan
The evaluation records multiple point-of-views or perspectives
Script:
A general participant-oriented approach to evaluation has the following characteristics:
The evaluation is dependent on inductive reasoning. Learning about the evaluated program is ususally done in a grassroots, or discovery manner.
The evaluation utilizes data from multiple sources. In order to create a vivid description of stakeholders’ needs, quantitative and qualitative data need to be collected, although participant-oriented evaluations tend to rely more on qualitative data.
The evaluation does not follow a standard plan. One key function of this type of evaluation is its ability to adapt to stakeholders and situations.
The evaluation records multiple point-of-views or perspectives. Participant-oriented evaluations generate views that may constitute varying realities.

4. An Evolved Approach
Other evaluation approaches have been adapted into participant-oriented approaches.
Script:
In response to the development of participant-oriented evaluation, many goal-based, goal-free, decision-making, theory-based, and other evaluation approaches have been adapted into participant-oriented approaches. These approaches encourage all evaluation efforts to attend to the interests and values of the participants.

5. Three Theorists Involved in Approach
Robert Stake
Egon Guba & Yvonne Lincoln
Michael Q. Patton
Script: We have chosen to comment on the research of Robert Stake, Egon Guba, Yvonne Lincoln, and Michael Patton for this project. Since we are the stakeholders in this presentation, a participant-oriented evaluation of this assignment would allow us to choose whichever researchers we believe to be important. This can also be seen as a disadvantage of this approach.

6. Robert Stake
First evaluation theorist to introduce Participant Oriented Evaluation into the field of education.
Introduced the theory of Responsive Evaluation which focuses on “re-directing data gathering and interpretative efforts around emerging issues of importance to program practitioners in the evaluation setting” (Abma, 2005)
Created the Countenance Framework in The model refers to the two faces of evaluation: description & judgment.
Description includes the evaluator’s observation and list of benchmarks for the activities being evaluated. Judgment is the evaluator’s overall rating of merit.
Script: Robert Stake is the “father” of the Participant-oriented evaluation movement. Stake’s views on responsive evaluation and his creation of the countenance model have provided the basis for the development and evolution of this evaluation approach. The countenance framework is particularly important in understanding the participant-oriented evaluation method, as it defines the major activities of evaluation as a full description and a judgment of that which is being evaluated.

7. Guba & Lincoln
Built on Stake’s Responsive Evaluation theory and came up with the method of fourth generation evaluation. This method was used when listing the generations of evaluation.
Generations include:
- 1st generation: evaluators measure participants
- 2nd generation: evaluators describe participant
- 3rd generation: evaluators judge participants
- 4th generation: evaluators respond to participants.
The evaluation includes 12 steps.
Script: Guba and Lincoln contributed to participant-oriented evaluations by defining the role of evaluation as responses to an audience’s requirements for information that take into account the various audience members’ value systems. They built on Stake’s ideas of responsive evaluation, and emphasized the importance of naturalistic inquiry.
Contacting
Organizing
Identifying Stakeholders
Developing within-group Joint Constructions
Enlarging joint stakeholder constrictions
Sorting out Resolved Claims, Concerns, and Issues
Prioritizing Unresolved Items
Collecting Information
Preparing for Negotiation
Carrying Out Negotiation
Reporting
Recycling

8. Michael Patton Identified more than 100 evaluations
Known for his words: “Research is aimed at truth. Evaluation is aimed at action”
He focuses on ‘identifying and working with key participants in organizations who can develop a vision for the value of gathering information and using it to improve the functions of the organization in ways responsive to each situation.’ (Williams, 2000)
Script: Michael Patton brings the idea of advocacy in evaluation to the front. Considering the approach used with this method, it is more apt to advocacy than many other approaches. There is also much to be said about the effectiveness of a program evaluated with a participant-oriented approach, as stakeholders will be more likely to take action if their values and perspectives are taken into consideration.

9. Key Considerations
(Fitzpatrick. Sanders, and Worthen, 2004), report that most proponents of participant-oriented evaluation view the participants in the object of evaluation as key to the evaluation.
As opposed to evaluating in a top-down manner, participant-oriented evaluation is conducted with a bottom-up approach
Qualitative methods are used much more than quantitative methods.
Jacobs (1985) describes that qualitative methods, “seek to describe not the frequency but the behaviors, beliefs, and feelings of the respondents in a setting” (p. 5).
Jacobs continues to describe qualitative data as “direct quotations, observations, personal impressions, analyses of records and documents, summaries of unobtrusive measures, case studies, and the like” (pp. 5-6).
Jacobs maintains the naturalistic design process in participant-oriented evaluation is emergent and the setting is changeable and natural.
As Fitzpatrick, et al. report, “the evaluator is in the role of the learner, and the stakeholders serve as teachers” (p. 149).
Script: Again, there are many key considerations to the participant-oriented approach to evaluation. Participants in the object of evaluation are seen as the most important part of evaluation, and more focus is placed on these stakeholders. As mentioned earlier, qualitative data is used more in this approach because of its subjective nature. Naturalistic inquiry is an important consideration in this approach, as evaluators tend to direct focus at non-technical audiences, use ordinary language, and rely more on informal than formal logic.

10. Advantages of This Approach
Gain new insights and usable theories
Educational, social or corporate programs
Flexibility
Addresses user training as a feature of the assessment
Consideration to related variables
Numerous data collection techniques
Empowers stakeholders
Reaches even those who are at times left out of the evaluation process
Add a political constituent
Cultivates activism
(Fitzpatrick, Sanders, and Worthen, 2004)
Script: There are several advantages to using the participant-oriented evaluations. New insights and useable theories can be found through the use of this process that might not be uncovered through the use of a different method. This approach is also extremely flexible, to the point that flexibility is one of its defining characteristics. Participant-oriented evaluations can also empower stakeholders that may otherwise be powerless. Finally, as mentioned previously, the nature of this approach fosters advocacy and activism, sometimes to a fault.

11. Disadvantages of This Approach
Subjective
Relies heavily on human observation and personal perspective
Expensive
Labor-intensive
Time-consuming and extensive
Evaluator loses objectivity
may become caught up in advocacy trends
(Fitzpatrick, Sanders, and Worthen, 2004)
Script: There are also several disadvantages to participant-oriented evaluation. Because of the detailed and exhaustive nature of this approach to evaluation, it is generally very labor-intensive, making it time-consuming and expensive. It relies heavily on the evaluator and stakeholders’ observations and perspectives, and is therefore, very subjective. Finally, evaluators can become so connected to stakeholders that the evaluator’s objectivity can be lost in the name of advocacy.

12. Implementation of Participant-Oriented Evaluation
When to use
When cost is not an issue
When time allows
When responding to stakeholders
When evaluator can take on the role of the learner
When not to use
When detailed quantitative data is essential
When there is “controlled, laboratory or contrived setting” (Jacobs, 1985).
Script: If time and budget are not constraints, participant-oriented evaluations can be considered, as well as when the key to the evaluation is a response to stakeholders. A participant-oriented evaluation would require the evaluator to take on the role of the learner. This approach should not be used if detailed quantitative data is essential, nor when the setting of the evaluation is a contrived situation in which the flexible nature of this approach would be rendered useless.

13. Scenarios for Use of Participant-Oriented Evaluation
Social reform
Evaluating programs developed to improve the environment
Education
Evaluating new and existing instructional programs on all levels
Evaluating instructional materials
Evaluating professional development
Corporations
Evaluating new and existing training programs
Government
Evaluating programs
Whenever qualitative formative evaluation is conducted
Script: Presented here are several scenarios that might call for the use of a participant-oriented evaluation. Note that each scenario would put the perspectives and values of the stakeholders as the highest priority. Think about how each of these scenarios could use participant-oriented evaluations, and then respond to the prompt on our discussion board. Thanks for your time.

14. References
Abma, T.A.(2005). Responsive Evaluation. Evaluation and Program Planning: Online version. Retrieved September 28, 2007 from World Wide Web
Fitzpatrick, J., Sanders, J., & Worthen, B. (2004). Program Evaluation: Alternative Approaches and Practical Guidelines, (3rd Ed.), Boston: Allyn & Bacon.
Jacobs, R. L. (1985). Naturalistic Inquiry and Qualitative Methods: Implications for Training and Development. Columbus, OH: Ohio State University. (ERIC Document Reproduction Service No. ED305518).
Williams, D. D. (2000). Evaluation of learning objects and instruction using learning objects. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects: Online Version. Retrieved September 27, 2007, from the World Wide Web: