1. Abstract Systematic research on evaluation can greatly improve a program. But in making decisions about which evaluation model to choose, there needs to be a larger pool of research literature across all evaluation types. One nearly unexamined type of evaluation is goal-free evaluation (GFE). GFE is designed to keep evaluators purposefully ignorant of the program’s stated goals and objectives. The idea behind GFE is that the evaluator may uncover unintended side effects which would have gone unnoticed looking solely for specific goals and objectives. In this analog experiment GFE is compared with goal-based evaluation (GBE). GFE and a GBE were simultaneously and independently used to evaluate the same training program. Thirty of the evaluations’ users completed an attitude survey and will participate in a focus group regarding the utility of each evaluation report. Tools and Methods Two evaluation tools were used in this study to collect quantitative and qualitative data on GFE and GBE: (1) an attitude survey known as a semantic differential, and (2) focus group. Semantic differentials use bipolar adjective pairs to collect opinions. The one used in this experiment was comprised of 25 pairs, and one open-ended question. EXAMPLE: Inconclusive ___ | ___ | ___ | ___ | ___ | ___ | ___ Conclusive Evaluation users indicated their opinion by marking where their attitude fell on the spectrum shown above. The qualitative tool used in this experiment was a focus group with the researchers and evaluations’ users. Acknowledgements Thanks to the GVSU’s Occupational Therapy faculty, staff and students for allowing us to observe their fieldwork. Thanks to Indian Trails Camp for cooperating with the evaluation team. Open-Ended Question Results Goal-Based Evaluation Report: 1.“This evaluation report was useful because it outlined how the information was actually used by counselors” 2.“Provided clear explanation of goals and results” Goal-Free Evaluation Report: 1.“Conclusions unclear, eval report was not as clearly organized” 2.“This evaluation was very helpful because it provided useful information regarding how goals can be better organized” Conclusions Both evaluation types elicited positive and negative responses from evaluation users. The 30 evaluation users' mean score on the semantic differential for the GBE report is 1.91 (SD =.66) while their score on the GFE report is 2.0 (SD =.42). Pros The goal-based evaluation was useful in that it provided a clear understanding on whether specific and predetermined goals of the program were met. The goal-free evaluation was useful in that there was more leniency in what the evaluation team observed, and whether the predetermined goals were actually useful. Cons The goal-based evaluation was reported by some to be over ambitious. It is important to note that while some evaluation users found the goal-free evaluation to be less organized, some found it to be more organized. Some evaluation users reported that the goal- free evaluation was not as clearly organized, as it was not created from a set of goals, and had its own organization style. Who Needs Goals? An Analog Experiment Comparing Goal-Based Evaluation and Goal-Free Evaluation Utility Summer Student Scholar: Alayna Zielinski Faculty Mentor: Brandon Youker Trustworthy Conclusive Relevant Believable Informative True Honest Complete Enlightening Objective Valid Effective Helpful Careful Specific Worthwhile Meaningful Correct Clear Useful Consistent Fair Reasonable Balanced Logical Untrustworthy Inconclusive Irrelevant Unbelievable Uninformative False Dishonest Incomplete Unenlightening Biased Invalid Ineffective Unhelpful Careless Vague Worthless Meaningless Incorrect Unclear Useless Inconsistent Unfair Unreasonable Unbalanced Illogical