1 Evaluation

2 Evaluating The Organization Effective evaluation begins at the organizational level. It starts with a strategic plan that has been carefully developed. The plan provides a clear set of measurable objectives, timelines, and activities to accomplish the objectives. Therefore organizational evaluation becomes a simple task of tracking the objectives and the related time lines and quantifying the results.

3 Timing The timing of organization evaluations is more long-term than the program/course evaluations because the objectives themselves are long term. The financial aspects of the operation may be evaluated monthly, quarterly, or annually. Professional development of the staff is usually evaluated annually.

4 Additional Methods In addition to the concept of self-evaluation (measuring your objectives against outcomes), there are other ways to conduct organizational evaluations. These efforts involve some form of peer evaluation and include: 1. An operational audit - Similar to a financial audit. A third party closely examines the entire CE process to determine how effectively they meet their objectives. 2. A team peer review - A team of CE professionals performs a brief, intensive review of the CE operation. Accrediting agencies use this model. However, it does not have to be limited to accrediting visits. For example, ACRU CE professionals could be invited as teams to visit your program. The benefit of this type evaluation is that both the reviewers and the team reviewed learn.

5 A typical list of the purposes of course evaluation, provided in a 1994 LERN publication, (Report 1J10) includes: 1. To find out how the class is doing. As an administrator, you cannot attend all your classes. The frequent question asked is "How are they doing?" An evaluation can help you answer this question. 2. To weed out the least effective teachers. Course evaluations are a good way to determine which teachers are ineffective and should not be teaching for you. They provide valuable feed back on which to base your evaluation of instructors. Program/Course Evaluation

6 Course Evaluation (Continued) 3. To improve or help teacher. This is one of the most valuable uses for course evaluations. Almost all of your teachers will perform well, but a few may experience some problems. Most problems can be solved. By helping teachers with their specific problems, you can significantly improve the quality of their classes. 4. To provide data on the quality of learning or participant satisfaction. Every educational organization must be able to gauge the quality of its programs and the level of learning that occurs.

7 Program/Course Evaluation Course Evaluations (continued) Again, the overall objectives of the course evaluations must be derived from the objectives defined in the strategic plan. Some of these general objectives may be: 1. quality control 2. improvement. Areas that may need improvement include: 1. marketing efforts 2. professional development 3. client communications 4. accountability to the institution

8 Happiness Index The most frequently used instrument to evaluate the courses is one referred to as the “happiness index.” It is usually administered at the end of the course and includes rating the instructor, rating the materials, rating the facilities, rating the presentation, and rating the content. It usually makes no effort to measure the progress toward accomplishing the intended objectives. The evaluation is positive if the students are "happy.”

9 Suggestions Given the limitations described above, the following suggestions can help with evaluation efforts: 1. Evaluations should be made as soon as possible after the event. In some cases, evaluations repeated at a later date also provide valuable information (Did the program actually change behavior?) 2. If the evaluation is a long one, divide it into two or more sections to be completed at different times. 3. For very large groups, ask participants at random to complete evaluations.

10 Suggestions Suggestions (continued) 4. Use brief, simple evaluation forms. More people are willing to respond to check-off lists, but more information may be obtained from open- ended questions. 5. Video taped sessions can be analyzed late for both content and methods.

11 Suggestions Suggestions (continued) 6. Programmers and administrators may obtain valuable information by informally interviewing participants at breaks and other convenient times. The following questions may be asked: * What did you expect from the event? * How well did the event meet your expectations? * Is anything missing from the event? * How will you use what you have learned? * What changes will you make based upon this experience? * How would you improve the program?

12 Types of Questions Evaluation instruments are usually made up of three types of questions: 1. Ones that may be checked off 2. Ones that ask participants to rank various elements 3. Ones that are open-ended and require the participants to provide their own answers.

13 Sequence The first questions should be broad-based and easy to answer. Their content should be about the overall quality of the program. As the participant moves further into the instrument, the questions may become more detailed such as various inquiries about the instructor and the program content. Open ended questions should be assigned to the end of the document. Make these questions optional to maximize the chances that participants will at least respond to earlier questions.

14 Typical Questions 1. Overall evaluation Were you satisfied with the course? Were the marketing materials for the course accurate? Was the course worth your time and money? What monetary value would you place on the course? Would you recommend the class to a friend?

15 Typical Questions 2. Teacher evaluation Was the instructor well prepared? Did the instructor know the subject? Did the instructor communicate the information well? Did the instructor manage the class well? Did the instructor encourage full class participation?

16 Typical Questions 3. Other Questions Were facilities comfortable and conductive to learning? Was the administrative support for the course positive? What did you like about the class? What, if anything, would you change?

17 Typical Questions 4. Testimonials Since no evaluation is typical, you should customize each evaluation to the extent possible and "sell" the participants on the concept that their input is extremely valuable and will be carefully reviewed and implemented.

18 Techniques For Return There are a number of techniques to encourage the participants to return completed evaluations. One example includes providing a gift to a participant based upon a randomly selected evaluation. Another is to provide additional subject matter such as a book or a writing tablet for everyone who completes and returns an evaluation.

19 Evaluation and Marketing Positive evaluations are one of the strongest course marketing elements that you can use to communicate the value of your program. When participants are willing to make positive comments that can be used in marketing materials, you are able to add credibility to your programs. It is important that they agree to be quoted. It is also important to identify their organization if they agree to provide that information.