1. COM 535, S08 Designing and Conducting Formative Evaluations April 7, 2008

2. COM 535, S08 How to Think About Formative Evaluations (FEs) FEs DO NOT… Evaluate learners’ ability to achieve performance objectives. Evaluate instructor delivery methods. Present a replica of what instruction will actually be like. FEs DO… Evaluate instructional materials. Generate feedback on methods of learner participation. Evaluate assessment instruments. Provide data that can shape instructional revision.

3. COM 535, S08 Phases (and Audiences) of Formative Evaluations Phases of FEs 1.One-to-One (clinical) evaluations 2.Small-Group evaluations (8-20 learners) 3.Field Trial (real- world testing) FE Audiences Subject Matter Experts (SMEs) Learners

4. COM 535, S08 Developing FE Tools Useful to frame FE tools using the instructional strategy developed in Phase 2. Pre-Instructional Activities Content Presentation/Examples Learner Participation Assessment Follow-through Activities Different audiences (i.e., SMEs, learners) require different tools

5. COM 535, S08 Focusing FEs for SMEs When developing criteria for SMEs to assess materials, consider: 1.Accuracy: Is the information/process/theory accurate? 2.Emphasis: Does the information emphasize the right components of the process? 3.Audience: Will the pretest measurements collect the necessary information we need about learners?

6. COM 535, S08 Focusing FEs for Learners When developing criteria for learners to assess materials, consider: 1.Clarity: Is the message/content clear to target learners? 2.Impact: What is the impact of the instruction on the learners’ attitudes? 3.Feasibility: How feasible is the instruction given the available resources?

7. COM 535, S08 FE Data Collection Measures The following research tools can be used to collect data: Interview (collected by designer) Observations (collected by designer and/or manager/supervisor) Sample Tests (completed by users) User questionnaire (completed by users) Others?

8. COM 535, S08 Managing the FE Process 1.Identify and contact users who will be tested. 2.Identify major issues/components of instruction that you’d like feedback on. 3.Construct FE tools to use with SMEs and learners. 4.Conduct FE. 5.Review results for areas of similarities and differences. 6.Revise ID based on FE feedback.

9. COM 535, S08 FOR WEDNESDAY Create one sample data collection tool that you plan to use with either your SME/learners.