2. Teaching in a Web-Based Distance Learning Environment: An Evaluation Summary Based on Four Courses Charles Graham, Joni M. Craner, Byung-ro Lim, & Kursat Cagiltay, Thomas M. Duffy Center for Research on Learning and Technology Indiana University

3. Evaluation Scenario You have just been asked by the VP of your institution to evaluate its online courses. She asks you if there is any difference in how you would evaluate an online course compared to a face-to-face course. What do you tell her? Center for Research on Learning and Technology, Indiana University

4. It happened to us... A similar scenario happened to the director of the Center for Research on Learning and Technology at Indiana University... Center for Research on Learning and Technology, Indiana University

5. Context for the Evaluation The assistant Dean in a large academic program in mid-west requested an evaluation of their online program Their evaluation objectives: Assess the quality of online instruction and provide recommendations for improvement to program directors Provide feedback to the instructors about the quality of their courses and provide recommendations for improvement Center for Research on Learning and Technology, Indiana University

6. Evaluation Background What was being evaluated? We evaluated 4 online courses Who was conducting the evaluations? A team of 4 evaluators from the Center for Research on Learning and Technology under the leadership of Tom Duffy Center for Research on Learning and Technology, Indiana University

7. Evaluation Criteria What Criteria was used? They use the 7 principles of Good Practice in Undergraduate Education (Chickering & Gamson, 1987) to guide their on-campus courses We decided to try and use the same criteria for online evaluations Center for Research on Learning and Technology, Indiana University

8. The Challenge The 7 Principles seemed to apply to both online and face-to-face courses. BUT... Specific evaluation strategies provided for each principle in a f2f context often did not apply to the online environment. Center for Research on Learning and Technology, Indiana University

9. The 7 Principles of Good Practice (American Association of Higher Education, 1987) Good practice: 1. Encourages student-faculty contact 2. Encourages cooperation among students 3. Encourages active learning 4. Gives prompt feedback 5. Emphasizes time on task 6. Communicates high expectations 7. Respects diversity Center for Research on Learning and Technology, Indiana University

10. Data Collection We evaluated the courses looking for examples and non-examples of the 7 principles of Good Practice We formulated a set of guidelines for each principle based on our findings How was data collected? Literature review Interviews, and document analyses (course materials, discussion forums, etc.) Center for Research on Learning and Technology, Indiana University

11. Findings These were our findings... Center for Research on Learning and Technology, Indiana University

12. Online Guidelines 1. Encourage Student-Faculty Contact Instructors should: Contact students (by phone or private email) who are not participating regularly to keep students engaged Clearly communicate email response policies (set student expectations) Center for Research on Learning and Technology, Indiana University

13. Online Guidelines 1. Encourage Student-Faculty Contact Encourage students to contact instructor through private email messages with learning problems Invite students to have on- or off-line appointments with instructor Be respectful to students in all interactions (listserv reply) Center for Research on Learning and Technology, Indiana University

14. Online Guidelines 2. Encourage Cooperation Among Students Begin course with a structured ice- breaking activity where students share interests/backgrounds/etc. Develop assignments that require meaningful peer interaction that fosters a sense of community/connectedness Encourage students to share their life experiences Center for Research on Learning and Technology, Indiana University

15. Online Guidelines 2. Encourage Cooperation Among Students Require and teach students how to evaluate each other’s work according to specific guidelines (Critical peers) Encourage students to praise and challenge each other Require group assignments and provide a mechanism for assessing individual contributions Center for Research on Learning and Technology, Indiana University

16. Online Guidelines 3. Encourage Active Learning Ask students to relate course content to real-world issues/experiences Require students to do authentic projects and assignments Require students to present their work to the class for peer feedback Center for Research on Learning and Technology, Indiana University

17. Online Guidelines 3. Encourage Active Learning Structure assignments and bulletin board discussions activities to require students to challenge the ideas of the instructor, of other students, or those presented in the readings or other course materials Provide a mechanism for students to add relevant resources for the class Center for Research on Learning and Technology, Indiana University

18. Online Guidelines 4. Give Prompt Feedback Provide information feedback on assignments and projects according to the written policy Provide acknowledgement feedback to students within 24 hours Monitor group bulletin boards regularly and give specific information feedback to students Center for Research on Learning and Technology, Indiana University

19. Online Guidelines 5. Emphasize Time on Task Structure assignments with deadlines that are spread throughout the semester Require students to participate in the online forum on a regular basis Emphasize the importance of regular work, self-directed learning, sound self- pacing, and scheduling. Center for Research on Learning and Technology, Indiana University

20. Online Guidelines 5. Emphasize Time on Task Use easily accessible resources and prepare sufficient resources. Avoid unnecessary time spent on finding materials. Center for Research on Learning and Technology, Indiana University

21. Online Guidelines 6. Communicate High Expectations Clearly list expectations from the students on the course Web site Provide examples of exemplary performance to students. Publicly call attention to excellent performance by the students Reinforce expectations during the duration of the course Center for Research on Learning and Technology, Indiana University

22. Online Guidelines 6. Communicate High Expectations Periodically discuss how well the class and individual groups are doing during the course of the semester Develop a mechanism for evaluating individual participation and contribution to group projects. Center for Research on Learning and Technology, Indiana University

23. Online Guidelines 7. Respect Diverse Talents and Ways of Learning Learn about the different backgrounds and interests of the students Allow students to shape their own coursework by choosing project topics Encourage students to express their diverse points of view Center for Research on Learning and Technology, Indiana University

24. Human Computer Interaction Recommendations Consistency of Web site layout and design Clear Organization and Presentation of Information Consistent and easy-to-use navigation Aesthetically Pleasing Design and Graphics Center for Research on Learning and Technology, Indiana University

25. Instructor Sharing and Collaboration Problem Asynchronous conferencing requires different methods and strategies from face-to-face conferencing. Center for Research on Learning and Technology, Indiana University

26. Guidelines for using Asynchronous Conferencing Effectively Require student participation (make grade dependent on it) Students need a task to focus their discussion The task should have a product or something that brings closure The task must engage the learner in the content The discussion structure should be carefully thought out Center for Research on Learning and Technology, Indiana University

27. Guidelines for using Asynchronous Conferencing Effectively (cont’) Discussion must be evaluated based on quality of content and not length of posting or number of postings Instructors should post examples of expectations for discussions Students must get some kind of feedback on the discussions Discussion groups should remain relatively small Center for Research on Learning and Technology, Indiana University

28. Course Management Problem Managing a course online was much different from managing a face-to-face course. Instructors start to burn out trying to manage interactions in the same way they might in the classroom. Center for Research on Learning and Technology, Indiana University

29. Online Course Management Strategies Increase emphasis on peer evaluation and feedback. Increase the emphasis on group work. Selectively evaluate discussions. Set clear feedback expectations. Center for Research on Learning and Technology, Indiana University

30. Questions? Center for Research on Learning and Technology, Indiana University

31. Contact Information Presentation Online at: http://php.indiana.edu/~crgraham/dl2000.ppt Paper Online at: http://crlt.indiana.edu/crlt00-13.pdf Email Contact: Charles Graham: crgraham@indiana.edu Byung-Ro Lim: byunlim@indiana.edu Joni Craner: jcraner@indiana.edu Center for Research on Learning and Technology, Indiana University