1. The Best of Both Worlds: Using Fifteen Features of Evaluation to Promote Collaboration Between Brick and Click Worlds Sandy Woolum Matt Coleman Carol Kivi Amanda Evans Bud McClure Sandy Woolum Matt Coleman Carol Kivi Amanda Evans Bud McClure University of Minnesota Duluth

2. Psychology Online University of Minnesota Duluth

4. Our Online Program Today  1200 student annually  $750,000 in annual tuition  18 courses  4 Staff  10 Online Learning Assistant’s (OLA’s)  10 faculty have developed courses

5. History  Developed Four Courses - 2000  4 courses (Transpersonal, Dreamworks, Psychology and Social Responsibility, Transpersonal Development)  Used a combination of tools  Dreamweaver  Test-pilot  ePortfolio  Spontaneous emergence

6. Recent Faculty Experience  Gene Grossman  Psy 3020: Statistical Methods  Joie Acheson  Psy 3601: Psychology of Personal Development  Paula Pedersen  Psy 3216: Human Sexuality

7. Gene Grossman Psy 3020: Statistical Methods

8. Joie Acheson Psy 3601: Psychology of Personal Development

9. Paula Pedersen Psy 3216: Human Sexuality

10. Course Structure  Independent study  15 unit courses  Exam every 3rd unit  Weekly thought questions  Threaded discussions  No incompletes  Course evaluations

11. Virtually Integrated Software  Student view  Instructor view  Drop box  Gradebook  Exams  Message center  Discussion  Evaluation

12. Transpersonal Psychology

13. Psychology & Social Responsibility

14. History & Systems of Psychology

15. Streaming Video

17. The Letters Project

19. Evaluation Fifteen Features

20.  To further conceptualize the many possibilities for evaluation  Can be included in a comprehensive model of an online program or a dual- delivery system  In a free choice environment, we have unusual opportunities to evaluate the choices of both students and faculty and to assess the way that our programs inform each other.

21. Fifteen Features The fifteen features represent a number of measures including: 1.demographics 2.course usage 3.student satisfaction 4.objective measures of performance 5.qualitative measures of performance 6.follow up on students as they enter additional courses 7.measures of learning style

22. Fifteen Features These measures are then used in a number of comparisons such as: 8) pre-post comparisons 9) comparisons on in-class and online courses 10) comparisons of online courses by optional features such as streaming 11) comparison by student variables such as demographics, learning style and course usage 12) comparisons by the perspectives of students, faculty and colleagues 13) comparisons over the curriculum from beginning courses to more advanced courses

23. Fifteen Features The results of these measures and then provide feedback to: 14) students as they make decisions about their education 15) faculty as they continue to revise and improve courses 16) the department as additional decisions are made for resources and curriculum

24. Evaluation for Courses

25. The online course presented challenging and stimulating material

26. This course is more difficult than other courses I have taken at UMD

27. Overall, I learned a lot in this course

28. If I had it to do again, I would take the course online

29. Learning and Behavior Exam 1

31. History and Systems Exam 1

33. History and Systems Exam 2

35. Developmental Homework 2

37. Thank You! Visit us at: http://www.d.umn.edu/psychology Visit us at: http://www.d.umn.edu/psychology