The Best of Both Worlds: Using Fifteen Features of Evaluation to Promote Collaboration Between Brick and Click Worlds Sandy Woolum Matt Coleman Carol Kivi Amanda Evans Bud McClure Sandy Woolum Matt Coleman Carol Kivi Amanda Evans Bud McClure University of Minnesota Duluth

Psychology Online University of Minnesota Duluth

Our Online Program Today  1200 student annually  $750,000 in annual tuition  18 courses  4 Staff  10 Online Learning Assistant’s (OLA’s)  10 faculty have developed courses

History  Developed Four Courses  4 courses (Transpersonal, Dreamworks, Psychology and Social Responsibility, Transpersonal Development)  Used a combination of tools  Dreamweaver  Test-pilot  ePortfolio  Spontaneous emergence

Recent Faculty Experience  Gene Grossman  Psy 3020: Statistical Methods  Joie Acheson  Psy 3601: Psychology of Personal Development  Paula Pedersen  Psy 3216: Human Sexuality

Gene Grossman Psy 3020: Statistical Methods

Joie Acheson Psy 3601: Psychology of Personal Development

Paula Pedersen Psy 3216: Human Sexuality

Course Structure  Independent study  15 unit courses  Exam every 3rd unit  Weekly thought questions  Threaded discussions  No incompletes  Course evaluations

Virtually Integrated Software  Student view  Instructor view  Drop box  Gradebook  Exams  Message center  Discussion  Evaluation

Transpersonal Psychology

Psychology & Social Responsibility

History & Systems of Psychology

Streaming Video

The Letters Project

Evaluation Fifteen Features

 To further conceptualize the many possibilities for evaluation  Can be included in a comprehensive model of an online program or a dual- delivery system  In a free choice environment, we have unusual opportunities to evaluate the choices of both students and faculty and to assess the way that our programs inform each other.

Fifteen Features The fifteen features represent a number of measures including: 1.demographics 2.course usage 3.student satisfaction 4.objective measures of performance 5.qualitative measures of performance 6.follow up on students as they enter additional courses 7.measures of learning style

Fifteen Features These measures are then used in a number of comparisons such as: 8) pre-post comparisons 9) comparisons on in-class and online courses 10) comparisons of online courses by optional features such as streaming 11) comparison by student variables such as demographics, learning style and course usage 12) comparisons by the perspectives of students, faculty and colleagues 13) comparisons over the curriculum from beginning courses to more advanced courses

Fifteen Features The results of these measures and then provide feedback to: 14) students as they make decisions about their education 15) faculty as they continue to revise and improve courses 16) the department as additional decisions are made for resources and curriculum

Evaluation for Courses

The online course presented challenging and stimulating material

This course is more difficult than other courses I have taken at UMD

Overall, I learned a lot in this course

If I had it to do again, I would take the course online

Learning and Behavior Exam 1

History and Systems Exam 1

History and Systems Exam 2

Developmental Homework 2

Thank You! Visit us at: Visit us at: