“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online

Assessing Online Instruction at Carteret Community College Donald Staub, Title III Director Carteret Community College ? ? ? ? ? ?

The Lineup Assessment of DL How We’re Doing CCC Assessment in DL Conclusions & Solutions Discussion

The Need to Assess DL SACS Common Sense it’s the right thing to do we need some answers (in order to improve)

“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” (CS 3.3.1) COC Distance Education Policy Statement education.pdf

How We’re Doing It QAP Success & Persistence Rates Grade Distributions ILLOs PLLOs

CCC FA08 seated: 270 sections; 73% success / 22% failed or withdrew (15% W) FA08 hybrid: 15 sections; 81% success / 18% D, F, WD (9% W) FA08 internet: 61 sections; 65% success / 33% D, F, WD (16% W) FA07 internet: 59 sections; 34% D, F, W FA06 internet: 48 sections; 34% D, F, W

The QAP (Quality Assessment Plan) Developed in-house (using models) Peer-reviewed by content specialist Stipends All current online courses have been reviewed All new online courses must be reviewed

The QAP Score Card

The QAP Score Card: Learning

Grade Distributions (Fall 2008)

ILLOs Communication Computer Literacy Critical Thinking Humanities & Fine Arts Information Literacy Personal Growth & Responsibility

Computer Literacy - CIS 110 Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers. Assessment: Final Exam in CIS 110. This exam consists of a hands-on skills assignment for Word, Excel, Access and PowerPoint and a 25 question Multiple choice assignment with questions regarding computer hardware, system software, internet browsers and other related computer terminology.

Computer Literacy 5 Sections: CIS WB; 2-IN Attrition:  IN01: 35%  IN02: 26%  WB03: 17%  WB04: 28%  WB05N: 18% Cross-section comparison (i.e. one course, different instructors)

Computer Literacy - CIS 110 Same instructor teaching one section of each

Personal Growth & Responsibility ACA 115 Outcome: Students demonstrate the ability to understand and manage self effectively.

Personal Growth & Responsibility ACA 115

Advisement Project In the fall of 2008, 79% of all ACA students (10 sections) scored 80 or above. For online sections (2 sections), 70% (7 out of 10) scored 80 or above.

Personal Growth & Responsibility PSY 150

PLLOs Same instructor teaching one section of each (take a look at a common assignment from each section) ACA 115 ACC 120 BUS 110 ECO 251 EDU 119

ACA Learning Styles Project Students identify and explain the strategy used to prepare for an exam.

ACC 120

BUS 110 ECO 250

EDU test test items related to: working/communicating with parents, parental rights, and regulations dealing with parental contact/information. n=20n=12

Assessment in DL “The largest issue confronting online assessments is that of the dynamics that occur when the instructor and student do not share the same immediate space.” Milam, et al. (2004). In, Developing and Implementing Assessment of Student Learning Outcomes

“In many instances, the assessment of an online course is not in alignment with the type of instruction that may have occurred in the course.” Palloff & Pratt (2009) Assessing the Online Learner

“…today’s college student is used to using the online environment in ways that depart from those involved in the usual and customary ways in which online courses are delivered. Involvement in social networking is different from participating in an online course.” Palloff & Pratt (2009) Assessing the Online Learner

7 Principles of Effective Online Assessment Palloff & Pratt (2009) Assessing the Online Learner

1. Design learner-centered assessments that require self-reflection regarding the lesson (e.g. “at what point this week did you feel most/least engaged?”) when conducting collaborative work - “describe the quantity and quality of your work with this group”

2. Design and include grading rubrics concrete can be used for self-reflection

3. Include collaborative assignments “Collaborative effort helps learners achieve a deeper level of knowledge generation while moving from independence to interdependence, thus strengthening the foundation of an online learning community…” “…collaborative activities are best assessed collaboratively.”

4. Encourage students to develop good feedback skills “The key to effective feedback is that it be constructive and encourage improvement.” use proper netiquette Instructor as model

5. Use effective assessment techniques “Many instructors feel that more authentic assessments - such as projects, papers, and artifacts that integrate course concepts - are more effective means by which to assess student learning online.”

6. Design assessments that are clear, easy to understand, and likely to work in the online environment. Group work, collaboration, high levels of interaction Assessments that require demonstration of skill acquisition and problem-solving ability use of alternative assessments

6. Ask for and incorporate student input. “Involving students in the development of assessments helps to move a learner from the role of student to that of reflective practitioner…”

Donald Staub Title III Director Carteret Community College