1. Part I: The 3 T’s of Online Assessment: Tools, Techniques, and (Saving) Time
Curtis J. Bonk, Professor, Indiana University
President, CourseShare
Vanessa Paz Dennen, Assistant Professor
Florida State University

2. Session Objectives Detail online assessment techniques
Discuss how to match learning activities with learner assessments
Examine instructor time and comfort issues
Discuss ways to limit and detect cheating and plagiarism
Document online tools and resources for assessment

3. Online Assessment Techniques
(with some time-saving
tips added in…)

4. Is this motivating? How would you feel?
You take an online class.
You read some Web pages.
Maybe you watch some videos or hear some audio clips.
Maybe you ponder some study review questions.
You take a multiple choice test online.
You receive an automated score on the test.
Class is over.

5. How about this scenario?
You take an online class.
You “meet” your fellow students on the d-board.
You read some materials. You find and share some materials too.
You participate in some discussions of course concepts.
You take a multiple choice test.
You receive an automated score on the test.
Class is over.

6. Commentary on Scenario 1
No interaction with peers.
Students don’t feel “missed” if they don’t participate.
Not clear why course is online (except perhaps for media elements).
Potential for immediate feedback is nice -- but assessment format is limiting.

7. Commentary on Scenario 2
Interaction with peers is great. Serves as a motivator.
Community is likely to develop.
Students will feel involved and important if they share examples and resources.
Assessment format may not be well aligned given the activities.
Class lacks closure in a manner appropriate to the activities.

8. Assessment and Learning
Course objectives, activities, and assessments should be in alignment
This tends to be an issue in courses regardless of medium.
Example:
In class students conduct a debate
Students are tested on their ability to recall facts

9. Mis-aligned Online Learning and Assessment
A not-uncommon scenario
Discussion is used as a learning activity
Students are required to participate
Participation is noted by how many messages were composed by a student
But does this method measure learning?

10. Common Online Assessment Complaints
Instructor perspective
There’s too much to assess!
I don’t know what activities to assess!
I don’t know if students really are ready for the test!
How do I know the student actually did the work/took the test?

11. Common Online Assessment Complaints
Student perspective
If they’re supposed to discuss, why doesn’t that count as part of their grade?
If they’re just supposed to do something, why does quality matter?
I just got a number, no feedback.
I didn’t get participation feedback.

12. The Feedback Issue
Students participating in online activities look for feedback to know:
A. the instructor is reading their contributions
B. their participation is valued
C. their participation is adequate, in terms of quality and quantity
Feedback need not be individualized to be effective
Whole class commentary provided on a regular basis was found to be just as satisfactory from the student point of view (Dennen, 2001)

13. The Assessment Issue Often, online activities go unassessed
“Add-on” syndrome: Adding an online activity to a previously designed class because it sounds like a good idea

14. The Assessment Issue
Students are more likely to participate when then know there is impact on their grade
Direct impact: graded on participation (quality, quantity or both)
Indirect impact: participation should bolster performance on other assessments
Students quickly become aware if an online activity is not related to assessed learning objectives

15. The Assessment Issue, Cont.
Sometimes the wrong things are assessed
Examples:
Assessing students’ online moderation skills when the course topic/learning objectives have nothing to do with online moderation
Assessing quantity of participation, but not quality
which, granted, is easier, but encourages sloppy message posting rather than thoughtful learning dialogues

16. Bloom’s Revised Taxonomy
A useful tool for checking alignment
Also great to guide your course design!

17. To Find Out More… A Taxonomy For Learning Teaching and Assessing
By Anderson
And Krathwohl

18. Assessment Techniques
Options…
Formative or Summative
Student-led (Self or Peer) or Teacher-led
Public or Private
Process or Product
Other issues to consider…
“Objective” or Interpretive
Rubric-based or Wholistic

19. Formative vs. Summative Assessments
Alleviate student anxieties re: expectations
Seem especially high in online classes
Encourage working toward mastery
Can be informal
Summative
Used for student’s grade
Assumed to be “best” effort

20. Example: Online Formative Assessment
Paper draft discussion forum
Start a discussion forum for papers-in-progress
Have each student start a thread and post elements of their papers as they complete them (e.g., topic, major claims, research sources)
A schedule for each element is useful
Monitor and provide feedback

21. Student vs. Teacher Led Teacher-led “Traditional” assessment
Most often summative
Student-led
Students may assess self or peers
May be formative or summative
Can greatly relieve instructor burden
Students reinforce concepts through feedback process

22. Examples: Online Self-Assessment
Self-tests: Use test tool to create self-tests (multiple choice, true false)
May wish to track student efforts
Can incentivize use (essentially, use as a learning tool)
Reflection papers: Have students submit brief, focused papers expressing the strengths and assessments of their assignment(s)

23. Example: Online Peer Assessment (Formative)
Feedback groups: Assign students in groups to provide formative feedback on projects and papers
Often raises quality of assignments
Need a structure with clear deadlines
Need prompts and models to guide students
May wish to assess feedback process/contributions

24. Example: Online Peer Assessment (Summative)
Conference Presentations: Have students “present” their work and ask questions/provide feedback to others.
In d-board, have students attach papers to messages; post a message with a synopsis; or attach a powerpoint presentation
Each student/team should have their own thread
Feedback should occur during a defined period of time.
May consider allowing students to rate assignments on certain dimensions

25. Public vs. Private Private Work is submitted to the teacher only
Entire burden of feedback is on teacher
Important if assessing at fact level
Public
Peers can see each others’ work (either in process or completed)
Peers may comment on each others’ work
Often increases quality of work submitted

26. Process vs. Product Product Process The end deliverable
Look for polish, accuracy
Process
How the student got there
Look for thoughtfulness of approach, intent

27. Assessing Process Easy to do
Many technology tools will archive student work/interactions
Students create a document trail in process
Helps students develop metacognitive knowledge
Instructors structure/model/encourage productive work processes
Students learn how to manage their own work processes

28. Why Assess Process? For the instructor …
Provides formative feedback on course (e.g., helps gather data about why students have difficulty with product-oriented assessments)
Clarifies who is doing most work in small group assignments
Helps prevent cheating

29. Why Assess Process? For the student …
Typically improves the quality of their products
Helps them develop productive work processes
Puts on a schedule
Shows that you care about individual growth

30. Assessment Project Cycle
From Classroom Assessment Techniques by Angelo & Cross (1993)
Step 1: Plan
Choose class
Focus on assessable question
Design project to answer question

31. Assessment Project Cycle [2]
Step 2: Implement
Teach target lesson
Collect assessment data
Analyze data
Step 3:
Interpret results
Communicate results
Evaluate assessment project

32. I. Term Papers How to do online:
Have students each start their own thread and post topic of interest
Peers and instructors give feedback
Students post thesis statements, research sources, etc., with iterations of feedback
Final paper is posted

33. Term Paper Assessments
Product: the paper
Process: quality and timeliness of student work from time when paper is assigned
Process: quality and timeliness of feedback provided to peers
Process: responsiveness to feedback received from instructor and peers

34. II. Discussion Assignments
1. Chain of thought
Have students develop a solution to a problem
Have students indicate what led them to a particular conclusion, method or approach
Can be done in a discussion board

35. Discussion Assignments
2. Theory to Practice
Have students match up theories you are learning about to actual problems
Present students with problems and have them explain what theories they would use to solve these problems and how they would approach it
Debrief the assignment

36. Discussion Assignment
3. Synthesizer (i.e., wrapper)
Have students take roles being the weekly synthesizer of class discussion
Add a “meta” level in which students narrate their own experiences while reading the weekly discussion
Reflect on how life relates to discussion

37. III. Group Projects
Tools used
Chat: brainstorming ideas, making group decisions, regular way to feel connected (should be archived)
Discussion board: commenting on drafts
quick feedback
File exchange: sharing project files
MS Word: Track changes

38. Group Project Assessments
Product: project files that are turned in
Process: online archive demonstrating
Who contributed what
Who provided peer feedback
Who worked in a timely manner
How collaborative a group was
Process: peer ratings
Process: interim instructor consultations

39. Group Project Assignments
1. Work Plans
Have students develop a plan of work for their project
Make them outline topic, schedule, resources needed, division of labor and anticipated form of final deliverables
At end of project, have students evaluate how well they followed their own plan and how useful it was

40. Project Assignments 2. Research Trail
Have students document the steps they took in the research process and the results
Ask for a brief reflection on how effective their process was and what they might change the next time

41. Project Assignments 3. Process Presentations
Have students focus on their process as well as their product in class presentations
To maintain focus, ask them to share 3 main lessons learned
Might ask for some process documents to be shared, like an early draft

42. Project Assignments 4. Design Journal
Have students maintain a journal of all ideas related to their project
Encourage sketches, lists, organizational charts, etc.
Require journals to be turned in with final projects

43. IV. Reflection Assignments
Have students keep a weekly journal of their thoughts on readings and course content AND real-world related instances that they noticed
May make these public, with each student having their own discussion thread

44. Making it Happen Learners need to see that process is valuable:
Model appropriate processes
Provide students with scaffolding (guide sheets) to structure their processes
Give students feedback on their process
Require students to reflect on their processes
Grade students on process

45. Vanessa’s Top Time-Saving Tips
Before you assign it, ask yourself “can I reasonably assess it?”
Rely on students/peers for providing some feedback
Let students know what to expect up front
Choose the right tool for the job
Get to know the editing and commenting features of your favorite programs
Use rubrics!
I make them in Word, and then while grading I highlight or bold the section that applies to the student’s projects. Add a few comments at the bottom = speedy grading!