© Subjective Metrics, Inc waypoint online assessment and structured peer review COAS: Drexel University Andrew J. McCann visiting professor of english, drexel university founder and president, subjective metrics, inc
© Subjective Metrics, Inc Agenda Background Introduce waypoint –Brief overview Drexel applications Live Demonstration –Evaluate –Manage –Libraries
© Subjective Metrics, Inc Problem: Feedback Handwritten and/or manually typed “Lost” Little accountability Consistency issues No data Needed: not AI – but a technological tool
© Subjective Metrics, Inc Waypoint: Evaluation Tool Encourages pre-written feedback: clearer, more detailed explanations of key concepts Facilitates sharing of assessments amongst instructors Quantifies evaluations by skill Archives all feedback Web-based: cross-platform backed up zero maintenance
© Subjective Metrics, Inc Waypoint Process
© Subjective Metrics, Inc Develop Assessment
© Subjective Metrics, Inc Develop Assessment
© Subjective Metrics, Inc Receive Paper or Exam (Hard copy shown here)
© Subjective Metrics, Inc Receive Paper or Exam (Hard copy shown here)
© Subjective Metrics, Inc Evaluate Against Skills
© Subjective Metrics, Inc Evaluate Against Skills
© Subjective Metrics, Inc Respond to Student
© Subjective Metrics, Inc Respond to Student
© Subjective Metrics, Inc Snapshot Analysis
© Subjective Metrics, Inc Snapshot Analysis
© Subjective Metrics, Inc Longitudinal Cohort Analysis
© Subjective Metrics, Inc Process Summary
© Subjective Metrics, Inc Drexel Applications tDec Humanities (Dr. Valarie Arms) –Evaluation of all major writing assignments (600 students) Writing Center (Harriet Millan) –Customized assessments for WITs College of Engineering (Kevin Scoles & Adam Fontecchio) –Evaluation of lab reports with WITs and TAs College of Business (Frank Linnehan) –Structured response to student writing & accreditation data generation Engineering Management (Mike Scheuerman) –Peer review
© Subjective Metrics, Inc Demonstration Evaluate –“Final Report” Peer Review –“Final Report” Manage –Quantification –Sorting –Data analysis
© Subjective Metrics, Inc Additional Features Self-assessment Collaborative assessment –Multiple instructors can contribute to evaluation
© Subjective Metrics, Inc Questions and Discussion
© Subjective Metrics, Inc Testimonials “I've been doing this for twenty years and have my own system of grading and commentary…and found little need to improve. And then Waypoint came along. Somehow--and I really can't even explain it--but my grading time has been cut down by more than half and my students are actually thanking me for the in- depth commentary.” –Professor Ken Bingham “Usually I have trouble criticizing a peer’s paper if I’m not given certain criteria to judge. During this peer review, I was actually focused and excited about judging a peer’s paper.” –Erin Williams, COE 2008 “If this peer review program was available in high school, I would have probably done a lot better in English.” –Ed Itaas, COE 2008
© Subjective Metrics, Inc Testimonials “An innovative teacher-friendly, student-friendly, efficient approach to grading writing—the most creative and time-saving method for evaluating writing I have ever found or used.” –Gayle, a high school English teacher with 35 years’ experience “I don’t know how I ran a writing program for three years without it.” –Harriet Millan, Director of the University Writing Program
© Subjective Metrics, Inc Quantitative Data Anonymous survey of Peer Review process –114 freshman engineers: 51 use waypoint, 63 the ‘old’ method 1-5 Scale (1 – strongly disagree, 5 – strongly agree) wp (N=51) Mean ‘old’ (N=63) Mean P value (t-test, 95% CL) 1. I received clear and helpful criticism of my draft p< Evaluating other students’ papers helped me better understand the assignment and the play p< I made significant changes to my first draft based on peer review feedback p< I made significant changes to my first draft independent of peer review feedback p=0.38