A CASE FOR LEVERAGING THE ONLINE GRADE BOOK John Fritz, Tim Hardy & Karin Readel University of Maryland, Baltimore County (UMBC) February 15, 2011

OVERVIEW Introductions, Stage Setting (John) Highlight: ECAR study of Undergraduates and IT, desire for profs to use grade book. UMBC’s Alternate Delivery Program (Karin) Effective Practice: Adaptive Release Walking the “Self Interest” Talk in ECON 122 (Tim) Highlight: How course design influences student performance Wrap Up, Q & A

About UMBC Founded in 1966 “Research extensive university” Carnegie classification Fall 2009 Stats 12,870 Students 9,947 undergrad, 2,923grad 730 Faculty 480 FT, 250 PT Selected Brags #1 “Up and Coming National University” U.S. News America’s Best Colleges 2010 & 2009 1st in undergrad chemistry degrees awarded to African Americans One of 50 Best Colleges for Women 7-time National College Chess Champions John

About Blackboard @ UMBC Blackboard Learn 9.1 As of Spring 2010 (began using in SP2000) 95% of all students 75% of all instructors 65% of all courses (1,645 FA2010) 356 Communities Includes all student, faculty and staff senates Support Staff: 2 FTE (Admin & Support) 1 Server Admin John

WHAT STUDENTS WANT FROM A CMS?

CURRENT PRACTICE @ UMBC Blackboard Courses with active grade books 49 percent (FA2010) 48 percent (SP2010) 47 percent (FA2009) Source: www.umbc.edu/blackboard/reports Why don’t more faculty use the grade book? What could happen if they did?

UMBC'S HYBRID WORKSHOP & ADP Workshop is not required, but recommended to meet ADP “requirements”

Program Goals Provide realistic outlook on what it will take to convert course to hybrid format Provide assistance with available technology Provide opportunity for feedback from peers Create network of “hybrid” faculty for support and feedback

SHIFTING RESPONSIBILITY FOR LEARNING Courtesy of Glen Macdougal, Algonquin College

ADAPTIVE RELEASE Create rules governing access to material ADP uses this in work with faculty to facilitate self-directed learning and demonstrate the approach. Syllabus or AI quiz before students can access assignments or turn in any work for credit). One of three “Effective Practices” emphasized during Hybrid Workshop and ADP.

TIM HARDY & ECON122 One of six faculty funded by the Provost’s FA2009 pilot to deliver a 50/50 hybrid course. ·    Joined the SU2009 cohort of the Alternate Delivery Program (ADP) ·    Attended the January 2009 Hybrid Course Redesign Workshop. ·    His UMBC Blackboard ranking for student-only activity has increased dramatically: FA2010 (1st overall, 2,970 average hits per student) * See note below * SP2010 (1st overall, 1,345 average hits per student) FA2009 (1st overall, 1,666 average hits per student) SU2009 (4th overall, 842 average hits per student) SP2009 (36th overall, 366 average hits per student) FA2008 (532nd overall, 78 average hits per student) SU2008 (38th overall, 334 average hits per student) SP2008 (76th overall, 494 average hits per student) FA2007 (151st overall, 326 average hits per student) * Note: His ECAC329 “Cost Accounting” course registered 5,536 avg. hits per student, 1st among all UMBC Bb courses (not just undergrad) for FA2010.

WHAT ARE WE GONNA DO TODAY? Background Final Exam “My” Hybrid Way The Keys The Results My Opinion What’s Next

BACKGROUND Economics Department ECON 122 – Accounting Principles II Required for five or six majors Approximately 500 students per year Three or four folks teach the course – all part time faculty Common final – department prepared Final exam grades are shared Results???

FINAL EXAM Same exam for all students Fifty questions Two hours All multiple choice Students MUST score at least 50% on the final exam or all other work is for naught. Department policy

“MY” HYBRID WAY Tell ‘em what you’re gonna tell ‘em. Tell ‘em Tell ‘em what you just told ‘em.

“MY” HYBRID WAY Read the chapter Consider EOC questions “Watch” the video – one per LO Take an on-line quiz on the LO “Watch” the next LO video Take the next on-line quiz for the LO Repeat ‘til finished. The student can’t see LO2 ‘til they’ve successfully completed LO1.

“MY” HYBRID WAY When all LOs are complete, take an on-line conceptual quiz – this is the first activity that “counts”. 30 minute time limit, one attempt only. Then do the homework problems – again, on-line. The class meets and we address any questions about the homework.

“MY” HYBRID WAY They take an “on-line” quiz on the problems just discussed. They are exactly the same problems just discussed, EXCEPT, each student has different values/numbers to use. 60 minute time limit, one attempt only.

“MY” HYBRID WAY Tests End of semester Four tests 25 questions each 1 hour time limit – strictly enforced All multiple choice End of semester Mock final exam – with key A few minutes on “how” to take the final exam

THE KEYS Small chunks. LO videos less than 10 minutes, less than 7 minutes is better. Adaptive Release on BlackBoard. A student can’t see the next activity ‘til they’ve successfully completed the current activity. Overlay Adaptive Release with Date/Time “open” and Date/Time “closed”.

ADAPTIVE RELEASE

THE KEYS Report results on BlackBoard – Grade Center/My Grades Very FAST feedback on each of the four tests. They know their results within an hour. The key is posted on BlackBoard. Keep a close eye on your email. Put a “step by step” on BlackBoard in Announcements/”Start Here” when you “open” the class.

THE KEYS Alert students about BlackBoard “Certified” operating system/browser combinations. Don’t be their “tech support”. TEST, TEST, TEST, TEST, TEST, TEST, TEST your site BEFORE the semester starts. TEST it as a student! You’ll have to get a student login to BB. Make nothing so valuable that it is fatal.

THE KEYS Syllabus quiz Advice to Students. Do all the parts and pieces Keep up Ask if you have a question

THE RESULTS All but one of my F2F students have successfully passed the departmental final exam – PROVIDED they “did all the parts”. Best success rate in the department. All Hybrid (and online) students have successfully passed the departmental final exam – PROVIDED they “did all the parts”.

THE RESULTS Fall 2009 departmental results. My section was “hybrid” All other sections were F2F (other instructors) All students took the same exam. My students, on average, did about 20% better on the final exam than the F2F students. I was delighted. So were they.

THE RESULTS – CONTINUED Spring 2010 and Fall 2010 departmental exam results. While the “gap” narrowed, “hybrid” delivery still produced the best results. Summer 2010 Course was offered for the first time completely “on-line”, though it did require an “in-person” final exam. Better final exam average score results than “hybrid”.

THE RESULTS – CONTINUED Winter 2010 and Winter 2011 “Hybrid” delivery in a three week “mini-mester” All who “did what was asked” succeeded. Winter 2011 average final exam scores were the best ever recorded.

MY OPINION I was quite nervous on my maiden voyage. ‘Tis “more comfy” now. It worked!!! Yup, I like it.

WRAP UP

BB ACTIVITY BY GRADE DISTRIBUTION

BB ACTIVITY OF D & F STUDENTS Based on voluntary participation by instructors in 131 courses, students earning a D or F tend to use Bb 39 percent less than students earning higher grades. SP2010 21 courses | 37 percent less FA2009 29 courses | 37 percent less SP2009 11 courses | 47 percent less FA2008 13 courses | 40 percent less SU2007 7 courses | 33 percent less SP2008 26 courses | 32 percent less FA2007 15 courses | 36 percent less Does the pattern hold true during the semester? What if students knew this information sooner? NEW: iStrategy analysis of 1,461 SP2010 courses shows D&F students used Bb 47% less than students earning C or higher. We did not have access to “official” grades in SIS, so we studied “unofficial” grades in the CMS.

MYUMBC CHECK MY ACTIVITY (CMA)

www.umbc.edu/blackboard/reports

fritz@umbc.edu thardy@umbc.edu readel@umbc.edu THANKS fritz@umbc.edu thardy@umbc.edu readel@umbc.edu