HOMEWORK REVIEW Establishing Baseline Assessments Student learning outcomes Course completion rates Traditional course costs (CPT)

C2R ASSESSMENT GOAL To establish the degree to which improved learning has been achieved as a result of the course redesign.

ASSESSMENT PLANNING Step 1. Establish the method of obtaining data. Step 2. Choose the measurement method.

ESTABLISH THE METHOD OF OBTAINING DATA Baseline “Before” (traditional) and “After” (redesign) Parallel Sections – Compare traditional sections and redesigned sections (Fall 2008)

CHOOSE THE MEASUREMENT METHOD: FIVE MODELS A. Comparisons of Final Exams B. Comparisons of Common Content Items Selected from Exams C. Comparisons of Pre- and Post- Tests D. Comparisons of Student Work using Common Rubrics E. Comparisons of Course Grades using Common Criteria

FIVE MEASUREMENT METHODS Common Finals: Coppin State U, Santa Fe College Pre- and Post-Tests: Edison State College Multiple Methods: El Paso CC (A & C), Morehead State U (A, C, E), UMES (A, B, E), UNC Charlotte (C & D) Unknown: Regis U, U of Minnesota, UW Bothell Common Finals: Saint Leo U, SELU – good Common Content Items: Oklahoma State U, Western Michigan U (Why not common exams? WMU – 10 questions – what %) Pre- and Post-Tests: Austin CC Multiple Methods: ASU (A & D), Auburn U--Physics) (A & B – don’t need B unless doing sub-analysis), NYIT (A, B & E), U of West Alabama (A & C; Really D - rubrics for essays), UCF (A, C, E) Grades: UMass Lowell - Grades based on 4 hourlies & a final; anything else? Unknown: Auburn U (Math) – need to decide

GRADES ARE NOT A SUFFICIENT MEASURE OF STUDENT LEARNING Lack of consistency Different coverage Different tests and exams Curving Inflation Grades: UMass Lowell - Grades based on 4 hourlies & a final; anything else? UCF, NYIT – have other measures, omit grades Use only for course completion!

More Is Not Better! Performance in follow-on courses Student attitude toward subject matter Student interest in pursuing further coursework in the discipline Differences in performance among student subpopulations Deep vs. superficial learning

If you have data If you do not have data Indicate the method you used to obtain data with specifics (# of students, # of sections) Indicate how you will measure learning with specifics (A – E) and report the results If you do not have data Indicate the method you intend to use with specifics (# of students, # of sections) Indicate how you will measure learning with specifics (A – E) and describe This is what we asked you to do for your homework. 6 of you have the data, 6 of you do not but only 2 of you reported the data (Austin CC and W Michigan U.) WMU – inscrutable – “Average of 10 embedded questions = 59.5% + 3.8%” Remember: 100 minimum in each group

Here is what we will ask you to submit with your final proposal in June. Go over each item. At least 100 students (not 2 sections of 25 each – not meaningful) Fall and spring problem

COURSE PLANNING TOOL A decision-making tool that enables institutions to compare the “before” activities and costs (the traditional course) and the “after” activities and costs (the redesigned course)

THE CPT - IS IT WORTH IT? Provides a structure for you to think about activities and costs Allows you to consider changes in specific instructional tasks Permits you to visualize duplication and waste Enables you to do a cost/benefit analysis re: type of personnel per task

ACTIVITIES AND COSTS Determine all personnel costs expressed as an hourly rate. Determine the specific tasks associated with offering a course. Determine how much time each person spends on each of the tasks. Calculate the total instructional costs. Redesign the course by task and re-calculate the costs. Compare the cost-per-student.

Instructional Costs per Hour

PERSONNEL COSTS Spreadsheet 1 Be sure to include benefits in personnel costs. Include all personnel types on sheet 1. Use an average salary by faculty type/role for one section rather than showing each individual salary. Show one average salary for each type rather than an aggregate. Use the same personnel costs on all 3 sheets. Calculation for faculty cost per hour should be for one person teaching one section.

Traditional Course Preparation

Traditional Course Delivery

COST-PER-STUDENT There may be good reasons for differences, but frequently big differences mean that you have made errors in your CPT draft. We suggest that you look at like institutions or disciplines to see if you can figure out what is the cause of the difference.

IN-CLASS TO OUT-OF-CLASS Page #1 Typically 45 in, 90 out, 135 total Most institutions have a rule-of-thumb expectation of something like two hours prep time (out-of-class) for each hour spent in class. Seems to be a trend in Round III of underestimating outside-of-class time – 2 for 1 was not a common ratio for this group.

DO #1 AND #2 CORRESPOND? The number of hours (in-class plus out-of-class) on the Instructional Costs per Hour sheet (#1) and the Instructional Costs of Traditional Course sheet (#2) should correspond. This correspondence helps guard against unrealistic estimates of how personnel spend their time on the individual course tasks. By using this expectation, we avoid the "self-reporting" problem. Establishing this rule-of-thumb expectation as a framework also encourages faculty members and others to be more realistic when estimating time associated with specific instructional tasks. Thus, the total on sheet #2 should roughly correspond to the total on sheet #1, which expresses the institutional “rule of thumb.”

Redesigned Course Preparation

Redesigned Course Delivery

MISCELLANEOUS CPT assumptions Section vs. whole course Read the instructions Be sure to consult the definitions Explain worksheets in CPT narrative Do not add worksheets to CPT except . . . When more than 3 worksheets are needed Section vs. whole course: If a simple extrapolation like UW - show section and multiply (Be careful re: multiple sections and prep issue) Otherwise show whole course e.g., diverse instructional arrangements – personnel, sections per semester  IX. When You Need More Than 3 Spreadsheets Speculation vs. fact (what to include in spreadsheet vs. narrative) So Maine conduct distance courses with savings; count revenue (no - speculative) If you plan to reduce attrition or increase academic success, please be as specific as you can as to how you plan to do it and why you think you can. (UCF 7% yes; MDCC no) Below the line items More than 3 spreadsheets UB - 2 redesigns U of Idaho - 3 courses into 1 Colorado - Tri model – C vs. Q– old and expensive, cheap but low quality, new, cheap and good Redesigns of redesign (Umass, Cal Poly & CMU)  

COURSE STRUCTURE FORM A formatted spreadsheet that enables institutions to compare the structure of the traditional course with the that of the redesigned course (types of sections, number of students enrolled and the kinds of personnel)

FOR MORE INFORMATION www.theNCAT.org