Evolution of Institutional Capacity to support the Assessment-Change- Effectiveness Cycle Course Content & Design Implementation Examples Assessment Strategies by Dr. Greg Witteman, U. Guam

The big questions What are students’ abilities before taking the class? –What Knowledge, Skills, or Abilities (KSAs) should the students enter the class with? What are the major objectives of the class? –What KSAs should the students leave the class with? What is the role of assessment in course design or delivery? –Can outcomes be improved? (and how would we know?)

Type of Course Content Self contained Content (no prerequisites) –Examples: Anatomy and Physiology I, and II Prerequisite knowledge needed –Examples: Conservation Biology and Comparative Vertebrate Biology labs Abilities (student) Objectives (Course) Assessment (results)

Self contained course content Anatomy and Physiology I & II –Assumes no prior knowledge –Each part of the course builds on knowledge gained earlier in the course (hierarchy) Abilities (student) Objectives (Course) Assessment (results)

Course content assumes KSAs e.g. Without general biology, math and ecology species population size, distribution and probability of extinction can not be meaningfully presented Cons. Bio. Previous courses Abilities (student) Objectives (Course) Assessment (results)

Course Design and Delivery Strategies Course content –(what type of subject matter and students?)  Course goals –(what do we hope to accomplish?) What should students be able to do to finish the course? What should they know after the course ends? –Delivery and Assessment methods (how and when do we know if we accomplished our goals ?)

The Ranges of KSA’s and goals for Science Courses No Knowledge limited background Have Knowledge from: H.S., College, other Lower Division non-Sci. Majors Lower Division Science Majors Upper Division Science Majors Entry KSAs Courses Course goals Abilities (student) Objectives (Course) Assessment (results)

Example 1: Anatomy and Physiology Non-science majors (freshmen/sophomore) –Extreme range of educational and cultural backgrounds Goal: “Remember and Understand”: –Content heavy (terminology for structures must be learned before subsequent structures, function & concepts can be presented) Two semester course, concurrent lab. (one grade) Large lecture hall, no blackboard (PowerPoint and overheads only). Typical lab facilities. No pre-requisites or assumed knowledge Abilities (student) Objectives (Course) Assessment (results)

Original Course Delivery model Abilities (student) Objectives (Course) Assessment (results) Concepts build on those previously presented in class. Students must leave course with knowledge and ability in every topic presented (which will be needed for subsequent courses and professional certification)

Advantages and Disadvantages (assessing this approach) Pro: consistent delivery of material (Year-to-year) Con: can not adapt to differences in student learning (can leave gaps in knowledge) Abilities (student) Objectives (Course) Assessment (results)

Problems with this approach Abilities (student) Objectives (Course) Assessment (results) Changes in content delivery (availability & access): Addition of supplemental materials on web and in library, adopted new texts.

Addressing these problems (adding adaptive assessment TO THIS approach) Weekly adaptive quizzes –Goal = insure that content is covered and retained by “recycling” or re-emphasizing poorly understood topics (subsequent content adapts to knowledge gaps indicated by objective measures of student performance). Pre-post tests (entry assessments, midterms and final) –Goal = overall course improvement by evaluating: successful presentations of individual topics, retention of material after completion of course (i.e.:of first semester material in second semester) Abilities (student) Objectives (Course) Assessment (results)

Adaptive Assessment Quiz 1 30 Neurons Neuroglia Function 20 Results 1 70% 80% 60% Quiz 2 40 Neurons Neuroglia 40 0 (N. Function) 20 Needs review? Review Abilities (student) Objectives (Course) Assessment (results)

Assessment “feedback” loop Pro: an adaptive approach can be used to close the gaps in student knowledge Con: takes time and effort by instructor to implement. Note: this can be used for both self contained and pre-requisite knowledge courses Abilities (student) Objectives (Course) Assessment (results)

Course Changes: Technology classroom (NIH-RISE) Interactive media & Video on demand –Dissections, models, background & supplemental information View PowerPoint lectures Print lecture notes and outlines Automated practice quizzes and tests –(Web-based, secure (CGI),with feedback (in TC only) Equal access to course materials

Example 2: Comparative Vertebrate Biology Upper division biology majors –Juniors/Seniors (pre-med, pre-vet, pre-graduate school) “Remember, Understand & Analyze” –Content and concept intensive Lecture & lab (20 students) Assumes basic understanding of lower- division general biology, basic laboratory skills, understanding of correct terminology. Abilities (student) Objectives (Course) Assessment (results)

Course design & delivery (lab) Lab exercises with questions –(non-adaptive, “you must know this…” and “how is this structure similar to …”). Assumes good study habits and skills. Student presentations of lecture summaries in lab –Identify difficult concepts with need for more in-depth coverage (identify gaps in knowledge) –Create participatory classroom dynamic Lab documentation projects –Students create class materials (self documenting review materials) –“problem based learning” Abilities (student) Objectives (Course) Assessment (results) NOTE: Student self- assessment/feedback is within lab prior to tests

Course changes: NIH-Rise-TC Visual review (Digital Video) of in-lab procedures, dissections, and specimens –A picture is worth a thousand words, and a video is worth a thousand pictures! Equal access to information resources –(“go find everything you can about…” on the web.)

Learner-centered approaches to assessment/improvement implemented Human Anatomy and Physiology –Use of adaptive weekly quizzes to “fine-tune” delivery of individual lectures (year-to-year), guarantee a sufficient knowledge foundation on which to build additional concepts (within the semester) –Entry and Exit assessments provide the information necessary to measure success in meeting the course goals Comparative Vertebrate Anatomy Lab –Use of self and peer assessment to identify and fill gaps in knowledge, skills and abilities before they are formally tested.

Application of NIH-Rise Technology Classroom capabilities Increases in information resources availability –Online resources (web) –Class notes, outlines and presentations. Dynamic content capabilities –Self serve video (TB server capable of 600+ hours of MP1 video) –Interactive content (practice exams, data exploration) Provide all the Information Technology tools needed for “Problem based learning” …and, let’s not forget Technology Development support and training for faculty

The bad news There is no free lunch –Incorporating adaptive assessment (in A&P) requires an additional 3 hours each week to quiz and lecture preparation and delivery (50% increase after initial development of course materials) –Adding self and peer assessment (2 hours additional planning and setup for TC use).

Still to be addressed... Why did I do this? –Student outcomes poor relative to objective standards Why should others do this? How can others incorporate these types of assessment features in their courses? –Drink more coffee –Give up sleep Conclusion: still some implementation obstacles