ICAT: Instructional Computer Adaptive Testing John Fountain Marine Earth and Atmospheric Sci. North Carolina State University

Active Learning Value documented over 20 years (Bonwell and Eison, 1991) Increasing active learning generally requires reduction in conventional lecture Background material often supplied through assignments outside of class time

Is External Material Retained? Must compete with distractions: computer games, social media, TV, partying etc. Easy for students to avoid necessary effort Quizzes: traditional approach to encourage attention to material Multimedia: videos etc. – to increase interest and hopefully use

ICAT Designed to increase mastery of out-of- class material Required Quiz: to maximize student use Instructional: Primary design objective, uses videos, images and text.

ICAT Instructional Computer Adaptive Test Online Computerized Test (quiz) Adaptive: wrong answers lead to additional questions on topic Instructional: instructional material presented after wrong answers Every question must eventually be answered correctly to complete exam

What Mineral is this? a.Pyroxene b.Feldspar c. Quartz d.Calcite

Wrong answers lead to additional questions on subject The quiz is designed to address knowledge needed for each answer This question requires a knowledge of mineral properties and identification of cleavage If they answer “quartz” then they either do not remember the properties of quartz or did not recognize cleavage

What mineral is this? a.Pyroxene b. Feldspar c. Quartz This wrong answer leads to : Does this sample show cleavage? a. yes b.no

Instructional Material Wrong answers lead to instructional material targeted to knowledge area In this case on cleavage origin, significance and recognition

Flow Chart Every “main” question has a flow chart that defines the sub-question sequence and instructional material Sub-questions designed to cover knowledge areas needed to answer main question

Basic Concepts Wrong answers lead to information –Can be text, images, video Only information relevant to answer is shown Every Question must be answered correctly – eventually.

Reporting Automatic report generated for each class 1.% first answer correct for each student 2.% answers correct for each question 3.Histogram of answers for each question Allows assessment of individual students, of each question and of extent of understanding

ICAT Initial design for online courses Equally applicable to seated courses Primary use to increase mastery of material Can be used as study aid to prepare for tests Or as review to reinforce material

Design Web based exam, run from any browser Giving exam takes one-line . Questions submitted on form Student Identities masked

ICAT Assessment Comparison of performance of cohorts taking intro geology lab Initial review by Teaching Assistants Second review by non-random group Modified program in response to each review thus results not yet appropriate for assessment

Scalable and Open Source Question bank database (MySQL), can handle any number of questions /courses/subjects Server (Apache Tomcat) –(Walmart > 100 million pages/day) Written in java (FX2): runs on any platform (except ipad)

Structure Client code is very small Requires minimal memory and computing power – any processor can handle it Secure – no access to database by client program

Future Plans Remainder of academic year for assessment and development Expand to other courses at NC State and Wake Tech Community Coll. Extend to other interested institutions

Acknowledgements National Science Foundation Opportunities for Enhancement of Diversity in Geoscience Program Dr. David McConnell Dr. John Bedward Katherine Ryker Geoscience Learning Process Laboratory, Dept. of Marine Earth and Atmospheric Sciences, NC State Doel Gonzalez, Computer Science Department NC State