Testing Process and Data Collection Ellen Iverson February 2018

By the end of this presentation, you should be able to: Testing Process and Collection By the end of this presentation, you should be able to: Find more information as you need it: web resources, webteam liaison, assessment consultant, team leads Know timing of piloting w/r/t authoring and rubric review Understand what data will be collected and why

Pilot Materials, Make changes Identify & Module Learning Goals Identify learning outcomes Determine how to assess and measure student success on goals and outcomes Design teaching resources and materials to match assessments Plan Instructional Strategies to implement teaching resources Pilot Materials, Make changes Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes?

Timeline for Materials Development Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes? *IRB verify at your institution and before summer

Data Collection Scheme Pre/Post project measures Student demographic data Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes?

What assessment data are collected? Assessment and Measurement What assessment data are collected? Measuring the level of understanding when learning related to that course/module goal is completed Important for adoption Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes?

What assessment data are collected? Assessment and Measurement What assessment data are collected? Pre- & Post-course Geoscience Literacy Assessment (GLE 8 MC questions) 2 from each literacy document 4 Single answer and 4 “select all that apply” Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes? http://serc.carleton.edu/integrate/info_team_members/currdev/documents/gle/index.html

Assessment and Measurement What assessment data are collected? Pre- & post-course motivation and career interest survey (related to project evaluation) URL created through SERC using course set-up form Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes? http://serc.carleton.edu/integrate/info_team_members/currdev/documents/course_form.html

Assessment and Measurement Course Information Demographic data Faculty Reflection Surveys Checkpoint 4 (after development) Notes while teaching (individual reporting page) Post-Pilot After Revisions Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes?

Data Collection Scheme Pre/Post project measures Student demographic data Students work together in small groups determined by experience, skills, or demographic factors (e.g., gender) Formative exercises during class used to assess student understanding and progress and to challenge students to think beyond comfort level Classes incorporate most or all of 7 good practices How can feedback devices produce learner-centered classes?

IRB: What you need to know Make contact with your Institution about IRB early (Checkpoint 1) Know that IRB procedures vary by institution Give project list of which students are under age* Carleton as IRB of record has already found GETSI to be exempt from IRB oversight for collection of student data and faculty reflections You may need one of the letters indicated EXEMPT Ask for help! * May not be required, dependent on Federal new adoption date of IRB rule

Data Collection Where can you find help? http://serc.carleton.edu/integrate/info_team_members/currdev/course_status.html Where can you find help?

What questions do you have about testing and data collection? Testing Process and Assessment What questions do you have about testing and data collection?