The Test Assessment Questionnaire Katherine M. Sauer William Mertens Metropolitan State College of Denver University of Colorado at Boulder

Slides:



Advertisements
Similar presentations
Rich Formative Assessment: Critical Component of Instruction
Advertisements

Characteristics of Descriptive Feedback
Introduction to Grading Grading is one of the most important activities a faculty member does. Many problems in teaching arise because of grading issues.
The Open Ended Response
Mary Jo Sariscsany Assessing Health- Related Fitness and Physical Activity 13 chapter.
Evaluation of Training
Self-Assessment Systems One way to convince students to take more ownership of their own learning process and set measurable and achievable goals. Dave.
STUDENT LEARNING OUTCOMES ASSESSMENT. Cycle of Assessment Course Goals/ Intended Outcomes Means Of Assessment And Criteria For Success Summary of Data.
Please CLOSE YOUR LAPTOPS, and turn off and put away your cell phones, and get out your note-taking materials. Today’s daily quiz will be given at the.
What do you know about Effective Teaching Behaviors?
EVALUATING WRITING What, Why, and How? Workshopping explanation and guidelines Rubrics: for students and instructors Students Responding to Instructor.
© 2006 McGraw-Hill Higher Education. All rights reserved. Chapter 19 Affective Behavior.
ICE Evaluations Some Suggestions for Improvement.
Formative and Summative Evaluations
Enhancing Student Learning Through Error Analysis
Techniques for Improving Student Learning Outcomes Lynn M. Forsythe Ida M. Jones Deborah J. Kemp Craig School of Business California State University,
TA End of Semester Survival Skills Lisa Kelly Center for Teaching October 31 st, 2013.
Data Collection and Preliminary Analysis Our survey addressed the first two of the questions presented in this study. The Qualtrics survey was framed by.
Reaffirmation of Accreditation: Institutional Effectiveness Southern Association of Colleges and Schools February 2008 Stephen F. Austin State University.
Governor’s Office of Early Childhood Teacher-Child Interactions in Early Childhood Settings CLASS is in session:
COMPASS National and Local Norming Sandra Bolt, M.S., Director Student Assessment Services South Seattle Community College February 2010.
Office of Institutional Research, Planning and Assessment January 24, 2011 UNDERSTANDING THE DIAGNOSTIC GUIDE.
Implementing Active Learning Strategies in a Large Class Setting Travis White, Pharm.D., Assistant Professor Kristy Lucas, Pharm.D., Professor Pharmacy.
LOCUS OF CONTROL Manishaa & Dayaanand.
AFT 7/12/04 Marywood University Using Data for Decision Support and Planning.
An Update on the MRSEC Cross-site REU Assessment with the Undergraduate Research Student Self Assessment Instrument Tim Weston University of Colorado,
Welcome to CS 3260 Dennis A. Fairclough. Overview Course Canvas Web Site Course Materials Lab Assignments Homework Grading Exams Withdrawing from Class.
Copyright WIDS Learning Design. Welcome! Performance Based Learning and Assessment Joseph “Mick” La Lopa, Ph.D. Purdue University Robin Soine, WIDS.
How to Evaluate Student Papers Fairly and Consistently.
Maximizing Learning Using Online Assessment 2011 SLATE Conference October 14, /12/ P. Boyles, Assistant Professor, Chicago State University,
Blackboard Learn Assessment and Feedback Ulster E-Learning Conference 20 th January 2011 Alan Masson & Fiona McCloy Access and Distributed Learning, University.
SPU Institute- Day 2 July 21, Session Overview Team Project Essential Conditions Thinking about implementation – Structures – Norms Formative Assessment.
Lack of Learning or Lack of Studying? An Inquiry into Low Exam Scores Katherine M. Sauer Metropolitan State College of Denver February.
Grading and Reporting Chapter 15
Acadia Institute for Teaching and Technology1 Creating a Balanced Course.
Exam Taking Kinds of Tests and Test Taking Strategies.
Student TAs in DSU Physics Labs Samuel Tobler, PhD Physics and Astronomy Teaching and Learning Conference April 2015.
THE RELATIONSHIP BETWEEN PRE-SERVICE TEACHERS’ PERCEPTIONS TOWARD ACTIVE LEARNING IN STATISTIC 2 COURSE AND THEIR ACADEMIC ACHIEVEMENT Vanny Septia Efendi.
Teacher Behaviors The teacher should allow the students to figure out the main idea of a lesson on their own. (SD, D, A, SA) –SD=4, D=3, A=2, SA=1 The.
Grade 4 Science.  Students will learn about:  Scientific Method  observing & inferring  Moon Cycles  Transportation  Electricity.
California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.
The Teaching Process. Problem/condition Analyze Design Develop Implement Evaluate.
Large-Scale Assessment of Student Learning: Valid, Reliable and Customizable Instruments for Courses, Departments, and Across Programs Stephen Carroll.
Connecting Course Goals, Assignments, and Assessment Faculty Development for Student Success at Prince George’s Community College William Peirce
Effective Grading Strategies Alison Morrison-Shetlar Faculty Center for Teaching and Learning Adapted from the book Effective Grading by Barbara Walvoord.
GATEWAY INITIATIVE Hillsborough Community College Fall 2007 Preliminary Results A Formative Evaluation.
Lessons from the Lab: Students’ Thoughts on ‘Right’ and ‘Wrong’ Dawn Del Carlo University of Northern Iowa 19 th BCCE, Purdue University Aug 31, 2006.
1 SUPPORTING PEDAGOGICAL CHANGE IN NMR SCHOOLS PROJECT Briefing of NMR secondary schools 11 February, 2010 Jean Russell, Graeme Jane, Graham Marshall.
Click to add title A Busy Professor’s Guide to Sanely Flipping Your Classroom Dr. Cynthia Furse Electrical & Computer Engineering.
The Test Assessment Questionnaire: A Tool for Student Self-Assessment After the Midterm Exam Katherine M. Sauer Metropolitan State University of Denver.
Strategies for Creating Success in College and in Life On Course Chapter 2 ACCEPTING PERSONAL RESPONSIBILITY.
Exploring Non-Cognitive Influences on College Success with New First-year Advisees Kristin Douglas Associate Dean of the College Mary Windeknecht Director.
The Use of Formative Evaluations in the Online Course Setting JENNIFER PETERSON, MS, RHIA, CTR DEPARTMENT OF HEALTH SCIENCES.
Syllabus Highlights CSE 1310 – Introduction to Computers and Programming Vassilis Athitsos University of Texas at Arlington 1.
Scoring Rubrics: Validity and Reliability Barbara M. Moskal Colorado School of Mines.
Idiom of the Day IN THE LOOP To keep someone informed and up-to-date about what’s happening – usually in the workplace.
DEVELOPED BY MARY BETH FURST ASSOCIATE PROFESSOR, BUCO DIVISION AMY CHASE MARTIN DIRECTOR OF FACULTY DEVELOPMENT AND INSTRUCTIONAL MEDIA UNDERSTANDING.
[School Name]’s Student Perception Survey Results This presentation is a template and should be customized to reflect the needs and context of your school.
USING STUDENT EVALUATIONS AT STOCKTON Heather McGovern Director of the Institute for Faculty Development Associate Professor of Writing January 2012.
Jacksonville, FL March 2013 Welcome, Bienvenido, Bienvenu Teaching Certification Programs Key Questions for Design & Refinement Judith Longfield Georgia.
Unit Seven Seminar Power Point Presentation. UNIT 7: Reading Importance Study Tips Steps to Success Improving Reading.
Jackson College CCSSE & CCFSSE Findings Community College Survey of Student Engagement Community College Faculty Survey of Student Engagement Administered:
ASSESSMENT OF STUDENT LEARNING
Classroom Assessment Validity And Bias in Assessment.
The Good The Bad & The Ugly Real-Life Examples of the SLO Assessment Report Form With Tips on How to Complete It August 21, 2012.
common expectations (5)
An Introduction to Study Skills
Using Family Survey Data for Program Improvement
Presentation transcript:

The Test Assessment Questionnaire Katherine M. Sauer William Mertens Metropolitan State College of Denver University of Colorado at Boulder March 29, 2012MBAA – ABE Annual Conference Chicago, IL

Students are Overconfident Walstad (2001) - suggests concepts from behavioral economics (e.g. overconfidence) could be used to explain student behavior Falchikov and Boud (1989) - students have grade expectations that are higher than the typical distribution for the course Grimes (2002) - has principles students predict exam scores and finds overconfidence in their understanding of material Nowell and Alston (2007) - instructor grading practices can influence the degree of overconfidence

What Happens When Students are Overconfident? Students may start to blame the instructor for the disconnect between (perceived) effort and desired grade. Students may label the instructor as “unfair” or the course simply “too hard”.

Grimes, Millea, and Woodruff (2004) - degree to which students accept personal responsibility for performance affects their evaluation of teaching effectiveness and course satisfaction Carrell and West (2010) - in course evaluations, students reward professors who increase achievement in the contemporaneous course, not those who facilitated deep learning for subsequent courses Millea and Grimes (2002) - instructors need not “water- down” courses in order to receive favorable course evaluations… they can positively influence evaluations by addressing negative student attitudes about forthcoming coursework

How can I help my students become more self-aware in their learning? Get them to focus on the type of mistake they are making instead of the number of mistakes they are making. Get them thinking about the link between their study activities and their exam performance.

The Test Assessment Questionnaire This instrument guides students through an analysis of their midterm exam mistakes. questions about exam preparation activities studying activities for the course in general formulate a study strategy for the rest of the course

The Version I’m Currently Using

Instructors can tailor the first set of questions to their course.

Logistics - TAQ worksheet is given to students when the exam is handed back - due one week later and counts as a homework grade - in the class period when it is due, debrief with study strategies for each type of common mistake (10 minutes) - students can come to office hours for additional guidance

Three Benefits 1) it guides students toward more self-awareness in their studies 2) it can be used as a part of assessment and assurance of learning efforts 3) may improve students’ final exam scores over the midterm

1) self-awareness in studies On the course evaluations: Please comment on the experience of completing the Test Assessment and/or meeting with me to discuss it. - It made me improve my studying skills. -Good, it gave me one-on-one time with you to figure out how I could improve. -Helpful in learning how I make mistakes and how to fix them better.

- It helped me to personally analyze my test taking and study habits. Very helpful! I have never had a teacher who cared enough to take the time to help me improve in this aspect. -Your tips helped me in other classes and got me running again and I don’t feel as stressed. My mom thanks you. There was no negative feedback on this question.

2) Assessment & Assurance of Learning More and more time is devoted to these activities. We assume that students are doing their part in the learning process. To what extent are they? Routinely, I find students who earn a D or F on an exam reveal that they simply didn’t study. - provides context to “does not meet expectations”

3) improve students’ final exam scores Experiment: Fall 2011 – University of Colorado All Students in Principles of Microeconomics 10 course sections, 1713 students 62 recitation sections

Design: Course instructors do not modify their class time or teaching activities in any way. Each TA that teaches two or more recitation sections has a control and experiment section assigned to them. The experiment will take place during the recitation class in which the midterm is handed back.

Control Sections: TAs spend the class time going over the exam as they normally would and will offer some general study strategies. Experiment Sections: Students will be given the correct answers and will spend the class time going over the exam by themselves while completing the TAQ. After the TAQs are completed, the TA discusses study strategies that complement each of the common mistake categories.

All students complete Duttweiler’s (1984) Internal Control Index. - many such instruments - argue this one is most reliable and valid one for adults - normed on college students - 28 questions, Likert-type scale

Half of the items are worded so that high internally oriented respondents are expected to answer at the "usually" end of the scale and the other half at the "rarely" end. The "rarely" response is scored as 5 points on items 1,2,8,11,14,17,19,22,23,24,26,27. This produces a possible range of scores from 28 to 140 with higher scores reflecting higher internal locus of control. For college students: MEAN:

Data: Two course instructors opted out of the experiment. n = treatment recitations, n = control recitations, n = 670 But not all sections were full.

We received 522 first-round ICI surveys from 28 different recitations. We received 488 second-round ICI surveys also from 28 different recitations (but not exactly the same recitations as those for the first-round surveys). We received 234 TAQs from 13 different recitations.

Treatment groups We received data that included both surveys and the TAQ for six recitations involving a maximum of 88 students having all 3 sheets (if they match). We received data that included one survey and a TAQ from seven recitations involving a maximum of 104 students having both sheets (if they match). Control groups We received data that included both surveys for ten recitations involving a maximum of 175 students having both sheets (if they match).

The student’s TAQs and ICIs are being scanned and the data entered into a spreadsheet. Control data is being obtained from Institutional Research.

Comments? Questions? Katherine M. Sauer Metropolitan State College of Denver March 29, 2012MBAA – ABE Annual Conference Chicago, IL