+ Old Reliable Testing accurately for thousands of years.

Slides:

Advertisements

Similar presentations

Parts of a Lesson Plan Any format that works for you and your JTEs is ok… BUT! Here are some ideas that might help you set up your LP format. The ALTs.

Advertisements

Assessing Student Performance

CHAPTER 8 Copyright © 2002 by the McGraw-Hill Companies, Inc.

ITEMS Grocery Store? Couples? No, language testing.

Consistency in testing

Topics: Quality of Measurements

Some (Simplified) Steps for Creating a Personality Questionnaire Generate an item pool Administer the items to a sample of people Assess the uni-dimensionality.

Types of Reliability.

Chapter 5 Reliability Robert J. Drummond and Karyn Dayle Jones Assessment Procedures for Counselors and Helping Professionals, 6 th edition Copyright ©2006.

Assessment Procedures for Counselors and Helping Professionals, 7e © 2010 Pearson Education, Inc. All rights reserved. Chapter 5 Reliability.

VALIDITY AND RELIABILITY

Lesson Six Reliability.

Reliability for Teachers Kansas State Department of Education ASSESSMENT LITERACY PROJECT1 Reliability = Consistency.

Testing What You Teach: Eliminating the “Will this be on the final

What is a Good Test Validity: Does test measure what it is supposed to measure? Reliability: Are the results consistent? Objectivity: Can two or more.

ACT Writing. 1 Question – 30 Minutes The ACT Writing prompt is an ARGUMENTATIVE topic. You will need to take a clear position on the topic (yes or no).

Making Your Assessments More Meaningful Flex Day 2015.

Can you do it again? Reliability and Other Desired Characteristics Linn and Gronlund Chap.. 5.

Reliability Analysis. Overview of Reliability What is Reliability? Ways to Measure Reliability Interpreting Test-Retest and Parallel Forms Measuring and.

Measurement Validity and Reliability. Reliability: The degree to which measures are free from random error and therefore yield consistent results.

Lesson Seven Reliability. Contents  Definition of reliability Definition of reliability  Indication of reliability: Reliability coefficient Reliability.

1 BASIC CONSIDERATIONS in Test Design 2 Pertemuan 16 Matakuliah: >/ > Tahun: >

Multiple Choice Test Item Analysis Facilitator: Sophia Scott.

Research Methods in MIS

Using statistics in small-scale language education research Jean Turner © Taylor & Francis 2014.

Comprehensive Assessment System Webinar #6 December 14, 2011.

Stages of testing + Common test techniques

Testing Writing. We have to : have representative sample of the tasks that we expect the students to perform. those task should elicit valid samples of.

Technical Issues Two concerns Validity Reliability

Validity and Reliability

Reliability, Validity, & Scaling

TEXAS TECH UNIVERSITY HEALTH SCIENCES CENTER SCHOOL OF PHARMACY KRYSTAL K. HAASE, PHARM.D., FCCP, BCPS ASSOCIATE PROFESSOR BEYOND MULTIPLE CHOICE QUESTIONS.

RELIABILITY BY DESIGN Prepared by Marina Gvozdeva, Elena Onoprienko, Yulia Polshina, Nadezhda Shablikova.

Standardized Testing (1) EDU 330: Educational Psychology Daniel Moos.

All you need to know about The Regents Exam. History of the Regents Test Regents Policy was created in 1972 by the Board of Regents of the University.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.

Reliability Lesson Six

LECTURE 06B BEGINS HERE THIS IS WHERE MATERIAL FOR EXAM 3 BEGINS.

Technical Adequacy Session One Part Three.

Induction to assessing student learning Mr. Howard Sou Session 2 August 2014 Federation for Self-financing Tertiary Education 1.

I Can Distinguish the types of validity Distinguish the types of reliability Identify if an example is objective or subjective Copyright © Allyn & Bacon.

How to Revise an Essay. Done-ness  After you finish the first draft of an essay, a sense of calm settles over your body. “At last,” you say, “I’m done.”

Tips for Editing an Essay Learning Assistance & Tutorial Center Mission College To view this presentation, click your space bar or arrow keys.

Assessment in Education Patricia O’Sullivan Office of Educational Development UAMS.

Listening to Bob Marley makes me think of statistics and standardized testing….. Said no one ever But… it is still your responsibility.

Chap. 2 Principles of Language Assessment

EDU 8603 Day 6. What do the following numbers mean?

Validity and Reliability Neither Valid nor Reliable Reliable but not Valid Valid & Reliable Fairly Valid but not very Reliable Think in terms of ‘the purpose.

Assessment. Workshop Outline Testing and assessment Why assess? Types of tests Types of assessment Some assessment task types Backwash Qualities of a.

What are the stages of test construction??? Take a minute and try to think of these stages???

Building Exams Dennis Duncan University of Georgia.

 Research Design Part 2 Variability, Validity, Reliability.

1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.

Reliability Ability to produce similar results when repeated measurements are made under identical conditions. Consistency of the results Can you get.

Measurement Experiment - effect of IV on DV. Independent Variable (2 or more levels) MANIPULATED a) situational - features in the environment b) task.

Imagine…  A hundred students is taking a 100 item test at 3 o'clock on a Tuesday afternoon.  The test is neither difficult nor easy. So, not ALL get.

Stages of Test Development By Lily Novita

Validity & Reliability. OBJECTIVES Define validity and reliability Understand the purpose for needing valid and reliable measures Know the most utilized.

RELIABILITY BY DONNA MARGARET. WHAT IS RELIABILITY?  Does this test consistently measure what it’s supposed to measure?  The more similar the scores,

Classroom Assessment Chapters 4 and 5 ELED 4050 Summer 2007.

Language Assessment Lecture 7 Validity & Reliability Instructor: Dr. Tung-hsien He

25 minutes long Must write in pencil Off topic or illegible score will receive a 0 Essay must reflect your original and individual work.

Reliability. Basics of test score theory Each person has a true score that would be obtained if there were no errors in measurement. However, measuring.

Assessment in Education ~ What teachers need to know.

Measurement Reliability and Validity

Series of Paragraphs Expressing an Opinion

RELIABILITY IN TESTING

The extent to which an experiment, test or any measuring procedure shows the same result on repeated trials.

Validity and Reliability II: The Basics

Series of Paragraphs.

Presentation transcript:

+ Old Reliable Testing accurately for thousands of years

+ Reliability Validity def.? Validity is how closely a test measures what it says it will measure. Reliability def.? Reliability is the consistency of your measurement,... or the degree to which an instrument measures the same way each time it is used… under the same conditions with the same subjects.

+ Why aren’t you more reliable? Imagine your students are taking a long multiple choice test, What are some common threats to reliability? Guessing Ambiguous questions Physical/emotional state of test takers Environmental distractions Mechanical problems, e.g. computer glitches, not filling in bubbles properly.

+ Why aren’t you more reliable? Imagine your students are taking an essay test, What are some common threats to reliability? Assessor or Inter-rater reliability Object/Person-related reliability Instrument-related reliability

+ Do we measure reliability??? No, we “estimate” it. How can we estimate reliability? Test-Retest Split-half What are the pros and cons of each?

+ Measuring reliability??? What do you get after you perform these tests? A reliability coefficient, like.91 or.78 What would be the range of reasonable reliability coefficients for a grammar test or vocabulary test?.9 to.99 How about for a speaking test?.7 to.79

+ Investigar Go to the class wiki located at Then click “Materials to be used in class” and follow the instructions under reliability of SAT.

+ Standard Error and True Score How high can you jump off of both feet with no steps? Average of several attempts = closer to “True Score” Can we really know what someone’s “True Score” truly is? Impossible, but... with lots of attempts, u get close.

+ Top Several Ways to Enhance Reliability Take enough samples Exclude items don’t distinguish between students Don’t give too much freedom Write unambiguous items/questions Provide clear instructions Ensure legibility, clarity of design, white space etc. Train Scorers Proved detailed scoring key/rubric/instructions Identify students/participants by number, not name Multiple raters for subjective stuff. Which of these work for teachers in a college or pub school setting?

+ Testing Tasks You are giving the following writing assignment to your students in Spanish Write a paper in Spanish giving your thoughts on a current event. With a partner, rewrite the instructions to improve reliability.

+ Testing Tasks Discuss with a partner… You are testing students’ oral proficiency as a final exam in your Spanish 3 high school class. What can you do to improve the reliability of scoring? You want to know which multiple choice questions on your test distinguish between stronger and weaker students. How might you analyze test results to determine this? Think of the main steps and then explain to a partner.

+