RELIABILITY IN TESTING

Slides:

Advertisements

Similar presentations

Questionnaire Development

Advertisements

Measurement Concepts Operational Definition: is the definition of a variable in terms of the actual procedures used by the researcher to measure and/or.

Reliability IOP 301-T Mr. Rajesh Gunesh Reliability  Reliability means repeatability or consistency  A measure is considered reliable if it would give.

Chapter 1 What is listening?

Reliability Presentation Test-Retest James Blackwood – AED 615 Fall Semester 2006.

Topics: Quality of Measurements

Chapter 5 Reliability Robert J. Drummond and Karyn Dayle Jones Assessment Procedures for Counselors and Helping Professionals, 6 th edition Copyright ©2006.

Assessment Procedures for Counselors and Helping Professionals, 7e © 2010 Pearson Education, Inc. All rights reserved. Chapter 5 Reliability.

Lesson Six Reliability.

1Reliability Introduction to Communication Research School of Communication Studies James Madison University Dr. Michael Smilowitz.

 A description of the ways a research will observe and measure a variable, so called because it specifies the operations that will be taken into account.

Reliability for Teachers Kansas State Department of Education ASSESSMENT LITERACY PROJECT1 Reliability = Consistency.

Testing What You Teach: Eliminating the “Will this be on the final

Evaluating tests and examinations What questions to ask to make sure your assessment is the best that can be produced within your context. Dianne Wall.

Can you do it again? Reliability and Other Desired Characteristics Linn and Gronlund Chap.. 5.

Reliability Analysis. Overview of Reliability What is Reliability? Ways to Measure Reliability Interpreting Test-Retest and Parallel Forms Measuring and.

Chapter 15 Conducting & Reading Research Baumgartner et al Chapter 15 Measurement Issues in Research.

Identification, Assessment and Re-classification of English Learners Initial Identification  Complete within 30 school days of enrollment Administer Home.

Lesson Seven Reliability. Contents  Definition of reliability Definition of reliability  Indication of reliability: Reliability coefficient Reliability.

1 BASIC CONSIDERATIONS in Test Design 2 Pertemuan 16 Matakuliah: >/ > Tahun: >

Basic Issues in Language Assessment 袁韻璧輔仁大學英文系. Contents Introduction: relationship between teaching & testing Introduction: relationship between teaching.

VALIDITY & RELIABILITY Raja C. Bandaranayake. QUALITIES OF MEASUREMENT DEVICES  Validity Does it measure what it is supposed to measure?  Reliability.

Research Methods in MIS

Using statistics in small-scale language education research Jean Turner © Taylor & Francis 2014.

Reliability of Selection Measures. Reliability Defined The degree of dependability, consistency, or stability of scores on measures used in selection.

Classical Test Theory By ____________________. What is CCT?

LG675 Session 5: Reliability II Sophia Skoufaki 15/2/2012.

Reliability, Validity, & Scaling

RELIABILITY BY DESIGN Prepared by Marina Gvozdeva, Elena Onoprienko, Yulia Polshina, Nadezhda Shablikova.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.

Unanswered Questions in Typical Literature Review 1. Thoroughness – How thorough was the literature search? – Did it include a computer search and a hand.

Reliability Lesson Six

+ Old Reliable Testing accurately for thousands of years.

Principles in language testing What is a good test?

Chapter 4: Test administration. z scores Standard score expressed in terms of standard deviation units which indicates distance raw score is from mean.

RELIABILITY AND VALIDITY OF ASSESSMENT

RELIABILITY Prepared by Marina Gvozdeva, Elena Onoprienko, Yulia Polshina, Nadezhda Shablikova.

Assessment. Workshop Outline Testing and assessment Why assess? Types of tests Types of assessment Some assessment task types Backwash Qualities of a.

McGraw-Hill/Irwin © 2012 The McGraw-Hill Companies, Inc. All rights reserved. Obtaining Valid and Reliable Classroom Evidence Chapter 4:

1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.

Nurhayati, M.Pd Indraprasta University Jakarta.  Validity : Does it measure what it is supposed to measure?  Reliability: How the representative is.

 A test is said to be valid if it measures accurately what it is supposed to measure and nothing else.  For Example; “Is photography an art or a science?

Reliability in Testing Is the test or assessment tool consistent and dependable? Student-related reliability Rater reliability Test administration reliability.

Reliability and Validity in Testing. What is Reliability? Consistency Accuracy There is a value related to reliability that ranges from -1 to 1.

Imagine…  A hundred students is taking a 100 item test at 3 o'clock on a Tuesday afternoon.  The test is neither difficult nor easy. So, not ALL get.

RELIABILITY BY DONNA MARGARET. WHAT IS RELIABILITY?  Does this test consistently measure what it’s supposed to measure?  The more similar the scores,

Reliability When a Measurement Procedure yields consistent scores when the phenomenon being measured is not changing. Degree to which scores are free of.

Language Assessment Lecture 7 Validity & Reliability Instructor: Dr. Tung-hsien He

The definition of table of specification. Table of specification is a chart that provides graphic representations of a related to the content of a course.

ESTABLISHING RELIABILITY AND VALIDITY OF RESEARCH TOOLS Prof. HCL Rawat Principal UCON,BFUHS Faridkot.

Professor Jim Tognolini

EVALUATING EPP-CREATED ASSESSMENTS

CHAPTER 3: Practical Measurement Concepts

Ministry of Defense of Georgia

Assessment Theory and Models Part II

RELIABILITY OF QUANTITATIVE & QUALITATIVE RESEARCH TOOLS

Tests and Measurements: Reliability

Reliability & Validity

مركز مطالعات و توسعه آموزش دانشگاه علوم پزشكي كرمان

Questionnaire Reliability

VALIDITY Ceren Çınar.

PSY 614 Instructor: Emily Bullock, Ph.D.

The extent to which an experiment, test or any measuring procedure shows the same result on repeated trials.

Chapter 4 Characteristics of a Good Test

By ____________________

Classroom Assessment A Practical Guide for Educators by Craig A. Mertler Chapter 8 Objective Test Items.

The first test of validity

Measurement Concepts and scale evaluation

Chapter 8 VALIDITY AND RELIABILITY

Presentation transcript:

RELIABILITY IN TESTING Büşra Nur Durmaz

CONTENTS Definition The Reliability Coefficient The Standard Error of Measurement and The True Score Scorer Reliability How to Make Tests more Reliable Reliability and Validity References

WHAT IS RELIABILITY? Consistency

Reliable or Not???

The Reliability Coefficient What is reliability coefficient?

The Reliability Coefficient What does the reliability coeffiecient do? compare the reliability of different tests The ideal reliability coefficient is……. (have no negative values) What is the relaibility coefficient for a good vocabulary, grammar and reading tests? What about listening and speaking? How do we come up with the value of the reliability coefficient? Test-retest method Alternate (equivalent) forms method Internal consistency: split-half method

The Standard Error of Measurement & The Trues Score How close the scores of the tests that we are comparing??? ( The true score) How can we arrive at this values? Standard Error of measurement (SEm): ‘’It’s how much measured test scores are spread around a “true” score.’’ It’s calculated based on the realiability coefficient. The higher the SEm the lower the reliability.

Scorer Reliability So far we considered the reliability without thinking about the scorer reliability. The perfect reliability of objective tests cannot be obtained in the subjective ones. For compositions the reliability is over 0.9

How to make tests more reliable Include more items. (Spearman- Brown Formula) They should be independent of each other. Not too long... Unambiguous items Clear & explicit instructions Well typed, legible tests Uniform and non-distracting conditions of administration Items should be as objective as possible A detailed scoring key Train scorers (esp. the scoring is subjective) Agree acceptable responses and scores Number the candidates Multiple, indepedent scoring (at least two scorers)

Reliability and Validity

REFERENCES Brown, J. D. (2005). Testing in Language Programs: A Comprehensive Guide to English Language Assessement. McGraw-Hill College. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (1993). How to design and evaluate research in education (Vol. 7). New York: McGraw-Hill. Hughes, A. (2002). Reliability. In Testing for Language Teachers (Cambridge Language Teaching Library, pp. 36-52). Cambridge: Cambridge University Press. doi:10.1017/CBO9780511732980.006 (On 13rd of March, 2018) Retrieved from: http://www.statisticshowto.com/standard-error-of- measurement/

THANK YOU!