Psychometrics of EGRA Gambian, Senegalese, and Nicaraguan pilots.

Slides:

Advertisements

Similar presentations

MCR Michael C. Rodriguez Research Methodology Department of Educational Psychology.

Advertisements

Issues of Reliability, Validity and Item Analysis in Classroom Assessment by Professor Stafford A. Griffith Jamaica Teachers Association Education Conference.

The Research Consumer Evaluates Measurement Reliability and Validity

Reliability and Validity checks S-005. Checking on reliability of the data we collect  Compare over time (test-retest)  Item analysis  Internal consistency.

Psychometrics William P. Wattles, Ph.D. Francis Marion University.

VALIDITY AND RELIABILITY

Part II Sigma Freud & Descriptive Statistics

VALIDITY vs. RELIABILITY by: Ivan Prasetya. Because of measuring the social phenomena is not easy like measuring the physical symptom and because there.

What is a Good Test Validity: Does test measure what it is supposed to measure? Reliability: Are the results consistent? Objectivity: Can two or more.

Part II Sigma Freud & Descriptive Statistics

Reliability or Validity Reliability gets more attention: n n Easier to understand n n Easier to measure n n More formulas (like stats!) n n Base for validity.

Lesson Eight Standardized Test. Contents Components of a Standardized test Reasons for the Name “Standardized” Reasons for Using a Standardized Test Scaling.

Measurement: Reliability and Validity For a measure to be useful, it must be both reliable and valid Reliable = consistent in producing the same results.

Lesson Seven Reliability. Contents  Definition of reliability Definition of reliability  Indication of reliability: Reliability coefficient Reliability.

Psych 231: Research Methods in Psychology

SELECTION RELIABILITY VALIDITY ETHICAL STANDARDS TEST TYPE TEST CONTENT.

Lesson Thirteen Standardized Test. Yuan 2 Contents Components of a Standardized test Reasons for the Name “Standardized” Reasons for Using a Standardized.

Validity, Reliability, & Sampling

Research Methods in MIS

Using statistics in small-scale language education research Jean Turner © Taylor & Francis 2014.

Principles of language testing

Questions to check whether or not the test is well designed: 1. How do you know if a test is effective? 2. Can it be given within appropriate administrative.

Technical Issues Two concerns Validity Reliability

PhD Research Seminar Series: Reliability and Validity in Tests and Measures Dr. K. A. Korb University of Jos.

1 Development of Valid and Reliable Case Studies for Teaching, Diagnostic Reasoning, and Other Purposes Margaret Lunney, RN, PhD Professor College of.

Building Effective Assessments. Agenda  Brief overview of Assess2Know content development  Assessment building pre-planning  Cognitive factors  Building.

Reliability and Validity what is measured and how well.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.

Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 14 Measurement and Data Quality.

LECTURE 06B BEGINS HERE THIS IS WHERE MATERIAL FOR EXAM 3 BEGINS.

Technical Adequacy Session One Part Three.

Assessment in Education Patricia O’Sullivan Office of Educational Development UAMS.

Student assessment AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.

Week 5 Lecture 4. Lecture’s objectives  Understand the principles of language assessment.  Use language assessment principles to evaluate existing tests.

Reliability vs. Validity.  Reliability  the consistency of your measurement, or the degree to which an instrument measures the same way each time it.

Experiment Basics: Variables Psych 231: Research Methods in Psychology.

Validity and Reliability Neither Valid nor Reliable Reliable but not Valid Valid & Reliable Fairly Valid but not very Reliable Think in terms of ‘the purpose.

RELIABILITY AND VALIDITY OF ASSESSMENT

Designs and Reliability Assessing Student Learning Section 4.2.

Validity Validity is an overall evaluation that supports the intended interpretations, use, in consequences of the obtained scores. (McMillan 17)

Validity and Item Analysis Chapter 4.  Concerns what instrument measures and how well it does so  Not something instrument “has” or “does not have”

Assessment. Workshop Outline Testing and assessment Why assess? Types of tests Types of assessment Some assessment task types Backwash Qualities of a.

McGraw-Hill/Irwin © 2012 The McGraw-Hill Companies, Inc. All rights reserved. Obtaining Valid and Reliable Classroom Evidence Chapter 4:

Introduction to Item Analysis Objectives: To begin to understand how to identify items that should be improved or eliminated.

MEASUREMENT: PART 1. Overview  Background  Scales of Measurement  Reliability  Validity (next time)

Reliability and Validity Themes in Psychology. Reliability Reliability of measurement instrument: the extent to which it gives consistent measurements.

Classroom Assessment A Practical Guide for Educators by Craig A. Mertler Chapter 4 Overview of Assessment Techniques.

Measurement Experiment - effect of IV on DV. Independent Variable (2 or more levels) MANIPULATED a) situational - features in the environment b) task.

Chapter 6 - Standardized Measurement and Assessment

Reliability a measure is reliable if it gives the same information every time it is used. reliability is assessed by a number – typically a correlation.

Validity & Reliability. OBJECTIVES Define validity and reliability Understand the purpose for needing valid and reliable measures Know the most utilized.

RELIABILITY BY DONNA MARGARET. WHAT IS RELIABILITY?  Does this test consistently measure what it’s supposed to measure?  The more similar the scores,

Classroom Assessment Chapters 4 and 5 ELED 4050 Summer 2007.

Language Assessment Lecture 7 Validity & Reliability Instructor: Dr. Tung-hsien He

Lesson Thirteen Standardized Test. Contents Components of a Standardized test Reasons for the Name “Standardized” Reasons for Using a Standardized Test.

1 Measurement Error All systematic effects acting to bias recorded results: -- Unclear Questions -- Ambiguous Questions -- Unclear Instructions -- Socially-acceptable.

Principles of Language Assessment

Lecture 5 Validity and Reliability

Questions Tool and Constructed Response Items

CHAPTER 3: Practical Measurement Concepts

Reliability and Validity in Research

Concept of Test Validity

Tests and Measurements: Reliability

Reliability & Validity

Week 3 Class Discussion.

پرسشنامه کارگاه.

Lesson 1 Foundations of measurement in Psychology

The first test of validity

How can one measure intelligence?

Chapter 8 VALIDITY AND RELIABILITY

Presentation transcript:

Psychometrics of EGRA Gambian, Senegalese, and Nicaraguan pilots

Psychometrics Theory and technique of educational and psychological measurement Construction of instruments and procedures for measurement Development and refinement of theoretical approaches to measurement

EGRA “The EGRA instrument…is designed to be a sample-based, system diagnostic measure…..Its purpose is to document student performance on early grade reading skills in order to inform ministries and donors regarding system needs for improving instruction.”

Questions We Would Like Answered Do students find the test too difficult, too easy, or just right? Does the test measure consistently? Does the test measure what it is supposed to measure?

Do students find the test too difficult, too easy, or just right?

Difficulty The test is difficult…

Students Items

… but tends to get easier across grades.

Nicaragua (WCPM)

Does the test measure consistently?

Reliability The consistency of test scores over different test administrations (split-half) multiple raters (inter-rater) different test questions (internal) How likely is it that a student would obtain the same score?

Reliability Measures Split half (.7+) Cronbach (.7+) Reliability: Inter-rater Gambia ? Senegal ? Nicaragua

Does the test measure what it is supposed to measure?

Validity Appropriateness or correctness of inferences, or decisions made from test results Sources of evidence Content Construct Criterion Consequences

Validity Evidence ContentConstructCriterionConsequences Gambia ?? Senegal ?? Nicaragua ??

3 Dimensions (Gambia as Example) Words/vocabulary/ reading comprehension Letters/sounds/ listening comprehension Physical

EGRA: An Identity Crisis? “The EGRA instrument…is designed to be a sample-based system diagnostic measure…..Its purpose is to document student performance on early grade reading skills in order to inform ministries and donors regarding system needs for improving instruction.” “Simple method for diagnosis and assessment of early reading difficulties” “Cost-effective scan of the early grade reading situation for the country”

Whither EGRA? Tool for teachers or for policymakers? Test questions best suited to former (DIBELS) Implementation and analysis design best suited to latter (NAEP)