Platzer, H.: "A criterion-referenced approach to the VLT."

Slides:

Advertisements

Similar presentations

L2 program design Content, structure, evaluation.

Advertisements

Survey Methodology Reliability and Validity EPID 626 Lecture 12.

1 COMM 301: Empirical Research in Communication Kwan M Lee Lect4_1.

Types of Reliability.

Reliability Definition: The stability or consistency of a test. Assumption: True score = obtained score +/- error Domain Sampling Model Item Domain Test.

© McGraw-Hill Higher Education. All rights reserved. Chapter 3 Reliability and Objectivity.

Reliability Analysis. Overview of Reliability What is Reliability? Ways to Measure Reliability Interpreting Test-Retest and Parallel Forms Measuring and.

Reliability & Validity.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If.

Part II Sigma Freud & Descriptive Statistics

MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.

General Information --- What is the purpose of the test? For what population is the designed? Is this population relevant to the people who will take your.

Methods for Estimating Reliability

CORRELATIONAL ANALYSES EDRS 5305 EDUCATIONAL RESEARCH & STATISTICS.

Reliability n Consistent n Dependable n Replicable n Stable.

Reliability Analysis. Overview of Reliability What is Reliability? Ways to Measure Reliability Interpreting Test-Retest and Parallel Forms Measuring and.

Psychometric Properties of the Job Search Self-Efficacy Scale Investigators: Jeff Christianson Cody Foster Jon Ingram Dan Neighbors Faculty Mentor: Dr.

Session 3 Normal Distribution Scores Reliability.

Research Methods in MIS

Chapter 7 Correlational Research Gay, Mills, and Airasian

Technical Issues Two concerns Validity Reliability

22nd International Conference STAR, July Palma de Mallorca, Spain The Perceived Stress Scale. Preliminary Psychometric Study with Spanish HIV+

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.

Unanswered Questions in Typical Literature Review 1. Thoroughness – How thorough was the literature search? – Did it include a computer search and a hand.

The Czech AUDIT: Internal Consistency and Latent Structure Ladislav Csémy and Hana Sovinová National Institute of Public Health, Czech Republic Inebria.

CRT Dependability Consistency for criterion- referenced decisions.

Presenter : Ching-ting Lin Instructor: Ming-puu Chen Developing a Usability Evaluation Method for E-learning Application: From Functional Usability to.

Instrumentation (cont.) February 28 Note: Measurement Plan Due Next Week.

Validity and Reliability THESIS. Validity u Construct Validity u Content Validity u Criterion-related Validity u Face Validity.

The API (Agent Persona Instrument) for assessing pedagogical agent persona Presenter: Wan-Ning Chen Professor: Ming-Puu Chen Date: May 18, 2009 Baylor,

The Correlational Research Strategy

Chapter 8 Validity and Reliability. Validity How well can you defend the measure? –Face V –Content V –Criterion-related V –Construct V.

1 Measurement and Data Collection  What and How?  Types of Scales Nominal Nominal Ordinal Ordinal Interval Interval Ratio Ratio.

Designs and Reliability Assessing Student Learning Section 4.2.

Review and Writers’ Workshop Class 7 Meetings at 11:30am Cathrine, Jeff, Alisha.

Review of Factorial ANOVA, correlations and reliability tests COMM Fall, 2007 Nan Yu.

The Correlational Research Strategy Chapter 12. Correlational Research The goal of correlational research is to describe the relationship between variables.

Reliability n Consistent n Dependable n Replicable n Stable.

Severity Indices for Personality Problems (SIPP): Factor structure, reliability, validity, and sensitivity to change Helene Andrea (PhD) Roel Verheul (PhD)

Measurement Experiment - effect of IV on DV. Independent Variable (2 or more levels) MANIPULATED a) situational - features in the environment b) task.

Classroom Assessment Chapters 4 and 5 ELED 4050 Summer 2007.

Design of online survey system with an advanced IPA discrimination index for customer satisfaction assessment Presenter: Mai, Yi-Ting ( 麥毅廷, 臺體大運管系 ) Paper.

Quality instrument* Questions are determined by objectives Resist the temptation to ask questions that are interesting but not relevant to your hypothesis.

1 Measuring Agreement. 2 Introduction Different types of agreement Diagnosis by different methods  Do both methods give the same results? Disease absent.

Survey Methodology Reliability and Validity

Comparing two vocabulary tests and TOEIC

THE SHORT VERSIONS OF FLOW SCALES: RELIABILITY AND VALIDITY STUDY

Lecture 5 Validity and Reliability

Chapter 7 Cooper and Schindler

Product Reliability Measuring

Reliability and Validity

Chapter 10 CORRELATION.

Assessment Theory and Models Part II

assessing scale reliability

Comparing TOEIC® and vocabulary test scores

Reliability Module 6 Activity 5.

Week 3 Class Discussion.

الاختبارات محكية المرجع بناء وتحليل (دراسة مقارنة )

AREA OF STUDY 2: INTELLIGENCE & PERSONALITY

First study published in JOGS.

תוקף ומהימנות של ה- Dementia Quality of Life בארה"בMeasure

Competency 007: E.

Leland Jacobs Seth Krivohlavek Shirley J. Mills

The first test of validity

How can one measure intelligence?

15.1 The Role of Statistics in the Research Process

The Czech AUDIT: Internal Consistency and Latent Structure

*** Series 3RS Version 1, ***

First Hour - How can one measure intelligence?

REVIEW Course Review.

Presentation transcript:

Platzer, H.: "A criterion-referenced approach to the VLT." Validated test instrument: Vocabulary levels test (VLT) Suite of five 30-item MC vocabulary tests Word levels: 2k, 3k, 5k, 10k, AWL Versions A, B Traditional versions published in: Nation (2001); Schmitt, Schmitt & Clapham (2001) Sample n=320 New, updated versions: VLT inflation? McLean & Kramer (2015) Peters, Velghe & van Rompaey (2015) Web, Sasao & Ballance (2017)

1. NR vs CR consistency of VLT a. Norm-referenced reliability estimates of VLT Cronbach's alpha mostly >0.9 b. Criterion-referenced consistency estimates of VLT (Version B) Word levels Phi lambda Kappa squared 2000 WL 0.83 0.85 3000 WL 0.80 0.88 AWL

2. Relationship of VLT scores and CEFR bands B2, C1/2  3000 WL AWL 2000 WL  A1/2, B1

Correlated test scores 3. Investigating potential gender bias H1: rS male (QPT, 2000 WL) = rS female (QPT, 2000 WL) H2: rS male (QPT, 3000 WL) = rS female (QPT, 3000 WL) H3: rS male (QPT, AWL) = rS female (QPT, AWL) Correlated test scores Male (n = 106) Female (n = 190) ZDifference p rS zr H1: QPT, 2000 WL 0.417 0.444 0.437 0.469 -0.199 0.841 H2: QPT, 3000 WL 0.590 0.678 0.551 0.620 0.471 0.638 H3: QPT, AWL 0.477 0.519 0.512 0.565 -0.378 0.704