Adapted from Educational Research: Competencies for Analysis and Applications (Gay, L.R., Mills, G. E., Airasian, P., 2006) M.Stewart M.Ed. 2011.

Slides:



Advertisements
Similar presentations
Assessment & Evaluation adapted from a presentation by Som Mony
Advertisements

Context-Based Learning in Physics. “New” processes for students Note: These skills may be new to Physics classes but they are not necessarily new to students.
L2 program design Content, structure, evaluation.
Testing What You Teach: Eliminating the “Will this be on the final
Gear-up: Content Literacy Jackson Independent/Breathitt Co Schools October 29, 2012
1 Sources:  SusanTurner - Napier University  C. Robson, Real World Research, Blackwell, 1993  Steve Collesano: Director, Corporate Research and Development.
Norm-referenced assessment Criterion-referenced assessment Domain-referenced assessment Diagnostic assessment Formative assessment Summative assessment.
Instructional Design Dr. Lam TECM 5180.
Assessing and Evaluating Learning
Understanding Validity for Teachers
Questions to check whether or not the test is well designed: 1. How do you know if a test is effective? 2. Can it be given within appropriate administrative.
Validity and Reliability Neither Valid nor Reliable Reliable but not Valid Valid & Reliable Fairly Valid but not very Reliable Think in terms of ‘the purpose.
What should be the basis of
performance INDICATORs performance APPRAISAL RUBRIC
Classroom Assessment A Practical Guide for Educators by Craig A. Mertler Chapter 9 Subjective Test Items.
Classroom Test and Assessment
Technical Issues Two concerns Validity Reliability
WHAT IS MAP? AMES Lunch and Learn.
Assessment Cadre #3: “Assess How? Designing Assessments to Do What You Want”
Objectives are statements of what we want students to learn as a result of the instruction we provide them. Objectives answer the question “What do we.
Language Assessment 4 Listening Comprehension Testing Language Assessment Lecture 4 Listening Comprehension Testing Instructor Tung-hsien He, Ph.D. 何東憲老師.
1 DEVELOPING ASSESSMENT TOOLS FOR ESL Liz Davidson & Nadia Casarotto CMM General Studies and Further Education.
ETEC 550 Final Project By: Taha Anjarwalla. Index Problem Context Statement of Problem Needs Assessment/Analysis Instructional Intervention Lessons Learned.
Welcome to Unit 6 Seminar: Learning The Language Learning and Assessment Strategies 1.
ENGLISH PRIMARY BENCHMARK COMPONENTS AND WEIGHTINGS SPEAKING – carrying 20% of the global mark (prepared by the Benchmark board and administered.
Authentic Assessment Principles & Methods
Oscar Vergara Chihlee Institute of Technology July 28, 2014.
TEACHING WRITING WITH “MORTAR” Ellen Levy Santa Cruz County Office of Education Wednesday, March 16th.
Higher-Level Cognitive Processes
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
Completion, Short-Answer, and True-False Items
Classroom Assessments Checklists, Rating Scales, and Rubrics
EDU 385 Education Assessment in the Classroom
CONSTRUCTING OBJECTIVE TEST ITEMS: MULTIPLE-CHOICE FORMS CONSTRUCTING OBJECTIVE TEST ITEMS: MULTIPLE-CHOICE FORMS CHAPTER 8 AMY L. BLACKWELL JUNE 19, 2007.
Sheltered Instruction Part III of III Presented by Office of English Language Learners
Chapter 10 Technology in Language Arts and Foreign Language Instruction Presented by: Melissa Dzubinski, Elizabeth Climer & Brandie Wolfe.
Chap. 2 Principles of Language Assessment
Stages 1 and 2 Wednesday, August 4th, Stage 1: Step 5 National and State Standards.
Assessments Matching Assessments to Standards. Agenda ● Welcome ● What do you think of assessment? ● Overview of all types of evidence ● Performance Tasks.
Week 5 Lecture 4. Lecture’s objectives  Understand the principles of language assessment.  Use language assessment principles to evaluate existing tests.
EDU 385 CLASSROOM ASSESSMENT Week 1 Introduction and Syllabus.
Introduction to Ethics Lecture 15 Writing Philosophy Papers By David Kelsey.
Selecting a Sample. Sampling Select participants for study Select participants for study Must represent a larger group Must represent a larger group Picked.
Validity and Reliability Neither Valid nor Reliable Reliable but not Valid Valid & Reliable Fairly Valid but not very Reliable Think in terms of ‘the purpose.
 Are more likely to use appropriate strategies when taking tests; and are more "test-wise" than their peers.  Have positive self-esteem  Have greater.
Common Formative Assessments for Science Monica Burgio Daigler, Erie 1 BOCES.
4-Day Agenda and Expectations Day 2 Building Formative Assessments linked to deconstructed content and skills Building Formative Assessments linked to.
Assessment Specifications Gronlund, Chapter 4 Gronlund, Chapter 5.
Validity Validity is an overall evaluation that supports the intended interpretations, use, in consequences of the obtained scores. (McMillan 17)
Grand Island K-8 SCIENCE Common Formative Assessments for Science Monica Burgio Daigler, Erie 1 BOCES.
What Are the Characteristics of an Effective Portfolio? By Jay Barrett.
Nurhayati, M.Pd Indraprasta University Jakarta.  Validity : Does it measure what it is supposed to measure?  Reliability: How the representative is.
Classroom Assessment A Practical Guide for Educators by Craig A. Mertler Chapter 12 Assessing Group Work.
Reliability performance on language tests is also affected by factors other than communicative language ability. (1) test method facets They are systematic.
COURSE AND SYLLABUS DESIGN
Standards That Count: Reading, Discussion, Writing, and Presentation.
Pedagogy As it relates to the field of linguistics.
Evaluation and Assessment Evaluation is a broad term which involves the systematic way of gathering reliable and relevant information for the purpose.
Dr. Lois McKee- Assistant Principal Curriculum Mr. William Scales- Testing Coordinator The Florida Standards: What Every Parent Should Know Seminole High.
 Good for:  Knowledge level content  Evaluating student understanding of popular misconceptions  Concepts with two logical responses.
Copyright © Springer Publishing Company, LLC. All Rights Reserved. DEVELOPING AND USING TESTS – Chapter 11 –
3 Chapter Needs Assessment.
Validity and Reliability
Validity.
Learning About Language Assessment. Albany: Heinle & Heinle
What are SMART Targets? S – Specific: precise or exact
Annotated Reading in Content Areas
Study Questions To what extent do English language learners have opportunity to learn the subject content specified in state academic standards and.
Paper 1: Tues 6th June GCSE English Language Paper 2: Mon 12th June
TESOL Materials Design and Development
Presentation transcript:

Adapted from Educational Research: Competencies for Analysis and Applications (Gay, L.R., Mills, G. E., Airasian, P., 2006) M.Stewart M.Ed. 2011

  “Validity is the degree to which a test measures what it is supposed to measure and, consequently, permits the appropriate interpretation of results” (134).  Validity depends on the context in which the assessment is used. In other words, it is designed for a specific purpose and a specific group of people.  So, a test that is valid one quarter, might not be the next. Validity M.Stewart M.Ed. 2011

  “Content validity is the extent to which a test measures the intended content area” (134).  Item validity- individual test items are relevant to intended content. If it wasn’t explicitly taught, it shouldn’t be tested.  *Sampling validity- how well the test samples the total content area being tested.  In other words, the questions on the test should be proportionate to the amount of coverage the content received in the classroom.  *most common area for instructors to have issues Types of Validity M.Stewart M.Ed. 2011

  Unclear test directions  Confusing/ambiguous test items  Using vocabulary too difficult for test takers  Overly difficult and complex sent. structures.*  Inconsistent and subjective scoring methods  Untaught items included on achievement tests Factors that threaten validity: M.Stewart M.Ed. 2011

  A question should only be as complex as the subject matter demands.  Making questions unnecessarily complicated does not mean that you are testing critical thinking, nor does it mean that you are “challenging” them.  In fact, an unnecessarily complicated question actually reduces the validity of the test because you are no longer testing the subject matter, but rather the students’ ability to decipher the complex sentence structure or instructions.  Simple, straightforward subject matter, should be tested with a simple, straightforward question. Either they know it or they don’t. Factors that threaten validity: Overly difficult and complex sentence structures M.Stewart M.Ed. 2011

  For second language students and students with some types of learning disabilities, some question types can be more difficult to understand than others. This can interfere with validity.  Form test questions in a positive rather than negative structure, whenever possible, especially for true/false questions.  For example, for a true/false statement, “Hormones are part of the endocrine system.” is much easier to comprehend than, “Hormones are not a component of the Endocrine system.” Helpful Hints M.Stewart M.Ed. 2011

  Expose students to test-like questions before the exam.  Including a few review questions in your PowerPoint lectures can be a great way to do a comprehension check over current material and gives students the opportunity to learn what is expected when answering certain kinds of questions. Helpful Hints M.Stewart M.Ed. 2011

  Collect all questions of the same type into their own section. For example, all of the true/false questions are together, all of the multiple-choice together, etc.  Don’t assume that students will know what to do. Every section of questions needs its own set of clear directions, even it seems obvious.  Give each question a point value, so students can apply test-taking strategies. Helpful Hints M.Stewart M.Ed. 2011

  Often, one of the duties of the basic skills instructor is to review the exams to look for issues with validity. To facilitate this, there are the following expectations:  1) there will be adequate time to collaborate and comment on the exam (two weeks before study guide is to be given)  2) once comments have been given, a final draft incorporating all changes will be sent to BS instructor  3) NO changes will be made to exam after final draft has been agreed upon/study guide created. Collaborating on Exams M.Stewart M.Ed. 2011

  Gray, L.R., Mills, G. E., Airasian, P. (2006). Selecting measuring instruments. Educational Research: Competencies for analysis and applications ( 8 th ed. ) (pp ). Upper Saddle River, NJ: Pearson, Merrill, Prentice-Hall. Works Cited M.Stewart M.Ed. 2011