Ecological Validity in Assessment Ecologically valid assessments are sensitive to – chronological and developmental ages – medical status – physical and.

Slides:



Advertisements
Similar presentations
Internal Assessment Your overall IB mark (the one sent to universities after the IB test) in any IB science course is based upon two kinds of assessments.
Advertisements

Issues of Technical Adequacy in Measuring Student Growth for Educator Effectiveness Stanley Rabinowitz, Ph.D. Director, Assessment & Standards Development.
1 Young Deaf Children and The Prediction of Reading and Mathematics
Chapter 13: Descriptive and Exploratory Research
Symbols Messaging Encoding. Symbol is “something that stands for or represents something else” (Vanderheiden & Yoder, 1986) “something else” = Referent.
Chapter 9 Flashcards. measurement method that uses uniform procedures to collect, score, interpret, and report numerical results; usually has norms and.
1.
READING ON A TABLET WILL INCREASE COMPREHENSION AMONGST STRUGGLING 2 ND GRADE READERS Nancy Ahmed & Catherine Arias CBSE 7201T Fall 2013.
Chapter 9 Instructional Assessment © Taylor & Francis 2015.
Descriptive Research. D Used to obtain information concerning the current status of a phenomena. D Purpose of these methods is to describe “what exists”
 reading nonsense words (e.g., plurp, fronkett) aloud to test phonetic word attack skills.  Word Attack measures skill in applying phonic and structural.
Technical Issues Two concerns Validity Reliability
Quantitative and Qualitative Research. Theory vs. Hypothesis From last class… Good research is informed by theory, or “a unified explanation for discrete.
Types of Research 1. Categorized by Practicality a. Basic research  done to satisfy a need to know with no intention of resolving an immediate social.
PSYCHOMETRICIANS: develop tests -try to make constructs measurable and quantifiable -purpose is to differentiate between test- takers 3 Qualities of Tests:
Instrumentation.
KEDC Special Education Regional Training Sheila Anderson, Psy.S
The World of Assessment Consider the options! Scores based on developmental levels of academic achievement Age-Equivalent scores.
Education 793 Class Notes Welcome! 3 September 2003.
Ottawa Area Intermediate School District Literacy Framework Unveiling.
Points made in discussions What interests and values in the landscape do we want to monitor? Integrated landscape approach takes account of different landscape.
WELNS 670: Wellness Research Design Chapter 5: Planning Your Research Design.
Nativity Catholic School IOWA Testing. 70 years of educational research and test development Form E Big Picture student’s classroom work homework projects.
Measuring School Readiness : The Early Development Instrument Washington, DC, 17 February 2005 Magdalena Janus, Ph.D.
Copyright ©2008 by Pearson Education, Inc. Pearson Prentice Hall Upper Saddle River, NJ Foundations of Nursing Research, 5e By Rose Marie Nieswiadomy.
Myers’ PSYCHOLOGY (7th Ed) Chapter 11 Intelligence James A. McCubbin, PhD Clemson University Worth Publishers.
Diagnostics Mathematics Assessments: Main Ideas  Now typically assess the knowledge and skill on the subsets of the 10 standards specified by the National.
Assessing Intelligence
Educational Psychology, 7 th edition Jeanne E. Ormrod © 2011 Pearson Education, Inc. All rights reserved. 1-1 Understanding research.
Reliability & Validity
CDIS 5400 Dr Brenda Louw 2010 Validity Issues in Research Design.
Overview Criteria Item Examples.  Create a sense of “measured urgency” ◦ Timeline is not immediate, but close ◦ Urgency in you and your teachers ◦ Difference.
Identify a Health Problem Qualitative Quantitative Develop Program -theory -objectives -format -content Determine Evaluation -design -sampling -measures.
ETHICS AND LAW FOR SCHOOL PSYCHOLOGISTS Chapters 6 and 8.
HOW DO WE USE DIBELS WITH AN OUTCOMES-DRIVEN MODEL? Identify the Need for Support Validate the Need for Support Plan Support Evaluate Effectiveness of.
Author(s)Donald D. Hammill & Stephen C. Larsen,(1996) defined Written Language as : “The term written language refers to the comprehension and expression.
Comprehensive Evaluations. Overview OBJECTIVES: Review Comprehensive Evaluation Process Provide Information On Selected Topics  Specific Learning Disability.
Visual-Graphic Symbol Acquisition by Pre-School Age Children with Developmental and Language Delays Andrea E. Barton 1, Mary Ann Romski, Ph. D. 2, Rose.
Creating Equity Dashboards to Monitor Racial, Ethnic and Linguistic Disparities in Health Care: Lessons from the Disparities Leadership Program DiversityRx.
Science Process Skills Vocabulary 8/17/15. Predicting Forming an idea of an expected result. Based on inferences.
Reading on a Tablet Will Increase Comprehension Amongst Struggling 2 nd Grade Readers Nancy Ahmed & Catherine Arias CBSE 7201T Fall 2013.
Assessment: Reliability & Validity. Reliability Refers to the repeatability of a given testing instrument The extent to which a student would be expected.
Criteria for selection of a data collection instrument. 1.Practicality of the instrument: -Concerns its cost and appropriateness for the study population.
GENERALIZING RESULTS: the role of external validity.
Chapter 6 - Standardized Measurement and Assessment
VALIDITY, RELIABILITY & PRACTICALITY Prof. Rosynella Cardozo Prof. Jonathan Magdalena.
Research Problem Students with poor quantitative reasoning skills are less successful in STEM fields.
Chapter 7: Assessment Identifying Strengths and Needs “Assessment is the process of gathering data for the purpose of making decisions about individuals.
Balancing on Three Legs: The Tension Between Aligning to Standards, Predicting High-Stakes Outcomes, and Being Sensitive to Growth Julie Alonzo, Joe Nese,
Career Counseling: A Holistic Approach
GRADING n n Grading that reflects the actual ability of the student n n Grading as a personal communication.
Educational Psychology Jeanne Ormrod Eighth Edition © 2014, 2011, 2008, 2006, 2003 Pearson Education, Inc. All rights reserved. Developing Learners.
ASSESSMENT METHODS – Chapter 10 –.
Does training in number knowledge improve arithmetic scores?
Chapter 1 INTRODUCTION.
Associated with quantitative studies
Copyright © American Speech-Language-Hearing Association
Myers’ PSYCHOLOGY Intelligence Worth Publishers.
Copyright © American Speech-Language-Hearing Association
Myers’ PSYCHOLOGY Unit 11 Intelligence Chris Dunn Spalding High School.
Research Methods: Concepts and Connections First Edition
Teaching and Educational Psychology
Research Methods: Concepts and Connections First Edition
Myers EXPLORING PSYCHOLOGY (6th Edition in Modules)
Goal of Psychology: to determine the Underlying “laws” of behavior.
Goal of Psychology: to determine the Underlying “laws” of behavior.
Myers’ PSYCHOLOGY (7th Ed)
Methodology Week 5.
Measurement Concepts and scale evaluation
Presentation transcript:

Ecological Validity in Assessment Ecologically valid assessments are sensitive to – chronological and developmental ages – medical status – physical and sensory abilities – education – occupation – cognitive level – and cultural, socioeconomic, and linguistic backgrounds

Criteria for Evaluating Ecological Validity (Franzen & Wilhelm, 1998) Objectification – To what extent can the results of the assessment be qualitatively or quantitatively specified? Generalizability – Given the assessment results, is one likely to make the same statements (predictions) about real-life behavior? Sufficiency – Will the information from the assessment allow us to draw reasonable conclusions about real-life phenomena or require additional information in the form of external variables?

Beyond Aesthetics Digital photography = Highest degree of realism possible More representative of how an individual would learn and rehearse vocabulary in the real world May offer a truer test of the breadth of an individual’s vocabulary knowledge than a test that uses line drawings

Iconicity & Augmentive Alternative Communication Fuller and Lloyd (1991) continuum of understanding the connections between symbols and their referents. – Transparent – Translucent – Opaque Color digital photographs have transparent to translucent iconicity. Color line drawings have more translucent and sometimes opaque iconicity.

That was then.This is now.

Digital or paper stimuli – Digital for use on your tablet or paper for traditional assessment Greater ecological validity – Full-color digital photographs provide the highest degree of realism possible Enhanced skill assessment – Composite and discrepancy scores enhance ability to assess differences between expressive and receptive skills

Assessment throughout the lifespan – Suitable for evaluating ages 2.5 to 95 years, offers both age- and grade-based norms Monitor effectiveness of interventions – Co-normed with equivalent parallel forms; provides change scores for measuring growth over time and in response to targeted interventions

Normative Sample Collected Summer of 2010 to Summer of 2012 Post-stratification weighting based on 2009 Census  N = 2678 individuals from 30 states 28 age groups – 2.5 to 95 years 14 grades – pre-K to 12 th grade – Split into fall (n = 835) and spring (n = 816)

Correlation between VAS and Other Picture Vocabulary Tests ScaleVAS-EVAS-R Vocabulary Composite PPVT-4.73**.68**.75** EVT-2.78**.66**.77** ROWPVT.83**.70**.82** EOWPVT.70**.67**.73** Note. N = 50. Correlation shown between standard scores. ** p <.01