Education Research: Quantitative, Qualitative and Mixed Approaches Johnson & Christensen SECOND EDITION Chapter 5 Standardized Measurement and Assessment Chapter Notes by Mellissa Harley

Chapter 5 Learning Objectives Explain the meaning of measurement Explain the different scales of measurement Articulate the 12 assumptions underlying testing and assessment Explain meaning of reliability Explain the differences among each method for computing reliability Explain the meaning of validity and validity evidence Identify the different types of standardized tests and sources of info on these tests Locate tests used to measure the constructs of interest to you

Definition of Measurement Assigning symbols or numbers according to a specific set of rules.

SCALES OF MEASUREMENT Nominal Ordinal Interval Ratio

NOMINAL SCALE CHARACTERISTICS - categorize, label, classify, name, or identify people or objects CATAGORICAL VARIABLES EXAMPLES - gender, race, political party, state, college, major, teaching method, counseling method, personality type

ORDINAL SCALE CHARACTERISTICS - Rank order objects or individuals from first to last or best to worst EXAMPLES - used frequently in admission selections for colleges DETERMINE RANKING, but not determine how much higher or lower one person is than another

INTERVAL SCALE CHARACTERISTICS - Rank order objects or individuals, equal intervals or distances between adjacent numbers EXAMPLES - Celsius and Fahrenheit temperature scale Absence of absolute zero point = no ratio statements

RATIO SCALE CHARACTERISTICS - Rank order objects or individuals with equal intervals between adjacent numbers Absolute zero point that permits forming ratio statements Ability to categorize, label, classify Highest level of quantitative measurement, rarely used in educational research Zero Point refers to absence of characteristic being measured

The 12 Assumptions of Assessment Psych traits and states exist. Psych traits and states can be quantified and measured. Various approaches can be useful. Can provide answers to some important questions of LIFE Can pinpoint phenomena that require further attention

The 12 Assumptions (continued) 6. Various sources of data enrich process 7. Various sources of error are part of process 8. Tests and other measures have strengths and weaknesses 9. Test-related behavior predicts non-test-related behavior 10. Present-day sampling predicts future behavior.

The 12 Assumptions (The Conclusion) 11. Testing and assessment can be fair and unbiased. And… 12. Testing and assessment benefit society!

Reliability and Validity RELIABILITY - The consistency or stability of the test scores. VALIDITY - The accuracy of the inferences or interpretations made from test scores.

RELIABILITY Several methods to calculate reliability: RELIABILITY COEFFICIENT TEST-RETEST RELIABILITY EQUIVALENT FORMS RELIABILITY INTERNAL CONSISTENCY SPLIT HALF RELIABILITY INTERSCORER RELIABILITY

VALIDITY VALIDITY EVIDENCE - empirical evidence and theoretical rationales that support the inferences or interpretations made from test scores. VALIDATION - Process of gathering evidence that supports inferences or interpretations made on the basis of test scores.

Methods of Collecting Validity Evidence Evidence based on content Experts measure, examine test content, construct Evidence based on internal structure Factor analysis - measure constructs Homogeneity of test items Evidence based on relations to other variables Relate test scores to a known criterion Correlate score to other measures with same construct and measures and different constructs Determine whether groups differ on test in expected ways

EDUCATIONAL AND PSYCHOLOGICAL TESTS Intelligence Tests Personality Tests Educational Tests Preschool Assessments Achievement Tests Aptitude Tests Diagnostic Tests

INTELLIGENCE TESTS Many types of intelligence Intelligence - The ability to think abstractly and to learn readily from experience. Not universally accepted definition Single prototype does not exist

PERSONALITY TESTS Personality - Relatively permanent patterns that characterize, classify a person Self-report Test - participant rate the degree to which characteristics are descriptive of themselves Performance Measures - participants perform task that is observed by researcher Projective Measures - participants respond to ambiguous stimuli (ex: ink blots)