Psychological Testing and Assessment Chapter 1 Psychological Testing and Assessment

Twelve Assumptions in Psychological Testing and Assessment Assumption 1: Psychological traits and states exist Observed (manifest) vs. latent Can you study something invisible? Logical positivism, Behaviorism Assumption 2: Psychological traits and states can be quantified and measured Can you really assign a numeric value to everything? Rasch: objective measurement Assumption 3: Various approaches to measuring aspects of the same thing can be useful (Methodology Triangulation)

Twelve Assumptions in Psychological Testing and Assessment Assumption 4: Assessment can provide answers to some of life’s most momentous questions Tracking (in the UK system) screening and placement Assumption 5: Assessment can pinpoint phenomena that require further attention or study National (?) assessment: No Child Left behind (The objective of assessment is…) International assessment: PISA, TIMSS

What is PISA? Not these PISA: Program for International Student Assessment is a series of assessments on reading literacy, mathematics literacy, and science literacy, sponsored by Organization for Economic Cooperation and Development (OECD), and administered internationally to 15-year-olds TIMSS: The Trends in International Mathematics and Science Study (TIMSS) is administered every four years internationally to both fourth and eighth grade students.

Source: OECD/PISA and Wikipedia

Source: TIMSS and Wikipedia

Obama's "Sputnik Moment" In 1957 the USSR launched the first artificial Earth satellite, Sputnik. In order to fill the technological gap, the US government awarded financial support to promote math and science education. In response to the performance gap between the US students and Asian/European students, President Obama introduced the phrase "Sputnik moment" to argue for more investment in science, technology, engineering, and technology (STEM) education.

Twelve Assumptions in Psychological Testing and Assessment (Cont’d) Assumption 6: Various sources of data enrich and are part of the assessment process (Data triangulation) Assumption 7: Various sources of error are part of the assessment process

Error is everywhere! Sources of error Types of error Measurement error Sampling error Types of error Bias: Systematic e.g. self-report height and weight Variance: Random

Twelve Assumptions in Psychological Testing and Assessment (Cont’d) Assumption 8: Tests and other measurement techniques have strengths and weaknesses Classical test theory Item response theory Rasch model

Twelve Assumptions Assumption 9: Test-related behavior predicts non–test-related behavior Assumption 10: Present-day behavior sampling predicts future behavior Academically adrift: Limited learning on college campuses -- After 4 years, 36% college graduates showed no gains in higher order thinking skills. US corporations complain that college graduates don’t have the critical thinking skill required by the jobs. Corporations launch their own assessment programs e.g. MCSE, CCNA.

Examples of industry certification/assessment program

Twelve Assumptions in Psychological Testing and Assessment (Cont’d) Assumption 11: Testing and assessment can be conducted in a fair and unbiased manner Assumption 12: Testing and assessment benefit society

Who, What and Why? Who Are the Parties? The test developer The test user The test taker Society at large

The test developer Subject matter expert/content expert: item authoring Psychometrician: item analysis, factor analysis Database programmer/IT personnel: item banking Program manager: put all together

Evaluating the Quality of Tests What is a “Good” Test? Reliability Validity Other settings Reference Sources for Test Information Test catalogues Test manuals Journal articles Online and CD-Rom databases Other sources

http://www.creative-wisdom.com/pub/efficacy.pdf

Exercise (2 points) Form a group consisting of 3-4 people. One or two must have a Web-enabled laptop. Access the APU library Look for Mental Measurements Yearbook (the electronic version) Search for instruments that measure psychological constructs (e.g. anxiety, depression, happiness…etc) Read the comments at the end Discuss which one is the best.