Measuring Achievement and Aptitude: Applications for Counseling Session 7

Definitions  Achievement tests –Provides information about what an individual has learned or acquired.  Provide information to help individuals understand their academic strengths and limitations.  Aptitude test –Predict future performance or ability to learn  Often come to counseling because they are trying to make a decision about their future.

Six Areas of Assessment Using Achievement  Survey achievement batteries  Individual achievement tests –Typically cover the areas of reading, math, and spelling  Diagnostic tests –Diagnosis learning disabilities –Assess achievement strengths and limitations

Six Areas--2  Criterion-referenced tests –Measure knowledge or comprehension to determine if certain criterion or standard has been met  Minimum-level skills –Measure skills for promotion, entrance, or graduation –Level is established before test is administered  Subject area tests –Example: test that covered knowledge of assessment strategies in counseling

Achievement Battery TerraNova  TerraNova –Administered to children K – 12. –Combination of norm and criterion referenced –Schools can select from basic battery –Schools can select format –Spanish version –Construction involved Item Response Theory  Information Provided –Norm-referenced –Criterion-referenced –Objective master –Performance level  Provides a profile  Scores provided –National percentiles –National percentile ranges –Stanines –Grade equivalents  Psychometric properties –Reliability r= mid.80 to mid.90 –Recently published not much on construct validity yet –Content validity  Item analysis

Discussion Would you use this test? Provide rationale? What do others think ?

Aptitude Assessments  Hood & Johnson (1997) argued that counselors in a variety of fields need to be knowledgeable about the predominant scholastic aptitudes tests.  If you do not have a working knowledge of these instruments may not be viewed as credible.

SAT  Scholastic Assessment Test –First adminstered in 1926 as the Scholastic Aptitude Tests –1994 was revised and renamed SAT –2004 added an essay—writing sample  Consists of two tests –SAT I: Reasoning test  3 hour test that measures verbal and mathematical reasoning abilities –SAT II: Subject tests  Consists of single subject tests –Writing –Math level 1 –Biology –French

Interpretation  Score ranges between 200 and 800 on both the Verbal and Mathematical sections of SAT I –Mean is 500 –SD is 100  ETS (Education Testing Services) uses complex formula to equate scores on different versions.

Psychometric Properties of Aptitude Tests  These tests are used to make decisions that often have a significant influence on people’s lives and, therefore, the validity of these instruments deserves analysis.  GRE –Combined scores have validity coefficients of –Undergraduate grade point averages have r = –Adding the GRE and Grade point results in r =

Discussion   Would you use this test?   Provide rationale?   What do others think?

Work Samples Assessment  Another way of assessing vocational/career aptitudes  Philosophical base –Work performance can best be assessed by using a sample of the actual work the individual would perform

Work Sample Assessments  Valpar Component Work Sample (VCWS) –23 individual work samples on computer –Criterion-referenced scoring –Speed test (completed in certain amount of time_ –Norms on nondisabled and disabled workers –Comes with built in computer scoring system  SAGE system –Non computer and computer versions

Test Preparation and Performance  Do workshops really increase scores? –Mixed review –Depends on individuals test taking sophistication—can they learn to learn to meet exams requirement or test format (logical problem solving) –Individual who have experience in taking standardized test have a distinct advantage (Anastasi, 1981) –Manuals and – sample tests have been constructed to level the playing field and provide persons with such experiences.  Coaching –Coaching programs do make a significant positive difference in scores (contested but not yet disproven) –Less expensive ways (travel experiences, tutoring, trips to museums –The closer the coaching material to the actual test content the greater the improvement in test scores –The more time individuals spend reviewing the material, the more likely it is that they will cover the material on the test.

Exam Results Mean = s.d. = Mode = {106,5; 108; 150} Median = Z= data point – mean standard deviation standard deviation

Raw and Z-scores Mean = s.d. = 17.08