Tests and Measurements Educational Research

Educational Measurement u technique used to measure data

Classifications u Cognitive versus Noncognitive u Commercial versus Teacher-made u Self-reporting versus Observation

Cognitive Vs Noncognitive u Noncognitive u examines personality, attitudes, values u Cognitive u measures what a person knows or is able to do mentally

Commercial or Standardized u Disadvantages u costs of using/ scoring u may not match objectives or needs u norms may not reflect local pop u Advantages u comparative norms u already put together u results compared to others in norms u known reliabilities and validities

Teacher-made u disadvantages u time involvement u unknown reliabilities and validities u advantages u use when inappropriate to use standardized test u can more closely relate to objectives u inexpensive costs

Self-reporting Versus Observation u Self-reporting u best way to obtain lots of information quickly u NOT always honest u do NOT always know u Observation

Important Terms u Raw score u # of (in)correct responses u amount of time u Standard score u raw scores transformed to normalized score (z scores, t scores, GREs)

u Grade equivalents u norm-referenced grade level performance u not interpreted well; limited meaning u Percentile ranks u describes how well an individual did on a test compared to a particular group u not interpreted well, esp. at extremes

u Reliability u degree of consistency/dependability of a measuring instrument u increase # items on test u range from 0 to +1.0 u Validity u credibility of measuring instrument u does instrument measure what it claims to measure?

Referenced-based Measurement Interpretation u Norm-referenced u compare one to the many u performance reported in %iles, standard scores, etc. u items produce wide range of scores u norms refer to typical/avg performance of a group of people

u Criterion-referenced u what the student CAN do without reference to others u in reference to a prespecified standard of performance u concerned with mastery u items selected to match instructional objectives

Types of Tests u Standard questions of cognitive knowledge or skills u Achievement Tests u Intelligence Tests u Aptitude Tests u Performance Assessment

Achievement Tests u measures mastery in different areas of knowledge u most common form of measurement in educational research

Intelligence Tests u look at performance in a specific area u NOT measures of innate or pure intelligence u dependent on background and schooling (more measures of scholastic aptitude u used to predict school/academic success; IVs

u Important persons and tests: u Army Alpha and Army Beta u first group IQ tests u Stanford-Binet u first individual IQ test u David Wechsler u Wechsler intelligence scales u most popular in use today

Aptitude Tests u aimed at measuring a person’s future performance in a specific skill or area of achievement u very close to intelligence

Performance Assessment u authentic assessment/alternative assessment u approach to evaluating students by directly examining performance on tasks with intrinsic value

Measures of Personality u Assess individual differences in aspects of personality such as traits, needs, psychological disorders, values, and attitudes. u Often self-reporting.

Types u Inventories u Projective Techniques u associationcompletion u role playingcreative/construction u Attitude Scales/Opinionnaires u Likert scalesThurstone Scale u Semantic Differential Technique

Inventories u attempt to yield a measure of the types of activities an individual likes or has a tendency to choose.

Projective Techniques u persons project thoughts, feelings, attitudes, needs onto vague stimulus u Rorschach Ink Blot u Thematic Apperception Test u AssociationCompletion u Role PlayingCreative/Construction

Attitude Scales/Opinionnaires u measures degree to which person possesses characteristic of interest u do NOT reflect success/failure or strengths/weaknesses u Likert Scale u Thurstone Scale u Semantic Differential Technique

Likert Scale u R. Likert u collection of statements about a given topic u half are positively stated and half are negatively stated u arranged in random order u measured on a scale of u (SA A U D SD)

Semantic Differential Scale u Circle one of the #s between each pair of adjectives to best indicate how closely one of the adjectives describes your attitude toward essay questions. u Good Bad u Pleasant Unpleasant u Fair Unfair u Positive Negative

Rating Scales u widely used measuring instrument u involves assessment by one person of another’s behavior/performance u Four types

Graphic Scale u uses a continuum u place a checkmark at the area to describe the performance/behavior u numerical values may be assigned to descriptive points

Numerical Scale u involves assignment of numbers to points on graphic scales

Categorical Scale u categories arranged in ordered series u rater selects category of person being rated

Comparative Rating Scale u make judgments compared to others u sometimes school systems use this method for references

Rating Errors u Halo Effect u generalized impression of person that slants the perception u (good manners, bad boy) u Generosity Error u giving subjects benefit of the doubt

Rating Errors u Error of Severity u tendency to rate all persons too low on all characteristics u Error of Central Tendency u avoiding the extremes of the scale, rate persons in middle

How to Reduce Errors u Thorough training of raters u BEFORE u making ratings

Sociometric Techniques u SOCIOGRAM u used to study social relationships present among various group members u study of choices made by each person in a group

Procedure u each member of group indicates other members with whom s/he would MOST like to engage in activity such as recess, lunch, or work time u choices will vary, depending upon the activity u choices are plotted

Sociogram Legend u boys=triangles u girls=circles u arrows show direction of selection

Sociogram Interpretation u star(s) u most frequently chosen member(s) u clique(s) u small subgroups of persons (3 or more) who mutually select each other u isolate u member(s) who receives no choices

Examples u Students asked to indicate two other children with whom they would MOST like to play: u Maria--Juan, PatJuan--Maria, Pat u Pat--Maria, JuanSue--Tony, Marco u Tony--Sue, MarcoLaura--Sue, Marco u Jackson--Tony, MarcoMarco-Laura,Tony

Examples u Students in an RSCH 7100 class were asked to specify the names of two other students with whom they would most like to work with on the midterm exam. Of the six students asked, the following are their selections.

Choices of students w/whom to work, on midterm exam u Herbert ---Sam, Julie u John --- Fred, Julie u Sam -- Julie, Herbert u Fred -- Cynthia, Julie u Julie -- Sam, Herbert u Cynthia -- Fred, Julie

Number of times selected u HerbertTwice (Julie, Sam) u JohnNone (isolate) u SamTwice (Herbert, Julie) u FredTwice (John, Cynthia) u Julie5X (all persons--star) u CynthiaOnce (Fred) u Clique: Herbert, Julie, and Sam mutually selected only each other

Observations u Purpose: determine extent to which a particular behavior(s) is present u used to study classroom behaviors; natural or contrived settings; with infants, preschool, and elementary school children u in quantitative and qualitative research

Observations u Disadvantages u expensive to use in terms of time and resources u presence of observer may alter behavior u observer bias u Advantages u observe persons’ behaviors in natural surroundings u may use w/those who cannot communicate through language

Interviews u form of data collection in which questions asked orally and participant’s responses are recorded

WE ARE DONE