1. Intelligence and Psychological Testing
Chapter 9 :
Intelligence and Psychological Testing
Copyright © 2007 by Nelson, a division of Thomson Canada Limited 1

2. Key Concepts in Psychological Testing
Standardization
Test norms
Standardization group
Reliability
Correlation coefficient
Validity
Content validity
Criterion-related validity
Construct validity
Standardization refers to the uniform procedures used in the administration and scoring of a test.
Test norms provide information about where a score on a psychological test ranks in relation to other scores on that test…allows a psychologist to determine how a person scores relative to other people.
The standardization group is the sample of people that the norms are based on.
Reliability refers to a test’s consistency; that is, repeated measurements should yield reasonably similar results.
Reliability estimates are based on the correlation coefficient. Two sets of scores from two administrations of the same test are correlated; the closer the correlation comes to +1.00, the more reliable the test.
Validity refers to the ability of a test to measure what it was designed to measure.
Content validity is the degree to which the content of a test is representative of the domain it is supposed to cover (physics questions on a psychology test…poor content validity).
Criterion-related validity is estimated by correlating subjects’ scores on a test with their scores on an independent criterion…predictive ability.
Construct validity is the extent to which there is evidence that a test measures a particular hypothetical construct…are we really measuring intelligence with an IQ test? 2

3. Principal Types of Psychological Tests
Mental ability tests
Intelligence – general
Aptitude – specific
Personality scales
Measure motives, interests, values, and attitudes
Psychological tests are standardized measures of behaviour. Most psychological tests fall under two broad categories, mental ability tests and personality scales.
Mental ability tests include intelligence tests, which are designed to measure general mental ability, and aptitude tests, which measure more specific mental abilities.
Personality measures are usually called scales, rather than tests, as there are no right or wrong answers.
Personality tests measure a variety of motives, interests, values, and attitudes. 3

4. Fig 9. 2 – Correlation and reliability
Fig 9.2 – Correlation and reliability. As explained in Chapter 2, a positive correlation means that two variables covary in the same direction; a negative correlation means that two variables covary in the opposite direction. The closer the correlation coefficient gets to either –1.00 or +1.00, the stronger the relationship. At a minimum, reliability estimates for psychological tests must be moderately high positive correlations. Most reliability coefficients fall between 0.70 and 0.95. 4

5. Fig 9. 3 – Criterion-related validity
Fig 9.3 – Criterion-related validity. To evaluate the criterion-related validity of a pilot aptitude test, a psychologist would correlate subjects’ test scores with a criterion measure of their aptitude, such as ratings of their performance in a pilot training program. The validity of the test is supported if a substantial correlation is found between the two measures. If little or no relationship exists between the two sets of scores, the data do not provide support for the validity of the test. 5

6. Fig 9. 4 – Construct validity
Fig 9.4 – Construct validity. Psychologists evaluate a scale’s construct validity by studying how scores on the scale correlate with a variety of variables. For example, some of the evidence on the construct validity of the Expression Scale from the Psychological Screening Inventory is summarized here. This scale is supposed to measure the personality trait of extraversion. As you can see on the left side of this network of correlations, the scale correlates negatively with measures of social introversion, social discomfort, and neuroticism, just as one would expect if the scale is really tapping extraversion. On the right, you can see that the scale is correlated positively with measures of sociability and self-acceptance and another index of extraversion as one would anticipate. At the bottom, you can see that the scale does not correlate with several traits that should be unrelated to extraversion. Thus, the network of correlations depicted here supports the
idea that the Expression Scale measures extraversion. 6

7. The Evolution of Intelligence Testing
Sir Francis Galton (1869)
Hereditary Genius
Alfred Binet and Theodore Simon (1905)
Binet-Simon Intelligence Scale
Mental age
Lewis Terman (1916)
Stanford-Binet Intelligence Scale
Intelligence Quotient (IQ) = MA/CA x 100
David Wechsler (1955)
Wechsler Adult Intelligence Scale
Sir Francis Galton, 1869, published Hereditary Genius, in which he put forth the notion that success runs in families because intelligence is inherited. He developed a crude mental abilities test based on sensory acuity.
Alfred Binet and Theodore Simon published the first intelligence test in 1905, a test designed to single out youngsters in need of special training…expressed a child’s score in terms of mental age…for example, a 4 year old child with a mental age of 6 performed like the average 6 year-old on the test.
Lewis Terman, at Stanford University, revised Binet’s test for use in the U.S….the Stanford-Binet.
Terman used a new scoring scheme, the intelligence quotient, dividing a child’s mental age by chronological age and multiplying by 100…this made it possible to compare children of different ages.
David Wechsler was the fist to devise an instrument to measure intelligence in adults. He later devised downward extensions of his scale for children. Wechsler is credited with two innovations in intelligence testing. First, his scales give more emphasis to nonverbal reasoning, yielding a verbal IQ, a performance IQ, and a full-scale IQ. Second, Wechsler devised a new scoring system based on the normal distribution…the deviation IQ. This scoring system is outlined on the next slide. 7

8. 8

9. those on the WAIS are shown on the right.
Fig 9.7 – Subtests on the Wechsler Adult Intelligence Scale (WAIS). The WAIS is subdivided into a series of tests that yield separate verbal and performance (nonverbal) IQ scores. Examples of low-level (easy) test items that closely resemble
those on the WAIS are shown on the right. 9

10. Fig 9. 8 – The normal distribution
Fig 9.8 – The normal distribution. Many characteristics are distributed in a pattern represented by this bell-shaped curve. The horizontal axis shows how far above or below the mean a score is (measured in plus or minus standard deviations). The vertical axis is used to graph the number of cases obtaining each score. In a normal distribution, the cases are distributed in a fixed pattern. For instance, 68.26% of the cases fall between +1 and –1 standard deviation. Modern IQ scores indicate where a person’s measured intelligence falls in the normal distribution. On most IQ tests, the mean is set at an IQ of 100 and the standard deviation at 15. Thus, an IQ of 130 means that a person scored 2 standard deviations above the mean. Any deviation IQ score can be converted into a percentile score, which indicates the percentage of cases obtaining a lower score. The mental classifications at the bottom of the figure are descriptive labels that roughly correspond to ranges of IQ scores. 10

11. Sternberg's (1981) categories:
Validity of IQ Tests
Are IQ Tests Valid?
Sternberg's (1981) categories:
Verbal
Practical
Social
Which do IQ tests measure?
Stanovich (2009) criticisms 11

12. Reliability and Validity of IQ Tests
Exceptionally reliable – correlations into the .90s
Qualified validity – valid indicators of academic/verbal intelligence, not intelligence in a truly general sense
Correlations:
.40s-.50s with school success
.60s-.80s with number of years in school
Predictive of occupational attainment, debate about predictiveness of performance
Although they are intended to measure potential, IQ tests inevitably assess both knowledge and potential.
IQ tests are, however, exceptionally reliable, with reliability coefficients into the .90s.
IQ tests are reasonably valid indicators of academic intelligence, as they predict school success and number of years in school.
They are not good measures of social or practical intelligence and do not measure intelligence in a truly general sense.
IQ scores are positively correlated with high status jobs; this may be related to the correlation with school success.
There is conflicting evidence regarding whether IQ scores predict occupational performance. Court rulings and laws now require that tests used in selection of employees measure specific abilities related to job performance. 12

13. Stability not seen in infancy/preschool ages
Stability and Culture
Stability not seen in infancy/preschool ages
Scores are stable around ages 7 yrs +
Changes can occur (e.g. Headstart Programs)
Used in Western Cultures and not in non- Western Cultures
Cross-cultural issues 13

14. Extremes of Intelligence: Mental Retardation
Diagnosis based on IQ and adaptive testing
IQ 2 or more SD below mean
Adaptive skill deficits
Origination before age 18
4 levels: mild, moderate, severe, profound
Mild most common by far
Causes:
Environmental vs. Biological
Mental retardation is a diagnosis reserved for individuals with subaverage general mental ability accompanied by deficiencies in adaptive skills, originating before age 18.
The vast majority of people with mental retardation have mild mental retardation and are not easily distinguished from the rest of the population.
Origins of mental retardation may include organic syndromes, as 350 biological conditions that can cause mental retardation have been identified. Diagnosticians are, however, only able to pin down an organic cause in <25% of cases.
In fact, cases of mild mental retardation tend to involve unknown origin. Environmental theories hold that unfavorable environmental factors may contribute to the development of mild mental retardation; things like neglect, inadequate nutrition and medical care, and lower quality schooling. 14

15. 15

16. Fig 9. 10 – The prevalence and severity of mental retardation
Fig 9.10 – The prevalence and severity of mental retardation. The overall prevalence of mental retardation is roughly 1 to 3% of the general population. The vast majority (85%) of the retarded population is mildly retarded. Only about 15% of the retarded population falls into the subcategories of moderate, severe, or profound retardation. 16

17. Fig 9. 11 – Social class and mental retardation
Fig 9.11 – Social class and mental retardation. This graph charts the prevalence of mild retardation (IQ 60 to 69) and more severe forms of retardation (IQ below 50) in relation to social class. Severe forms of retardation are distributed pretty evenly across the social classes, a finding that is consistent with the notion that they are the product of biological aberrations that are equally likely to strike anyone. In contrast, the prevalence of mild retardation is greatly elevated in the lower social classes, a finding that meshes with the notion that mild retardation is largely a product of unfavorable environmental factors. (Source: Adapted from Popper and Steingard, 1994) 17

18. Extremes of Intelligence: Giftedness
Identification issues – ideals vs. practice
IQ 2 SD above mean standard
Creativity, leadership, special talent?
Stereotypes – weak, socially inept, emotionally troubled
Lewis Terman (1925) – largely contradicted stereotypes
Ellen Winner (1997) – moderately vs. profoundly gifted
Giftedness and high achieving – beyond IQ
Renzulli (2002) – intersection of three factors
Simonton (2001) – drudge theory and inborn talent
“Hidden gifted”
There are discrepancies between ideals and practice regarding how gifted children are identified in the U.S. Usually, identification occurs based on IQ of 130 or higher, although creativity, leadership, and special talents are recommended for use in identification as well.
Gifted individuals are often stereotyped as weak, sickly, socially inept, and emotionally troubled “bookworms.”
Lewis Terman initiated a study in the early 1920’s with 1500 children with IQs of 150 or higher. These children were followed throughout their lives. As a group, these subjects exhibited better than average physical health, emotional stability, and social satisfaction through their adult years.
Ellen Winner (1997) claims that a distinction needs to be made between moderately gifted (IQ ) and profoundly gifted (IQ above 180) individuals, asserting that profoundly gifted children are often introverted and isolated.
Studies of giftedness and achievement in life suggest that more than IQ determines high achievement. Joseph Renzulli theorizes that there is a more rare form of giftedness, based in an intersection of 3 factors, that leads to genuine greatness…high intelligence, high creativity, and high motivation.
Drudge theory is captured in the reaction of one talented violinist after a critic termed him a genius – “A genius! For 37 years I’ve practiced fourteen hours a day, and now they call me a genius!” – While clearly obsessive hard work is important in this case, it can be argued that inborn ability allowed him to work harder because he found his efforts more rewarding. Simonton proposes an elaborate theory of talent development that gives roles to both innate ability and environmental factors. 18

19. Intelligence: Heredity or Environment?
Family and twin studies
Heritability estimates
Environment
Adoption studies
Cumulative deprivation hypothesis
The Flynn effect
Interaction
The concept of the reaction range
This issue has far-reaching sociopolitical implications and continues to be a complex controversy.
Family studies determine only whether genetic influence on a trait is plausible, not whether it is certain. Family members also share environments.
Twin studies provide evidence regarding the role of genetic factors. The basic rationale is that identical and fraternal twins develop under similar environmental conditions, but identical twins share more genes…if identical twins end up more similar on a given characteristic, it must be genetic.
A heritability ratio is an estimate of the proportion of trait variability in a population that is determined by variations in genetic inheritance. A heritability estimate is a group statistic and cannot be meaningfully applied to individuals.
Adoption studies provide evidence that upbringing plays an important role in mental ability, as adopted children show some resemblance to their foster parents. Also, siblings reared together are more similar in IQ than siblings reared apart. In fact, entirely unrelated children who are reared together show resemblance in IQ.
The cumulative deprivation hypothesis holds that children raised in deprived environments will experience a gradual decline in IQ as they grow older. Conversely, children removed from deprived environments and placed in homes that are more conducive for learning show IQ increases.
The Flynn effect is the trend, all over the developed world, for IQ scores to increase from one generation to the next. Hypotheses for why this occurs focus on environmental variables, as evolution does not operate in a generation.
Clearly, heredity and environment both influence intelligence. Theorists use the term “reaction range” to refer to genetically determined limits on IQ. The environment determines whether a person will fall at the upper or lower end of their genetically determined range. 19

20. Fig 9. 13 – Studies of IQ similarity
Fig 9.13 – Studies of IQ similarity. The graph shows the mean correlations of IQ scores for people of various types of relationships, as obtained in studies of IQ similarity. Higher correlations indicate greater similarity. The results show that greater genetic similarity is associated with greater similarity in IQ, suggesting that intelligence is partly inherited (compare, for example, the correlations for identical and fraternal twins). However, the results also show that living together is associated with greater IQ similarity, suggesting that intelligence is partly governed by environment (compare, for example,
the scores of siblings reared together and reared apart). (Data from McGue et al., 1993) 20

21. Fig 9. 14 – The concept of heritability
Fig 9.14 – The concept of heritability. A heritability ratio is an estimate of the portion of variation in a trait determined by heredity—with the remainder presumably determined by environment—as these pie charts illustrate. Typical heritability estimates for intelligence range between a high of 70% and a low of 50%, although some estimates have fallen outside this range. Bear in mind that heritability ratios are estimates and have certain limitations that are discussed in the text. 21

22. Fig 9.16 – Reaction range. The concept of reaction range posits that heredity sets limits on one’s intellectual potential (represented by the horizontal bars), while the quality of one’s environment influences where one scores within this range (represented by the dots on the bars). People raised in enriched environments should score near the top of their reaction range, whereas people raised in poor-quality environments should score near the bottom of their range. Genetic limits on IQ can be inferred only indirectly, so theorists aren’t sure whether reaction ranges are narrow (like Ted’s) or wide (like Chris’s). The concept of reaction range can explain how two people with similar genetic potential can be quite different in intelligence (compare Tom and Jack) and how two people reared in environments of similar quality can score quite differently (compare Alice and Jack). 22

23. Cultural Differences in IQ
Heritability as an Explanation
Aurthur Jensen (1969)
Herrnstein and Murray (1994) – The Bell Curve
Rushton “Race, Evolution, and Behaviour”
Environment as an Explanation
Kamin’s cornfield analogy – socioeconomic disadvantage
Steele (1997) - stereotype vulnerability
Arthur Jensen argued that cultural differences in average IQ are largely due to heredity.
The authors of The Bell Curve, by implying that we are moving toward a meritocracy based on intellect, ignited the same controversy.
These arguments have been challenged on a number of grounds. First, even if IQ is largely due to heredity, group differences may not be. Social class and socioeconomic disadvantage are correlated with ethnicity, so environmental variables are not equal between groups.
Philippe Rushton’s book Race, Evolution, and Behavior (1995) argued that genetics were the cause for intellectual and behavioural differences between cultural groups. His book was criticized as being based on bad science, sloppy reasoning, and inaccuracies.
Kamin’s cornfield analogy, presented on the next slide, depicts this issue.
Claude Steele argues that derogatory stereotypes create feelings of vulnerability in the educational domain, undermining group members’ achievement and performance on tests. 23

24. Fig 9. 17 – Genetics and between-group differences on a trait
Fig 9.17 – Genetics and between-group differences on a trait. Kamin’s analogy (see text) shows how between-group differences on a trait (the height of corn plants) could be due to environment, even if the trait is largely inherited. The same reasoning presumably applies to ethnic group differences in the trait of human intelligence. 24

25. New Directions in the Study of Intelligence
Increased emphasis on specific abilities
Moving beyond Spearman’s g
Guilford’s 150 distinct mental abilities.
Fluid vs. crystallized intelligence
Biological Indexes of Intelligence
Reaction time and inspection time
Cognitive Conceptualizations of Intelligence
Sternberg’s triarchic theory and successful intelligence
Expanding the Concept of Intelligence
Gardner’s multiple intelligences
Goleman’s emotional intelligence
Increased emphasis is being placed on specific abilities rather than a general mental ability that Charles Spearman labeled g. Spearman used a statistical procedure called factor analysis to determine intercorrelated, specific mental talents (s) (concluding that all cognitive abilities share a common core). In contrast, J.P. Guilford asserts that intelligence is made up of as many as 150 distinct mental abilities.
Cattell and Horn suggest that g should be divided into fluid intelligence, which consists of reasoning ability, memory capacity, speed of information processing, and crystallized intelligence, which consists of the ability to apply acquired knowledge and skills in problem solving.
Other researchers, such as Aurthur Jensen, are searching for physiological indicators of general intelligence. Reaction time has been used in these studies, although the “fast is smart” idea is modest at best. Other measures studied include inspection time, which is an assessment of how long it takes to make simple perceptual discriminations that meet a certain criterion of accuracy. Higher correlations with IQ have been found with this measure, although much work remains to be done to discover why.
For over a century, intelligence was approached from a testing perspective. In contrast, the cognitive perspective focuses on how people use their intelligence. Robert Sternberg’s triarchic theory of intelligence consists of three parts: the contextual, experiential, and componential subtheories. In more recent years (1999, 2000), Sternberg has asserted that there are three aspects of what he calls “successful intelligence” – analytical intelligence, creative intelligence, and practical intelligence.
Other theorists propose an expansion of the concept of intelligence. Howard Gardner argues that IQ tests emphasize verbal and mathematical skills and exclude other important skills. He suggests the existence of a number of human intelligences, listed in Table 9.3.
Daniel Goleman and others argue for the concept of emotional intelligence, which is the ability to perceive and express emotion, assimilate emotion in thought, understand and reason with emotion, and regulate emotion. 25

26. Fig 9.5 – Spearman’s g. In his analysis of the structure of intellect, Charles Spearman found that specific mental talents (S1, S2, S3, and so on) were highly intercorrelated. Thus, he concluded that all cognitive abilities share a common core, which he labeled g for general mental ability. 26

27. Fig 9. 6 – Guilford’s model of mental abilities
Fig 9.6 – Guilford’s model of mental abilities. In contrast to Spearman (see Figure 9.18), J. P. Guilford concluded that intelligence is made up of many separate abilities. According to his analysis, people may have as many as 150 distinct mental abilities that can be characterized in terms of the operations, contents, and products of intellectual activity. 27

28. Fig 9. 20 – Sternberg’s triarchic theory of intelligence
Fig 9.20 – Sternberg’s triarchic theory of intelligence. Sternberg’s model of intelligence consists of three parts: the contextual subtheory, the experiential subtheory, and the componential subtheory. Much of Sternberg’s research has been devoted to the componential subtheory, as he has attempted to identify the cognitive processes that contribute to intelligence. He believes that these processes fall into three groups: metacomponents, performance components, and knowledge-acquisition components. 28

29. Divergent vs. Convergent thinking Measuring
Creativity
Defined
Insight
Divergent vs. Convergent thinking
Measuring
Remote Associates Test (1967)
Predictive ability
Correlates
Personality, Intelligence & Mental Illness 29