Abstract The Validity of Psychological Tests as Measures of Aggressive Behavior: A Review of the Literature E.M. Farrer & J.L. Mihura University of Toledo This study reviews the empirical literature on the validity of psychological tests as measures of aggressive behavior. The psychological tests were categorized into two groups: (a) self-report questionnaires (e.g., BDHI, JI, PAI) and (b) performance personality tests (e.g., Rorschach and Hand Test). For criterion variable of aggressive behavior, only studies using observational measures are included in the review (e.g., ward reports, patient records, chart reviews). The effect sizes of the psychological tests compared to observational measures are presented and then compared using a monotrait-multimethod approach. Also of interest are similar studies using a multitrait-multimethod approach, comparing and contrasting similar constructs (e.g., aggression, anger, antisocial behavior) using the same or different methods (self-report measures, performance personality tests, and observational measures). The goal of the study is therefore twofold: (1) to review the literature on the validity of psychological tests as measures of aggressive behavior and (2) to place this aggression literature in a psychometric context regarding more general issues of monomethod-heteromethod approaches to validity. The observational measures needed to be clear in how they measured the aggressive behavior. Records of past aggressive behavior (e.g., chart reviews, criminal file reviews) also had to have a well-defined way of measuring aggressive behavior including “objective” systems like the number of institutional infractions for forensic samples. Only studies written and conducted in English with no clear criterion contamination, (e.g., behavior ratings blind to psychological test data) were included. For comparative purposes, the findings are reported in effect sizes, converted where necessary to use Pearson r as the common metric. As a rule of thumb, the magnitude of effect sizes (r) can be classified as (a) small =.10, (b) moderate =.30, and (c) large =.50 Figure 1. Multitrait-Multimetehod Table Table 2. Aggressive Behavior as Measured by Performance Personality and Self-Report Tests: Summary Statistics Method cont. Introduction Most often in psychology the general notion of a person’s level of functioning and personality aspects is obtained by the word of mouth of the person him- or herself. Ways this can be done is by using self- report measures, such as the Personality Assessment Inventory (PAI), or by performance personality tests, such as the Rorschach. These measures, however, rely heavily on the respondent as the source of information, whereas behavior measures rely on others as the source of information. Results cont. Measurement MethodkNrwrw Performance Personality Test Self-Report Tests Total Tests Note: k = number of effect sizes included in the summary statistic Table 3. MTMM Results: Weighted Mean Effect Sizes Construct Overlap Measurement MethodModerateHigh Different.16 (k = 8, N = 924).28 (k = 9, N = 814) Same.46 (k = 6, N = 1,771).77 (k = 4, N = 371) Joni L Mihura: Construct Overlap Measurement MethodModerateHigh Different E.g., Self-report anger measure compared to aggressive behavior Major Study Question: Self report or performance personality aggression measures compared to aggressive behavior Same E.g., self-report anger measure compared to self- report aggression measure E.g., Self-report aggression measure compared to self-report aggression measure

References Many self-report measures used for screening are broadband inventories such as the PAI or the Jesness Inventory (JI). Several are also specifically designed to measure the construct of interest. The construct of particular interest to this review is aggression. Aggression can be defined as “the act or practice of attacking without provocation, “ (Coccaro et al., 1997). Aggression can be verbal or physical and, for this study, directed outwardly. The reliance on self-report measures and performance personality tests of aggression is of particular interest due to the implications that could arise if the aggression is carried out. How well can self-report measures and performance personality tests designed to measure aggression actually predict aggressive behavior? Further, aggression also has similar constructs with similar implications. Anger and antisocial behavior are among those. How well do tests specifically measuring those related constructs predict aggressive behavior? Also, how well do the same constructs measured by different methods compare? This multitrait-multimethod approach is of particular interest to the study. According to Campbell and Fiske (1959) the same construct measured by different methods should agree and should agree better than different constructs measured by different methods. Thus, the study has two goals. The first is to review how self- report and performance personality measures of aggression compare to observable behavior. The second is to compare similar but slightly different constructs to themselves and each other using the same and different methods. Results For studies that reported more than one effect size, these were were averaged to report as one effect size per study. Table 1. Aggressive Behavior as Measured by Performance Personality and Self-Report Tests StudyMeasuresSampleNr Performance Personality Tests 3. Rorschach AgCC Hand Test AOS&ACT-MOVMR Hand Test AOS&ACT-MOVMR Self-Report Tests 18. PAI AGGF Aggression Questionnaire PAS BDHI AssaultF BDHI AssaultC PAI AGG-PF Note: C = Clinical; MR = Mentally Retarded; F = Forensic; S = Student According to Jacob Cohen (1988), “…when one looks at near- maximum correlation coefficients, of personality measures…with real- life criteria, the values one encounters fall at the order of r =.30.” This corresponds to the findings above. The values were obtained using personality measures and with the same or highly overlapping constructs as compared to the real-life criteria in question. Self-report and performance personality measures do not differ in their effect sizes either. For the Campbell and Fiske’s (1959) MTMM approach, the data correspond quite well. Measuring moderate construct overlap using different methods will result in low effect sizes. On the other hand, using high construct overlap and the same methods, the correlation is quite high and what one would expect for test-retest reliability. This also corresponds with Meyer et al.’s (2001) findings that a single measure will only represent a certain portion of one’s personality and that different sources of information tend to provide their own unique interpretation of someone’s personality or behavior. Future information is yet to come. While performance personality tests and self-report measures were compared to each other, to behavioral measures, and to themselves; the next step would be to see how well observational measures compare to themselves. Discussion 1.Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, Cohen, J. (1988). Set correlation and contingency tables. Applied Psychological Meaurement, 12(4), Meyer, G.J., Finn, S.E., Eyde, L.D., Kay, G.G., Moreland, K.L., Dies, R.R., Eisman, E.J., Kubiszyn, T.W., & Reed, G.M. (2001). Psychological testing and psychological assessment. American Psychologist, 56(2), See Handout for the List of Reviewed Studies. Table 1 shows the effect sizes for performance personality tests and self-report measures as compared to behavior that range from.20 TO.57. Summary statistics were also computed for self-report and performance personality test effect sizes. This was done by taking the mean of each grouping of tests weighted by N. As shown in Table 2, both performance personality and self-report tests had overall medium effect size relationships with aggressive behavior—r =.31 and r =.27, respectively. The next table shows the results from the question of what happens to the effect sizes when slightly different constructs are measured using different methods and when the same constructs are measured by the same methods. Again, the effect sizes in the table are weighted to compensate for the varying sample sizes. Method Studies were located by conducting a PsycINFO search of articles published within the past 30 years with either Aggressive Behavior or Antisocial Behavior or Violence as Subject terms. These were further limited by a classification code of personality scales and inventories or clinical psychological testing. The articles were limited due to the high volume retrieved without the classification code— 19,651. The limit reduced the number of articles to 387. These remaining articles were kept or eliminated based on the following criteria. The tests in the study had to contain a self-report or performance personality measures of aggression, anger, or antisocial behavior. The next criterion for the study was an observational measure used that could be correlated with the self-report or performance personality test.