Bibliometrics and Altmetrics for Special Education and EBD-Related Journals Bryan G. Cook, University of Hawaii John Wills Lloyd, University of Virginia

How We Could or Ought to Measure Influence Influence on research questions, sampling plan, settings, IVs, instruments, measurement plan, data analyses Influence on practice Change in practices’ application in applied settings Change in student outcomes Influence on developing and refining theory

Just a Note Not “impact” Try “influence” or “affect” “Impact” is popular. “Impact” means “to hit”; we don’t mean that. One of us wants to influence the use of the word “impact” negatively. Sigh. Too bad it’s become common. Try “influence,” “affect,” or some other better word.

How We Do Measure Influence How many times are papers cited in a data base Traditional influence measures do not indicate quality or positive influence Wakefield et al. (1998) cited 2,649 times; 1,170 since 2011 (retracted in 2010):  Ayllon et al. (1975) cited 81 times, despite having >20 more years and many more replications:  Ayllon, T., Garber, S., & Pisor, K. (1975). The elimination of discipline problems through a combined school-home motivational system. Behavior Therapy, 6(5), 616-626. Ayllon, T., & Roberts, M. D. (1974). Eliminating discipline problems by strengthening academic performance. Journal of applied behavior analysis, 7(1), 71-76.

Traditional Measures of Journal Influence All citation based Vary on years considered, data bases, and formulas Journal Citation Reports’ (JCR) Journal Impact Factor (JIF) # of citations in year x of articles in years x-1 and x-2/# articles published in years x-1 and x-2; Web of Science data base

Traditional Measures of Journal Influence JCR’s Immediacy Index (II) # of citations in year x of articles published in years x/# articles published in year x; Web of Science data base Google Scholar’s h(5)-index the largest # such that at least h articles in that publication were cited at least h times each in previous 5 years; Google Scholar data base

Traditional Measures of Journal Influence SCImago Journal Rank (SJR) Computed similarly to JIF, but over previous 3 years and citations weighted by journal prestige; Scopus data set Source Normalized Impact per Paper (SNIP) Similar to SJR, but adjusts for “citation potential” of field and subfield of journal; Scopus data set

A Composite Influence Index Bradshaw and Brook (2016) recommended using a composite index We created a 2016 composite index from the five previously described influence indices For journals (a) listed as special education by JCR or Google Scholar or (b) CEC-affiliated journals, AND Had > four of five influence scores available (n=40)

Correlations Between Measures JIF II H SJR SNIP .44 .50 .29 .81 .56 .60 .83 .53 .87 JIF=Journal Impact Factor, II=Immediacy Index, H=H-factor, SJR=SCImago Journal Rank, SNIP=Source Normalized Impact per Paper

Composite Influence Index Rankings and Scores 1. Remedial and Special Education, 1.91 2. Exceptional Children, 1.63 3. Journal of Learning Disabilities, 1.18 4. Journal of Emotional and Behavioral Disorders, 1.16 9. Journal of Positive Behavior Intervention, 0.90 13. Research in Autism Spectrum Disorders, 0.44 22. Focus on Autism and Other DDs, -0.12 31. Behavioral Disorders, -0.65 32. Education and Treatment of Children, -0.65

Variability Across Indices (reported in z-scores) Journal of Emotional and Behavioral Disorders JIF=1.31, II=2.20, h=-0.04, JCR=0.88, SNIP=1.16 Research in Autism Spectrum Disorders JIF=0.45, II=-0.29, h=2.28, JCR=0.17, SNIP=0.42 Behavioral Disorders JIF=-0.45, II=-0.95, JCR=-0.61, SNIP=-0.61

Citation Distributions Distributions may be a more transparent representation of citations (Lariviere et al., 2016) Indices often influenced by a few highly cited publications Many published papers are never cited

Citation Distributions We examined 2016 citations of all substantive 2015 publications in seven non-categorical and EBD-related journals (N=168) Exceptional Children, Remedial and Special Education, Journal of Special Education, Journal of Emotional and Behavioral Disorders, Journal of Positive Behavior Intervention, Behavioral Disorders, Focus on Autism and Other Developmental Disabilities M=1.43 citations, mode=0, median=1 SD=1.92, range=0-12

Exceptional Children Number of Articles Number of Citations

Remedial and Special Education Number of Articles Number of Citations

Journal of Special Education Number of Articles Number of Citations

Journal of Emotional and Behavioral Disorders Number of Articles Number of Citations

Journal of Positive Behavior Interventions Number of Articles Number of Citations

Behavioral Disorders Number of Articles Number of Citations

Focus on Autism and Development Disabilities Number of Articles Number of Citations

Total for Seven Journals Number of Articles Number of Citations

Altmetrics A class of alternative influence metrics reflecting influence on social media Altmetrics now a company that reports an altmetric index reported for individual articles on many journals’ websites

Altmetric Attention Scores Three sources Volume of mentions Only one per person counts (first) Source They are weighted; mention in a newspaper article worth more than in a blog, which is worth more than in a tweet Authors Independent authors count more than a publisher’s mention

Altmetric Donuts

Altmetric Scores We examined altmetric scores for all substantive 2015 publications in seven non-categorical and EBD-related journals (N=168) Exceptional Children, Remedial and Special Education, Journal of Special Education, Journal of Emotional and Behavioral Disorders, Journal of Positive Behavior Intervention, Behavioral Disorders, Focus on Autism and Other Developmental Disabilities M=2.06, mode=0, median=1 SD=4.63, range=0-40

Correlations # of 2016 citations # of downloads Altmetric .45 .30 .23

High Alt-High Cite Social-Emotional Learning Program to Reduce Bullying, Fighting, and Victimization Among Middle School Students With Disabilities, RASE (40 altmetric, 18 citations) Comparative Study of Bullying Victimization Among Students in General and Special Education, EC (31 altmetric, 14 citations) Context Blindness in Autism Spectrum Disorder: Not Using the Forest to See the Trees as Trees, Focus on A & DD (20 altmetric, 6 citations)

High Alt-Low Cite Increasing Compliance in Students With Intellectual Disabilities Using Functional Behavioral Assessment and Self-Monitoring, RASE (15 altmetric, 2 citations) Unequal Opportunities: A Profile of the Distribution of Special Education Teachers, EC (13 altmetric, 3 citations) Involvement of Children With Autism Spectrum Disorder (ASD) in the Child Protection System, Focus on A & DD (9 altmetric, 2 citations)

Low Alt-High Cite Relationships Between Self-Determination and Postschool Outcomes for Youth With Disabilities, JSPED (1 altmetric, 27 citations) Is Performance Feedback for Educators an Evidence-Based Practice? A Systematic Review and Evaluation Based on Single-Case Research, EC (0 altmetric, 18 citations) A Systematic Evidence Review of the Check-In/Check-Out Program for Reducing Student Challenging Behaviors, JPBI (0 altmetric, 10 citations) Self-Monitoring Interventions for Students with Behavior Problems: A Systematic Review of Current Research, BD (0 altmetric, 10 citations)

Low Alt-Low Cite 16 articles with 0 citations and 0 altmetric scores

Conclusions Citation-based journal metrics differ and may not be robust indicators of true influence The modal article in special education/EBD journals is not cited in a given year Altmetrics appear to represent a different construct of influence Thanks!