1. Bibliometrics and Altmetrics for Special Education and EBD-Related Journals
Bryan G. Cook, University of Hawaii
John Wills Lloyd, University of Virginia

2. How We Could or Ought to Measure Influence
Influence on research questions, sampling plan, settings, IVs, instruments, measurement plan, data analyses
Influence on practice
Change in practices’ application in applied settings
Change in student outcomes
Influence on developing and refining theory

3. Just a Note Not “impact” Try “influence” or “affect”
“Impact” is popular.
“Impact” means “to hit”; we don’t mean that.
One of us wants to influence the use of the word “impact” negatively.
Sigh. Too bad it’s become common.
Try “influence,” “affect,” or some other better word.

4. How We Do Measure Influence
How many times are papers cited in a data base
Traditional influence measures do not indicate quality or positive influence
Wakefield et al. (1998) cited 2,649 times; 1,170 since 2011 (retracted in 2010): 
Ayllon et al. (1975) cited 81 times, despite having >20 more years and many more replications: 
Ayllon, T., Garber, S., & Pisor, K. (1975). The elimination of discipline problems through a combined school-home motivational system. Behavior Therapy, 6(5),
Ayllon, T., & Roberts, M. D. (1974). Eliminating discipline problems by strengthening academic performance. Journal of applied behavior analysis, 7(1),

5. Traditional Measures of Journal Influence
All citation based
Vary on years considered, data bases, and formulas
Journal Citation Reports’ (JCR) Journal Impact Factor (JIF)
# of citations in year x of articles in years x-1 and x-2/# articles published in years x-1 and x-2; Web of Science data base

6. Traditional Measures of Journal Influence
JCR’s Immediacy Index (II)
# of citations in year x of articles published in years x/# articles published in year x; Web of Science data base
Google Scholar’s h(5)-index
the largest # such that at least h articles in that publication were cited at least h times each in previous 5 years; Google Scholar data base

7. Traditional Measures of Journal Influence
SCImago Journal Rank (SJR)
Computed similarly to JIF, but over previous 3 years and citations weighted by journal prestige; Scopus data set
Source Normalized Impact per Paper (SNIP)
Similar to SJR, but adjusts for “citation potential” of field and subfield of journal; Scopus data set

8. A Composite Influence Index
Bradshaw and Brook (2016) recommended using a composite index
We created a 2016 composite index from the five previously described influence indices
For journals (a) listed as special education by JCR or Google Scholar or (b) CEC-affiliated journals, AND
Had > four of five influence scores available (n=40)

9. Correlations Between Measures
JIF II H
SJR
SNIP
.44
.50
.29
.81
.56
.60
.83
.53
.87
JIF=Journal Impact Factor, II=Immediacy Index, H=H-factor, SJR=SCImago Journal Rank, SNIP=Source Normalized Impact per Paper

10. Composite Influence Index Rankings and Scores
1. Remedial and Special Education, Exceptional Children, Journal of Learning Disabilities, Journal of Emotional and Behavioral Disorders, Journal of Positive Behavior Intervention, Research in Autism Spectrum Disorders, Focus on Autism and Other DDs, Behavioral Disorders, Education and Treatment of Children, -0.65

11. Variability Across Indices (reported in z-scores)
Journal of Emotional and Behavioral Disorders
JIF=1.31, II=2.20, h=-0.04, JCR=0.88, SNIP=1.16
Research in Autism Spectrum Disorders
JIF=0.45, II=-0.29, h=2.28, JCR=0.17, SNIP=0.42
Behavioral Disorders
JIF=-0.45, II=-0.95, JCR=-0.61, SNIP=-0.61

12. Citation Distributions
Distributions may be a more transparent representation of citations (Lariviere et al., 2016)
Indices often influenced by a few highly cited publications
Many published papers are never cited

13. Citation Distributions
We examined 2016 citations of all substantive 2015 publications in seven non-categorical and EBD-related journals (N=168)
Exceptional Children, Remedial and Special Education, Journal of Special Education, Journal of Emotional and Behavioral Disorders, Journal of Positive Behavior Intervention, Behavioral Disorders, Focus on Autism and Other Developmental Disabilities
M=1.43 citations, mode=0, median=1
SD=1.92, range=0-12

14. Exceptional Children
Number of Articles
Number of Citations

15. Remedial and Special Education
Number of Articles
Number of Citations

16. Journal of Special Education
Number of Articles
Number of Citations

17. Journal of Emotional and Behavioral Disorders
Number of Articles
Number of Citations

18. Journal of Positive Behavior Interventions
Number of Articles
Number of Citations

19. Behavioral Disorders
Number of Articles
Number of Citations

20. Focus on Autism and Development Disabilities
Number of Articles
Number of Citations

21. Total for Seven Journals
Number of Articles
Number of Citations

22. Altmetrics
A class of alternative influence metrics reflecting influence on social media
Altmetrics now a company that reports an altmetric index reported for individual articles on many journals’ websites

23. Altmetric Attention Scores
Three sources
Volume of mentions
Only one per person counts (first)
Source
They are weighted; mention in a newspaper article worth more than in a blog, which is worth more than in a tweet
Authors
Independent authors count more than a publisher’s mention

24. Altmetric Donuts

31. Altmetric Scores
We examined altmetric scores for all substantive 2015 publications in seven non-categorical and EBD-related journals (N=168)
Exceptional Children, Remedial and Special Education, Journal of Special Education, Journal of Emotional and Behavioral Disorders, Journal of Positive Behavior Intervention, Behavioral Disorders, Focus on Autism and Other Developmental Disabilities
M=2.06, mode=0, median=1
SD=4.63, range=0-40

32. Correlations
# of 2016 citations
# of downloads
Altmetric
.45
.30
.23

33. High Alt-High Cite
Social-Emotional Learning Program to Reduce Bullying, Fighting, and Victimization Among Middle School Students With Disabilities, RASE (40 altmetric, 18 citations)
Comparative Study of Bullying Victimization Among Students in General and Special Education, EC (31 altmetric, 14 citations)
Context Blindness in Autism Spectrum Disorder: Not Using the Forest to See the Trees as Trees, Focus on A & DD (20 altmetric, 6 citations)

34. High Alt-Low Cite
Increasing Compliance in Students With Intellectual Disabilities Using Functional Behavioral Assessment and Self-Monitoring, RASE (15 altmetric, 2 citations)
Unequal Opportunities: A Profile of the Distribution of Special Education Teachers, EC (13 altmetric, 3 citations)
Involvement of Children With Autism Spectrum Disorder (ASD) in the Child Protection System, Focus on A & DD (9 altmetric, 2 citations)

35. Low Alt-High Cite
Relationships Between Self-Determination and Postschool Outcomes for Youth With Disabilities, JSPED (1 altmetric, 27 citations)
Is Performance Feedback for Educators an Evidence-Based Practice? A Systematic Review and Evaluation Based on Single-Case Research, EC (0 altmetric, 18 citations)
A Systematic Evidence Review of the Check-In/Check-Out Program for Reducing Student Challenging Behaviors, JPBI (0 altmetric, 10 citations)
Self-Monitoring Interventions for Students with Behavior Problems: A Systematic Review of Current Research, BD (0 altmetric, 10 citations)

36. Low Alt-Low Cite
16 articles with 0 citations and 0 altmetric scores

37. Conclusions
Citation-based journal metrics differ and may not be robust indicators of true influence
The modal article in special education/EBD journals is not cited in a given year
Altmetrics appear to represent a different construct of influence
Thanks!