1. Disciplinary structure and topical complexity in SSH—the IMPACT EV mission
Sándor Soós, András Schubert, Zsófia Vida

2. The IMACT EV challange(s)

3. The IMACT EV challange(s)
Evaluating the outcomes of SSH research, FP7
Metrics challange: to find evaluation methods that are a better fit for SSH project assessment (funded via EU FPs)
Age-old issues for SSH bibliometrics (and evaluation):
Different communication channels (no-journal regime)
More diverse disciplinary background (multidisciplinarity), various citation regimes, problem even for normalization-based assessment („MNCS”)
Our research questions: is that still so?

4. The concept and the idea
Contrasting SSH fields with „STEM” fields in terms their evolution of subject composition (long-term, until today)
Recent trends?
Technically: measuring the development of multidisciplinarity (subject diversity of fields) and analysing it
Main idea
SSH fields are defined by field reviews, their references (relevant, selected etc.
„Supervised” method
Expert validation is a built-in feature
WoS? Turning a bug into feature

5. Data and materials Sampling SSH and STEM disciplines
Selecting 5 "protoypic" fields (Subject Area) per Social Sciences and Humanities (SSH) and per Sciences (STEM)
Collecting all review papers for each field during (20-year time window)
Filtering out references of reviews published max. 5 years before the review → contemporary "state of the art"
Aggregated 5-year references of review cohorts per Subject Area → field definition

6. Data and materials
Selected fields

7. Data and materials Indicators of multidisciplinarity
Mapping between (journal) references and WoS Subject Areas (based on pre-defined journal assignment in WoS)
Obtaining a Subject (Area) profile for each "field" (aggregated reference set) → disciplinary composition of the field
Filtering out low frequency occurences of Subject Areas (threshold around 2% of max. frequency)

8. Methods – Modelling disciplinary composition
Framework of science overlay mapping (Rafols, Porter, Leydesdorff, 2010)

9. Methods – IDR measures Framework of Stirling on aspects of diversity
Similarity:
Degree of
overlap

10. Methods – Visualization of Subject development
Change of Subject composition on the science map Contrasting 2004 – 2009 and 2009 – 2014 (5-year windows)
Coloring of nodes:
red=appearance,
rose=growth,
dark blue=disappearence,
light blue=decrease,
size=share in last period.

11. Analytic strategies
Detailed view of multidisciplinarity development in SSH vs. STEM: individual fields
Contrasting evolutionary profiles of individual fields, i.e. time series of IDR measures for "prototypic" SSH and STEM ( ) areas
(Visual) Contrasting the change in Subject profile via science overlay maps (last two 5-year windows)
To get the "big" picture: a two-step clustering of fields based on mulltidisciplinarity characteristics and its dynamics
1) Clustering fields based on the similarities along dimensions of multidisciplinarity, i.e, IDR measures obtained for 5-year time periods (focal years: 1999, 2004, 2009, 2014) → "primary clusters"
2) Clustering fields based on their co-membersip in primary clusters, i.e. to the degree they tend to cluster together with other fields throughout their history → "secondary clusters"
Fields are grouped according to their behavior in consecutive time periods, i.e. their similarities in multidisciplinarity dinamics ("feature based" clustering of multivariate time series)

12. Results – SSH vs. STEM evolution by IDR metrics
SSH: Social psychology
decreasing concentration
decreasing proximity
increasing variety
recombination of subs a 40%
Coloring of nodes: red=appearance, rose=growth,
dark blue=disappearence, light blue=decrease

13. Results – SSH vs. STEM evolution by IDR metrics
SSH: economics
variety higher than soc.psy (steady)
concentration lower (steady)
proximity at low level (steady)
increasing recombination (40-50%)
Coloring of nodes: red=appearance, rose=growth,
dark blue=disappearence, light blue=decrease

14. Results – SSH vs. STEM evolution by IDR metrics
STEM: soil science
variety goes up, reaches SSH level
proximity is comparable to SSH
disparity gets even lower
similarity: consolidation process
Coloring of nodes: red=appearance, rose=growth,
dark blue=disappearence, light blue=decrease

15. Results – SSH vs. STEM evolution by IDR metrics
STEM: nanoscience
variety remains steady
concentration, proximity is not very different from SSH in general
recombination initially high then vanishes recently
Coloring of nodes: red=appearance, rose=growth,
dark blue=disappearence, light blue=decrease

16. Results – typology by evolution of multidisciplinarity
Secondary clusters: Cluster tree of fields based on… …yearwise co-membership in primary clusters (shared patterns of multidisciplinarity)

17. Results – overall pattern of multidisciplinarity
Cluster centers by IDR measures

18. Main conclusions
SSH fields are more similar to each other in their evolution of disciplinary composition (multidisciplinarity) than to STEM fields. The same applies to STEM fields.
BUT
Similarities between SSH fields are much lighter than between STEM fields
STEM fields are very similar to SSH fields in terms of multidisciplinarity: variety (10% of all Subjects), balance (high), and disparity (relatively high) throughout the time window ( ).
SSH fields are continuously „recombinant”, recomposing their basis of Subject Categories in each period, reflected in much lower similarity values.
Exceptions appear in each period (sociology, nanoscience and technology.

19. Social networks as a potential source of bias in peer review
Thank you for your attention!