Is there anybody out there? Searching the social space for signs of intelligence Mike Taylor Research Specialist

For much of the the last century, our only measurement of the impact of scholarly research has been through the counting and analysis of citation: one authoring researcher acknowledging the contribution of another authoring researcher. Significant, certainly, given these caveats, but in a wider social context citation analysis begins to look like an edge case. Can the measurement of sharing on social networks provide a wider view of how research is consumed in society, or is it all chatter and noise – and how do we detect the conversations of true significance?

The world according to citation “I am writing an article and wish to cite another article”

The world according to alternative metrics “I’m writing an article and might cite this” “You should read this article if you’re interested in #thistopic” “My PI wrote this paper” “My daughter wrote this and I’m so proud” “This article has a titivating title. Anyway, it made me laugh” “These scientists are going to cure cancer”

The phenomenally rich world of alternative metrics Social activity indicators: Twitter, Facebook, Delicious, Pintrest, Google+ Scholarly activity indicators: Mendeley, Citeulike, Zotero Scholarly articles: blogs, reviews Mass media: news papers, TV Re-usage indicators: data, code, graphics

An example from 2013 Huge potential for social impact Press campaign: front page story on much of the UK press Great publisher support from Nature 1000s of tweets But what’s missing?

The phenomenally poor world of alternative metrics Current alternative metrics don’t count or model: Poorly referenced mass media Stories about stories The flow of the story Social media about stories, replies, re-tweets Influence on professional bodies Representation to Government, Government policy

Not only are alternative metrics bigger than citations, they’re also different Public vs private Anonymous vs attributable Persistent vs fleeting Positive vs negative (counts and sentiment) Real time vs slower But article driven, formal links

The different characteristics of alternative metrics Citation: one class of activity, with many sub- classes Alternative metrics: several types of activity, with many classes and countless sub-classes (all vying with each other)

The power of intelligent conversation Elevator pitch > monograph A word in the ear of a President versus Engaging with millions Patient-power Lobbying interests

The chatter of (how shall we say this?) less than intelligent conversation Not all communication is equal Not all communication is between equals Noise is not meritocratic But is Twitter just meaningless noise?

The myth of social networks Often assumed to be trivial, with a focus on titillating articles An analysis of 13.5k papers revealed striking differences: Top 0.5% of social activity – strong emphasis on policy, funding, areas where science and government overlap (stem cells, CERN, etc) Top 0.5% of scholarly activity – primary research

The academic networks are building Orcid / ODIN / THOR Data DOIs RDA data citation Data metrics Usage APIs / data Open data, open articles

Mapping academic influence is becoming easier Heading towards a paradigm shift in mapping academic influence Academics probably won’t create negative links This is a matter-of-fact network, flat, a statement of “what is” Insufficient to understand social impact

Science in society Open science, citizen science, open access, open data, cloud infrastructure, open source code, virtualization Social networks, easy access to scholars Too hard => too easy? Explosion in communication and access

A partial view Moving towards a more complete scholarly network Data exists to get an idea of how research is being consumed in society Too much missing to extrapolate Almost entirely devoid of political context

Correlations Not an even picture, there are threads of correlations – blogs – tweets – mass media We can’t make simple conclusions We don’t have enough data to make complex conclusions

Bigger data Deeper: Talking about people, departments, companies, movements More sensitive: Going from “being spoken about” to “what is being said” Wider: “who is speaking”, “to whom are they speaking” Further: “ 他們在中國說了什麼？ ”

The role of sociologists and economists Social potential, professional and academic perspectives Is $$$ a good reflection of impact? People like to think of the “return on investment” model, but it’s not that easy, and the conclusions may be uncomfortable seen in isolation

Is a social impact index computable? Social impact index = f(social capacity), f(social accessibility), f(social reach) If has no capacity for effecting social change, if is incomprehensible, if no-one is aware of it… We need the data and the maths to identify the intelligent versus the influential