南京理工大学信息管理系 Research Methodology 科学研究方法 Class 5 April 11, 2017

Overview of Today’s Class Major data collection techniques III Bibliometrics Aka Informetrics and Scientometrics Webometrics Aka Cybermetrics Altmetrics Ethnography/fieldwork Historical research Delphi study - Practice RM-Class 5

Bibliometrics Definition Components Hulme Chu Counting and analyzing various facets of written communication Chu Analysis of data (publication data, references/citations, and other related data) recorded in formal communication Components Bibliometric laws (e.g., Bradford’s Law) Citation analysis Citation-dependent measures (e.g., impact factor, h-index) RM-Class 5

Written/Formal Communication Facets of written communication Publication data (e.g., author, title, publication year) Text itself (e.g., words, location of words, word frequency) Reference/Citation data (e.g., author, title, document type, language) Relationship to information management Information management → Written/Recorded information (e.g., books, periodicals) → Written communication RM-Class 5

Citations or References? Citing Author/Item Making a reference Cited Author/Item Receiving a Citation RM-Class 5

Bibliometric Laws – Bradford’s Law If periodicals contributing to a subject area are ranked and then grouped in such a way that each group contributes the same # of articles, the #s of periodicals in each group increase geometrically, i.e., k : kn : kn2 ... Aka “law of scattering” or “law of concentration” Bradford type of distribution RM-Class 5

Bradford-type Distribution RM-Class 5

Bradford Distribution of Publications in Applied Physics & Lubrication (k=9, n=5) # of Journals k:kn:kn2 … # of Articles 9 k → 9 429 59 kn → 9x5=45 499 258 kn2 → 9x52=225 404 … RM-Class 5

Bibliometric Laws – Lotka’s Law Suppose n is the number of papers published, k is the number of authors who published a single paper, the number of people publishing n articles is about k/n2. That is, if k people publish a single paper, k/22 people publish 2 papers, k/32 people publish 3 papers, … Aka "Inverse square law of authors' productivity" An example RM-Class 5

Lotka’s Law: An Example If K=100 in k/n2 Then When n=1, k/n2 =100/12= 100 → 100 people publish 1 paper in their life time When n=2, k/n2 =100/22= 25 → 25 people publish 2 papers in their life time When n=3, k/n2 =100/32= 11 → 11 people publish 3 papers in their life time When n=4, k/n2 =100/42= 6 → 6 people publish 4 papers in their life time … RM-Class 5

Bibliometric Laws – Zipf’s Law The principle of least effort If words are ranked (r) according to their frequency of occurrence (f), the product of r and its corresponding f is equal to a constant: rf = C Sample applications Language learning/acquisition Indexing & retrieval RM-Class 5

Zipf’s Law vs. Indexing & Retrieval C D RM-Class 5

Citation Analysis Assumptions Sources of citation data Citation indexes (Eugene Garfield & ISI) → Web of Science Scopus, Microsoft Academic Search, … CSSCI, CSCI, 百度学术，… Citation analysis methods Direct counting Bibliographic coupling (书目对、书目耦合） Co-citation analysis （共引分析） RM-Class 5

Bibliographic Coupling vs. Co-citation Analysis RM-Class 5

Impact Factor (IF – 影响因子) Journal Citation Reports (JCR) Journal IF=A/B A = # of times the “citable items” (e.g., articles, reviews, proceedings or notes) published in two years (e.g., 2013-2014) were cited in indexed journals during the third year (e.g., 2015) B = # of citable items published in the first two years (e.g., 2013-2014) RM-Class 5

h-index & Other Variations Suggested by Jorge E. Hirsch in 2005 A scientist has index h if h of his Np papers have at least h citations each, and the other (Np - h) papers have at most h citations each. That is, h = citation #s one receives for each paper = paper #s the same person publishes h5-index, h10-index, … Journal Scholar Metrics RM-Class 5

Webometrics Coined by Peter Ingwersen in 1997 Webpages/websites ≈ publications Links → (Sitations) ≈ Citations Incoming links (in-links) Co-cited links (co-links) Web impact factor (WIF) Examples Ranking of universities in the world: http://www.webometrics.info/ MDS map of 53 LIS schools RM-Class 5

MDS Map for 53 LIS Schools (Innovation) (Technology) RM-Class 5

Bibliometrics & Webometrics: A Summary Features Unobtrusive Rich data for both quantitative and qualitative analysis Widely used for evaluating individuals, groups, institutions, and artifacts such as publications and websites; for exploring the intellectual development and structure of a specialty/field; … Criticisms Hard to determine reasons for citations/linking Self-citations/links and citation/link manipulation Only first author was counted until recent years What to do with publications of multiple authors? RM-Class 5

Altmetrics Altmetrics, short for alternative metrics, are new measurements for the impact of scholarly content. Altmetrics go beyond counts from citation databases by considering other aspects of the impact of a work, such as article views, downloads, or # of followers/mentions/reviews in social media. Increasingly used as a method to complement bibliometric & webometric research See Gallican and Dyas-Correia (2013) for more. RM-Class 5

More on Altmetrics Reasons for using social media Common academic social networks 中国科技论文在线: http://www.paper.edu.cn 学术汇: http://www.xuehuile.com 天玑网(SoScholar): http://www.soscholar.com Academia.edu: https://www.academia.edu ResearchGate: https://www.researchgate.net Mendeley: https://www.mendeley.com RM-Class 5

Bibliometrics, Webometrics & Altmetrics Dos and Don’ts Select trusted sources for collecting citation, link & altmetric data. Apply sampling techniques for data collection wherever possible. Use citation/link/altmetric counts as a rough indicator of quality. Analyze citation/link/altmetric data in context, both quantitatively and qualitatively. Use altmetric and other techniques to supplement bibliometric & webometric research Don’t assume all citations/links/altmetric data are equal. Don’t over-interpret citation/link/altmetric data. RM-Class 5

What is Ethnography? Ethnos Ethnography (人种/民族志学) Features People of the same race or nationality who share a distinctive culture (from WordNet) Ethnography (人种/民族志学) The branch of anthropology that provides scientific description of individual human societies Features Involves an ongoing attempt to place specific encounters, events, and understandings into a fuller, more meaningful context Combines various methods of inquiry to produce historically, politically and personally situated accounts, interpretations, and representations of human lives Enters into close and relatively prolonged interaction with people RM-Class 5

Approaches to Ethnography Fieldwork The most often used approach in ethnography Need to employ specific techniques for data collection Participant observation Interview, questionnaire Photo survey Various other techniques described in Delcore, et al. (2009) such as floating reference, student day-mapping, … RM-Class 5

Fieldwork: Stages & Questions How to gain access to less or least accessible fields? How to become acceptable after entering the field? How to become invisible in the field while not causing ethical concerns? How to watch, listen and learn in a field? Shall the researcher be value-free or value-laden? RM-Class 5

Ethnography: Applications General areas Cultural and social anthropology, sociology, communication studies, … Specific areas Cultural studies, literary theory, folklore, nursing (i.e., ethnonursing), law, … Information management Information seeking behavior (e.g., information source selection, library use) of users, information needs of special groups, online communities, organization, … RM-Class 5

Historical Research What is historical research Features Study a research problem by examining, analyzing (synthesize, summarize) & interpreting existing published/unpublished materials Features No discrete phases. Rather, hypothesis formation, collection and interpretation of evidences (data) are all integrated with one another Mainly qualitative General research steps Data sources and evaluation Primary, secondary, tertiary RM-Class 5

Steps in Historical Research Recognizing a historical problem or identifying a need for certain historical knowledge Gathering as much pertinent information as possible about the problem or topic as possible If appropriate, forming hypotheses that tentatively explain relationships between historical factors (variables) Rigorously collecting and organizing evidence, and verifying the authenticity and veracity of information and its sources Selecting, organizing, and analyzing the most pertinent collected evidence, and drawing conclusions Recording conclusions in a meaningful narrative RM-Class 5

More on Historical Research Dos and don’ts Be thorough in data collection Be diligent in verifying the authenticity of data sources Be truthful in interpreting the data collected Try to take into consideration of the contextual factors in existence when analyzing the data collected When to use the historical research method? RM-Class 5

Delphi Study What is Delphi study? Features of the Delphi method? Use data collected from a selected group of anonymous experts during several "rounds" of information exchange in order to reach consensus. Features of the Delphi method? Compare with focus groups Other features Controlled feedback in the form of statistical summaries of group opinions Allow the researcher to ask participants to provide explanations for their choices and if their feedback fall outside of the range of group consensus. RM-Class 5

Delphi Study: General Process Select experts Develop data collection instrument Conduct the first round of Delphi study Gather data using the data collection instrument Analyze, summarize and incorporate the results into the data collection instrument before moving to the next round Repeat the same process in the 2nd, 3rd, … round How many rounds? Example Zhang & Salaba, 2009 RM-Class 5

More on Delphi Study Dos and don’ts Keep time intervals between rounds to a reasonable length Summarize and present group responses properly Don’t omit, misinterpret what is gathered Don’t impose one’s own opinion to research participants When to use the Delphi method in our field? Practice: Research fronts in information management RM-Class 5

Research Fronts in Info Management Access to quality info Artificial intelligence Big data Cloud computing Database manag’mt Digital libraries HCI/UX Information networks Information retrieval Information security Info seeking behavior Information systems Internet of Things (IoT) Knowledge manag’mt Knowledge organiz’tn Linked data Mobile technologies Natural language IR Organizing the web Security systems Social media Ubiquitous computing Web 2.0 Your suggestions: RM-Class 5

Next class (4/12, 2-5pm) Major data collection techniques IV Think aloud protocols Transaction log analysis Research diaries Other methods Preparations for data analysis Quantitative research vs. qualitative research Major quantitative data analysis techniques (Q&A only) Descriptive statistics Inferential statistics & Big Data analytics RM-Class 5