1. Objectives of theory-mining reviews Theory development with systematic literature reviews Chitu Okoli for ICT University, Fall 2015

What is a literature review? “A systematic, explicit, [comprehensive,] and reproducible method for identifying, evaluating, and synthesizing the existing body of completed and recorded work produced by researchers, scholars, and practitioners” Fink (2005 pp. 3,17) Systematic Follows a rigorous procedure Explicit Explains what is done, how and why Comprehensive Attempts to include all relevant literature Reproducible Sufficient detail that others could do the same thing This point is controversial, since it assumes a positivistic research epistemology

What is theory? “An integrated collection of explanations about the relationships between one or more pairs of concepts that represent real-world phenomena under specified conditions; such explanations might be accompanied with predictions and implications for intervention and action” (Okoli 2015 TMR) Concepts (What?) Distinct things that vary in value, temporal order or state Relationships (How?) How concepts are linked to each other Explanations (Why?) The reason why the relationship exists Bounding conditions (Who? Where? When?) Under which conditions the relationships are valid

Variance, Process and Systems theories (Burton-Jones et al. 2014) Variance theory Concepts are constructs (measurable things, whether physical or abstract) When the value of concept A varies, the value of concept B varies accordingly Process theory Concepts are events that either occur or don’t occur Event A is required in order for event B to subsequently occur Systems theory Concepts are systems or sub-systems within other systems If concept A is a system and concept B is a subsystem of A, then the theory could be that when the state (values of properties) of B changes, then the state of A (values of properties other than B) changes accordingly

Theory-mining review (TMR) A literature review that extracts theoretical concepts from its constituent primary studies as a key aspect of the synthesis It might also extract and synthesize: the relationships between these concepts the explanations of these relationships the bounding contextual conditions of the extracted relationships. At the very minimum, it extracts concepts

Why is theory important? Theory is important for theory-oriented disciplines Natural sciences Medicine Physics, biology, chemistry Social sciences Management, information systems, communication studies Public health, nursing Education, sociology, psychology It is not so important for non-theory oriented disciplines Engineering and computer science Humanities (arts, literature, law, etc.)

Rigour vs. Relevance Rigour Relevance Theory-oriented disciplines Following detailed, appropriate methods to assure that your results give reliable, confident knowledge Relevance Asking research questions that give meaningful answers to ordinary people (non-scholars) Theory-oriented disciplines Rigour is paramount; relevance sometimes suffers Non-theory-oriented disciplines Relevance is paramount; rigour sometimes suffers Rigorous and Relevant We should strive to always be both rigorous and relevant Always start with relevant research questions (questions that matter to practitioners, not just scholars) Use rigorous research methods to answer these questions, so that people (both scholars and practitioners) have confidence in the results

Why are theory-mining reviews important? For theory-oriented disciplines, TMRs help extract and synthesize the core elements of theory from primary studies In my study of 98 lit reviews in information systems, TMRs had more citations than other kinds of literature reviews Even when controlling for journal reputation Even when controlling for author seniority Citations can indicate how valuable a lit review is to other scholars

Three kinds of TMR Theory-landscaping review (TLR) Focuses on a theme, a topic or a single concept as its scope, but not on theoretical relationships. Theory-contending review (TCR) Focuses explicitly on theoretical relationships (at least two concepts) as the scope of the review, but without empirical verification Theory-testing review (TTR) Tests theoretical relationships (at least two concepts) using empirical secondary data

Dissemination targets: where can you publish your TMR? Standalone lit review Protocol: Conference poster session, or regular article Pilot study: Conference regular article Completed study: Journal article Parts of a doctoral thesis TLR: literature review chapter TCR: literature review chapter plus theory development chapter TTR: Usually only if doctoral dissertation is a meta-analysis, in which case the entire dissertation would be a TTR

For a standalone review, what kind of TMR should I do?

Can I frame my research topic in theory structure? What kinds of concepts am I dealing with? (Burton-Jones et al. 2014) Variance: values that vary Process: events in a temporal order Systems: systems with sub-systems None of the above: themes, topics, ideas Do not artificially or prematurely set bounding conditions For example, be wary about building theory that supposedly applies only to one country or region Why does your theory apply only in that country, but not in others? In contrast, it is perfectly fine to propose a general theory, and then test it only in one country In this case you would state your limited scope of testing as a limitation, but you are not obligated to argue a different theory In particular, if the only thing you do is apply existing theory to a new country, that is not a significant contribution to knowledge

Sources Burton-Jones, A., McLean, E.R., and Monod, E. Theoretical perspectives in IS research: from variance and process to conceptual latitude and conceptual fit. European Journal of Information Systems, Forthcoming, (September 2014). Fink, A. (2005). Conducting research literature reviews: From the Internet to paper (2nd ed.). Thousand Oaks, CA: Sage. Okoli, Chitu (TMR). The view from giants’ shoulders: guidelines for developing theory with theory-mining literature reviews. Working paper, 2015.