September 20, 2012 CONDUCTING A SYSTEMATIC REVIEW: DEFINING A PROBLEM LITERATURE REVIEW

Stages of Systematic Review 1.Define the Problem 2.Literature Search 3.Data Evaluation 4.Data Analysis 5.Interpretation of Results 6.Presentation of Results

Define the Problem What research evidence will be relevant to the problem or hypothesis of interest in the review? Define variables of interest Define how variables are related (can be a hypothesis) Define search criteria

Define the Problem: Defining Variables Conceptual vs Operational Conceptual – describes qualities of the variable in general or qualitative terms Example: Intelligence – “a capacity for learning, reasoning, understanding, and similar forms of mental activity” Operational – describes observable events used to measure the variable Example: Intelligence – Intelligence Quotient (IQ) score

Defining Variables: Primary research vs systematic review Primary research: conceptional and operational definitions must be precisely defined before experiment Systematic review: can begin with a broad conceptual definition and known operational measures and further define during course of review

Define the Problem: Relationship between variables How do the conceptual and operational definitions fit together in the literature? Are there multiple operational definitions for one conceptual variable? Example: intelligence may be measured by many tests (IQ, ACT, SAT, GRE)

Define the Problem: Determine Appropriate Research Designs 1.Should the results of the research be expressed in numbers or narrative? 2.Is the problem or hypothesis seeking to uncover a description of an event, an association between events, or an explanation of an event? 3.Does the problem or hypothesis seek to understand how a process unfolds within an individual unit over time, or what is associated with or explains variation between units or groups of units?

Descriptive: ask what is happening Measurement or description of characteristics of an event Example: reporting of mean IQ score in Tufts freshmen Associations: what events happen together Measurement of a correlation Example: is IQ score in Tufts freshmen associated with major? Explanations: explain why an event happens Construction of a predictive model Example: modeling predictive factors in IQ score in Tufts freshmen Define the Problem: Determine Appropriate Research Designs

Literature Search What procedures should be used to find relevant research? Identify sources for research Identify search terms used

Literature Search: Sources and Search Methods Reference databases (pubmed, etc) Footnote chasing Consultation with colleagues Library browsing Citation searches

Literature Search: Search Terms Begin search with broad conceptual definitions as search terms Want to identify largest number of potentially relevant articles Narrow search using operational definitions later If you find numerous operational definitions in initial search Which operational definitions to include? Example: using term intelligence instead of searching for IQ, SAT, GRE

Develop exhaustive list of terms used for conceptual definition Consider different levels of a term: scientific vs lay person Consider synonyms Consider spelling in different countries (estrus vs oestrus) Terms vary by source Examples? Literature Search: Search Terms

Logic Models A systematic, visual representation of a plan

Logic Model for Systematic Review InputActivitiesOutputsOutcomeImpact Based on definition of problem: Variables Search terms Literature search Data evaluation Data analysis Quantitative or qualitative results of review Conclusions Resolve conflict New information for field Drive new research Change in policy

Questions on Articles for Discussion 1.How do the authors define the research problem? 2.What variable(s) are defined in the review? a.Conceptual definition b.Operational definition c.Effect vs outome 3.Describe the review’s search methodology a.Search terms – exhaustive? b.Sources of information for review c.Study eligibility – types of studies included