1. Study Objectives and Questions for Observational Comparative Effectiveness Research Prepared for: Agency for Healthcare Research and Quality (AHRQ) www.ahrq.gov

2. This presentation will:  Characterize the primary uses and users of the scientific evidence that will be generated by the study (stakeholders) and how the evidence may be used  Articulate the main study objectives in terms of a highly specific research question or set of related questions that the study will answer  Synthesize the literature and characterize the known effects of the exposures and interventions on patient outcomes  Provide a conceptual framework  Delineate study limitations that stakeholders and investigators are willing to accept a priori  Describe the meaningful magnitude of change in the outcomes of interest as defined by stakeholders Outline of Material

3.  A study’s objectives and questions form the foundation for research protocols.  All aspects of study design and analysis are based on objectives and questions stated in the protocol.  Stakeholders are individuals or organizations that use scientific evidence for decisionmaking and, therefore, have an interest in the results of new research.  Involvement of stakeholders in formulating the research question facilitates appropriate translation of the results. Introduction

4.  Clearly articulate decisions or actions for which stakeholders seek new scientific evidence  State purpose and time frame of the study  Describe context in which the decision will be made or actions will be taken  Describe how evidence will be generated  Clarify the number of analyses that will be necessary to generate evidence Identify Decisions, Decisionmakers, Actions, and Context

5.  Relevant questions to assess the current knowledge base for developing an observational comparative effectiveness research protocol:  What are the most relevant studies and guidelines about the interventions, and why are these studies relevant to the protocol (e.g., because of the study findings, time period conducted, populations studied, etc.)?  Are there differences in recommendations from clinical guidelines that would indicate clinical equipoise?  What else is known about the expected effects of the interventions based on current understanding of the pathophysiology of the targeted condition?  What do clinical experts say about gaps in current knowledge? Synthesize Current Knowledge Base

6.  Collaborate with stakeholders to determine major objectives  Describe potential relationships between intervention and important health outcomes  Enumerate all major assumptions that affect the conceptualization of the research problem  Apply scientific theory in designing the study protocol and developing the analytic plan  Develop a conceptual model or framework to guide implementation of the study Conceptualize the Research Problem

7.  Stage research in a way that allows for interim decisions and sequentially more rigorous studies  Below is a potential categorization for the stage of knowledge development. 1.Descriptive analysis 2.Hypothesis generation 3.Feasibility studies/proof of concept 4.Hypothesis supporting 5.Hypothesis testing Determine the Stage of Knowledge Development

8. Based on a conceptual model, define study questions using PICOTS:  Population: What is the patient population of interest? What subgroups will be considered in terms of age, sex, ethnicity, et cetera?  Intervention: What are the interventions of interest (drug, device, procedure, or test)?  Comparator: What are the alternatives? What is the gold standard? What is usual care? How does usual care vary? Are decisionmakers interested in superiority or noninferiority?  Outcomes: What are the outcomes of interest?  Timing: What is the time frame of interest for assessing outcomes? Are decisionmakers interested in short-term or long-term outcomes?  Setting: What is the clinical setting of interest? Defining and Refining Study Questions Using the PICOTS Framework

9.  The following are suggested questions to discuss with stakeholders to help elicit the amount of uncertainty that is acceptable to them.  What level of new scientific evidence is needed by the decisionmaker to make a decision or to take action?  What quality of evidence is needed for the decisionmaker to act?  What level of certainty of the outcome is needed by the decisionmaker(s)?  How specific does the evidence need to be?  Will decisions require the consensus of multiple parties? Discuss the Evidentiary Need and Uncertainty

10.  Discuss the magnitude of effect stakeholders believe represents a meaningful difference between treatment options a priori.  How do patients and other stakeholders define a meaningful difference between interventions?  How do previous studies and reviews define a meaningful difference?  Are patients and other stakeholders interested in superiority or noninferiority as it relates to decisionmaking? Specify Magnitude of Effect

11.  Study objectives and questions form the basis for observational comparative effectiveness research studies.  Establishing a study objective framework can help conceptualize the research problem and the design of a study.  Implementing the framework involves collaboration between investigators and stakeholders.  Challenges include difficulty identifying, engaging, or managing all interested stakeholders. Conclusion

12. Summary Checklist (1 of 2) GuidanceKey Considerations Characterize the primary uses and users of the scientific evidence that will be generated by the study (stakeholders) and how the evidence may be used Explain specific decisions or actions by stakeholders that will potentially be informed by the study results Describe the evidentiary need of the stakeholders Articulate the main study objectives in terms of a highly specific research question or set of related questions that the study will answer Write research questions by identifying the population, intervention, comparator, outcomes, timing, and settings (PICOTS) of interest to the decisionmakers Discuss operational definitions and measures to meet the study objectives with stakeholders Synthesize the literature and characterize the known effects of the exposures and interventions on patient outcomes

13. Summary Checklist (2 of 2) GuidanceKey Considerations Provide a conceptual framework Describe hypothesized relationships between interventions and outcomes and key covariates Include appropriate figures or diagrams as needed Delineate study limitations that stakeholders and investigators are willing to accept a priori Describe the meaningful magnitude of change in the outcomes of interest as defined by stakeholders Provide the rationale for why a particular difference is hypothesized to be meaningful Discuss differences that may exist among stakeholders in terms of what is meaningful to different stakeholders