FOCUS GROUP OR INTERVENTION? PUBLIC RESPONSES TO COST EFFECTIVENESS INFORMATION Shoshanna Sofaer, Dr.P.H. School of Public Affairs Baruch College

June 3, 2007Shoshanna Sofaer, Dr.P.H.2 OVERVIEW Background Research Questions Design for study public responses Methodological issues

June 3, 2007Shoshanna Sofaer, Dr.P.H.3 BACKGROUND Other members of the Research Team: Marthe R. Gold, M.D., M.P.H., Sophie Davis Medical College, City University of New York Taryn Siegelberg, M.P.A., Sophie Davis Medical College, City University of New York Peter Franks, Ph.D., UC Davis (statistical analysis)

June 3, 2007Shoshanna Sofaer, Dr.P.H.4 BACKGROUND Cost effectiveness analysis is used in other developed countries as an input to coverage decisions, particularly for new technologies. In the US, it has been assumed that it will be unacceptable to the public and many senior officials have made it clear that it will not be used, for example, in making decisions for the Medicare program.

June 3, 2007Shoshanna Sofaer, Dr.P.H.5 BACKGROUND Concerns about the use of cost effectiveness analysis (CEA) seem to be related to three kinds of concerns: The sense that it will be viewed as a form of rationing, and therefore unacceptable on its face The backlash against managed care plans which were perceived as making individual decisions about “medical necessity” on the basis of the cost of the treatments and tests in question The real ethical and normative issues that inevitably arise when using CEA

June 3, 2007Shoshanna Sofaer, Dr.P.H.6 BACKGROUND However, there has been virtually no empirical research conducted to confirm the existence and nature of concerns about CEA The purpose of this studies, therefore, was to conduct exploratory empirical research to examine these issues

June 3, 2007Shoshanna Sofaer, Dr.P.H.7 RESEARCH QUESTIONS Our first study conducted (with support from AHRQ) focused on the responses of members of the public. We will discuss this study today. A second study conducted (with direct support from the California Health Care Foundation and indirect support from The Commonwealth Fund’s Harkness Fellowship Program) focused on the responses of key decision makers in the public and private sectors in California. We are presenting findings of that study on Tuesday morning.

June 3, 2007Shoshanna Sofaer, Dr.P.H.8 RESEARCH QUESTIONS With respect to the public, questions were: Can members of the public gain a basic understanding of CEA? Do members of the public recognize, and how do they respond to, ethical and normative issues related to CEA? Are members of the public willing and able to take on a “social decision maker” role and set hypothetical priorities for coverage of particular treatments by the Medicare program?

June 3, 2007Shoshanna Sofaer, Dr.P.H.9 RESEARCH QUESTIONS More questions for the public: When presented with cost effectiveness information about a range of treatments for particular conditions, are people willing and able to set priorities such that some are not covered? Do their priorities change when they receive information not only about treatment effectiveness, but about treatment costs, cost effectiveness, and impact on the Medicare budget? How do they respond to the opportunity to be involved in this kind of deliberation on priorities for coverage?

June 3, 2007Shoshanna Sofaer, Dr.P.H.10 DESIGN & DATA COLLECTION Participants: members of the public chosen from the jury pool in Manhattan; demographically diverse; had to have very basic math skills Pair of focus groups/workshops, two weeks apart: six pairs, two with people whose education level was high school or less

June 3, 2007Shoshanna Sofaer, Dr.P.H.11 DESIGN & DATA COLLECTION First session: Presentation of information on health care costs in the US Discussion of possible drivers of health care costs -- ideas generated first from the group with additional ideas presented by researchers Presentation and discussion of what CEA is, with a focus on the calculation of Quality Adjusted Life Years (QALYs) and CE ratio

June 3, 2007Shoshanna Sofaer, Dr.P.H.12 DESIGN & DATA COLLECTION First session, continued: Presentation of hypothetical “choices” among different treatments or treatments for different groups of people designed to elicit ethnical and normative concerns that arise in the context of CEA, including The rule of rescue The concept of ‘fair innings” The issue of personal responsibility The potential for discrimination against those already quite ill or with disabilities The relative value of improvements in quality of life v. extension of life

June 3, 2007Shoshanna Sofaer, Dr.P.H.13 DESIGN & DATA COLLECTION At the end of the first session, participants given a list of 14 treatment/condition pairs, which were described in lay terms Descriptions incorporated available information about the effectiveness of each treatment with respect to the Medicare population Participants asked to put on a “social decision maker” hat and, as a homework assignment, identify 5 treatments they would definitely fund, 5 they would probably fund, and 4 they would not fund

June 3, 2007Shoshanna Sofaer, Dr.P.H.14 DESIGN & DATA COLLECTION Second session: Review of CEA Presentation of aggregate responses to the 14 treatment/condition pairs Discussion of areas of consensus and dissensus in the responses

June 3, 2007Shoshanna Sofaer, Dr.P.H.15 DESIGN & DATA COLLECTION Second session, continued: Distribution of information about the CE ratios associated with each of the 14 Second round of priority setting (individually) Elicitation of new priorities Discussion of reasons why people did and did not change their priorities in response to the information

June 3, 2007Shoshanna Sofaer, Dr.P.H.16 DESIGN & DATA COLLECTION Second session, continued: Distribution of information about the impact coverage of each treatment would have on the Medicare budget Charge to group to presume a constrained budget that would require that at least some treatments not be funded Identification of a member of the group as a facilitator for next segment, which was Attempt of group (with no participation from researchers) to build consensus on what to fund and not fund

June 3, 2007Shoshanna Sofaer, Dr.P.H.17 DESIGN & DATA COLLECTION Second session, continued: Final discussion, questions, etc. Quantitative component: Pre-group survey, with questions about attitudes drawn in part from existing public opinion polls on health care issues Post-group survey, with same questions, more questions about the use of CEA, plus questions about the group and the idea of having members of the public participate in setting coverage decisions in Medicare

June 3, 2007Shoshanna Sofaer, Dr.P.H.18 DESIGN & DATA COLLECTION Analysis: Groups videotaped, transcribed, and key sections coded using NVivo Surveys analyzed using traditional descriptive and inferential statistics (not n was not large) Calculation of the cost implications of priority setting decisions and changes in those decisions

June 3, 2007Shoshanna Sofaer, Dr.P.H.19 METHODOLOGICAL ISSUES In many ways, this looks like a focus group study, with these features: A more extensive pre-group survey Exercises using “stimulus materials” and that generated data that could be quantified A much more extensive post-group survey Thus, the process of data collection was HIGHLY integrated

June 3, 2007Shoshanna Sofaer, Dr.P.H.20 METHODOLOGICAL ISSUES However, a lot of “education” was done in these groups The education could be considered an “intervention” Even the discussions and the exercises could be considered an “intervention”

June 3, 2007Shoshanna Sofaer, Dr.P.H.21 METHODOLOGICAL ISSUES So in what ways is this a focus group study, and in what ways is this a study that uses key elements of focus group methodology as the basis for carrying out an intervention and measuring its impact? Note, however, that the impact is discerned not just through quantifiable results of surveys and exercises, but through classic qualitative analysis of discussions (both moderated and unmoderated)

June 3, 2007Shoshanna Sofaer, Dr.P.H.22 METHODOLOGICAL ISSUES Questions for the audience: Is this a qualitative or mixed method study? What is the potential of this kind of method within health services and policy research, especially vis a vis exploring the views of members of the public on key issues?