Rigor AND Relevance Kurt C. Stange, MD, PhD Editor, Annals of Family Medicine Gertrude Donnelly Hess, MD Professor of Oncology Research Professor of Family Medicine, Epidemiology & Biostatistics, Sociology and Oncology Case Western Reserve University

Overview Reflections on Ornstein & Ruhe presentations The problem An integrative epistemology A transdisciplinary whole systems approach QI “laboratories” - PBRNs Participatory approaches to both R & D Mixed methods – quantitative & qualitative A complexity science underpinning Putting it together

Reflections on Presentations by Ornstein & Ruhe Responding to common stimuli Intervention approaches Evaluation Methods

Responding to Common Stimuli Real-world primary care –Incompatible reimbursement/business model –Competing demands / opportunities –PBRNs Diversity –Undesirable (low rates of effective practices) –Desirable (local adaptation) Melding research & quality improvement –Resistance to randomization Peer review / categorical funding

Intervention Approaches Multifaceted –Multiple processes, tools & outcomes –Moving toward multilevel Chronic Care Model Individualization of shared best practices –Outside facilitation / consultation –Shared learning Complexity science principles

Evaluation Approaches Mixed methods designs –Group RCT → pre/post design –Concurrent qualitative process evaluation Tailoring of intervention Measures –Quantitative process & outcome Inductive approach –Observation, interviews –Discovery of what works locally

The Problem Rigor vs. relevance Internal vs. external validity Isolation of a phenomenon from context (so that it can be more rigorously studied) when context matters Our methods, theories, world view don’t match the problems or apparent solutions

4 Ways of Knowing Adapted from: Wilber, K. Sex, Ecology, Spirituality. 1995/2000, Boston: Shambhala Publications, Inc. Wilber, K. A Brief History of Everything. 1996, Boston: Shambhala Publications, Inc. Inner Reality Outer Reality Individual“I” “It” Collective “We”“It”

4 Ways of Knowing About Health & Health Care Adapted from: Stange KC, Miller WL, McWhinney I. Developing the knowledge base of family practice. Fam Med. 2001; 33(4): Clinician, Patient, Worker, Policymaker 4 Disease, Treatment 2 Family, Practice, Community 3 Systems, Organization

4 Ways of Knowing About Health & Health Care Borders 1-2 Relationship 2-3 Justice 3-4 Prioritization 4-1 Information mastery 1-3 Collaboration 2-4 Illness 1 Clinician, patient, worker, policymaker 4 Disease, Treatment 2 Family, Practice, Community 3 Systems, Organization

Multiple Ways of Knowing SYNTHESIS of ways of knowing The craft of policy & general practice is the integration and application of knowledge of biomedicine, health care systems, individuals, families, communities & self. 1-2 Relationship 2-3 Justice 3-4 Prioritization 4-1 Information mastery 1-3 Collaboration 2-4 Illness 1 Clinician, patient, worker, policymaker 4 Disease, Treatment 2 Family, Practice, Community 3 Systems, Organization BORDERS among ways of knowing

Multiple Ways of Knowing

Honoring Different Ways of Knowing Acknowledging different perspectives –In planning studies –In interpreting findings Multiple groups working separately but communicating Paying attention to (or at least considering) other ways of knowing, even if only working on one.

Integrating Different Ways of Knowing Sequential studies Simultaneous studies –Multiple viewpoints –Multiple methods Studies of the border regions Studies of the integrative function

Transdisciplinary, Whole Systems Collaboration can create abundance by sharing scarcity. Crabtree BF, Miller WL, Adison RB, Gilchrist VJ, Kuzel A. Exploring Collaborative Research in Primary Care. Thousand Oaks, California: Sage Publications; 1994.

Leadership for Collaborative Research, Development and Quality Improvement Whole system –Top down –Bottom-up Thomas P, Griffiths F, Kai J, O'Dwyer A. Networks for research in primary health care. BMJ. 2001;322: Thomas P. Integrative Primary Health Care: Leading, Managing, Facilitation. London: Radcliff Press, 2006.

A Typology of Collaboration Multidisciplinary Interdisciplinary Transdisciplinary Crabtree BF, Miller WL, Adison RB, Gilchrist VJ, Kuzel A. Exploring Collaborative Research in Primary Care. Thousand Oaks, California: Sage Publications; 1994.

Multidisciplinary Research Multiple disciplines Each contributes their piece to solving a problem Like an edited book or separate presentations by multiple experts

Interdisciplinary Research A conversation between and among disciplines Working together on solving a common problem Like a collaborative health care team

Transdisciplinary Research A sustained conversation across and beyond disciplinary boundaries Creates a new shared language Such as the emergence of family systems medicine

6 Stages of Collaboration Acceptance / validation Shared expectations Declaring group process Action consensus Common space Sustained common action

Barriers to Collaborative Process Rhetorical stones –Power heaping –Shaming –Jargon hurling Powerful hegemony –Rationality vs. / and –Wonder, confession gratitude, receptivity to gift and mystery Tension between pragmatism and reflection Tension between individual & systems focus

Tactics for Advancing the Process Brainstorming Humor Storytelling Silence and time out for play

The Actual Process Non-linear Blurring and blending of levels Back and forth between levels Destabilized by time pressure Enhanced by flexibility, tact, patience and persistence

Dangers of Collaborative Research Relationships Sloppiness from training down Suppression of individuality Cultism Political nature of groups Minimizing these requires self- reflection and challenging

6 Ingredients for Successful Collaborations Linkage perspective Local context and action Problem-focused Appropriate methods Critical multiplism Coordination by a generalist researcher

Leadership for Collaborative Research In the face of change and uncertainty –Animation Provide initial direction Encourages updating Facilitates respectful interaction - trust, trustworthiness –Improvisation A hunch held lightly is a direction to be followed, not a decision to be defended –Lightness “I don’t know” –Authentication Sensemaking –Learning Weick KE. Leadership as the legitimation of doubt. In Bennis W, Spreitzer GM, Cummings TG, (eds). The Future of Leadership. San Francisco: Jossey-Bass, 2001.

R &D &QI “Laboratories” Whole systems e.g. HMO Research Network NIH Research Center Model e.g. Cancer Centers Practice-based research networks (PBRNs)

PBRNs Affiliated practices devoted to patient care Often academic or other partners Engage frontlines wisdom to –Develop or frame questions –Gather data –Interpret findings –Implement findings More generalizable patient populations, & theoretically, more transportable research

Primary Care PBRNs Laboratories for primary care research 111 networks in 44 states Translate research into practice Translate practice into research Nutting P, Beasley J, Werner J. Practice-based research networks answer primary care questions. JAMA. 1999;281: Thomas P, Griffiths F, Kai J, O'Dwyer A. Networks for research in primary health care. BMJ. 2001;322: Lanier D. Primary care practice-based research comes of age in the United States. Ann Fam Med. 2005; 3(suppl 1):S2-S4.

Primary Care PBRNs Answer important questions Challenging current environment With support, ready for prime-time Answer to the call of the NIH Roadmap Contemporary Challenges for Practice-Based Research Networks. Ann Fam Med; 2005; 3 (suppl 1). Nutting PA, Stange KC, eds. Prescription for Health: Changing Primary Care Practice to Foster Healthy Behaviors. Ann Fam Med; 2005; 3 (suppl 2). Full text free at: Green LA, Dovey SM. Practice based primary care research networks. They work and are ready for full development and support. BMJ.2001;322: Nutting PA, Beasley J, Werner JJ. Practice-based research networks answer primary care questions. JAMA. 1999;281:

The Ecology of Medical Care

Community Participatory Research Knowledge, resources, involvement of communities are key to success of research and its transportability & sustainability 3 primary features –Collaboration –Mutual education –Acting on results developed from research questions relevant to the community Macaulay AC, Commanda LE, Freeman WL, et al. Participatory research maximises community and lay involvement. BMJ. 1999;319:

Facilitation of Participatory Research Continuity Timeliness Flexibility Clear and explicit expectations Appropriate & instructive accountability A vision for participatory research Tailored technical assistance Enhancing partner collaboration & support Mercer SL, MacDonald G, Green LW. Participatory research and evaluation: from best practices for all states to achievable practices within each state in the context of the Master Settlement Agreement. Health Promot Pract. 2004;(3 Suppl):167S-178S.

Multimethod Approaches Increasingly accepted Efficient for generating new knowledge Stange KC, Zyzanski SJ. Integrating qualitative and quantitative research methods. Fam Med, 1989; 21: Stange KC, Miller WL, Crabtree BF, O’Connor PJ, Zyzanski SJ. Multimethod research: Approaches for integrating qualitative and quantitative methods. J Gen Intern Med. 1994; 9: Crabtree BF, Miller WL. Doing Qualitative Research. 2nd Ed. Thousand Oaks, California: Sage Publications, Crabtree BF, Miller WL, Stange KC. Understanding practice from the ground up. J Fam Pract, 2001; 50: Borkan JM. Mixed methods studies: a foundation for primary care research. Ann Fam Med. 2004;2(1):4-6 NCI Conference on Mixed Methods Research, July, 2004, Denver, CO.

Multimethod Research Quantitative methods Counting descriptions Testing a priori hypotheses Seek to isolate phenomenon from context Qualitative methods Rich descriptions Discovery; testing evolving hypotheses Seek to understand meaning and context Integrated use Qualitative, then quantitative Quantitative, then qualitative Simultaneous Stange KC, Miller WL, Crabtree BF, O’Connor PJ, Zyzanski SJ. Multimethod research: Approaches for integrating qualitative and quantitative methods. J Gen Intern Med. 1994; 9:

Mixed methods allow you to have your cake and eat it too.

Additional Theoretical Underpinnings Complexity science principles Chronic care model Re-Aim Framework for complex interventions Gree nhalgh

Practices as Complex Adaptive Systems Complex behavior emerges from relationships among agents Simple rules Recurrent patterns Co-evolution Dependence on initial conditions Non-linearity Strategies for intervention Joining Transforming Learning Miller WL, Crabtree BF, McDaniel RA, Stange KC. Understanding primary care practice: A complexity model of change. J Fam Pract, 1998; 46: Miller WL, McDaniel RA, Crabtree BF, Stange KC. Practice Jazz: Understanding variation in family practices using complexity science. J Fam Pract, 2001; 50: Stroebel CK, McDaniel RR Jr, Crabtree BF, Miller WL, Nutting PA, Stange KC. Using complexity science to inform a reflective practice improvement process. Jt Comm J Qual Patient Saf, 2005; 31:

Three Key Insights Practices are co-creative participants in dynamic fitness landscapes. There are multiple ways to achieve effective health care delivery in practice. The best strategies for improving practice pay attention to improving relationships among stakeholders. McDaniel RA, 2005

Chronic Care Model Community Health care system Resources, policies & organization –Self-management support –Delivery system design –Decision support –Clinical information systems Interaction –Informed, activated patient –Prepared, proactive practice team Wagner EH. Chronic disease management: What will it take to improve care for chronic illness: Effective Clinical Practice. 1998;1:2-4. Glasgow RE, Orleans CT, Wagner EH, Curry SJ, Solberg LI. Does the chronic care model serve also as a template for improving prevention? Millbank Q. 2001;79:

Re-Aim Reach Efficacy/effectiveness Adoption Implementation Maintenance Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. 2001;44: Glasgow R, Magid D, Beck A, Ritzwoller D, Estabrooks P. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care. 2005;43:

Design & Evaluation of Complex Interventions Campbell M, Fitzpatrick R, Haines A, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):

Next Generation of Diffusion of Health Service Innovations Theory-driven Process rather than ‘package’ oriented Ecological Addressed with common definitions, measures, tools Collaborative & coordinated Multidisciplinary & multimethod Meticulously detailed Participatory Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):

An Incremental Approach Work on or pay attention to multiple levels Pursue research, development & shared learning Develop participatory relationships that transcend single projects Integrate qualitative & quantitative methods Reconsider enabling the current dysfunctional system versus fostering its transformation