1 Strengths and Challenges of Action Research Carol VanDeusen Lukas, EdD QUERI Implementation Seminar June 26, 2008

2 Seminar objectives Outline the concepts and constructs of action research Outline the concepts and constructs of action research Explore the challenges of conducting action research Explore the challenges of conducting action research

3 Action research… “engages the researcher in an explicit program to develop new solutions that alter existing practice and then test the feasibility and properties of the innovation.” “engages the researcher in an explicit program to develop new solutions that alter existing practice and then test the feasibility and properties of the innovation.” Kaplan, RS. Innovation action research: creating new management theory and practice. Journal of Management Accounting Research :89.

4 Traditional research design RCT is gold standard RCT is gold standard Detailed replication is expected/assumed Detailed replication is expected/assumed intention to treat design intention to treat design Context is controlled away Context is controlled away Sites/subjects are recipients of treatment Sites/subjects are recipients of treatment

5 Contrast with action research RCT is gold standard RCT is gold standard Rigorously test theories in natural organizational setting Rigorously test theories in natural organizational setting Detailed replication is assumed - intention to treat design Detailed replication is assumed - intention to treat design Implement innovation while tailoring to meet local needs Implement innovation while tailoring to meet local needs Context is controlled away Context is controlled away Reflect in-depth on system being changed Reflect in-depth on system being changed Sites/subjects are recipients of treatment Sites/subjects are recipients of treatment Collaboration of researchers with managers Collaboration of researchers with managers

6 Key features of action research Rigorously test theories in natural organizational setting Rigorously test theories in natural organizational setting Implement innovation while tailoring to meet local needs Implement innovation while tailoring to meet local needs Reflect in-depth on system being changed Reflect in-depth on system being changed Collaboration of researchers with managers Collaboration of researchers with managers

7 But action research is not easy Challenges & strategies for addressing examined in context of a case example

8 Case example “Strengthening organizations to implement evidence-based clinical practices” (IMV ) “Strengthening organizations to implement evidence-based clinical practices” (IMV ) Collaboration of Network leaders in VISNs 1, 10, 23 and COLMR Collaboration of Network leaders in VISNs 1, 10, 23 and COLMR

9 Project team Carol VanDeusen Lukas, EdD Carol VanDeusen Lukas, EdD Ryann L. Engle, MPH Ryann L. Engle, MPH Sally K. Holmes, MBA Sally K. Holmes, MBA Marjorie Nealon-Seibert, MBA Marjorie Nealon-Seibert, MBA Vicky Parker, DBA Vicky Parker, DBA Michael Shwartz, PhD Michael Shwartz, PhD Jennifer L. Sullivan, PhD Jennifer L. Sullivan, PhD Preet Ghuman Preet Ghuman Jason Silver Jason Silver Jeannette Chirico-Post, MD Jeannette Chirico-Post, MD Michael Miller, MD, PhD Michael Miller, MD, PhD Robert A. Petzel, MD Robert A. Petzel, MD Barry Graham, MD Barry Graham, MD Sheila Gelman, MD, MPA Sheila Gelman, MD, MPA

10 Research objectives Test effectiveness of an organizational model (VISN 23) vs. data feedback model (VISNs 1 & 10) in improving compliance with evidence-based guidelines Test effectiveness of an organizational model (VISN 23) vs. data feedback model (VISNs 1 & 10) in improving compliance with evidence-based guidelines Identify factors that affect success of model implementation and effectiveness Identify factors that affect success of model implementation and effectiveness

11 Organizational model for improving evidence-based clinical practices

12 Study design: Intervention in VISN 23 Innovation = the organizational model Innovation = the organizational model Clinical focus is hand hygiene Clinical focus is hand hygiene Project aim is building organizational capabilities Project aim is building organizational capabilities Hand hygiene data collection & reporting Hand hygiene data collection & reporting External facilitation to medical centers External facilitation to medical centers Shared learning group Shared learning group Leadership consortium Leadership consortium

13 Study design: Intervention in VISNs 1 and 10 Innovation = data feedback Innovation = data feedback Clinical focus is hand hygiene Clinical focus is hand hygiene Hand hygiene data collection & reporting Hand hygiene data collection & reporting Annual interviews Annual interviews

14 Study design: Evaluation Is the organizational model implemented with high fidelity to the model design? Is the organizational model implemented with high fidelity to the model design? Is the organizational model when implemented with fidelity more effective than comparisons in improving HH compliance? Is the organizational model when implemented with fidelity more effective than comparisons in improving HH compliance? Why is the organizational model successful or not successful in terms of implementation & effectiveness? Why is the organizational model successful or not successful in terms of implementation & effectiveness? Are the activities used by the research team to introduce & support the organizational model successful? Are the activities used by the research team to introduce & support the organizational model successful?

15 Challenges of action research: Lessons from the case example Rolling out the intervention Rolling out the intervention Operationally defining the innovation Operationally defining the innovation Engaging leadership Engaging leadership Engaging middle & front-line staff Engaging middle & front-line staff Conducting rigorous research Conducting rigorous research Measuring implementation fidelity Measuring implementation fidelity Measuring performance Measuring performance Documenting intervention, context and changes Documenting intervention, context and changes Balancing roles as researcher and change facilitator Balancing roles as researcher and change facilitator

16 Rolling out the intervention 1: Operationally defining the innovation Communicating key elements Communicating key elements Tailoring innovation to each site Tailoring innovation to each site Reinforcing the innovation Reinforcing the innovation Case strategies Preliminary data as part of introduction Preliminary data as part of introduction Implementation plan Implementation plan Regular feedback in site visits Regular feedback in site visits

17 Rolling out the intervention 2: Engaging leadership Reaching multiple levels of leadership Reaching multiple levels of leadership Grappling with competing priorities Grappling with competing priorities Clarifying leadership roles Clarifying leadership roles Case strategies Model introduction at ELC Model introduction at ELC Leadership consortium across network Leadership consortium across network Quadrad champion Quadrad champion Communication between visits Communication between visits

18 Rolling out the intervention 3: Engaging middle & front-line staff Staff skills and buy in Staff skills and buy in Site experience with systematic quality improvement and redesign teams Site experience with systematic quality improvement and redesign teams Case strategies Redesign teams Redesign teams Improvement efforts linked with larger organization - quadrad champion Improvement efforts linked with larger organization - quadrad champion Shared learning group Shared learning group NCOD consultation NCOD consultation

19 Conducting rigorous research 1: Measuring implementation fidelity Defining key elements of the innovation Defining key elements of the innovation Developing tools Developing tools Case strategies Research teams document and rate fidelity Research teams document and rate fidelity

20 Conducting rigorous research 2: Measuring performance Developing local measures Developing local measures Systematic Systematic Consistent Consistent Sustainable Sustainable Case strategy Work with individual sites Work with individual sites Measurement subgroup of shared learning group Measurement subgroup of shared learning group VISN coordination of data collection VISN coordination of data collection

21 Conducting rigorous research 3: Documenting intervention, context & changes Research team facilitation and support Research team facilitation and support Changes in innovation Changes in innovation Organizational context Organizational context Case strategies Notes on all interviews, meetings, telephone conversations, Notes on all interviews, meetings, telephone conversations, Impressions journal Impressions journal

22 Balancing roles as researcher & change facilitator Researchers are used to being objective observers Researchers are used to being objective observers How do you balance being actively involved in the project with stepping back and letting the project process happen naturally? How do you balance being actively involved in the project with stepping back and letting the project process happen naturally?