Research in the Real World: Rebecca Etz Rebecca Etz Deborah Cohen Deborah Cohen Nicole Isaacson Nicole Isaacson Bijal Balasubramanian Bijal Balasubramanian Benjamin Crabtree Benjamin Crabtree Alfred Tallia Alfred Tallia Lessons Learned from Prescription for Health

Let’s Get Real Translation Translation Realize the shift from efficacy to effectiveness Realize the shift from efficacy to effectiveness Dissemination Dissemination Capture context and strategy as well as outcomes Capture context and strategy as well as outcomes Public Health Impact Public Health Impact Create partnerships in the community Create partnerships in the community

Conceptualization Conceptualization Studying implementation as way to evaluate what ‘works’ in real-life setting (i.e. effectiveness) Studying implementation as way to evaluate what ‘works’ in real-life setting (i.e. effectiveness) Methods Used Methods Used Practice Surveys Practice Surveys Site Visits/Interviews Site Visits/Interviews Online diary Online diary Evaluation of P4H The Analysis Team’s Focus

Evaluation The Analysis Team’s Focus Data Analysis – Qualitative Data Data Analysis – Qualitative Data 1 st : Read the diary data 2 nd : Read it again 3 rd : Really read the data

Evaluation The Analysis Team’s Focus Data Analysis – Quantitative Data Data Analysis – Quantitative Data 1 st : Find missing data 2 nd : Fix any inconsistencies 3 rd : Convert numbers into words

Let’s Get Real Translation Realize the shift from efficacy to effectiveness Realize the shift from efficacy to effectiveness PBRNs are an ideal environment PBRNs are an ideal environment Research oriented, but not unusual 85 practices (10 PBRNs) 92% Family medicine practices Staff turnover of 8-12%

PBRNs and Getting Real REAIM as our lens REAIM was proactively for the first time in practice-based, primary care research REAIM was proactively for the first time in practice-based, primary care research Across all 10 projects in round 2, P4H Across all 10 projects in round 2, P4H Reach, Effectiveness, Adoption, Implementation, Maintenance

PBRNs and Getting Real REAIM as our lens REAIM and our qualitative data allowed us to ask different kinds of questions: How do we take something that we know works, or that we think will work, and put that thing into practice? How do we take something that we know works, or that we think will work, and put that thing into practice? How did you get it to work? How did you get it to work? Why did it work and can we get it to work elsewhere? Why did it work and can we get it to work elsewhere?

PBRNs and Getting Real REAIM as our lens Two themes emerged from the diary data – the tensions between… Flexibility & Fidelity …an issue of translation …an issue of translation Quality Improvement & Research …an issue of dissemination …an issue of dissemination

PBRNs and Getting Real Fidelity vs Flexibility Fidelity and Flexibility In the primary care setting, interventions are not plug and play In the primary care setting, interventions are not plug and play They are tailored to the contours of the practice They are tailored to the contours of the practice “… Dr made the comment that we need to come up with something that will work at a 1 nurse office like hers”

PBRNs and Getting Real Fidelity vs Flexibility Flexibility is a reflection of the inescapable realness of the real world ‘ The other striking thing is the varied ways that the practices are implementing the HRA: Keep it at reception, given to patients at check in Keep it at reception, given to patients at check in Give it to one staff member, he administers all Give it to one staff member, he administers all Given to all patients of the in house diabetes educator Given to all patients of the in house diabetes educator Keep it at nurses station, used primarily by ‘one care giving team’ Keep it at nurses station, used primarily by ‘one care giving team’

PBRNs and Getting Real Fidelity vs Flexibility Understanding how research is integrated into real life practice is essential Understanding how research is integrated into real life practice is essential PBRNs facilitate that translation with all the muckiness that it implies PBRNs facilitate that translation with all the muckiness that it implies

Let’s Get Real Translation Realize the shift from efficacy to effectivenessRealize the shift from efficacy to effectivenessDissemination Capture context and strategy as well as outcomes Capture context and strategy as well as outcomes Share solutions for navigating the tension between QI and research Share solutions for navigating the tension between QI and research

PBRNs and Getting Real Quality Improvement vs Research Research is about establishing/following protocols, establishing/following protocols, assessing outcomes Enrolling the ‘right’ type and number of patients Enrolling the ‘right’ type and number of patients Having a bounded data collection period Having a bounded data collection period Securing funds to pay for research and sometimes the intervention Securing funds to pay for research and sometimes the intervention Maintaining fidelity – implementing the intervention according to an established protocol Maintaining fidelity – implementing the intervention according to an established protocol

PBRNs and Getting Real Quality Improvement vs Research Quality Improvement is about understanding organization and setting Tailoring improvement to fit the protocols and procedures of the practice Tailoring improvement to fit the protocols and procedures of the practice Adapting to setting/context, allowing for flexibility Adapting to setting/context, allowing for flexibility Institutionalizing the improvement, sustainability Institutionalizing the improvement, sustainability

PBRNs and Getting Real Quality Improvement vs Research Front line PCPs are overwhelmed – this is reality Front line PCPs are overwhelmed – this is reality Choices for recruitment are often about choosing practices positioned for improvement Choices for recruitment are often about choosing practices positioned for improvement PBRNs allow us to see how those that actually take the change on are able to make it happen PBRNs allow us to see how those that actually take the change on are able to make it happen

PBRNs and Getting Real Quality Improvement vs Research On the ground, team members manage patient expectations “[Staff] report [this] is the first thing the patient asks: what do I get for participation?” Projects that were seamless – where increased service was in the foreground, research in the background – faired best

PBRNs and Getting Real Quality Improvement vs Research QI is ‘institutional’ where research is finite in terms of time & resources “We had practices take a two week break from referring patients … It has been more difficult than I had anticipated to get them back up and running...” “We tried to fine tune our message for the staff... That is, one patient enrollment per [staff member] per day.” PBRNs showed us institutionalization was manageable but involved sacrifices

Investigating with REAIM Where’s the Tension – QI and Research Front line practices don’t have a lot of slack Front line practices don’t have a lot of slack Research teams bring resources and it’s hard to hold up when they are gone Research teams bring resources and it’s hard to hold up when they are gone “This practice [does] prevention because it’s important, not because it’s reimbursed. [We] offered to keep the intervention on at this practice (but without free services). They pulled the plug on the project.”

Investigating with REAIM Where’s the Tension – QI and Research PBRN based research allows practices to build a line of inquiry that: shows the value of what they did shows the value of what they did wins funding enabling translation of research into practice wins funding enabling translation of research into practice PBRNs are an environment that makes translation happen

Let’s Get Real Translation Realize the shift from efficacy to effectivenessRealize the shift from efficacy to effectivenessDissemination Capture context and strategy, share solutionsCapture context and strategy, share solutions Public Health Impact Create partnerships in the community Create partnerships in the community Fill in the missing elements Fill in the missing elements

In the Real World … “linking to community resources is easier said than done” Primary Care Elements Identify patients at risk Identify patients at risk Capacity to refer patients Capacity to refer patients Know how to access community resources Know how to access community resources Community Resource Elements Availability Availability Affordability Affordability Accessibility Accessibility Perceived as valuable Perceived as valuable

Primary Care Elements Identifying Patients at Risk Many practices did not have processes in place to identify patients at risk…

Primary Care Elements Identifying Patients at Risk …Project teams were able to help them develop what was needed “[This] practice had, with the help of our facilitator, modified their vital signs template in the EHR (eClinical Works) to include smoking. His staff review smoking and update status at every visit. The inclusion of smoking as a vital sign plus verbal reminders by the nurses keeps the issue in front of [the doctor] and appears to have increased referrals to the Quit Line and his use of brief counseling strategies.”

Primary Care Elements Making the Referral Limited capacity for referrals was a barrier... Limited capacity for referrals was a barrier... “If we need to print referrals for faxing…, maybe someone in the [research team’s] office could do it (some concern about stressing the referral person...)” “If we need to print referrals for faxing…, maybe someone in the [research team’s] office could do it (some concern about stressing the referral person...)” Project teams offered different strategies for simplifying referrals Project teams offered different strategies for simplifying referrals “Having the direct real-time link to the designated outreach person…makes it very easy for the physician to make the referral… quick and easy.”

Primary Care Elements Making the Time Limited time was another barrier... Limited time was another barrier... “Talked with [Doctor] 6:30 am... had read material, [said] it sounded like a good study, as long as it didn’t take too much time would be happy to do it (like 30 seconds).” “Talked with [Doctor] 6:30 am... had read material, [said] it sounded like a good study, as long as it didn’t take too much time would be happy to do it (like 30 seconds).” And project teams helped practices met these time constraints And project teams helped practices met these time constraints “The rooming staff person reported that she did the P4H pop-up as part of the vital signs process. She reports it was not extra work/time.”

Primary Care Elements Knowing How to Access/Use Resources Community resources are fluid, many, and difficult to track Projects addressed this in a number of ways Projects addressed this in a number of ways Web-tools, HIT, resource cards People as portals to community resources Tapping into national, state, or health systems Building relationships with resources

Community Resource Elements Availability, Affordability… Many practices, particularly those in rural and low income areas, reported a scarcity of community resources Many practices, particularly those in rural and low income areas, reported a scarcity of community resources When resources did exist in communities, they were often not affordable When resources did exist in communities, they were often not affordable “A barrier for counseling is patients affording it - many would be interested in group visits if free, reason for interest in the study… ” “Currently they will refer to some nutritionists, but sporadic and dependant on what insurance will cover, more often for diabetic patients”

Community Resource Elements Accessibility Grounded and informed by patient need, projects teams took many creative approaches Offering in-house counseling Offering in-house counseling Developing website and IVR systems Developing website and IVR systems Offering and paying for branded services Offering and paying for branded services Leveraging their relationships with community resources Leveraging their relationships with community resources

PBRNs are Real P4H projects, and the use of REAIM, show PBRNs as an ideal setting in which to innovate Providing real-world effectiveness Providing real-world effectiveness Balancing internal and external validity Balancing internal and external validity Fostering proactive intervention designs Fostering proactive intervention designs Greater public appeal and rapid adoption Greater public appeal and rapid adoption Enabling view of potential public impact Enabling view of potential public impact Benefit of qualitative and quantitative in unison Benefit of qualitative and quantitative in unison

PBRNs are Real Translation Translation Integrate : Full integration of RE-AIM model into research and intervention design from project conception Integrate : Full integration of RE-AIM model into research and intervention design from project conception Dissemination Dissemination Context : qualitative and quantitative work together to suggest potential Context : qualitative and quantitative work together to suggest potential Public Health Impact Public Health Impact Adapt : Interventions must be responsive to site specific needs – functional, cultural, emotional Adapt : Interventions must be responsive to site specific needs – functional, cultural, emotional