1. What comprises ‘organizational context’ in implementation research
What comprises ‘organizational context’ in implementation research? A systematic integrative review
Shelly-Anne Li, MSc, PhD Student
Melanie Barwick, PhD, CPsych
Lianne Jeffs, RN, PhD
Bonnie Stevens, RN, PhD

2. What is organizational context?

3. What is the problem?
>> The multiple representations of organizational context in implementation research Organizational contextual factors have received a lot of attention by implementation researchers for over a decade. But there is currently little agreement on which factors represent the organizational context
While some implementation researchers described organizational context as the customs, practices, and values within an organization, others consider organizational context to comprise factors that are extrinsic to the organization  (e.g., sociopolitical environment, economic climate, and interorganizational collaborations.
The conceptualization of organizational context in implementation research appears to be abstract and vague, often being described as an all-encompassing concept with a multitude of different elements that reside within or external to an organization.
Several validated tools are reported to measure organizational context in implementation research. However, there is inconsistency in how these tools define and measure organizational context. Although the Alberta Context Tool and the Context Assessment Index were both informed by the PARiHS framework both tools measure different sets of features from the PARiHS framework. Furthermore, the developers of the Implementation Climate Scale (ICS: Erhart, Aarons, & Farahnak, 2014) consider climate as tantamount to organizational context (Ehrhart et al., 2014). In contrast, the CAI and ACT do not include climate in their measures. The Organizational Social Context Measure (OSCM: Glisson et al., 2008) considers organizational context as mainly comprised of culture and climate.
Multiple, distinct factors are reported to be representative of organizational context, suggesting that implementation researchers have not yet reached consensus on a core set of factors that represent this concept.
If org context is nebulous and there is little agreement on what it is exactly (e.g., some say internal factors, others say external factors), then what exactly has been acknowledged as being important?!
The importance of OC is acknowledged by researchers but we don’t even know what this thing is. Avoid saying importance has been acknowledged by; just say:
The issue surrounding the implementation, or the contextual factors that surrounds the implementation has received a lot of attention.
Even context, there’s so many different ways that they define context (in PARIHS)
The idea that there are issues surrounding the environment, contextual factors has been receiving increasing interest. This is something that has been growing (appreciation for this factor). Without being saying that this unified concept has been of interest.

4. Why is this review important?
To understand how organizational context is conceptualized and operationalized in implementation studies across health care settings We aim to identify factors that implementation researchers commonly explore when they use the term ‘organizational context’
The variability between researchers as to what factors constitute organizational context, and the lack of a consistent operational or conceptual definition of organizational context may have hindered the development of coherent and consistent evidence bases for organizational contextual factors that influence implementation efforts

5. Integrative review method
Combines findings in studies that use diverse methodologies (qualitative, quantitative)
To establish methodological rigor, we followed 5 phases of conducting integrative reviews:
problem identification
literature search
data evaluation
data analysis
results presentation
This will enhance our understanding of the terminology and taxonomy for organizational context by allowing for a synthesis of evidence on how organizational context is operationally defined, described and examined
Whittemore and Knafl’s (2005)

6. Methods – Literature search
MEDLINE, CINAHL, EMBASE, PsycINFO, Cochrane databases searched
Date parameters: 2005 to 2016
Hand-searched Implementation Science, reference lists of relevant articles
Searched grey literature
Inclusion and exclusion criteria applied
Inclusion:
Peer-reviewed, published literature
Study described, explained, measured, or explored the implementation phase of any evidence-based initiative in healthcare service settings
Abstract explicitly mentioned the term ‘organizational context’, ‘context of an organization’, ‘organizational contextual’ factors or characteristics as the main or one of the main objectives
Empirical studies of all research designs
Exclusion:
Study that do not refer specifically to organizational context in scope of the implementation phase
Do not concern the healthcare field
Editorials, opinions, discussions, textbooks, conceptual papers, articles that assessed validity and reliability of tools and frameworks
Languages other than English

7. Methods – Data evaluation
2 reviewers independently and in duplicate:
Screened the titles and abstracts
Performed data extraction and quality appraisal
Third-party adjudication
Quality appraisal: Mixed Methods Appraisal Tool (Pluye et al., 2011)
Citations that did not provide the abstract during the title and abstract screening were retained for the full-text screening to establish eligibility. Citations that met the eligibility criteria were included for full-text screening. The full-text screening followed the same calibration strategy as the title and abstract screening to ensure inter rater reliability.
Abstracted variables included: study design, country, objectives of the study, definition of organizational context, sample size and characteristics, setting, data collection method, the framework used (if applicable), and organizational context factors (main findings).
We held one meeting to resolve five discrepancies that occurred during full text abstraction. Three discrepancies were based on the framework that was used or created (Harvey, Jas, Walsche, & Skelcher, 2014; Livet, Courser & Wandersman 2008; Olstad, Downs, Raine, Berry, & Mccargar, 2011), and two were related to the data collection method (Livet et al., 2008; Olstad et al., 2011).
MMAT is a checklist that was developed to provide quality appraisal for qualitative, quantitative and mixed-methods studies

8. Methods – Data analysis
Qualitative content analysis
Data reduced to focus on relevant findings
Reading and rereading of the articles to create codes
Categories collapsed into themes
Themes used to further analyze and interpret findings

9. Results 12 relevant articles
50% guided by implementation framework
Organizational context reported as representation of multiple combinations of over 30 factors
Consolidated Framework for Implementation Research (CFIR: Damschroder et al., 2009), PARiHS (Kitson et al., 1998), Rogers’s Diffusion of Innovation (Rogers, 1962), and the Multi-Tiered Model of Diffusion of Complex Innovations (Greenhalgh, 2004). One study was guided by a framework developed by the study authors using conceptual mapping. One study used a non-implementation framework—Absorptive Capacity (Cohen, 1990). For authors who did not use frameworks, most of the organizational contextual factors that they identified could be found in the Inner Setting and Outer Setting domains of CFIR.

10. Results: Characteristics of included studies
Studies published between 2008 to 2016
Based in at least 6 different countries
Total of 625 participants
physicians, nurses, front-line clinicians, senior managers and allied health professionals from various healthcare settings

11. Is organizational context defined?
25%
Included definition of context
17%
Distinguished organizational context from social, political, and environmental contexts
Described organizational context as the inner setting of existing implementation research frameworks
25% included definition of context (Bergstrom et al., 2012; Harvey et al., 2015; Marchionni et al., 2008)
2 considered organizational context as a synonym for context (Bergstrom et al., 2012; Marchionni et al., 2008)
1 neither defined organizational context nor context, but used both terms synonymously (Powell et al., 2009)
17% distinguished organizational context from social, political, and environmental contexts (Hofstede et al., 2013; Krein et al., 2010)
17% described organizational context as the inner layer or inner setting of existing implementation research frameworks
Only one study provided definition of organizational context (Thomas et al., 2011)
One study provided definition of organizational context

12. Commonly reported organizational contextual factors
Resources (75%)
Leadership (67%)
Communication and networks (58%)
Culture (50%)
These factors reside in the inner settings domain of the Consolidated Framework for Implementation Research (CFIR)

13. Other organizational contextual factors reported

14. We must aim to advance clarity and achieve common terminology
Conclusions
Use of CFIR as a guiding framework for investigating organizational contextual factors
We must aim to advance clarity and achieve common terminology
Shared commonality in how we define, assess and measure organizational context adds to generalizability of studies.
The definitions and conceptualizations of ‘organizational context’ are broad and somewhat vague. A handful of implementation frameworks are beginning to consider the salience of context, and some frameworks have begun to categorize ‘context’ into inner contexts and outer contexts to demonstrate situational factors residing within and external from the organization under study (e.g., CFIR: Damschroder et al., 2009; i-PARiHS: Harvey & Kitson, 2016). However, the situational factors described in these two broad categories vary from framework to framework.
Researchers can consider using the CFIR as a guiding framework to investigate the organizational contextual factors in implementation projects
This integrative review adds to the conceptual clarity on organizational context, and advances our understanding on how the term organizational context is utilized in implementation research.
Shared commonality in how we define, assess and measure organizational context adds to the generalizability of future studies.
By agreeing on a core set of organizational contextual factors that frequently influence implementation, researchers can focus their research efforts to further understand these factors.
We must aim to advance clarity and achieve common terminology.
By facilitating the appropriate application of the term organizational context relevant to implementation studies, we foster cross-disciplinary dialogue among implementation researchers.

15. Limitations Limited to… English peer-reviewed journal articles
Implementation phase studies
Researchers that use the terms ‘organizational context’, ‘context of the organization’ (very specific)
Absence of a consistent definition of organizational context and little uniformity in its measurement across the studies

16. Next steps
Which organizational contextual factors are most important to the implementation of evidence-based practices in healthcare settings?
How do organizational contextual factors influence implementation?

17. Translating research into practice
CFIR can be an appropriate framework for change leaders and implementation researchers who aim to assess or improve the organizational context in their setting.
Use CFIR as a framework to develop comprehensive measurements for organizational context

18. shellyanne.li@mail.utoronto.ca Questions? Thoughts?
What most interested you?
What are the existing challenges?
Funding Sources
Pain in Child Health (PICH): An Innovative, International, Trans-Disciplinary Research Training Consortium
Canadian Institute for Health Research and Dr. Bonnie Stevens (CIHR-funded investigator)

19. Methods – Literature Search
Inclusion criteria
Exclusion criteria
Peer-reviewed, published literature
Study described, explained, measured, or explored the implementation phase of any evidence-based initiative in healthcare service settings
Abstract explicitly mentioned the term ‘organizational context’, ‘context of an organization’, ‘organizational contextual’ factors or characteristics as the main or one of the main objectives
Empirical studies of all research designs
Study that do not refer specifically to organizational context in scope of the implementation phase
Do not concern the healthcare field
Editorials, opinions, discussions, textbooks, conceptual papers, articles that assessed validity and reliability of tools and frameworks
Languages other than English

20. Methods – Data Evaluation
Quality appraisal: Mixed Methods Appraisal Tool (MMAT; Pluye et al., 2011) is a checklist that was developed to provide quality appraisal for qualitative, quantitative and mixed-methods studies
Quantitative and qualitative studies are each appraised by four criteria with overall scores varying from 0% (no criterion met) to 100% (all four criteria met).
For mixed methods studies, three components were appraised: qualitative, quantitative and mixed methods component; with the overall score determined by the lowest component score.
In keeping with integrative review methods (Ganong, 1987; Whittemore & Knafl, 2005), all records were retained in the analysis, regardless of score. The analysis suggested that inclusion of these lower quality articles did not alter the review’s findings.
. Of the 12 studies, six received MMAT score of 75% (moderately high quality), four received score of 50% (moderate quality), and one received 100% (high quality). Of note, the criterion to accurately and fully report study findings was not part of MMAT. Therefore, even though the MMAT scores reflected moderate to high quality, readers must take into consideration the limitations of the MMAT.