1. NPT: Potential uses in Quantitative Research Tracy Finch Institute of Health and Society, Newcastle University

2. Objectives  To describe the development of a structured research instrument (questionnaire) to measure constructs of the NPM/T in relation to a particular intervention (E-health)  To consider wider issues for use of NPT in Quantitative research

3. Understanding Normalisation in relation to E-health  ‘Understanding the implementation and integration of e- health services’. Mair, May, Finch, Murray, O’Donnell, Wallace et al. NHS SDO Funded study. (April 2006 – Jan 09).  Focused on barriers and facilitators to use of e-health systems by health professionals within the NHS  One work-package (out of 4) involved developing a structured research instrument to assess ‘readiness’ for e- health  The NPM and NPT framed data collection and analysis throughout project (Began with NPM – finished with NPT)

4. WP3 ‘Technology Adoption Readiness Scale’ (TARS) Objective: To develop a structured, predictive instrument to test the contextual readiness of a health care setting for uptake and routine use of a specific ehealth system by health professionals. Contextual readiness: combination of individual and organisational factors Perspective: Professionals

5. TARS Study Structure  Phase 1: Instrument development  included online survey of experts  Phase 2: Generic instrument  ‘factors’ rated in importance  Phase 3: Specific instrument  ‘questions’ about specific e-health system being used

6. TARS Phase 1: Item development  Aim was to develop an initial item set, using expert review  Generate potential items through translation of construct statements into plain language and single dimensions

7. Item development: Example…. Step 1Factor/ Issue: Concerns about security and confidentiality and standards Step 2Question: Does X affect your confidence in knowledge available to you? (Relational Integration) Step 3Question for ehealth study: How important are the following factors in affecting the use of e-Health systems in the everyday work of health professionals: ….how much the e-Health system affects the users’ confidence in their ability to conduct their work safely and efficiently (scale: Not at all important – Extremely important)

8. Phase 1: Expert Survey Survey:  27 Items for inclusion (IW, RI, SW, CI)  5pt scale of ‘importance’ (+ ‘don’t know)  Free-text space for ‘any factors missed’  Conducted electronically Sample:  Authors of reviews relevant to field Recruitment:  Email invitation

9. Phase 1: Results Response:  63 completed surveys (25% of 252 invitations presumed received) Data analysis:  Descriptive (mean ratings, correlations) Findings:  Importance of items generally highly endorsed  Most correlations low-moderate (little redundancy) and higher within than across NPM constructs  Free-text responses useful in identifying further factors to include

10. Phase 2: Development of TARS Generic Aimed to:  Refine factor set using expert survey data  Test ‘relative importance’ from perspective of professionals Revisions to item set (31 Items):  Some items re-worded, dropped or combined  New items added from free-text responses (phase 1)  New items for NPT: Coherence, Cognitive Participation, Reflexive Monitoring

11. Phase 2: Testing TARS Generic  Survey of health professionals’ perspectives of importance of factors affecting uptake and use of e- health  Sample: Regional NHS Hospitals Trust (potentially 10,000+ respondents). Extensive use of e-Health.  Recruitment: Via site contact (Technical Director)  Response: Extremely low (51 responses)  Analysis: Not particularly useful, but suggestive of different patterns of response between experts and professionals

12. Phase 3: TARS Specific Aimed to:  Develop a version containing questions framed for an individual’s assessment of a particular e-health system  Test through data collection at (2) different sites: Site 1: Community nurses using PDAs (relatively new) Site 2: Established use of several e-health systems as basis of work (algorithms, information resources, etc)

13. TARS Specific Instrument  Demographics (eg. Professional role, system/s used, length of time using)  Comfort with using computer-based technology  TARS Items: 30 Items rated from ‘agree strongly’ to ‘disagree strongly’ (7 pt) (Items on hand-out)  ‘Normalisation’ questions:  Whether system was ‘not at all’, ‘partly’, or ‘completely’ in routine use  Perceived likelihood of it becoming routine (5 pt scale)

14. Phase 3: Sample SITE 1% (n)SITE 2% (n) Response: 46/243 (19%)Response: 231/1351 (17%) Age: 72% aged 45+; 29% aged <35Age: 40% aged 45+; 29% aged <35 Sex: All FemaleSex: 86% Female Working role: Community Enrolled Nurse Community Staff Nurse District Nursing Sister/Charge Nurse Practice Development Nurse Senior Nurse 0 (0) 28 (13) 61 (28) 9 (4) 2 (1) Working role: Call handlers Nurse advisors Team leaders Health Information advisors Other 47 (109) 24 (56) 9 (21) 3 (7) 16 (38) Perceived level of routinisation of e-Health Not at all Partly Completely 0 (0) 68 (30) 32 (14) Perceived level of routinisation of e-Health Not at all Partly Completely 1 (2) 17 (35) 83 (174)

15.  Non-parametric (cross-tab) analysis  Groups perceiving e-health as ‘completely’ rather than ‘partly’ routine differed in expected direction:  on 12/30 Items at Site 1 (CI=4; RI=4; IW=1; Co; CP; RM  on 9/30 Items at Site 2 (CI=3; RI=3; SW=1; Co; RM)  At Site 2, comparison of call handlers with nursing & related staff indicated differences on 4 items Key results: Normalisation Perceptions

16. Summary of Results of TARS  Development of NPM/T based questionnaire for assessment of individual’s perceptions of factors relating to normalisation of e-health  Operationalising of NPM/T constructs into plain language questions  Support for NPM/T in terms of constructs – patterns of relations between items  Potential of items representing NPM/T constructs for discriminating between levels of perceived normalisation of e-health

17. TARS - Limitations  Low response rates – insufficient for scale development work (statistical properties)  Constraints of ‘real’ environments:  ‘Readiness’ assessment is dependent on timing and site characteristics  Lack of access to participants/control over survey reminders etc  TARS should be used/tested in further studies, in sites where predictive utility of TARS can be assessed prospectively

18. Using NPM/T in Quantitative Research: Wider issues  Potential Benefits  Challenges: 1.Translating theory into plain language 2.Addressing multiple perspectives 3.Standardisation vs specification 4.Operationalising ‘normalisation’  Summary

19. Using NPM/T Quantitatively: Potential Benefits  The ‘How much?’ question:  Structured surveys have the potential to collect data efficiently, and on a large scale  The ‘what is likely to happen?’ question:  Surveys, used prospectively, may have some predictive utility with respect to outcomes  Potentially useful in comparative research  Surveys are appealing to practitioners and researchers - facilitate take-up of the Theory!

20. Challenge 1: Translating constructs into plain language Example: How important are the following factors in affecting the normalisation of e-health….. “…… the extent to which organizational effort is allocated to an ehealth system in proportion to the work that the system is intended to do” (CI) Problem: Multi-dimensional constructs difficult to capture in single questions/statements Possible solutions?  Clear definitions of terms (eg effort) or  Establish understanding of terms of reference (eg agreement on what the system is intended to do?) and use several questions to build understanding

21. Challenge 2: Addressing multiple perspectives  Questions not to be framed around ‘intention’ – instead reflect judgements about others/the organisation  Which stakeholder groups should be included? How do we combine/weight their ratings?  Need for customising questions (or question sets) for different stakeholder groups

22. Challenge 3: Standardisation vs specification Quantitative validation of NPT would be facilitated by:  Focused effort on scale development in appropriate settings and with adequate resources, and  Comparative analysis of quantitative research using NPT survey across different settings However.....  Can we develop a useful ‘generic’ NPT based structured survey instrument that is useable across settings?  (and if we do, are we denying the complexity that the NPT embraces?)

23. Challenge 4: How to operationalise ‘normalisation’?  Does the NPM/T yet define ‘normalisation’ adequately for quantitative measurement of it as an ‘outcome’?  Are ‘perceptions’ of how much an intervention/technology/ practice has become ‘part of everyday work/life’ sufficient to test the constructs of the model?  i.e. Are ‘objective’ measures needed also?

24. Summary & Final Thoughts  Quantitative use of NPT brings challenges, but potential benefits are huge  TARS represents a useful starting point in developing quantitative use of the NPM/T  Need for more focused effort on scale development and validation (MRC Methodology Programme grant planned)  Other quantitative studies are underway (eg. May, Rapley et al. ‘BSPAR Survey’; Newton et al. Midwifery, Melbourne).