1. Establishing HTA Impact Evaluation From Day One Ruth Louise Poole
Establishing HTA Impact Evaluation From Day One Ruth Louise Poole*, Sophie Hughes, Lauren Elston, Matthew Prettyjohns, Susan Myles Health Technology Wales, Velindre University NHS Trust, Wales, UK
Introduction Health Technology Wales (HTW) was formally established in 2017 as a new national Health Technology Assessment (HTA) agency. Our remit is to provide a strategic, streamlined approach to the identification, appraisal and adoption of non-medicines technologies (NMT) throughout Wales, UK. From the beginning, we planned to develop methods for monitoring and recording our impact across these three key areas in consultation with stakeholders.
Methods Matter of Focus, external experts in Contribution Analysis methods, facilitated sessions and interviews with various stakeholders. This exercise generated a series of logic models aligned with HTW’s objectives, taking into consideration anticipated risks and assumptions. Members of the main HTW committees (Assessment Group and Appraisal Panel) were involved, as well as core HTW staff and Welsh Government commissioners. The resulting HTW Evaluation Plan has been widely disseminated, with further input being sought from additional stakeholders, including patient and public representatives. Evidence collection activities are ongoing, and summary data are being regularly uploaded to a bespoke online repository ( An Evaluation Steering Group has been set up to oversee this work.
Results
Having theoretical frameworks in place from day one has helped to target our evidence collection activities. Our data collection tools generate evidence to validate HTW’s impact in the specific areas of interest identified.
HTW staff are conscious of the need to routinely record supporting information. This systematic, prospective evidence collection facilitates the timely production of reports (such as annual reports and impact statements).
Conclusions
This theory-based, co-ordinated approach is helping HTW to demonstrate the value offered by our HTA activities.
As a dynamic, ‘living’ framework, our outcome maps will continue to be iteratively refined, adapted and used beyond 2020.
Conclusions:
This novel evaluation approach prioritises technologies with potential for positive
impact for patients and health services. It is anticipated that involving local clinicians in decision-making will also facilitate adoption.
If these prioritisation methods prove successful, HTW will co-ordinate similar exercises for other specialties, and develop
methodological resources for widespread use.
Figure 1. An example of an outcome map for our evidence appraisal activities. Each ‘stepping stone’ is associated with one or more success criteria, and evaluated using pre-specified sources of evidence (such as surveys, reports, minutes or social media feedback). Colour coding and shading is used to track progress, showing ‘at a glance’ where our impact evidence gaps could be strengthened.
@HealthTechWales
Health Technology Wales is funded by, but independent of, Welsh Government