Background The United Kingdom Department of Health introduced single-use instruments (SUI) for tonsil and adenoid surgery in January 2001, based on the advice of the Spongiform Encephalopathy Advisory Committee, to minimise the theoretical risk of transmitting variant Creutzfeldt-Jakob Disease (vCJD) via surgical instruments. Concerns were raised about potential intra-operative hazards due to the quality of the instruments and increased risk of postoperative complications, especially haemorrhage. In Wales, new highly specified SUI for tonsil and adenoid surgery were introduced in February 2003 together with a surveillance system to monitor both instrument problems and complications. Objective To evaluate the effectiveness of the Single-use Instrument Surveillance Programme (SISP). Methods The Centres for Disease Control and Prevention (CDC) Updated Guidelines for Evaluating Public Health Surveillance Systems [1] methodology was used as an evaluation framework. The evaluation included a description of the system and analysis of surveillance system data, reports and summaries; semi-structured interviews and discussions with the SISP team; independent hospital activity data (PEDW) to externally validate the system; and a questionnaire based survey of system users, Ear Nose and Throat (ENT) surgeons. The evaluation covered the period from February 2003 to March 2005; the report was completed in April Results Description of the Surveillance System Description of the public health importance of the health-related event under surveillance The public health importance of the use of specified SUI for tonsil and adenoid surgery is that they reduce the risk of prion transmission Description of the purpose and operation of the Surveillance System The surveillance system was designed to monitor all surgery performed with the specified SUI and post-operative complications The events under surveillance are number of tonsil and adenoid surgery, post-operative complications, instrument usage and associated problems, and number of reported problems by types of instruments The information is reported from the ENT departments to the National Public Health Service for Wales (NPHS); instruments with problems are also sent to the Surgical Material Testing Laboratory (SMTL) Regular feedback is given to the ENT departments and to the Welsh Assembly Government (WAG) Performance of the Surveillance System The response rate of the survey of the ENT surgeons was 72% (56/78) (Table) Usefulness The rapid response is due to the information obtained of the problems arising with specified SUI during surgery. Between February 2003 and March 2004, 4,151 procedures were recorded using 41,376 instruments; there were 335 types of problems with instruments. Timeline analysis showed peaks at which problems with specific instruments were noted (Figure 1). The instrument problems identified by SISP and actions taken include, Draffin rods and Knot pusher, company redesigned the instrument; Eves Snare and Gag & Blade, instruments withdrawn and replaced in large numbers. System Attributes Simplicity Forms are easy to complete by the ENT surgeon, the workload is minimal as it is been done in Theatre, the main form is part of the operative details and a carbon copy is kept for the patient’s clinical records Flexibility The system has been changing from the beginning and has adapted to new requirements like logistic constraints Acceptability All ENT departments take part in SISP, with 100% (10/10) of hospitals reporting each month since December 2003 Sensitivity Even though SISP started at the beginning of 2003, it was only in July 2003 that highly specified SUI became available throughout Wales. The proportion of cumulative procedures detected by SISP increased from 5% in March 2003 to 85% in December 2003 and 96% in March 2004 (Figure 2) Representativeness The All Wales operative sheet return in 2004 was 95%. Two out of 10 hospitals have a return of 65% and 74% respectively Conclusions SISP appears to be an effective surveillance system. It is simple, flexible, meets its objectives, attains the standards set, and is acceptable and useful to the majority of users. It shows that it is possible to monitor surgical outcomes and advice on the impact of changes in instrumentation within a short time period, and provides an alternative approach to the safe introduction of new health technology on a large scale. It demonstrates that a surveillance system can be used to: assess a given surgical procedure or the instruments used to perform that procedure safely introduce a new health technology if evidence on its safety is unavailable or inconclusive provide reassurance to all key stakeholders in these circumstances Recommendations Surveillance system to continue as long as specified SUI are in use and vCJD continues to be a theoretical risk Lessons identified and learnt in the set up and evaluation of this surveillance system should be applied to future introduction of SUI Consider extending this approach to other areas of health care Reference [1] Guidelines Working Group. Updated guidelines for evaluating public health surveillance systems. Recommendations from the Guidelines Working Groups. MMWR 2001; 50(RR13): 1-35 Evaluation of a Surveillance System for Single-Use Instruments in Tonsil Surgery. Can a Surveillance System Monitor the Safety of Single-use Surgical Equipment? S. De Martin 1,2 *, A. Tomkinson 3, J.M.F. Temple 1, M.R. Evans 1,4, W. Harrison 1, S. Harris 1, V. McClure 1 1.National Public Health Service for Wales. 2.All Wales Higher Training Programme. 3.University Hospital of Wales. 4.Department of Epidemiology, Statistics and Public Health, University of Cardiff. * All Wales Higher Training Programme, European Programme for Intervention Epidemiology Training and European Centre for Disease Prevention and Control Usefulness Do you think the SISP has been useful so far? Yes 83% System Attributes Simplicity Do you have any problems with the forms used? No 91% Do you have any problems in the data collection and collation to NPHS? No 94% Acceptability While it remains government policy to continue using SUI, do you think SISP should continue? Yes 59% Fig. 1. Problems with specific single-use instruments Table. ENT surgeons survey results Fig. 2. Tonsillectomies recorded by PEDW and SISP