1. 25.8.2015Dr Andy Brooks1 Lecture 4 Therac-25, computer controlled radiation therapy machine, that killed people. FOR0383 Software Quality Assurance

2. No official inquiry Five Therac-25 machines were installed in the U.S and six in Canada. Between June 1985 and January 1987, Therac-25 massively overdosed six people. No official inquiry was undertaken, but Nancy Leveson investigated what happened from “law suits and depositions, government records, and copies of correspondence and other material obtained from the U.S. Food and Drug Administration (FDA) which regulates such devices ”. 25.8.2015Dr Andy Brooks2

3. 25.8.2015Dr Andy Brooks3 Medical linear accelerators High energy beams destroy tumors with minimal impact on surrounding healthy tissue. Relatively shallow tissue is treated with electrons. Deeper tissue is treated by converting the electron beam into X-ray photons. Dual-mode machines are more economic.

4. 25.8.2015Dr Andy Brooks4 Therac-6 (6 MeV) & Therac-20 (20 Mev) Therac-6 produced X rays only. Therac-20 was dual-mode. Software functionality was limited. Both machines had industry standard hardware safety features. Some Therac-6 software was re-used in the Therac-20. Production of the machines was a joint venture between AECL (Atomic Energy of Canada Limited) and the French Company CGR.

5. 25.8.2015Dr Andy Brooks5 Therac-25 (25 MeV) The Therac-25 was solely developed by AECL as a dual- mode device. AECL took advantage of computer control and decided not to duplicate all the existing hardware safety features. The first commercial version was available in late 1982. Some software was re-used from the previous machines. AECL´s quality assurance manager apparently was unaware of the re-use of software from Therac-20. Bugs in the Therac-20 software were recognised only afterwards when Therac-25 came under investigation: the hardware safety features in Therac-20 had prevented any injuries.

6. Software Testing A safety analysis of Therac-25 was undertaken by AECL in March 1983 which apparently excluded the software. At a Therac-25 user´s meeting, a quality assurance manager claimed that Therac-25 sofware had been tested for 2,700 hours. When questioned further, however, he clarified that he meant 2,700 hours of use. The same quailty assurance manager could only report that a “small amount” of software testing was done on a simulator. The FDA had difficulty getting an adequate test plan from AECL. There was no public evidence of any regression testing. 25.8.2015Dr Andy Brooks6

7. 25.8.2015Dr Andy Brooks7 The basic hazard of dual-mode machines. Equipment is rotated into the beam path to produce the two therapeutic modes. For electron therapy, scanning magnets spread the beam. For X-ray therapy, a beam flattener is used to produce a uniform treatment field. The flattener is a very efficient attenuator, so a very high input dose rate (of electrons on a target) is required. If a beam flattener is not in position, a high output dose results. For X-ray therapy, the only energy level is 25MeV. In the Therac-25, there was also a mirror and light source to help correctly position the patient. The operator can see exactly where the beam will strike.

8. 25.8.2015Dr Andy Brooks8 Upper turntable assembly Electron mode scan magnets. X-ray mode target and flattener. Mirror. Plunger. Microswitches monitor the position of the turntable.

9. 25.8.2015Dr Andy Brooks9 Operator interface In response to operator complaints that it took too long to enter a treatment plan, AECL modified the software before the first Therac-25 was installed. Instead of re-entering treatment details, operators could just use a quick series of carriage returns to complete the data entry. Because of timing issues in the software and how it controlled the machine, under particular circumstances, if an operator went very fast through the series of carriage returns, the machine could deliver an overdose.

10. 25.8.2015Dr Andy Brooks10 From Nancy Leveson, Software: System Safety and Computers, copyright Addison-Wesley 1995. x or e

11. 25.8.2015Dr Andy Brooks11 After one incident, a memorandum from the FDA stated: “The operator´s manual supplied with the machine does not explain nor even address the malfunction codes. The Maintenance [sic] Manual lists the various malfunction numbers but gives no explanation. The materials provided give no indication that these malfunctions could place a patient at risk.” I wonder what MALFUNCTION 54 means? Not to worry, I have been told there are many safety mechanisms in place.

12. 25.8.2015Dr Andy Brooks12 The memorandum from the FDA also stated: “The program does not advise the operator if a situation exists wherein the ion chambers used to monitor the patient are saturated, thus are beyond the measurement limits of the instrument. This software package does not appear to contain a safety system to prevent parameters being entered and intermixed that would result in excessive radiation being delivered to the patient under treatment.”

13. 25.8.2015Dr Andy Brooks13 East Texas Cancer Center, March 1986 The intended treatment was a 22MeV electron beam of 180 rads. The operator entered the treatment details but noticed she had typed “x” rather than “e”. She used the up-arrow key to replace “x” with “e” and hit the return key several times as the other parameters were to remain unchanged. A MALFUNCTION 54 message was displayed but the dose monitor display indicated a substantial underdose. She hits the P key to proceed. –It was common to do this in response to quirks of the machine. A video display of the patient was unplugged and the audio monitor was broken. –There was no way of being alerted of any patient difficulty.

14. 25.8.2015Dr Andy Brooks14 The patient... He felt a thump and heat and heard a buzzing sound. He moved to get up from the table but then felt as if his arm had been electrocuted and that his hand was leaving his body. He pounded on the treatment room door, visibly upset. Unknown at the time, he had received a dose of 16,500 to 25,000 rads in less than 1 second. Over the weeks that followed he lost function of his left arm and suffered nausea and vomiting. He then got paralysis in both legs and also could not speak. He developed a lesion in his left lung and recurrent skin infections. He died five months later. East Texas Cancer Center, March 1986

15. Response to the incident... AECL engineers spent a day testing the machine but could not reproduce a MALFUNCTION 54. An AECL engineer is reported as having given assurances that it was not possible for Therac-25 to overdose a patient. An ETCC physicist asked if AECL knew of other overdoses by Therac-25. AECL personnel denied any knowledge of previous incidents and suggested that an electrical problem had caused the fault. An engineering firm ruled out any electrical problem. The ETCC physicist found the calibration of the machine to be satisfactory and put the machine back in service on April 7, 1986. 25.8.2015Dr Andy Brooks15 East Texas Cancer Center, March 1986

16. The same operator who was involved in the first incident prepared a patient for his treatment on April 11. The operator entered the treatment details, and as before, she noticed that she had to change “x” to “e”. She used the up-arrow key to replace “x” with “e” and hit the return key several times as the other parameters were to remain unchanged. The display showed MALFUNCTION 54. The distressed patient asked: “What happened to me, what happened to me?” The patient died from the overdose on May 1, 1986. 25.8.2015Dr Andy Brooks16 East Texas Cancer Center, April 1986

17. 25.8.2015Dr Andy Brooks17 Race conditions. Therac-25 did not employ a standard operating system. Instead, Therac-25 had a custom real- time treatment operating system written in PDP- 11 assembly language. The implementation of multitasking allowed race conditions to result. i.e. the sequence and timing of events were critical. This played a big part in the overdosing of patients.

18. 25.8.2015Dr Andy Brooks18 The whole software development process was deficient: requirements, design, implementation, testing, maintenance.