Chapter 7: Computer Reliability Ethics for the Information Age Forth Edition by Michael J. Quinn

Chapter Overview Introduction Data-entry or data-retrieval errors Software and billing errors Notable software system failures Therac-25 Computer simulations Software engineering Software warranties

Introduction Computer systems are sometimes unreliable Erroneous information in databases Misinterpretation of database information (similar names) Malfunction of embedded systems (fatal errors) Effects of computer errors Inconvenience (incorrect bill) Bad business decisions Fatalities

Data-Entry or Data-Retrieval Errors A computerized system may fail because wrong data entered into it A computerized system may fail because people incorrectly interpret data they retrieve

Disfranchised Voters November 2000 general election Florida disqualified thousands of voters Reason: People identified as felons Cause: Incorrect records in voter database Consequence: May have affected election’s outcome

Thu 5-12 False Arrests Sheila Jackson Stossier mistaken for Shirley Jackson at airport. Arrested and spent five days in detention Roberto Hernandez mistaken for another Roberto Hernandez Arrested twice and spent 12 days in jail Terry Dean Rogan arrested after someone stole his identity Arrested five times, three times at gun point

Position of Privacy Advocates Number of records is increasing More erroneous records  more false arrests Accuracy of crime records more important than ever

Sun 26-4 Software and Billing Errors Assume data correctly fed into computerized system System may still fail if there is an error in its programming (a bug)

Errors Leading to System Malfunctions Qwest sends incorrect bills to cell phone customers ($600/minute) A bill of more than $57,000 for a customer Spelling and grammar error checkers increased errors (University documents) BMW on-board computer failure (Thailand finance minister was trapped in his BMW)

Tue 27-11 Errors Leading to System Failures Temporarily out-of-control Boeing 777 Malaysian plane over the Indian ocean-autopilot error (2005) Japan’s air traffic control system went down for an hour (2003)  some flight were delayed Emergency calls were put on hold for 30 minutes, ambulances took 3 hours to respond  20 people died because the computerized dispatch system of London has a failure.

Analysis: E-Retailer Posts Wrong Price, Refuses to Deliver In (2005) Amazon.com in its British website offered iPaq (handheld computers – pocket PCs) for £7 instead of £275 Orders flooded in Amazon.com shut down site, refused to deliver unless customers paid true price Was Amazon.com wrong to refuse to fill the orders?

Mon 14-7 Notable Software System Failures Patriot Missile (28 soldiers killed in 1991) – insufficient precision in an floating-point variable (system clock) Ariane 5 is a satellite launch vehicle self-destructed in 1996 ($500 million not insured) – software error In 1990, AT&T long-distance network (70 million calls couldn’t be made and 60,000 people lost service because routing switches crashed) In 2000, direct recording electronic voting machines registered either no or multiple choice. In 1987, Therac-25 (3 patients killed out of 6 patients) - software error resulted in overdoses.

Sun 23-4 Direct Recording Electronic Voting Machines After problems with 2000 election, punch card voting systems were replaced by direct recording electronic (DRE) voting machines – touch screen Brazil and India have run national elections using DRE voting machines exclusively In November 2006 1/3 of U.S. voters used DRE voting machines

Issues with DRE Voting Machines Voting irregularities (due to programming errors) Failure to record votes Overcounting votes Misrecording votes Lack of a paper audit trail Vulnerability to tampering Source code a trade secret, can’t be examined to find out how secure it is. Possibility of widespread fraud through malicious programming – change programs

Moral Responsibility of the Therac-25 Team Conditions for moral responsibility Causal condition: actions (or inactions) caused the harm Mental condition Actions (or inactions) intended or willed -OR- Moral agent is careless, reckless, or negligent Therac-25 team morally responsible They constructed the device that caused the harm They were negligent

Sun 22-4 Uses of Computer Simulations Simulations replace physical experiments. Why? Experiment too expensive or time-consuming Experiment unethical Experiment impossible (evolution of the universe) Examples of uses of simulations: Model past events (Evolution of the universe) Understand world around us (Search for Oil) Predict the future (weather predictions)

Thu 11-12 Validating Simulations Erroneous Simulation because of : Bugs in SW Model uses SW is flawed Solutions: Validation: Does the model accurately represent the real system? Validation is to check whether software meets the customer expectations and requirements (Specs) Verification: Does program correctly implement model? Verification is to check whether the software conforms to specifications (Implementation) Validation methods Make prediction, wait to see if it comes true (Car crash) Predict the present from old data (Weather forecast) Test credibility with experts and decision makers

Validating Simulations Validation: Are we building the right system? Verification: Are we building the system right? Validation is concerned with checking that the system will meet the customer’s actual needs, while verification is concerned with whether the system is well-engineered, error-free, and so on. Verification will help to determine whether the software is of high quality, but it will not ensure that the system is useful.

Wed 16-7 Software Engineering SE is a four steps process: Specs: determine the functions to be performed Development: produce SW that meet specs. Validation: testing the SW Evolution: Modify SW to meet change requirements

Software Engineering: Specification Followed SW Crises in 1960s Specifications of SE: Determine system requirements Understand constraints Determine feasibility (Budget and schedule) End products High-level statement of requirements (Summary) Mock-up of user interface Low-level requirements statement (Detailed)

Software Engineering: Development Create high-level design Discover and resolve mistakes, omissions in specification CASE tools to support design process Object-oriented systems have advantages After detailed design, actual programs written Result: working software system

Software Engineering: Validation (Testing) Ensure software satisfies specification Ensure software meets user’s needs Challenges to testing software Noncontinuous responses to changes in input. No identical data set of inputs Exhaustive testing impossible. Infinite number of different inputs Testing reveals bugs, but cannot prove none exist Test modules, then subsystems, then system

17-7 Software Quality Is Improving Standish Group tracks IT projects Situation in 1994 1/3 projects cancelled before completion 1/2 projects had time and/or cost overruns 1/6 projects completed on time / on budget Situation in 2006 1/6 projects cancelled 1/3 projects completed on time / on budget

Shrinkwrap Warranties Some say you accept software “as is” Some offer 90-day replacement or money-back guarantee None accept liability for harm caused by use of software

Moral Responsibility of Software Manufacturers If vendors were responsible for harmful consequences of defects Companies would test software more They would purchase liability insurance Software would cost more Start-ups would be affected more than big companies and thus Less innovation in software industry Software would be more reliable Making vendors responsible for harmful consequences of defects may be wrong Consumers should not have to pay for bug fixes