1. CS 5150 Software Engineering Lecture 20 Reliability 1 (and a little Privacy)

2. CS 5150 2 Administrivia Milestone 3 this week Presentation Report Teammate feedback

3. CS 5150 3 SE in the News Side-channel attacks

4. CS 5150 4 Privacy If you remember nothing else about privacy from 5150: Store as little data about people as you can Consult a lawyer before launching a publicly- accessible service that retains personal information

5. CS 5150 5 Privacy Computers challenge assumptions about the division between public and private Targeted advertising Government records Cameras and microphones in public places RFID everywhere Network monitoring

6. CS 5150 6 Dependable and Reliable Systems: The Royal Majesty From the report of the National Transportation Safety Board: "On June 10, 1995, the Panamanian passenger ship Royal Majesty grounded on Rose and Crown Shoal about 10 miles east of Nantucket Island, Massachusetts, and about 17 miles from where the watch officers thought the vessel was. The vessel, with 1,509 persons on board, was en route from St. George’s, Bermuda, to Boston, Massachusetts." "The Raytheon GPS unit installed on the Royal Majesty had been designed as a standalone navigation device in the mid- to late1980s,...The Royal Majesty’s GPS was configured by Majesty Cruise Line to automatically default to the Dead Reckoning mode when satellite data were not available."

7. CS 5150 7 The Royal Majesty: Analysis The ship was steered by an autopilot that relied on position information from the Global Positioning System (GPS). If the GPS could not obtain a position from satellites, it provided an estimated position based on Dead Reckoning (distance and direction traveled from a known point). The GPS failed one hour after leaving Bermuda. The crew failed to see the warning message on the display (or to check the instruments). 34 hours and 600 miles later, the Dead Reckoning error was 17 miles.

8. CS 5150 8 The Royal Majesty: Software Lessons All the software worked as specified (no bugs), but... Since the GPS software had been specified, the requirements had changed (stand alone system now part of integrated system). The manufacturers of the autopilot and GPS adopted different design philosophies about the communication of mode changes. The autopilot was not programmed to recognize valid/invalid status bits in message from the GPS (NMEA 0183). The warnings provided by the user interface were not sufficiently conspicuous to alert the crew. The officers had not been properly trained on this equipment.

9. CS 5150 9 Basic Definitions The reliability of software is related to how frequently it does what it is supposed to do The robustness of software is related to how badly things go wrong when it fails to do exactly what it is supposed to do

10. CS 5150 10 Key Factors that Promote Reliability Organization culture that expects quality Software design and implementation that hides complexity (e.g., structured design, object-oriented programming) Precise, unambiguous specification Software tools that restrict or detect errors (e.g., strongly typed languages, source control systems, debuggers) Programming style that emphasizes simplicity, readability, and avoidance of dangerous constructs Continuous validation (e.g., reviews, unit testing) Systematic testing (test plan, test cases, etc.) Change management (regression testing)

11. CS 5150 11 Good Organizational Practices Acceptance of the group's style of work (e.g., meetings, preparation, support for juniors) Visibility – tell people your problems Issue tracking Completion of a task before moving to the next (e.g., documentation, comments in code)

12. CS 5150 12 Quality Management Processes Standard terminology (requirements, specification, design, etc.) Software standards (coding standards, naming conventions, etc.) Regular builds of complete system Internal and external documentation Reporting procedures

13. CS 5150 13 When Time/Money is Short Pay even more attention to the early parts of the design process “Measure twice, cut once” Programmers tend to prefer to get something working, then refine/reimplement Investing in early design tends to lead to an acceptable product sooner

14. CS 5150 14 Change Management Source code management and version control Tracking of change requests and bug reports Procedures for changing requirements specifications, designs and other documentation Regression testing Release control

15. CS 5150 15 Reliability Metrics Informally speaking Probability of a software system failing in real-world use Perceived reliability User behavior Sets of inputs Pain of failure

16. CS 5150 16 Traditional Metrics For online systems Mean time between failures Availability (up time) Mean time to repair Market measures Complaints Customer retention