Software Engineering Ethics and Future Responsibilities Barry Boehm CSCI 510, Fall 2018

Outline Definitions and context Principles and examples Power to do public harm or good ACM/IEEE Software Engineering Code of Ethics Principles and examples Rawls’ Theory of Justice Relation to stakeholder win-win CS 577 ethics situations Ethics challenges for your future careers Global threats, nanosecond decisions Coping with AI, Internets of Things, DevOps, Systems of Systems Fall 2018 © USC-CSSE

Definition of “Ethics” -Webster, 1993 The discipline dealing with what is good and bad And with moral duty and obligation A theory, system, or set of moral principles or values The principles of conduct governing an individual or group Professional ethics Fall 2018 © USC-CSSE

Context Software engineers will have vastly increasing power to do public harm or good System safety and security, intellectual property, privacy, quality of work, fairness, liability, risk disclosure, conflict of interest, unauthorized access Professional societies have developed codes of ethics Integrating value-based ethics into value-neutral software engineering practices via ICSM Future software engineers and managers (you) will need to cope with complex global threats, nanosecond decisions, AI, Internets of Things, DevOps, systems of systems Fall 2018 © USC-CSSE

Power to Do Public Harm or Good – I Intellectual Property: use without credit; use copyrighted material Privacy: credit, health, personal information Confidentiality: competitive information, political sensitivity Quality of work: many dimensions; see table Fall 2018 © USC-CSSE

Quality Concerns Vary by Stakeholders Role Developers, Maintainers System Dependents Stakeholder Classes System Controllers **Critical *Significant 0 Insignificant or indirect Info Suppliers Info Brokers Administrators Acquirers Information Consumers   Mission - Protection critical uncrit. Safety **  ** Security * Privacy Robustness Reliability Availability Survivability Quality of Service Performance Accuracy, Consistency Accessibility, ease of use; difficulty of misuse Evolvability Interoperability Correctness Cost Schedule Reusability Fall 2018 © USC-CSSE

Example: Confidentiality Government agency hires company to support SW procurement Provides data under nondisclosure agreement Employee and company consultant prepare cost estimate Employee: “ I don’t see how anyone can do all this for $8M” Consultant provides $8M target cost to some bidders Government agency angry with company for leak Whose fault? How could it be avoided? Fall 2018 © USC-CSSE

Power to Do Public Harm or Good - II Fairness: equality of opportunity/treatment; fair reward system Liability: accountability; parity of authority and responsibility Risk Disclosure: safety tests, COTS capabilities; schedule slips Conflict of Interest: source selection; personnel or product reviews Unauthorized Access: reading, copying, modifying; denial of service Fall 2018 © USC-CSSE

Examples: Fairness Enron software to schedule power outages, raise prices E.g., air conditioning for homes, offices, hospitals during heat waves Suppose you had been asked to develop it? Urban fire dispatching system Inefficient old system caused $700M property loss New-system spec. includes dispatching algorithm to minimize property loss Any fairness issues? Fall 2018 © USC-CSSE

Example: Safety Tests Your company is delivering a drug prescription fulfillment system Reusing software from a warehouse inventory system You are the quality assurance manager With company responsibility for certifying product safety The software has passed all the contracted tests But many off-nominal conditions untested Some have shown unsafe outcomes You feel more off-nominal testing if necessary Company president says if you don’t certify safety by delivery date, company may go out of business What should you do? Fall 2018 © USC-CSSE

ACM/IEE Software Engineering Code of Ethics -Table of Contents Products: achievable goals, realistic estimates, high quality Public: safety, respect of diversity, public interest first Judgment: objectivity, no bribes or conflicts of interest Client and Employer: no employer-adverse interests, surface problems Management: fair, ethical work rules, due process for violations Profession: support profession and ethics code, don’t misrepresent software Colleagues: credit colleagues’ work, give colleagues a fair hearing Self: improve your technical and ethical knowledge and practices Fall 2018 © USC-CSSE

Code of Ethics 2. Public 2.01 Disclose any software-related dangers 2.02 Approve only safe, well tested software 2.03 Only sign documents in area of competence 2.04 Cooperate on matters of public concern 2.05 Produce software that respects diversity 2.06 Be fair and truthful in all matters 2.07 Always put the public’s interest first 2.08 Donate professional skills to good causes 2.10 Accept responsibility for your own work Fall 2018 © USC-CSSE

Code of Ethics 4. Client and Employer 4.01 Provide services only where competent 4.02 Ensure resources are authentically approved 4.03 Only use property as authorized by the owner 4.04 Do not use illegally obtained software 4.05 Honor confidentiality of information 4.06 Raise matters of social concern 4.07 Inform when a project becomes problematic 4.08 Accept no detrimental outside work 4.09 Represent no interests adverse to your employer Fall 2018 © USC-CSSE

Outline Definitions and context Principles and examples Power to do public harm or good ACM/IEEE Software Engineering Code of Ethics Principles and examples Rawls’ Theory of Justice Relation to stakeholder win-win CS 577 ethics situations Ethics challenges for your future careers Global threats, nanosecond decisions Coping with AI, Internets of Things, DevOps, Systems of Systems Fall 2018 © USC-CSSE

Rawls’ Theory of Justice (1971) -Following Collins et al Rawls’ Theory of Justice (1971) -Following Collins et al., “How Good Is Good Enough?” Comm.ACM, Jan. 1994 Fair rules of conduct Principles of justice Participants and obligations Provider (developer) Buyer (acquirer) User(s) Penumbra (general public) Negotiate mutually satisfactory (win-win) agreements Fall 2018 © USC-CSSE

Rawls’ Theory of Justice - II Fair rules of conduct Negotiation among interested parties Veil of ignorance (about what affects whom) Rationality Principles Least Advantaged - don’t increase harm to them Harm = probability x magnitude (~risk exposure) Risking harm - don’t risk increasing harm Don’t use “low-threat” software in “high-threat” context Publicity test - defensible with honor before an informed public Use for difficult cost-benefit tradeoffs Fall 2018 © USC-CSSE

Obligations of the Software Provider Fall 2018 © USC-CSSE

Obligations of the Software Buyer Fall 2018 © USC-CSSE

Obligations of the Software User Fall 2018 © USC-CSSE

Obligations of the Software Penumbra Fall 2018 © USC-CSSE

CS 577 Ethics Accountability -1 Assuming your priorities match those of other stakeholders Users: GUI; quality factor priorities Maintainers: programming language, reuse, documentation Customers/Owners: legacy compatibility, advanced vs. mature technology, full business case Treating stakeholders equally on each issue Users on GUI; owners on legacy compatibility: developers on cost/schedule/risk Promising more than you can deliver Borrowing from other projects without credit Suppressing or delaying bad news Fall 2018 © USC-CSSE

CS 577 Ethics Accountability -2 Delivering just source code vs. including architecture, regression test cases, documentation for users, maintainers Honoring commitments to CS 577b Misinterpreting the Golden Rule Do unto others as you would have others do unto you Building programmer-friendly GUIs for doctors, lawyers, administrators Pop-up windows for curiosity-driven vs. closure-driven users Platinum Rule Do unto others as they would be done unto Fall 2018 © USC-CSSE

Outline Definitions and context Principles and examples Power to do public harm or good ACM/IEEE Software Engineering Code of Ethics Principles and examples Rawls’ Theory of Justice Relation to stakeholder win-win CS 577 ethics situations Ethics challenges for your future careers Global threats, nanosecond decisions Coping with AI, Internets of Things, DevOps, Systems of Systems Fall 2018 © USC-CSSE

Ethics challenges for your future careers Reference: Thomas Friedman, Thanks for Being Late Moore’s Law will continue 2x computer-communications speed, data capacity every 2 years, at half the price By 2038, factors of 1024 “Software is Eating the World” Marc Andreessen, Netscape: 2012 Much more complex Internets of Things Challenges of compatible DevOps evolution AI, computers making decisions in nanoseconds Farewell to the Principle of Human Primacy Powerful capabilities falling into the wrong hands Denial of service, perversion of objectives Fall 2018 © USC-CSSE

Robotics: Implementing Asimov’s Laws 1. Do not injure a human or allow a human to come to harm. 2. Obey orders given by humans, except when orders conflict with Law 1. 3. Protect its own existence, except when this conflicts with Laws 1 and 2. Hard to achieve, even in simple, stable situations Fall 2018 © USC-CSSE

Perversion of Objectives: Example Zika mosquitoes carry deadly virus One bite usually fatal Genetic and software engineers propose gene-splicing approach to disable Zika mosquitoes to reproduce Suppose this approach falls into the hands of organizations wishing to eliminate races of humans Suppose this approach is mishandled and applied to all humans Hopefully there is time to create a countermeasure Fall 2018 © USC-CSSE

Conclusions Software engineers will have vastly increasing power to do public harm or good Rawls’ Theory of Justice enables constructive approach for integrating ethics into daily software engineering practice ICSM Stakeholder win-win with least-advantaged system dependents as success-critical stakeholders Responsibility challenges for your future careers Global threats, nanosecond decisions Coping with AI, Internets of Things, DevOps, Systems of Systems Try to build in reversibility of bad decisions Fall 2018 © USC-CSSE