University of Southern California Center for Software Engineering C S E USC © USC-CSSE1 Barry Boehm CS 577a, 510 Fall 2010 Software Engineering Ethics 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE2 Outline Definitions and context –Power to do public harm or good –ACM/IEEE Software Engineering Code of Ethics Principles and examples –Rawls’ Theory of Justice –Relation to stakeholder win-win –Case study: Mercy Hospital Integrating ethics into daily software engineering practices –Incremental Commitment Spiral Model –CS 577 ethics situations 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE3 Definition of “Ethics” -Webster, 1993 The discipline dealing with what is good and bad –And with moral duty and obligation A theory, system, or set of moral principles or values The principles of conduct governing an individual or group –Professional ethics 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE4 Context Software engineers have increasing power to do public harm or good –Intellectual property, privacy, confidentiality, quality of work, fairness, liability, risk disclosure, conflict of interest, unauthorized access Professional societies have developed codes of ethics Hard to integrate value-based ethics into value- neutral software engineering practices VBSE/MBASE/Win Win Spiral enable ethics integration 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE5 Power to Do Public Harm or Good – I Intellectual Property: use without credit; use copyrighted material Privacy: credit, health, personal information Confidentiality: competitive information, political sensitivity Quality of work: many dimensions; see table 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE6 Example: Confidentiality Government agency hires company to support SW procurement –Provides data under nondisclosure agreement Employee and company consultant prepare cost estimate –Employee: “ I don’t see how anyone can do all this for $8M” Consultant provides $8M target cost to some bidders Government agency angry with company for leak –Whose fault? How could it be avoided? 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE7 Power to Do Public Harm or Good - II Fairness: equality of opportunity/treatment; fair reward system Liability: accountability; parity of authority and responsibility Risk Disclosure: safety tests, COTS capabilities; schedule slips Conflict of Interest: source selection; personnel or product reviews Unauthorized Access: reading, copying, modifying; denial of service 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE8 Examples: Fairness Enron software to schedule power outages, raise prices –Suppose you had been asked to develop it? Urban fire dispatching system –Inefficient old system caused $700M property loss –New-system spec. includes dispatching algorithm to minimize property loss Any fairness issues? 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE9 ACM/IEE Software Engineering Code of Ethics -Table of Contents 1.Products: achievable goals, realistic estimates, high quality 2.Public: safety, respect of diversity, public interest first 3.Judgment: objectivity, no bribes or conflicts of interest 4.Client and Employer: no employer-adverse interests, surface problems 5.Management: fair, ethical work rules, due process for violations 6.Profession: support profession and ethics code, don’t misrepresent software 7.Colleagues: credit colleagues’ work, give colleagues a fair hearing 8.Self: improve your technical and ethical knowledge and practices 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE10 Code of Ethics 2. Public 2.01 Disclose any software-related dangers 2.02 Approve only safe, well tested software 2.03 Only sign documents in area of competence 2.04 Cooperate on matters of public concern 2.05 Produce software that respects diversity 2.06 Be fair and truthful in all matters 2.07 Always put the public’s interest first 2.08 Donate professional skills to good causes 2.10 Accept responsibility for your own work 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE11 Code of Ethics 4. Client and Employer 4.01 Provide services only where competent 4.02 Ensure resources are authentically approved 4.03 Only use property as authorized by the owner 4.04 Do not use illegally obtained software 4.05 Honor confidentiality of information 4.06 Raise matters of social concern 4.07 Inform when a project becomes problematic 4.08 Accept no detrimental outside work 4.09 Represent no interests adverse to your employer 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE12 Outline Definitions and context –Power to do public harm or good –ACM/IEEE Software Engineering Code of Ethics Principles and examples –Rawls’ Theory of Justice –Relation to stakeholder win-win –Case study: Mercy Hospital Integrating ethics into daily software engineering practices –VBSE/MBASE/Win Win Spiral Model –CS 577 ethics situations 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE13 Rawls’ Theory of Justice (1971) - Following Collins et al., “How Good Is Good Enough?” Comm.ACM, Jan Fair rules of conduct Principles of justice Participants and obligations –Provider (developer) –Buyer (acquirer) –User(s) –Penumbra (general public) Negotiate mutually satisfactory (win-win) agreements 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE14 Rawls’ Theory of Justice - II Fair rules of conduct –Negotiation among interested parties –Veil of ignorance (about what affects whom) –Rationality Principles –Least Advantaged - don’t increase harm to them Harm = probability x magnitude (~risk exposure) –Risking harm - don’t risk increasing harm Don’t use “low-threat” software in “high-threat” context –Publicity test - defensible with honor before an informed public Use for difficult cost-benefit tradeoffs 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE15 Obligations of the Software Provider 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE16 Obligations of the Software Buyer 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE17 Obligations of the Software User 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE18 Obligations of the Software Penumbra 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE19 Case Study: Mercy Hospital Pharmacy System -Collins et al., 1994 Growing hospital –Manual pharmacy information system reaching overload Spec developed for PC-based information system –Rachel: VP, Records & Automation –George: Chief Pharmacist System developed by consultants –Hired by George –Rachel: test procedures –Based on mature warehouse inventory system –Budgeted 50% more testing than other bidders Installation & Training discovers problems –Helen: consultant in charge of installation & training –Ann: skeptical nurse cross-checking computer outputs 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE20 Mercy Hospital Pharmacy System: Problems Dosage problems from data entry errors –10x dosage; wrong patient Cross-checking incomplete; not trusted by some doctors Heavier data-entry load –Formalizing automated procedures  more info. needed –Pharmacy info > warehouse info Helen: Should go back to old system during cleanup George:- Is old system less risky? -How do we ensure cleanup will get it right? -How much will cleanup cost? Future practice: How to anticipate, avoid similar problems? 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE21 Outline Definitions and context –Power to do public harm or good –ACM/IEEE Software Engineering Code of Ethics Principles and examples –Rawls’ Theory of Justice –Relation to stakeholder win-win –Case study: Mercy Hospital Integrating ethics into daily software engineering practices –Incremental Commitment Spiral Model –CS 577 ethics situations 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE22 Mercy Hospital : Use of ICSM Results chain –Add patient safety outcome, patient stakeholder representative –Rework-business-workflows initiative, including safety checks; add clerical-staff stakeholder Stakeholder Win Win –Patient representative: safety criteria; parallel-operation phase- in –Clerical staff: prototype GUI, including safety-check support Business Case: includes added safety costs and benefits Risk Management: assess warehouse package safety, effects of workflow changes. 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE23 Use of ICSM -- II Concurrent Engineering –Concurrently address business workflows, GUI prototypes, COTS alternatives, feature prioritization, cost/schedule/benefits analysis, other risks –Prepare to pass LCO, LCA, CCD, and IOC anchor point milestone reviews Monitoring and Control: Use Balanced Scorecard to track progress with respect to plans; apply corrective actions as necessary Change as Opportunity: Look for emerging COTS pharmacy-related fulfillment systems 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE24 CS 577 Ethics Situations Assuming your priorities match those of other stakeholders –Users: GUI; quality factor priorities –Maintainers: programming language, reuse, documentation –Customers/Owners: legacy compatibility, advanced vs. mature technology, full business case Favoring stakeholders who agree with you –Excessive privacy protection: customers vs. users Weighting stakeholders equally on each issue –Users on GUI; owners on legacy compatibility: developers on cost/schedule/risk Promising more than you can deliver Borrowing from other projects without credit Suppressing or delaying bad news QR for LCO Package 11/02/2010

University of Southern California Center for Software Engineering C S E USC © USC-CSSE25 Conclusions Software engineers have increasing power to do public harm or good Value-based codes of ethics are hard to integrate with value-neutral software engineering practices Rawls’ Theory of Justice enables constructive approach for integrating ethics into daily software engineering practice –Stakeholder win-win with least-advantaged system dependents as success-critical stakeholders –ICSM provides daily-practice framework 11/02/2010