1. Software Engineering Ethics
Barry Boehm
Fall 2017

2. Outline Definitions and context Principles and examples
Power to do public harm or good
ACM/IEEE Software Engineering Code of Ethics
Principles and examples
Rawls’ Theory of Justice
Relation to stakeholder win-win
Case study: Mercy Hospital
Integrating ethics into daily software engineering practices
VBSE/MBASE/Win Win Spiral Model
Fall 2017
© USC-CSSE

3. Definition of “Ethics” -Webster, 1993
The discipline dealing with what is good and bad
And with moral duty and obligation
A theory, system, or set of moral principles or values
The principles of conduct governing an individual or group
Professional ethics
Fall 2017
© USC-CSSE

4. Context
Software engineers have increasing power to do public harm or good
Intellectual property, privacy, confidentiality, quality of work, fairness, liability, risk disclosure, conflict of interest, unauthorized access
Professional societies have developed codes of ethics
Hard to integrate value-based ethics into value-neutral software engineering practices
VBSE/MBASE/Win Win Spiral enable ethics integration
Fall 2017
© USC-CSSE

5. Power to Do Public Harm or Good – I
Intellectual Property: use without credit; use copyrighted material
Privacy: credit, health, personal information
Confidentiality: competitive information, political sensitivity
Quality of work: many dimensions; see table
Fall 2017
© USC-CSSE

6. Example: Confidentiality
Government agency hires company to support SW procurement
Provides data under nondisclosure agreement
Employee and company consultant prepare cost estimate
Employee: “ I don’t see how anyone can do all this for $8M”
Consultant provides $8M target cost to some bidders
Government agency angry with company for leak
Whose fault? How could it be avoided?
Fall 2017
© USC-CSSE

7. Quality Concerns Vary by Stakeholders Role
Developers, Maintainers
System Dependents
Stakeholder Classes
System Controllers
**Critical
*Significant
0 Insignificant or
indirect
Info Suppliers
Info Brokers
Administrators
Acquirers
Information Consumers
Mission -
Protection
critical
uncrit.
Safety **
 **
Security *
Privacy
Robustness
Reliability
Availability
Survivability
Quality of Service
Performance
Accuracy, Consistency
Accessibility, ease of use;
difficulty of misuse
Evolvability
Interoperability
Correctness
Cost
Schedule
Reusability
Fall 2017
© USC-CSSE

8. Power to Do Public Harm or Good - II
Fairness: equality of opportunity/treatment; fair reward system
Liability: accountability; parity of authority and responsibility
Risk Disclosure: safety tests, COTS capabilities; schedule slips
Conflict of Interest: source selection; personnel or product reviews
Unauthorized Access: reading, copying, modifying; denial of service
Fall 2017
© USC-CSSE

9. Examples: Fairness
Enron software to schedule power outages, raise prices
Suppose you had been asked to develop it?
Urban fire dispatching system
Inefficient old system caused $700M property loss
New-system spec. includes dispatching algorithm to minimize property loss
Any fairness issues?
Fall 2017
© USC-CSSE

10. Example: Safety Tests
Your company is delivering a drug prescription fulfillment system
Reusing software from a warehouse inventory system
You are the quality assurance manager
With company responsibility for certifying product safety
The software has passed all the contracted tests
But many off-nominal conditions untested
Some have shown unsafe outcomes
You feel more off-nominal testing if necessary
Company president says if you don’t certify safety by delivery date, company may go out of business
What should you do?
Fall 2017
© USC-CSSE

11. ACM/IEE Software Engineering Code of Ethics -Table of Contents
Products: achievable goals, realistic estimates, high quality
Public: safety, respect of diversity, public interest first
Judgment: objectivity, no bribes or conflicts of interest
Client and Employer: no employer-adverse interests, surface problems
Management: fair, ethical work rules, due process for violations
Profession: support profession and ethics code, don’t misrepresent software
Colleagues: credit colleagues’ work, give colleagues a fair hearing
Self: improve your technical and ethical knowledge and practices
Fall 2017
© USC-CSSE

12. Code of Ethics 2. Public
2.01 Disclose any software-related dangers 2.02 Approve only safe, well tested software 2.03 Only sign documents in area of competence 2.04 Cooperate on matters of public concern 2.05 Produce software that respects diversity 2.06 Be fair and truthful in all matters 2.07 Always put the public’s interest first 2.08 Donate professional skills to good causes 2.10 Accept responsibility for your own work
Fall 2017
© USC-CSSE

13. Code of Ethics 4. Client and Employer
4.01 Provide services only where competent 4.02 Ensure resources are authentically approved 4.03 Only use property as authorized by the owner 4.04 Do not use illegally obtained software 4.05 Honor confidentiality of information 4.06 Raise matters of social concern 4.07 Inform when a project becomes problematic 4.08 Accept no detrimental outside work 4.09 Represent no interests adverse to your employer
Fall 2017
© USC-CSSE

14. Outline Definitions and context Principles and examples
Power to do public harm or good
ACM/IEEE Software Engineering Code of Ethics
Principles and examples
Rawls’ Theory of Justice
Relation to stakeholder win-win
Case study: Mercy Hospital
Integrating ethics into daily software engineering practices
VBSE/MBASE/Win Win Spiral Model
CS 577 ethics situations
Fall 2017
© USC-CSSE

15. Rawls’ Theory of Justice (1971) -Following Collins et al
Rawls’ Theory of Justice (1971) -Following Collins et al., “How Good Is Good Enough?” Comm.ACM, Jan. 1994
Fair rules of conduct
Principles of justice
Participants and obligations
Provider (developer)
Buyer (acquirer)
User(s)
Penumbra (general public)
Negotiate mutually satisfactory (win-win) agreements
Fall 2017
© USC-CSSE

16. Rawls’ Theory of Justice - II
Fair rules of conduct
Negotiation among interested parties
Veil of ignorance (about what affects whom)
Rationality
Principles
Least Advantaged - don’t increase harm to them
Harm = probability x magnitude (~risk exposure)
Risking harm - don’t risk increasing harm
Don’t use “low-threat” software in “high-threat” context
Publicity test - defensible with honor before an informed public
Use for difficult cost-benefit tradeoffs
Fall 2017
© USC-CSSE

17. Obligations of the Software Provider
Fall 2017
© USC-CSSE

18. Obligations of the Software Buyer
Fall 2017
© USC-CSSE

19. Obligations of the Software User
Fall 2017
© USC-CSSE

20. Obligations of the Software Penumbra
Fall 2017
© USC-CSSE

21. Case Study: Mercy Hospital Pharmacy System -Collins et al., 1994
Growing hospital
Manual pharmacy information system reaching overload
Spec developed for PC-based information system
Rachel: VP, Records & Automation
George: Chief Pharmacist
System developed by consultants
Hired by George
Rachel: test procedures
Based on mature warehouse inventory system
Budgeted 50% more testing than other bidders
Installation & Training discovers problems
Helen: consultant in charge of installation & training
Ann: skeptical nurse cross-checking computer outputs
Fall 2017
© USC-CSSE

22. Mercy Hospital Pharmacy System: Problems
Dosage problems from data entry errors
10x dosage; wrong patient
Cross-checking incomplete; not trusted by some doctors
Heavier data-entry load
Formalizing automated procedures  more info. needed
Pharmacy info > warehouse info
Helen: Should go back to old system during cleanup
George: - Is old system less risky?
How do we ensure cleanup will get it right?
How much will cleanup cost?
Future practice: How to anticipate, avoid similar problems?
Fall 2017
© USC-CSSE

23. Outline Definitions and context Principles and examples
Power to do public harm or good
ACM/IEEE Software Engineering Code of Ethics
Principles and examples
Rawls’ Theory of Justice
Relation to stakeholder win-win
Case study: Mercy Hospital
Integrating ethics into daily software engineering practices
VBSE/MBASE/Win Win Spiral Model
CS 577 ethics situations
Fall 2017
© USC-CSSE

24. Mercy Hospital : Use of VBSE/MBASE/Win Win Spiral
Results chain
Add patient safety outcome, patient stakeholder representative
Rework-business-workflows initiative, including safety checks; add clerical-staff stakeholder
Stakeholder Win Win
Patient representative: safety criteria; parallel-operation phase-in
Clerical staff: prototype GUI, including safety-check support
Business Case: includes added safety costs and benefits
Risk Management: assess warehouse package safety, effects of workflow changes.
Fall 2017
© USC-CSSE

25. Use of VBSE/MBASE/Win Win Spiral-II
Concurrent Engineering
Concurrently address business workflows, GUI prototypes, COTS alternatives, feature prioritization, cost/schedule/benefits analysis, other risks
Prepare to pass LCO, LCA, CCD, and IOC anchor point milestone reviews
Monitoring and Control: Use Balanced Scorecard to track progress with respect to plans; apply corrective actions as necessary
Change as Opportunity: Look for emerging COTS pharmacy-related fulfillment systems
Fall 2017
© USC-CSSE

26. Conclusions
Software engineers have increasing power to do public harm or good
Value-based codes of ethics are hard to integrate with value-neutral software engineering practices
Rawls’ Theory of Justice enables constructive approach for integrating ethics into daily software engineering practice
Stakeholder win-win with least-advantaged system dependents as success-critical stakeholders
Win Win Spiral Model/MBASE/VBSE provides daily-practice framework
Fall 2017
© USC-CSSE