1. Software Engineering Ethics and Future Responsibilities
Barry Boehm
2/16/2019
© USC-CSSE

2. Outline Definitions and context Principles and examples
Power to do public harm or good
ACM/IEEE Software Engineering Code of Ethics
Principles and examples
Rawls’ Theory of Justice
Relation to stakeholder win-win
CS 577 ethics situations
Ethics challenges for your future careers
Global threats, nanosecond decisions
Coping with AI, Internets of Things, DevOps, Systems of Systems
2/16/2019
© USC-CSSE

3. Definition of “Ethics” -Webster, 1993
The discipline dealing with what is good and bad
And with moral duty and obligation
A theory, system, or set of moral principles or values
The principles of conduct governing an individual or group
Professional ethics
2/16/2019
© USC-CSSE

4. Context
Software engineers will have vastly increasing power to do public harm or good
System safety and security, intellectual property, privacy, quality of work, fairness, liability, risk disclosure, conflict of interest, unauthorized access
Professional societies have developed codes of ethics
Integrating value-based ethics into value-neutral software engineering practices via ICSM
Future software engineers and managers (you)
will need to cope with complex global threats, nanosecond decisions, AI, Internets of Things, DevOps, systems of systems
2/16/2019
© USC-CSSE

5. Power to Do Public Harm or Good – I
Intellectual Property: use without credit; use copyrighted material
Privacy: credit, health, personal information
Confidentiality: competitive information, political sensitivity
Quality of work: many dimensions; see table
2/16/2019
© USC-CSSE

6. Quality Concerns Vary by Stakeholders Role
Developers, Maintainers
System Dependents
Stakeholder Classes
System Controllers
**Critical
*Significant
0 Insignificant or
indirect
Info Suppliers
Info Brokers
Administrators
Acquirers
Information Consumers
Mission -
Protection
critical
uncrit.
Safety **
 **
Security *
Privacy
Robustness
Reliability
Availability
Survivability
Quality of Service
Performance
Accuracy, Consistency
Accessibility, ease of use;
difficulty of misuse
Evolvability
Interoperability
Correctness
Cost
Schedule
Reusability
2/16/2019
© USC-CSSE

7. Example: Confidentiality
Government agency hires company to support SW procurement
Provides data under nondisclosure agreement
Employee and company consultant prepare cost estimate
Employee: “ I don’t see how anyone can do all this for $8M”
Consultant provides $8M target cost to some bidders
Government agency angry with company for leak
Whose fault? How could it be avoided?
2/16/2019
© USC-CSSE

8. Power to Do Public Harm or Good - II
Fairness: equality of opportunity/treatment; fair reward system
Liability: accountability; parity of authority and responsibility
Risk Disclosure: safety tests, COTS capabilities; schedule slips
Conflict of Interest: source selection; personnel or product reviews
Unauthorized Access: reading, copying, modifying; denial of service
2/16/2019
© USC-CSSE

9. Examples: Fairness
Enron software to schedule power outages, raise prices
E.g., air conditioning for homes, offices, hospitals during heat waves
Suppose you had been asked to develop it?
Urban fire dispatching system
Inefficient old system caused $700M property loss
New-system spec. includes dispatching algorithm to minimize property loss
Any fairness issues?
2/16/2019
© USC-CSSE

10. Example: Safety Tests
Your company is delivering a drug prescription fulfillment system
Reusing software from a warehouse inventory system
You are the quality assurance manager
With company responsibility for certifying product safety
The software has passed all the contracted tests
But many off-nominal conditions untested
Some have shown unsafe outcomes
You feel more off-nominal testing if necessary
Company president says if you don’t certify safety by delivery date, company may go out of business
What should you do?
2/16/2019
© USC-CSSE

11. ACM/IEE Software Engineering Code of Ethics -Table of Contents
Products: achievable goals, realistic estimates, high quality
Public: safety, respect of diversity, public interest first
Judgment: objectivity, no bribes or conflicts of interest
Client and Employer: no employer-adverse interests, surface problems
Management: fair, ethical work rules, due process for violations
Profession: support profession and ethics code, don’t misrepresent software
Colleagues: credit colleagues’ work, give colleagues a fair hearing
Self: improve your technical and ethical knowledge and practices
2/16/2019
© USC-CSSE

12. Code of Ethics 2. Public 2.01 Disclose any software-related dangers
2.02 Approve only safe, well tested software
2.03 Only sign documents in area of competence
2.04 Cooperate on matters of public concern
2.05 Produce software that respects diversity
2.06 Be fair and truthful in all matters
2.07 Always put the public’s interest first
2.08 Donate professional skills to good causes
2.10 Accept responsibility for your own work
2/16/2019
© USC-CSSE

13. Code of Ethics 4. Client and Employer
4.01 Provide services only where competent
4.02 Ensure resources are authentically approved
4.03 Only use property as authorized by the owner
4.04 Do not use illegally obtained software
4.05 Honor confidentiality of information
4.06 Raise matters of social concern
4.07 Inform when a project becomes problematic
4.08 Accept no detrimental outside work
4.09 Represent no interests adverse to your employer
2/16/2019
© USC-CSSE

14. Outline Definitions and context Principles and examples
Power to do public harm or good
ACM/IEEE Software Engineering Code of Ethics
Principles and examples
Rawls’ Theory of Justice
Relation to stakeholder win-win
CS 577 ethics situations
Ethics challenges for your future careers
Global threats, nanosecond decisions
Coping with AI, Internets of Things, DevOps, Systems of Systems
2/16/2019
© USC-CSSE

15. Rawls’ Theory of Justice (1971) -Following Collins et al
Rawls’ Theory of Justice (1971) -Following Collins et al., “How Good Is Good Enough?” Comm.ACM, Jan. 1994
Fair rules of conduct
Principles of justice
Participants and obligations
Provider (developer)
Buyer (acquirer)
User(s)
Penumbra (general public)
Negotiate mutually satisfactory (win-win) agreements
2/16/2019
© USC-CSSE

16. Rawls’ Theory of Justice - II
Fair rules of conduct
Negotiation among interested parties
Veil of ignorance (about what affects whom)
Rationality
Principles
Least Advantaged - don’t increase harm to them
Harm = probability x magnitude (~risk exposure)
Risking harm - don’t risk increasing harm
Don’t use “low-threat” software in “high-threat” context
Publicity test - defensible with honor before an informed public
Use for difficult cost-benefit tradeoffs
2/16/2019
© USC-CSSE

17. Obligations of the Software Provider
2/16/2019
© USC-CSSE

18. Obligations of the Software Buyer
2/16/2019
© USC-CSSE

19. Obligations of the Software User
2/16/2019
© USC-CSSE

20. Obligations of the Software Penumbra
2/16/2019
© USC-CSSE

21. CS 577 Ethics Accountability -1
Assuming your priorities match those of other stakeholders
Users: GUI; quality factor priorities
Maintainers: programming language, reuse, documentation
Customers/Owners: legacy compatibility, advanced vs. mature technology, full business case
Treating stakeholders equally on each issue
Users on GUI; owners on legacy compatibility: developers on cost/schedule/risk
Promising more than you can deliver
Borrowing from other projects without credit
Suppressing or delaying bad news
2/16/2019
© USC-CSSE

22. CS 577 Ethics Accountability -2
Delivering just source code vs. including architecture, regression test cases, documentation for users, maintainers
Honoring commitments to CS 577b
Misinterpreting the Golden Rule
Do unto others as you would have others do unto you
Building programmer-friendly GUIs for doctors, lawyers, administrators
Pop-up windows for curiosity-driven vs. closure-driven users
Platinum Rule
Do unto others as they would be done unto
2/16/2019
© USC-CSSE

23. Outline Definitions and context Principles and examples
Power to do public harm or good
ACM/IEEE Software Engineering Code of Ethics
Principles and examples
Rawls’ Theory of Justice
Relation to stakeholder win-win
CS 577 ethics situations
Ethics challenges for your future careers
Global threats, nanosecond decisions
Coping with AI, Internets of Things, DevOps, Systems of Systems
2/16/2019
© USC-CSSE

24. Ethics challenges for your future careers
Reference: Thomas Friedman, Thanks for Being Late
Moore’s Law will continue
2x computer-communications speed, data capacity every 2 years, at half the price
By 2038, factors of 1024
“Software is Eating the World”
Marc Andreessen, Netscape: 2012
Much more complex Internets of Things
Challenges of compatible DevOps evolution
AI, computers making decisions in nanoseconds
Farewell to the Principle of Human Primacy
Powerful capabilities falling into the wrong hands
Denial of service, perversion of objectives
2/16/2019
© USC-CSSE

25. Robotics: Implementing Asimov’s Laws
1. Do not injure a human or allow a human to come to harm.
2. Obey orders given by humans, except when orders conflict with Law 1.
3. Protect its own existence, except when this conflicts with Laws 1 and 2.
Hard to achieve, even in simple, stable situations
2/16/2019
© USC-CSSE

26. Perversion of Objectives: Example
Zika mosquitoes carry deadly virus
One bite usually fatal
Genetic and software engineers propose gene-splicing approach to disable Zika mosquitoes to reproduce
Suppose this approach falls into the hands of organizations wishing to eliminate races of humans
Suppose this approach is mishandled and applied to all humans
Hopefully there is time to create a countermeasure
2/16/2019
© USC-CSSE

27. Conclusions
Software engineers will have vastly increasing power to do public harm or good
Rawls’ Theory of Justice enables constructive approach for integrating ethics into daily software engineering practice
ICSM Stakeholder win-win with least-advantaged system dependents as success-critical stakeholders
Responsibility challenges for your future careers
Global threats, nanosecond decisions
Coping with AI, Internets of Things, DevOps, Systems of Systems
Try to build in reversibility of bad decisions
2/16/2019
© USC-CSSE