A Safe Ethical System for Intelligent Machines Mark R. Waser

Either ethical percepts, such as justice and human rights, are independent of human experience or else they are human inventions. E. O. Wilson Centuries of debate on the origin of ethics comes down to this:

In nature, cooperation appears wherever the necessary cognitive machinery exists to support it Vampire bats (Wilkinson) Blue Jays (Stephens, McLinn, & Stevens) Cotton-Top Tamarins (Hauser, et al)

Be nice/don’t defect Retaliate Forgive Axelrod's Evolution of Cooperation and decades of follow-on evolutionary game theory provide the theoretical underpinnings. “Selfish individuals, for their own selfish good, should be nice and forgiving”

Assuming that ETHICAL ACTIONS are defined as those which are the necessary basis for optimizing cooperation Acting ethically is an attractor in the state space of intelligent goal-driven systems. (assuming they interact with other intelligent goal-driven systems on a long-term ongoing basis)

Optimizing  Utilitarianism So why do the vast majority of people believe that it is unethical to grab someone off the street to serve as an involuntary organ donor for six dying patients?

Ethics is as much a human invention... Natural physical laws dictate the design of the optimal steam engine... as the steam engine.... and the same is true of ethics.

ONE non-organ donor SIX dying patients > + ONE unbroken principle Credit to: Eric Baum What Is Thought?

A Safe Ethical System for Intelligent Machines Not innately inimical (unfair to humans) Not a push-over (unfair to itself) Not unfair to others (which could save us)

A sense of fairness also appears wherever the necessary cognitive machinery exists to support it Dogs (Range et al.) Monkeys (Brosnan & de Wall

Altruistic Punishment & Outrage (the fastest, safest route to ethical warbots)

Baar’s Global Workspace Theory Most of cognition is implemented by a multitude of small, local, special purpose processes, that are almost always unconscious Coalitions of these processes compete, whenever necessary, for conscious attention (access to a limited capacity global workspace) Attention then serves as an I NTEGRATION P OINT that allows us to deal with novel or challenging situations that cannot be dealt with efficiently, or at all by local, routine, unconscious processes (Also Perlis 2008)

Sloman’s architecture for a human-like agent (Sloman 1999)

Reprise Simple, Safe, Stable, Self-Correcting, and Sensitive to current human thinking, intuitions, and feelings

Next.... A Safe Motivational System for Intelligent Machines Copies of this powerpoint available from