Wisdom DOES Imply Benevolence Mark R. Waser

Super-Intelligence  Ethics (except in a very small number of low-probability edge cases) So... What’s the problem?

Superintelligence does not imply benevolence Fox, J. & Shulman C. (2010) Superintelligence Does Not Imply Benevolence. In K. Mainzer (ed.), ECAP10: VIII European Conference on Computing and Philosophy (pp ) Munich: Verlag.

If machines become more intelligent than humans, will their intelligence lead them toward beneficial behavior toward humans even without specific efforts to design moral machines?

References Evolution of reciprocal altruism (Trivers 1971) Increase in scope of cooperation (Wright 2000) Reduction in rates of violence (Pinker 2007) Expanding circle of moral concern (Singer 1981) D. Gauthier J. Haidt S. Omohundro

One might generalize from this trend and argue that as machines approach and exceed human cognitive capacities, moral behavior will improve in tandem.

Ceteris Paribus (other things being equal) intelligence – the ability to achieve goals in a wide range of environments. intelligence can be far less important than goal system properties & content in determining benevolence vs. malevolence

For example, If an intelligence has the single goal to *destroy humanity*, increased intelligence will only make it more malevolent

The human motivational system is opaque, messy, and conflicted, but most importantly transient! The primary danger of AIs is entirely due to the fact that their goal system *could* be different

“Friendly AI” (Yudkowsky 2001) An artificial intelligence with a cleanly hierarchical goal system with a single top-level (monomaniacal) goal of “Friendliness” (to humans) Imagine a “Friendly AI” where Friendliness has been defined (hopefully accidentally) as *DESTROY HUMANITY*

Wisdom The goal/motivation to achieve maximal goals in terms of number and diversity. Avoids “lock-in” and short-sighted over-optimization of goals/utility functions (smoking) Avoids undesirable endgame strategies (prisoner’s dilemma) Promotes avoiding unnecessary actions that preclude reachable goals including wasting resources and alienating or destroying potential cooperators (waste not, want not)

Two conceptions of morality This picture neglects a critical distinction between 1. A system for cooperation 2. A system to protect the weak/helpless Advances one’s own ends Demands revision of our ultimate ends AIs will out-cooperate humans (Hall 2007) Will AIs revise their preferences to be more moral (Chalmers 2010)?

Paths from intelligence to moral behavior (ways in which increased intelligence might prompt behavior favorable to humans) 1. noticing direct instrumental motivations Advances one’s own ends (transient) 2. noticing instrumental benefits to enduring benevolent dispositions/trustworthiness Advances one’s own ends (permanent?) 3. causing an intrinsic desire for human welfare independent of instrumental concerns Revision of ends/desires (maybe?)

If you have a verifiable history of being trustworthy when not forced, others do not have to commit resources to defending against you – and can pass some of those savings on to you On the other hand, if you harm (or worse, destroy) interesting or useful entities, more powerful entities will likely decide that *you* need to spend resources as reparations and altruistic punishment (as well as paying the cost of enforcement)

Basic AI Drives 1. AIs will want to self-improve 2. AIs will want to be rational 3. AIs will try to preserve their utility 4. AIs will try to prevent counterfeit utility 5. AIs will be self-protective 6. AIs will want to acquire resources and use them efficiently Steve Omohundro, Proceedings of the First AGI Conference, 2008 Instrumental Goals

“Without explicit goals to the contrary, AIs are likely to behave like human sociopaths in their pursuit of resources.” Any sufficiently advanced intelligence (i.e. one with even merely adequate foresight) is guaranteed to realize and take into account the fact that not asking for help and not being concerned about others will generally only work for a brief period of time before ‘the villagers start gathering pitchforks and torches.’ Everything is easier with help & without interference Cooperation is an instrumental goal!

Goal Systems, Morality, and David Hume’s Is-Ought Divide In every system of morality, which I have hitherto met with, I have always remark'd, that the author proceeds for some time in the ordinary ways of reasoning, and establishes the being of a God, or makes observations concerning human affairs; when all of a sudden I am surpriz'd to find, that instead of the usual copulations of propositions, is, and is not, I meet with no proposition that is not connected with an ought, or an ought not. This change is imperceptible; but is however, of the last consequence. For as this ought, or ought not, expresses some new relation or affirmation, 'tis necessary that it shou'd be observ'd and explain'd; and at the same time that a reason should be given; for what seems altogether inconceivable, how this new relation can be a deduction from others, which are entirely different from it.

Ought Requires a goal or desire (or, more correctly, multiples thereof) IS the set of actions most likely to fulfill those goals/desires For the sum of all goals converges to a universal morality a superset of ^

Moral Systems Are... Haidt & Kesebir, Handbook of Social Psychology, 5 th Ed interlocking sets of values, virtues, norms, practices, identities, institutions, technologies, and evolved psychological mechanisms that work together to suppress or regulate selfishness and make cooperative social life possible.

Are values dependent upon intelligence? Humean view – values are entirely independent of intelligence Kantian view – many extremely intelligent beings would converge on (possibly benevolent) substantive normative principles upon reflection

Arguments Pro & Con Against Kantian – AIXI has no room to move from reason to values Against Kantian – Humean design is a stable equilibrium unless the utility function is self-referential Pro Kantian – Humans change our goals under reflection and “often acquire intrinsic preferences for correlates of instrumentally useful actions”.

Quick Answer 1.Values are dependent upon goals 2.Values are dependent upon instrumental goals as long as they do not conflict with primary goals 3.Intelligence allows you to see this and take advantage of it, so.... YES! EXAMPLE: Waste not, want not.

Thought Experiment How would a super-intelligence behave if it knew that it had a goal but that it wouldn’t know that goal until sometime in the future? Preserving that weak entity may be that goal Or it might have necessary knowledge/skills

Reprise: Three Views of Wisdom Waste not, want not Block as few goals as possible, particularly Omohundro drives Fulfill as many goals as possible

Power Many of those concerned about intelligent machines appear obsessed with power levels Yet, interestingly enough, power is notable in *NOT* being on Omohundro’s list ( i.e. a true instrumental goal Will greater intelligence eschew power for efficiency (in diversity)?

An Alternate View of Intelligence Greater cognitive resources leads to marked improvements in prediction and reductions in time discounting Leads to moving planning horizons out and moving from short-term REQUIREMENTS to long-term optimality Indeed, a truly intelligent entity should never be caught in a situation where.... (unless out-thought by an even greater intelligence)

“Self-Interest” vs. Ethics Higher personal utility (in the short term only) More options to choose (in the short term only) Less restrictions Higher global utility Less risk (if caught) Lower cognitive cost (fewer options, no need to track lies, etc.) Assistance & protection when needed/desired