Download presentation
Presentation is loading. Please wait.
1
Some thoughts on the future
Stuart Russell University of California, Berkeley
6
What’s going on? Have we thought enough about what that would mean?
Eventually, AI systems will make better* decisions than humans Taking into account more information, looking further into the future The biggest event in human history: Everything we have is the product of intelligence An end to war, disease, poverty, ecological degradation Shaping our own destiny in the universe Have we thought enough about what that would mean?
7
Why is there a problem? Obviously, smarter-than-human AI systems are intrinsically hard to predict and control In particular, AI systems that are incredibly good at achieving something other than what we* really* want AI, economics, statistics, operations research, control theory all assume utility to be exogenously specified I.e., it’s someone else’s job to get it right!
8
Value misalignment If we use, to achieve our purposes, a mechanical agency with whose operation we cannot interfere effectively … we had better be quite sure that the purpose put into the machine is the purpose which we really desire Norbert Wiener, 1960 King Midas, c540 BCE
9
Responses so far: It’ll never happen It’s too soon to worry about it
See Rutherford, 9/11/33, Szilard 9/12/33 It’s too soon to worry about it 2066 asteroid collision: too soon to worry? When should we have worried about climate change? It’s like worrying about overpopulation on Mars We are spending billions moving humanity to Mars with no plan for what to breathe Just don’t have explicit goals for the AI We need to steer straight, not remove the steering wheel Don’t worry, we’ll just have human-AI teams Value misalignment precludes teamwork
10
Responses so far (contd.):
You can’t control research Yes, we can and have done for genetic engineering and biosafety protocols You’re just Luddites Fusion researchers are Luddites if they point out the need for containment? Alan Turing, Norbert Wiener, Bill Gates, and Elon Musk are Luddites? Don’t worry, we can just switch it off Sure, and we can just checkmate Deep Blue Only humans care about self-preservation Wrong
11
Self-preservation A natural subgoal of almost any objective
Death isn’t bad per se. It’s just hard to get the milk after you’re dead
12
Time for AI to get serious
Like physics and biology, AI can significantly impact humanity AI must expand its concerns to ensure the best possible outcome Value alignment research Corrigibility research etc.
13
Cooperative inverse reinforcement learning
Learn a multiagent value function whose Nash equilibria optimize the payoff for humans Vast amounts of evidence for human behavior and human attitudes towards that behavior We need value alignment even for subintelligent systems in human environments; strong economic incentives! Humans are irrational, inconsistent, weak-willed, computationally limited, and heterogeneous
14
Wiener, contd. [this work] requires an imaginative forward glance at history which is difficult, exacting, and only partially achievable. … We must always exert the full strength of our imagination to examine where the full use of our new modalities may lead us
15
OK, so…. Foresight on development paths and barriers for integrated general AI Design templates for safe AI Value function industry Inculcating ethics/common sense in AI community (cf. civil engineering) Ethics research contributes at systemic level Professional societies speaking for AI and helping to develop policy
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.