1. When and How to Violate Norms
Trevor Bench-Capon and Sanjay Modgil
University of Liverpool
King’s College, London

2. Overview Value-Based Practical Reasoning
Value-Based Reasoning and Norms
When (and how) to violate norms

3. Practical Reasoning v Theoretical Reasoning
Direction of fit:
In theoretical reasoning we fit our beliefs to the world
In practical reasoning we (try to) fit the world to our desires
choice

4. Value Based Practical Reasoning
Uses a state transition diagram
Transitions are labelled with values they promote and demote. Values and their ordering represent individual aims, aspirations and preferences of agents
1,1 +M -M +M
3.3
1,1
5,0
0,5

5. Justifying an Action Practical Reasoning Argumentation Scheme:
In the current circumstances R
I should do action A
To produce new circumstances S
Which will realise a goal G
Which promotes Value V
Eg. In 1,1 I should defect to move to 5,0 which increases my money and promotes my value of wealth
But this depends on the other agent cooperating
The value explains why
G is a goal
and is my reason
to perform A

6. Ant and Grasshopper and/or Parable of the Prodigal Son
Work or play?
Give or refuse?

7. The Choices without Norms
Agent will choose to play if it prefers pleasure to the risk of death (a short life, but a merry one)
The agent will refuse to give if it values its own pleasure over the other’s life (you have made your bed now lie in it)
The father in the parable chooses to give
but there is a family relationship
the son promises not to do again

8. Sustainable cycles But the play-give cycle allows for Freeloading.
Unsustainable in
The long term
Sustainability
Requires that
Agents punish
And that failure
to punish itself
be punished

9. Resulting Norms
In order to produce a sustainable society we need two norms:
Work in Summer
Refuse to give food to non-workers
From these norms abstract values arise (so that punishing doesn’t look like pure selfishness):
Justice: Punish people who violate norms
Mercy: Don’t punish violators
but there must be limits, (one or two strikes and you are out)

10. Norms and Values
I have sketched how value based reasoning can lead to norms
The norms represent a set of values, and an ordering on values
People with the right values will make the right choices
Obedience from choice not from following rules
This is important because norms contain the possibility of violation as a necessary feature:
Obeying the norms generally promote the right values
Norms are good as heuristics But not always best
The “must” of obligation is not the “must” of necessity
Rules are made to be broken

11. Standard Driving Norm (UK)
You must drive on the left
Norm of Cooperation to promote Safety and Progress
Why would anyone want to violate it?
Overtaking
Passing parked Cars
Road works
Avoiding pedestrians

12. Multi Agent Systems Very often multi-agent systems either
Ensure obedience: e.g. simply block forbidden actions
Reason with norms as rules: but rigid obedience can be counter productive (sci-fi Robot threats)
But using value-based reasoning can recognise when norms should be disobeyed
When they would demote the values they are designed to promote
How they should be violated
So as to promote the values (or minimise demotion)

13. Recognising the Value Order
A value ordering can be discovered by examining the norms and the actions of agents
What values do the norms promote?
What choices do the agents make?
What violations go unpunished?

14. AATS for Driving Example
Relevant Fragment:
States
Whether there is an obstruction and whether it is moving or stationary (0= clear, 1 = slow moving, 2 = stationary).
Whether there is an on-coming vehicle vehicle and whether it can stop or not (0 = no on-coming, 1 = can stop, 2 = cannot stop).
Whether our own vehicle can stop safely (0= can stop, 1 = cannot stop).
Whether there is a collision (0 = no collision, 1(x,y) = x collides with y).

15. AATS for Driving Example II
Actions:
For actions, both our own vehicle and the on-coming will be able to
continue, change lane, stop, slow, or mount the pavement.
Values:
our own progress and safety (P(s) and S(s)),
the progress and safety of the on-coming (P(o) and S(o)) and
the safety of any pedestrians in the area (S(p)).

16. Potentially 25 Joint Actions
Consider 2100 : there is a stationary obstacle,
and both ourselves and
the on-coming could stop,
and there is no collision (yet)

17. Consequences I
J1 – J10 (continue or slow down): norm is obeyed, but self hits obstacle, demoting both progress and safety
J11 –J15 (stop): norm is obeyed but progress is demoted
Thus compliance demotes one or both values
Violation may demote different values, according to what others do

18. Consequences 2 Changing lane demotes Mounting the pavement
The safety and progress of both ourselves and on-coming if on-coming if on-coming does not stop
If on-coming slows, its progress is demoted
If on-coming changes lane both its progress and safety are demoted
If on-coming mounts pavement, pedestrian safety may be demoted
Mounting the pavement
Risks demoting pedestrian safety
So change lane
And hope the on-coming
Is less reckless than we are
Or better stop, and demote progress – unless there is no on-coming, when the norm can be violated without risk

19. Summary Autonomous agents need to be able to reason morally
Which means violating norms when they are not appropriate, but
Violating them is the most acceptable way
Value based reasoning
Can explain and justify norms
Identify when they should be violated
Identify how to violate them