Computing Optimal Randomized Resource Allocations for Massive Security Games Presenter : Jen Hua Chi Advisor : Yeong Sung Lin

Agenda  Introduction  Stackelberg games  Compact Security Game Model  Algorithms: ERASER, ORIGAMI, ORIGAMI-MILP, ERASER (-C)  Evaluation 2

Introduction  Motivation : Providing security for transportation systems, computer networks, and other critical infrastructure.  Recent work ： Paruchuri et. al. uses a game-theoretic approach to create randomized security policies for traffic checkpoints and canine patrols at the Los Angeles International Airport (LAX). 3

Introduction  A limitation of existing solution methods ： 1. Size 2. Computing infeasible  Now: subway systems, random baggage screening, container inspections at ports, and scheduling for the Federal Air Marshals Service.  US commercial airlines fly 27,000 domestic flights and over 2000 international flight each day. The Federal Air Marshal Service (FAMS) has law enforcement authority for commercial air transportation.  Flights should not necessarily have equal weighting in a randomized schedule. 4

Introduction  Key questions ： 1.How to efficiently allocate resources to protect against a wide variety of potential threats? 2.The adversarial aspect of security domains poses unique challenges for resource allocation.  Randomization => Game Theory 5

Problems  An individual air marshal’s potential departures are constrained by their current location, and schedules must account for flight and transition times.  Normal form of Stackelberg game model can only present the the cost of a combinatorial explosion in the size of the strategy space and payoff representation. => Compact Security Game Model 6

Introduction  New techniques: 1. Randomized security resource allocation 2. First algorithm (ERASER) Dramatically reduce both space and time requirements for the multiple-resource case 3. Two additional algorithms: (ORIGAMI, ORIGAMI-MILP) Improving performance 4. Incorporating additional scheduling and resource constraints into the model: ERASER-C 7

Agenda  Introduction  Stackelberg games  Compact Security Game Model  Algorithms: ERASER, ORIGAMI, ORIGAMI-MILP, ERASER (-C)  Evaluation 8

Stackelberg Games  Stackelberg Games  A ‘leader’ moves first, and the ‘follower’ observes the leader’s strategy before acting.  Related games: border patrolling, computer job scheduling, security patrolling.  It can models the capability of malicious attackers to employ surveillance in planning attacks. 9

Stackelberg Games  Normal form: 1. Two players: a defender, Θ an attacker, ψ 2. Pure strategy set: σ Θ Є Σ Θ, σ ψ Є Σ ψ 3. Mixed strategy set: δ Θ Є Δ Θ, δ ψ Є Δ ψ 4. Payoffs for each player: Ω Θ ： Σ ψ x Σ Θ  R 5. Attacker’s strategy function: F ψ ： Δ Θ  Δ ψ 10

Stackelberg Equilibrium  It is a form of subgame perfect equilibrium.  Subgame ： partial sequences of actions  Two types of unique Stackelberg equilibria: 1. Strong 2. Weak  A strong Stackelberg equilibrium exists in all Stackelberg games, but a weak Stackelberg equilibrium may not. 11

Definition 1 Attacker 根據 defender 行為， 選擇對應的策略 Attacker 從 mixed strategy set 中任意選擇 12

Agenda  Introduction  Stackelberg games  Compact Security Game Model  Algorithms: ERASER, ORIGAMI, ORIGAMI-MILP, ERASER (-C)  Evaluation 13

Compact Security Game Model  A set of targets that may be attacked: T = {t 1, …, t n }  A set of resources available to cover these targets, R = {r 1,..., r m } (all resources are identical)  Four payoffs of each target:  Example payoffs for an attack on a target. CoveredUncovered Defender Attacker CoveredUncovered Defender5-20 Attacker

Compact Security Game Model  Restrict to attack a single target with probability 1 15 VectorMeaning CA coverage vector ctct The probability that each target is covered AThe probability of attacking a target

Compact Security Game Model CoveredUncovered 16

Compact Security Game Model  In a strong Stackelberg equilibrium, the attacker selects the target in the attack set with maximum payoff for the defender.  Let t ∗ denote this optimal target  PlayersThe expected SSE payoff Defender Attacker 17

Compact Security Game Model  Any security game represented in this compact form can also be represented in normal form.  Attack vector A  attacker’s pure strategies  For the defender, each possible allocation of resources corresponds to a pure strategy in the normal form.  There are n choose m ways to allocate m resources to n targets. 18

Compact Security Game Model 19

Compact Security Game Model FormPlayerStrategyPayoff function size Compact form Defendern continuous variables 4n variables Attackern continuous variables Normal form Defendern Choose m variablesn(n Choose m) Attackern continuous variables 20

Agenda  Introduction  Stackelberg games  Compact Security Game Model  Algorithms: ERASER, ORIGAMI, ORIGAMI-MILP, ERASER (-C)  Evaluation 21

ERASER  ERASER algorithm (Efficient Randomized Allocation of SEcurity Resources) 1. Input: a security game in compact form 2. A mixed-integer linear program (MILP) 22

ERASER max d (5) a t ∈ {0, 1} ∀ t ∈ T (6) Σ t ∈ T a t = 1 (7) c t ∈ [0, 1] ∀ t ∈ T (8) Σ t ∈ T c t ≤ m (9) d − U Θ (t,C) ≤ (1 − a t ) Z ∀ t ∈ T (10) 0 ≤ k − U Ψ (t,C) ≤(1 − a t ) Z ∀ t ∈ T (11) U Ψ (t,C) ≤ k 23

ERASER  THEOREM 1. For any feasible ERASER coverage vector, there is a corresponding mixed strategy δ Θ that implements the desired coverage probabilities.  THEOREM 2. A pair of attack and coverage vectors (C,A) is optimal for the ERASER MILP correspond to at least one SSE of the game. 24

Compact Security Game Model 25

Compact Security Game Model 26

ORIGAMI algorithm 27

ORIGAMI algorithm 28

ORIGAMI algorithm  THEOREM 3. ORIGAMI computes a coverage vector C that is optimal for the ERASER MILP, and is therefore consistent with a SSE of the security game. 29

ORIGAMI- MILP  ORIGAMI algorithm + MILP algorithm min k (12) γ t ∈ {0, 1} ∀ t ∈ T (13) c t ∈ [0, 1] ∀ t ∈ T (14) Σ t ∈ T c t ≤ m (15) U Ψ (t,C) ≤ k ∀ t ∈ T (16) k − U Ψ (t,C) ≤ (1 − γ t ) ・ Z ∀ t ∈ T (17) c t ≤ γ t ∀ t ∈ T (18) 30

ORIGAMI- MILP  THEOREM 4. ORIGAMI-MILP generates an optimal solution for the ERASER MILP. 31

Resource constraints  Modeling air marshals as resources, flights as targets, with payoffs defined by expert risk analysis.  Resource types can be used to specify different sets of legal schedules for each resource.  Adding these constraints effectively reduces the space of feasible coverage vectors.  Example:

ERASER-C (constrained) . Adding the capability to represent certain kinds of resource and scheduling constraints  The first extension allows resources to be assigned to schedules covering multiple targets.  The second extension introduces resource types,Ω = {ω 1,..., ω v }. 33

ERASER-C (constrained)  Variables MeaningVariables/ Functions The relationship between targets and schedules M : S × T → {0, 1} The number of available resources of each type R(ω) Coverage capabilities for each type Ca : S × Ω → {0, 1} The total probability that is assigned to each schedule by all resource types q The probability assigned to a schedule by a specific type of resource h Large constant, relative to the maximum payoff Z 34

ERASER-C (constrained) max d (19) a t ∈ {0, 1} ∀ t ∈ T (20) c t ∈ [0, 1] ∀ t ∈ T (21) q s ∈ [0, 1] ∀ s ∈ S (22) h s,ω ∈ [0, 1] ∀ s, ω ∈ S × Ω (23) Σ t ∈ T a t = 1 (24) Σ ω ∈ Ω h s,ω = q s ∀ s ∈ S (25) Σ s ∈ S q s M(s, t) = c t ∀ t ∈ T (26) Σ s ∈ S h s,ω Ca(s, ω)≤ R(ω) ∀ ω ∈ Ω (27) h s,ω ≤ Ca(s, ω) ∀ s, ω ∈ S × Ω (28) d − U Θ (t,C) ≤ (1 − a t ) ・ Z ∀ t ∈ T (29) 0 ≤ k − U Ψ (t,C)≤ (1 − a t ) ・ Z ∀ t ∈ T (30) 35

Agenda  Introduction  Stackelberg games  Compact Security Game Model  Algorithms: ERASER, ORIGAMI, ORIGAMI-MILP, ERASER (-C)  Evaluation 36

Evaluation  DOBSS  The ordering of the algorithms in terms of the size of the class of games: ORIGAMI/ORIGAMI-MILP ⊂ ERASER ⊂ ERASER-C ⊂ DOBSS.  First set of experiments: Compares the performance of DOBSS, ERASER, and ERASER-C on random game instances.  Next comparison: ERASER, ORIGAMI, and ORIGAMI-MILP on much larger instances that DOBSS is unable to solve.  Final experiment: Compares the algorithms on relevant example games for the LAX and FAMS domains. 37

Evaluation (The first set of tests) (a) Runtimes for DOBSS, ERASER, and ERASER-C (b) Memory use of DOBSS, ERASER, and ERASER-C 38

Evaluation (The first set of tests)  Comparing the performance of ERASER-C and DOBSS on games.  Random game instances now include schedules, resource types, and coverage mappings.  We test games with 3 resource types, and availability of [3, 3, 2] for each type.  There are twice as many schedules as targets, and each schedule covers a randomly-selected set of two targets. (c) Runtimes for DOBSS and ERASER-C (d) Memory use of DOBSS and ERASER-C 39

Evaluation (The second set of tests)  Comparing the performance of ERASER, ORIGAMI, and ORIGAMI-MILP on very large games well beyond the limits of DOBSS. (e)Runtime scaling of ERASER, ORIGAMI, and ORIGAMI-MILP (f)Runtime scaling of ORIGAMI, and ORIGAMI-MILP Comparing the runtimes of the three algorithms on games with 25 resources and up to 3000 targets. Comparing the runtimes of the two algorithms on games with 1000 resources and up to 4000 targets. 40

Evaluation (Real data) ActionsDOBSSERASER (-C) LAX (6 canines) s0.23s FAMS (small)~6, s0.09s FAMS (large)~85, s*1.57s Table 2: Runtimes on real data. Both examples cover a one week period, but cover different foreign and domestic airports to generate "small" and "large" tests. 41

Limitation  Additional constraints are necessary if there are odd cycles possible in the schedules. 42

Thanks for your attention