1. An Accelerated Gradient Method for Multi-Agent Planning in Factored MDPs Sue Ann HongGeoff Gordon CarnegieMellonUniversity

2. Multi-agent planning Optimize Shared constraints resources Individual constraints Individual objective

3. Individual constraints Individual objective Want: an efficient, distributed solver Factored MDPs [Guestrin et al., 2002] MDP: maximize linear reward Piece-wise linear constraints on shared resources Optimize Shared constraints resources Fast solver: value iteration

4. Distributed optimization Lagrangian relaxation How to set the prices? Gradient-based methods. Resource 1 @ $100 NO 1 2 2 $100 $50 $200 $80 $300 Solve in a distributed fashion

5. FISTA for factored MDPs linear objective  : augment with a strongly convex function: causal entropy [Ziebart et al., 2010] – Usually regularization towards a more uniform policy – Retains a fast individual planner (softmax value iteration) – Introduces smoothing error (to the linear objective) We show that the gain in convergence can outweigh the approximation (smoothing) error.