Zach Ramaekers Computer Science University of Nebraska at Omaha Advisor: Dr. Raj Dasgupta 1

Modular Self-Reconfigurable Robots (MSRs) – What and Why An MSR is a type of robot that is composed of identical modules The modules connect together to form larger robots capable of performing complex tasks Why MSRs? Inexpensive and Simple Highly Adaptable Three main types of MSRs: Chain, Lattice, Hybrid 2

ModRED (Modular Robot for Exploration and Discovery) Novel 4 degrees of freedom design Gives improved dexterity Allows maneuver itself and get out of tight spaces’ ModRED sensors and actuators Arduino processor (for doing computations) IR sensors(for sensing how far obstacles are) Compass (which direction am I heading) Tilt sensor (what is my inclination) XBee radio (for wireless comm.) Designed by Dr. Nelson’s group, Mechanical Engineering, UNL 3 CAD diagram of robot Simulated robot in Webots

ModRED Movements in Fixed Configuration 4 All these movements are in a fixed configuration

Problem Addressed: Dynamic Self- Reconfiguration by MSRs 5 Why does an MSR need to reconfigure dynamically? Problem Statement: How can an MSR that needs reconfigure (e.g., after encountering an obstacle) determine 1. which other modules to combine with, and 2. the best configuration to form with those modules?... in an autonomous manner.

Coalition Games for Dynamic MSR Reconfiguration We propose a novel, coalition game theory based approach to address the problem of MSR self-reconfiguration A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselves Teams are guranteed to be stable: once teams are formed no one will want to change teams 6 Game Theoretic Layer (Coalition Game) Controller Layer (Gait-tables) Mediator Dynamic self- reconfiguration Movement in Fixed Configuration

Coalition Games for Dynamic MSR Reconfiguration We propose a novel, coalition game theory based approach to address the problem of MSR self-reconfiguration A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselves Teams are guranteed to be stable: once teams are formed no one will want to change teams 7 Game Theoretic Layer (Coalition Game) Controller Layer (Gait-tables) Mediator Dynamic self- reconfiguration Movement in Fixed Configuration For our scenario: Each module of an MSR is provided with embedded software called an agent that does the coalition game related calculations

Coalition Games for Dynamic MSR Reconfiguration We propose a novel, coalition game theory based approach to address the problem of MSR self-reconfiguration A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselves Teams are guranteed to be stable: once teams are formed no one will want to change teams 8 Game Theoretic Layer (Coalition Game) Controller Layer (Gait-tables) Mediator Dynamic self- reconfiguration Movement in Fixed Configuration For our scenario: Each module of an MSR is provided with embedded software called an agent that does the coalition game related calculations Our problem: How can we determine these teams or partitions or coalitions for our MSR problem?

Determining the Partitions Enumerate all possible partitions that includes all the agents - called coalition structures 9 {1} {2}{3}{4}{1, 2}{3}{4}{1} {2,3}{4} Example coalition structures with 4 agents  Each coalition structure is associated with a value  V(CS i ) = sum of utilities of each coalition in CS i  All the possible coalition structures are represented as a coalition structure graph V(CS i ) = u(1,2) + u(3) + u(4) V(CS i ) = u(1) + u(2,3) + u(4) V(CS i ) = u(1) + u(2) + u(3) + u(4)

Coalition Structure Graph (CSG) 10 Possible partitions of 4 modules (agents) Problem: Find the node (coaliltion structure) that has the highest value in this graph

Coalition Structure Graph (CSG) Finding the optimal coalition structure node in this graph is not easy! CSG has  (n n ) nodes, the search problem is hard (NP-complete) Approximation algorithm used to find the optimal CSG node (Sandholm 1999, Rahwan 2007, etc.): in exponential time. 11 Possible partitions of 4 modules (agents) Problem: Find the node (coaliltion structure) that has the highest value in this graph

Dynamic Reconfiguration under Uncertainty 12 I need to form another configuration We are here And we are here Which modules should I join with? Ridge in planned path

Modeling Uncertainty in Coalition Formation 13 {1} {2}{3}{4}{1, 2}{3}{4}{1} {2,3}{4} Possible Coalition Structures Conventional CSG V(CS i ) = u(1,2) + u(3) + u(4) (additive reward value) {1, 2}{3}{4} CSG with uncertainty V(CS i ) < u(1,2) + u(3) + u(4) (subadditive) V(CS i ) = u(1,2) + u(3) + u(4) (additive) V(CS i ) > u(1,2) + u(3) + u(4) (superadditive) {1, 2}{3}{4}

Modeling Uncertainty in Coalition Formation Dealing with uncertainty: Markov Decision Process (MDP) provide a mathematical model for robots or agents to determine their actions in the presence of uncertainty 14 {1} {2}{3}{4}{1, 2}{3}{4}{1} {2,3}{4} Possible Coalition Structures Conventional CSG V(CS i ) = u(1,2) + u(3) + u(4) (additive reward value) {1, 2}{3}{4} CSG with uncertainty V(CS i ) < u(1,2) + u(3) + u(4) (subadditive) V(CS i ) = u(1,2) + u(3) + u(4) (additive) V(CS i ) > u(1,2) + u(3) + u(4) (superadditive) {1, 2}{3}{4}

MDP-Based CSG Solution 15 Conventional CSG without uncertainty Modified CSG with uncertainty

Algorithm to find optimal coalition structure Pruning – used to reduce the number of nodes that are searched by the algorithm in the coalition structure graph Three strategies explored: Keep the optimal and two least optimal children; keep the optimal, median, and least optimal children; keep three random children 16 Set of modules information Generate Coalition Utility Values Generate Coalition Structure Graph Run Value Iteration and Determine Policy Run MDP Traversal to Find Optimal CS Optimal Coalition Structure

Simulation Results 17

Conclusions and Future Work Developed coalition game theory based algorithm for MSR self-reconfiguration Validated to work on accurate model of MSR called ModRED within Webots robotic simulator To the best of our knowledge First application of game theory to MSR self- reconfiguration problem First attempt at combining planning under uncertainty (MDP) with coalition games* Future work Investigate distributed models of planning under uncertainty (MMDPs, DEC-MDPs, etc.) Simulation of exploration and coverage on realistic terrains Integrate with hardware of ModRED robot 18 *: Suijs et al.(1999) defined a framework called stochastic coalition game, but using a different model involving agent types, which wasn’t validated empirically.

ModRED Self-Reconfiguration Simulation Demo 19