K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Behavior Recognition and Opponent Modeling in Autonomous Multi-Robot Systems Behavior Recognition and Opponent Modeling in Autonomous Multi-Robot Systems Keith J. O’Hara College of Computing Georgia Institute of Technology

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Introduction Recognizing and modeling behavior from low-level action thru high-level strategy. –Single agent primitive action –A sequence of single agent actions –Group behavior To understand opponents To understand teammates –No Communication –Communication troublesome or dangerous –Speak different “languages” Operate based on a different behavior vocabulary

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Outline 2 Approaches –Intille and Bobick (MIT) Application of bayesian belief networks for American football play recognition. –Han and Veloso (CMU) Behavior Hidden Markov Models for robot soccer behavior recognition.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Important Themes Single/Multi agent Recognition of agents and primitive actions Agent subgoals, goals, intentions Group subgoals, goals, intentions Online recognition Uncertainty in Perception Uncertainty/Flexibility of Plan Use of probabilistic techniques to deal with uncertainty. Completely described action and observation spaces.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct “Recognizing Multi-Agent Action from Visual Evidence” Recognition of American football plays from real games. –Assumes we have labeled participants with rough position and orientation estimates. Properties of the domain: –Complex –Complex: partially ordered causal events –Multi-agent –Multi-agent: parallel event streams –Uncertain: Uncertainty in –Uncertain: Uncertainty in both data and model Other domainsOther domains –Sports, military, traffic, robotics

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Method Method inspired by model- based object recognition techniques. Database of plays (temporal structure descriptions) described by temporal and logical relationships of events. Construct “visual network” to detect individual goals (primitive actions) from visual evidence.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Temporal Structure Descriptions Individual Goal Action Components Object Assignment Temporal Constraints

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Visual Networks Construct belief network (visual network) based upon visual evidence.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Multi-Agent Belief Network Multi-Agent Belief Network Multi-Agent Networks normally contain at least 50 belief nodes and 40 evidence nodes Conditional and prior probabilities are determined automatically

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Results System of 29 tracked plays, 10 temporal play descriptions 21/25 were recognized correctly False positives are a problems. (plays that aren’t defined) Recognized single-agent behavior and multi-agent plays. Handled fuzzy temporal relationships (around, before). Not evaluated online. Assumes tracking/labeling/localization problem is solved. (Manually done in this work.) Must know entire domain of observations (player states), and all possible plans (play book).

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct “Automated Robot Behavior Recognition” Robot Soccer –Adaptable Strategy –Narrative Agents –Coaches Formalism –Agent R is the observed robot –Agent O is the observing robot –R acts according to a known set of behaviors h(i) –O has a model of the set of the possible behaviors. –O must decide which h(i), R is performing. Must be online algorithm. One observed robot and one observed ball.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Use Hidden Markov Models (HMMs) to recognize behaviors –Motivated by success of HMMs in other “recognition” tasks. (e.g. speech, gesture) A Behavioral HMM() for each behavior –Set of States Initial, intermediate, accept, reject –Observations Space Absolute/Relative Position, Dynamic (velocity) –State Transition Matrix –Observation Probabilities –Initial State Distribution P(this state | observations, ) Method(1) s1 s2 s3 s4 O1O2, O3O3 O1 Go-To-Ball O1 O2 O3

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct The BHMM() –Set of States –Observations Space –State Transition Matrix –Observation Probabilities –Initial State Distribution Method(2) s1 s2 s3 s4 O1O2, O3O3 O1 Go-To-Ball O1 O2 O3

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Results Online algorithm Applied to robotics domain (simulation/real-robots) Implemented everyone’s favorite behaviors –Go-To-Ball, Go-Behind-Ball, Intercept-Ball, Goalie-Align-Ball Not much quantitative evidence. Only single agent case. Assume each behavior to be a sequence of state traversals. BHMM and behavior initial states must match up, or use a timeout/restart mechanism. –Mentioned by Intille and Bobick as a problem with treating temporal constraints as first-order markovian.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Conclusions New and hard problem. Use of probabilistic techniques to deal with uncertainty in perception and the plan. Completely described action and observation spaces.