1. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20021 Behavior Recognition and Opponent Modeling in Autonomous Multi-Robot Systems Behavior Recognition and Opponent Modeling in Autonomous Multi-Robot Systems Keith J. O’Hara College of Computing Georgia Institute of Technology kjohara@cc.gatech.edu

2. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20022 Introduction Recognizing and modeling behavior from low-level action thru high-level strategy. –Single agent primitive action –A sequence of single agent actions –Group behavior To understand opponents To understand teammates –No Communication –Communication troublesome or dangerous –Speak different “languages” Operate based on a different behavior vocabulary

3. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20023 Outline 2 Approaches –Intille and Bobick (MIT) Application of bayesian belief networks for American football play recognition. –Han and Veloso (CMU) Behavior Hidden Markov Models for robot soccer behavior recognition.

4. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20024 Important Themes Single/Multi agent Recognition of agents and primitive actions Agent subgoals, goals, intentions Group subgoals, goals, intentions Online recognition Uncertainty in Perception Uncertainty/Flexibility of Plan Use of probabilistic techniques to deal with uncertainty. Completely described action and observation spaces.

5. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20025 “Recognizing Multi-Agent Action from Visual Evidence” Recognition of American football plays from real games. –Assumes we have labeled participants with rough position and orientation estimates. Properties of the domain: –Complex –Complex: partially ordered causal events –Multi-agent –Multi-agent: parallel event streams –Uncertain: Uncertainty in –Uncertain: Uncertainty in both data and model Other domainsOther domains –Sports, military, traffic, robotics

6. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20026 Method Method inspired by model- based object recognition techniques. Database of plays (temporal structure descriptions) described by temporal and logical relationships of events. Construct “visual network” to detect individual goals (primitive actions) from visual evidence.

7. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20027 Temporal Structure Descriptions Individual Goal Action Components Object Assignment Temporal Constraints

8. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20028 Visual Networks Construct belief network (visual network) based upon visual evidence.

9. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 20029 Multi-Agent Belief Network Multi-Agent Belief Network Multi-Agent Networks normally contain at least 50 belief nodes and 40 evidence nodes Conditional and prior probabilities are determined automatically

10. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 200210 Results System of 29 tracked plays, 10 temporal play descriptions 21/25 were recognized correctly False positives are a problems. (plays that aren’t defined) Recognized single-agent behavior and multi-agent plays. Handled fuzzy temporal relationships (around, before). Not evaluated online. Assumes tracking/labeling/localization problem is solved. (Manually done in this work.) Must know entire domain of observations (player states), and all possible plans (play book).

11. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 200211 “Automated Robot Behavior Recognition” Robot Soccer –Adaptable Strategy –Narrative Agents –Coaches Formalism –Agent R is the observed robot –Agent O is the observing robot –R acts according to a known set of behaviors h(i) –O has a model of the set of the possible behaviors. –O must decide which h(i), R is performing. Must be online algorithm. One observed robot and one observed ball.

12. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 200212 Use Hidden Markov Models (HMMs) to recognize behaviors –Motivated by success of HMMs in other “recognition” tasks. (e.g. speech, gesture) A Behavioral HMM() for each behavior –Set of States Initial, intermediate, accept, reject –Observations Space Absolute/Relative Position, Dynamic (velocity) –State Transition Matrix –Observation Probabilities –Initial State Distribution P(this state | observations, ) Method(1) s1 s2 s3 s4 O1O2, O3O3 O1 Go-To-Ball O1 O2 O3

13. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 200213 The BHMM() –Set of States –Observations Space –State Transition Matrix –Observation Probabilities –Initial State Distribution Method(2) s1 s2 s3 s4 O1O2, O3O3 O1 Go-To-Ball O1 O2 O3

14. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 200214 Results Online algorithm Applied to robotics domain (simulation/real-robots) Implemented everyone’s favorite behaviors –Go-To-Ball, Go-Behind-Ball, Intercept-Ball, Goalie-Align-Ball Not much quantitative evidence. Only single agent case. Assume each behavior to be a sequence of state traversals. BHMM and behavior initial states must match up, or use a timeout/restart mechanism. –Mentioned by Intille and Bobick as a problem with treating temporal constraints as first-order markovian.

15. K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct 10 200215 Conclusions New and hard problem. Use of probabilistic techniques to deal with uncertainty in perception and the plan. Completely described action and observation spaces.