2. Tracking Social Insects Ashok Veeraraghavan Rama Chellappa

3. Problem Statement  Behavior analysis of insects has led to advances in navigation, control systems etc.  Bulk of the time in behavioral analysis is spent in manually tracking and labeling insect motions.  Goal: To automate tracking and labeling of insect motions i.e., track the position and the behavior of insects.

4. Challenges  Extreme Clutter  Presence of hundreds of other insects in the surroundings.  Insects are capable of making surprisingly fast movements.  Complex interactions among the various insects.  Drastic Appearance Changes

5. General Principles for Tracking Insects  Anatomical (or Structural) Modeling  Behavioral Modeling  Interaction Modeling

6. Anatomical Modeling  All insects have similar anatomy.  Hard Exoskeleton, soft interior.  Three major body parts- Head, Thorax and abdomen.  Each body part modeled as an ellipse.  Anatomical modeling ensures Physical limits of body parts are consistent. Accounts for structural limitations. Accounts for correlation among orientation of body parts Insects move in the direction of their head.

7. Behavioral Modeling  Insects display very specific behaviors.  Behavior Modeling Vs Motion Modeling (Level of abstraction)  Modeling behavior explicitly improve Tracking performance Behavioral understanding  Position tracking and behavioral analysis in a unified framework.

8. Modeling Interactions  Insects communicate through behavioral interactions.  Modeling these interactions is important to study behavior based communication in insects.

9. Tracking a Bee in a Hive  Several bees moving and interacting in a hive.  Simultaneously track and analyse the behavior of bees.  Specifically interested in bees performing the waggle dance.  Structural Model -  Three Ellipses.

10. Waggle Dance- A behavior  Foragers perform waggle dance.  Orientation of waggle axis  Direction of Food source.(with respect to sun).  Intensity of waggle dance  Sweetness of food source.  Frequency of waggle  Distance of food source.  Recruits follow the dancer.  Behavior Modeling: Markov Model on basic motions.

11. Tracking- Shape and Motion Encoded Particle Filter  Tracking based on particle filter.  Behavioral model in addition to motion model in the normal particle filter framework.  Track both position and orientation of various body parts and the behavior exhibited by the bee.  Observation model: Mixture of Gaussians. 5 Exemplars for the appearance of the bee.  Maximum Likelihood estimate for both position and behavior.

12. Results

13. Results (Contd.)  Successfully tracked the dancer.  Identified the various actions exhibited by the dancer.  Currently working on modeling interaction between dancer and follower.