Bio-inspired Learning for a Colony of Microsystems Rama Chellappa UMD

Who am I? A computer vision researcher Interested in representation and recognition. Over the past 10 years heavy involvement with understanding video sequences A participant in the ARO MURI on Micro air vehicles Collaborations with Prof. M. Srinivasan.

Pattern Recognition and Bees Behavior analysis of insects has led to advances in navigation, control systems etc. Goal: To automate the tracking and labeling of insect motion i.e., track the position and the behavior of insects. Joint work with Prof. M. Sreenivasan of ANU To appear in IEEE Trans. PAMI

Anatomical Modeling All insects have similar anatomy. Hard Exoskeleton, soft interior. Three major body parts- Head, Thorax and abdomen. Each body part modeled as an ellipse. Anatomical modeling ensures –Physical limits of body parts are consistent. –Accounts for structural limitations. –Accounts for correlation among orientation of body parts –Insects move in the direction of their head.

Waggle Dance- 1 Orientation of waggle axis  Direction of Food source.(with respect to sun). Intensity of waggle dance  Sweetness of food source. Frequency of waggle  Distance of food source. Parameters of interest in the waggle dance –Waggle Axis : Average orientation of Thorax during Waggle. –Duration of Waggle : Number of frames of waggle in each segment of the dance.

Mixture Modeling for Behaviors Low level Motion states –Straight, motionless, waggle, turn. Each Behavior is a Markov model on such motion states. Switching between behaviors is modeled as another Markov Process. Detect frames of waggle dance by looking at –Rate of change of Abdomen Orientation –Average absolute motion of center of abdomen in the direction perpendicular to the axis of the bee.

Shape, Motion and Behavior Encoded Particle Filter Tracking using a particle filter. Behavioral model in addition to motion model in the normal particle filter framework. Track both position and orientation of various body parts and the behavior exhibited by the bee. Observation model: – Mixture of Gaussians. – 5 Exemplars for the appearance of the bee. Maximum Likelihood estimate for both position and behavior.

Result

The Grand Challenge - 1 More and more MAVs and Microrobots will be employed for a variety of applications. Microsystems/pupil ratio will increase at an alarming rate. We need to figure out how a colony of such systems (less than 100) can organize themselves for carrying out a few well-defined tasks. –Landmark-based navigation –Terminal guidance –Perching –Surveillance –Looking for anomalies Much is known about how honeybees carry out tasks related to navigation, perching, hunting for honey, etc. –Prof. Srinivasan and several others

The Grand Challenge – 2 Examples of problems to be studied. Sensors for MAVs and Microrobots How to keep track of other microsystems in the colony –Tracking/Tagging a large number of movers in a restricted space –Bees are always on the move. Microsystems need to move only when needed. –Keeping an account of who went out and who came in –How to organize a sub-group of micro-systems for carrying out a specific task? –Activity recognition –Waggle dancing by MAVs! How to nourish microsystems? –Power, communication issues –Self-evaluation of their well being –Call for help. How will humans interact/interface with the colony?