1. MULTISENSOR INTEGRATION AND FUSION Presented by: Prince Garg

2. CONTENT INTRODUCTION INTRODUCTION TYPES OF SENSORS TYPES OF SENSORS WHAT DOES MULTIPLE SENSORS MEAN? WHAT DOES MULTIPLE SENSORS MEAN? MULTI SENSOR INTEGRATION VS. FUSION MULTI SENSOR INTEGRATION VS. FUSION ROLE OF MULTI SENSOR INTEGRATION AND FUSION IN A SYSTEM ROLE OF MULTI SENSOR INTEGRATION AND FUSION IN A SYSTEM ADVANTAGES ADVANTAGES APPLICATION APPLICATION LASER RADAR AND FORWARD LOOKING INFRARED SENSORS LASER RADAR AND FORWARD LOOKING INFRARED SENSORS ADVANCEMENT ADVANCEMENT REFERENCES REFERENCES

3. INTRODUCTION Sensor is a device that detects or senses the value or changes of value of the variable being measured. Sensor is a device that detects or senses the value or changes of value of the variable being measured. The term sensor some times is used instead of the term detector, primary element or transducer. The term sensor some times is used instead of the term detector, primary element or transducer.

4. TYPES OF SENSORS CONTACT SENSORS: A contact sensor measure the response of a target to some form of physical contact.this group of sensors responds to touch, force torque,pressure,temperature or electrical quantities. CONTACT SENSORS: A contact sensor measure the response of a target to some form of physical contact.this group of sensors responds to touch, force torque,pressure,temperature or electrical quantities. NONCONTACT SENSOR: A non contact sensor measures the response brought by some form of electromagnetic radiation. This group of sensors responds to light, x-ray, acoustic, electric or magnetic radiation. NONCONTACT SENSOR: A non contact sensor measures the response brought by some form of electromagnetic radiation. This group of sensors responds to light, x-ray, acoustic, electric or magnetic radiation.

5. WHAT DOES MULTIPLE SENSORS MEAN? For these system to use effectively the multiple sensors,we require some method for integrating the information provided by these sensors into the operation of the system. For these system to use effectively the multiple sensors,we require some method for integrating the information provided by these sensors into the operation of the system. In many multi sensor system, the information from each sensor serve as the separate input to the system. In many multi sensor system, the information from each sensor serve as the separate input to the system. The actual combination or fusion of information prior to its use in the system has been a particularly active area of research. The actual combination or fusion of information prior to its use in the system has been a particularly active area of research. Typical of the application that can benefit from the use of multiple sensor are industrial task like assembly,military command and for battlefield management. Typical of the application that can benefit from the use of multiple sensor are industrial task like assembly,military command and for battlefield management.

6. MULTISENSOR INTEGRATION VS FUSION Multi sensor integration refers to the synergistic use of the information provided by the multiple sensory devices to assist in the accomplishment of a task by a system. Multi sensor integration refers to the synergistic use of the information provided by the multiple sensory devices to assist in the accomplishment of a task by a system. Multi sensor fusion refers to any stage in the integration process where there is an actual combination of different sources of the information into one representational format. Multi sensor fusion refers to any stage in the integration process where there is an actual combination of different sources of the information into one representational format. Multisensor integration is discussed in terms of basic integration functions whereas multisensor fusion in terms of the different levels at which fusion can take place. Multisensor integration is discussed in terms of basic integration functions whereas multisensor fusion in terms of the different levels at which fusion can take place. This difference leads to the conclusion that in many multi sensor integrated system, The output from one sensor may be used to guide the operation of other sensor in the system without ever actually fusing the sensor information. This difference leads to the conclusion that in many multi sensor integrated system, The output from one sensor may be used to guide the operation of other sensor in the system without ever actually fusing the sensor information.

7. ROLE OF MULTISENSOR INTEGRATION AND FUSION IN A SYSTEM The role of multi sensor integration and fusion can be described by knowing that which type of information multiple sensor provide to the system. The role of multi sensor integration and fusion can be described by knowing that which type of information multiple sensor provide to the system. In figure 1, n sensors are integrated to provide information to the system. In figure 1, n sensors are integrated to provide information to the system. Fusion of the information takes place at the node as indicated in the fig. Fusion of the information takes place at the node as indicated in the fig. The output x1 and x2 from the first two sensors are fused into new one indicated as x1,2. The output x1 and x2 from the first two sensors are fused into new one indicated as x1,2. The output x3 from the third sensor could then be fused with x1,2 at the next node resulting in x1,2,3,which might then be fused with the higher node in the structure. The output x3 from the third sensor could then be fused with x1,2 at the next node resulting in x1,2,3,which might then be fused with the higher node in the structure.

8. GENERAL PATTERN OF MSIF

9. The dashed lines from the system to the node represent any of the signal sent from the integrating functions. 3 FUNCTION IN THE INTEGRATING SYTEM SENSOR SELECTION : It selects the most appropriate group of sensors in response to changing conditions. WORLD MODEL: It represents all the sensory information. DATA TRANSFORMATION: From this all the information taken from the sensors need to be transformed before representing in the world model.

10. ADVANTAGES OF USING MULTIPLE SENSOR REDUNDANCY COMPLEMENTARY INFORMATION TIMELINESS LESS COST OF INFORMATION

11. APPLICATION OF MULTI SENSOR INTEGRATION AND FUSION ROBOTICS: ROBOTICS:  Robots with multi sensor fusion and integration enhance their flexibility and productivity in industrial application such as material handling, part fabrication, inspection and assembly. Mobile robot present one of the most important application areas for multi sensor fusion and integration. When operating in an uncertain or unknown environment, integrating and tuning data from multiple sensors enable mobile robots to achieve quick perception for navigation and obstacle avoidance. Mobile robot present one of the most important application areas for multi sensor fusion and integration. When operating in an uncertain or unknown environment, integrating and tuning data from multiple sensors enable mobile robots to achieve quick perception for navigation and obstacle avoidance.

12. perception, position location, obstacle avoidance vehicle control, path planning, and learning are necessary functions for an autonomous mobile robot. Multi sensor fusion and integration of vision,tactile,thermal,range,laser radar, and forward looking infrared sensors play a very important role for robotic system.

13. VISION: Visual information is the most powerful single source of sensory information available to a system. Many different types of non visual sensor are used in combination with vision sensors to compensate for some of the difficulties encountered in the machine processing of visual information. Recent research efforts aimed at combining intensity and range features derived from visual images to determine the structure of 3-d objects.

14. TACTILE:  Combined vision and tactile sensor have been used with a great extent by industrial robot to perform both assembly and inspection task.  STEP 1: Initially 2-d vision is applied to determine bounded regions. The centroid of each region is calculated by using the matched 3-d so that it can be used as the starting point for tactile exploration of the region.  STEP 2: The tactile sensing system explores each region to determine if it is a hole,cavity or surface. The tactile sensor used in this system is an octagonal cylinder covered with conducting surfaces mounted perpendicular to the mounting plate of the robot.

15.  STEP 3: For the surface region,the tactile sensing are integrated to create 3-d surface patch that can be matched with model data base. Starting from the location of the contact with the surface,the tactile sensors traces the surface. The points along each trace are combined into cubic least square curves. These curves are than used to fill in areas of the surface.  STEP 4 The surface patches and closed curves are matched against the model data base to find an object that is consistent with the sensory information. If more than one object is found, a probabilistic curve is used to order the objects for verifications.

16. LASER RADAR AND FORWARD LOOKING INFRARED SENSORS Recent research efforts have developed the method of fusing complementary information provided by laser radar and infrared sensors to segment and enhance features of man made objects in a cluttered back ground. Once separated from the back ground, the laser radar image provides range information for areas with in afield of view and the infrared image can show special feature of the area. The gradients of both images are used for both object enhancement and segmentation.

17. HONDA HUMANOID ROBOT

19. Military application Remote sensing

20. ADVANCEMENT IN MSIF The 2010 IEEE International Conference on Multi sensor Fusion and Integration for Intelligent Systems will take place at historical Fort Douglas, University of Utah, Salt Lake City, Utah,USA, 5-7 September 2010. The 2010 IEEE International Conference on Multi sensor Fusion and Integration for Intelligent Systems will take place at historical Fort Douglas, University of Utah, Salt Lake City, Utah,USA, 5-7 September 2010. The theme of MFI 2010 is “Cognitive Sensor Fusion.” A major goal of multi sensor fusion systems is to achieve human-like performance in terms of perception, knowledge extraction and situation assessment, exploiting symbolic and/or dynamical systems approaches. The theme of MFI 2010 is “Cognitive Sensor Fusion.” A major goal of multi sensor fusion systems is to achieve human-like performance in terms of perception, knowledge extraction and situation assessment, exploiting symbolic and/or dynamical systems approaches. Innovative results on multi sensor fusion topics are invited such as distributed methods, sensor/actuator networks, bio-based systems, evolutionary methods, biomedical applications and many more. Innovative results on multi sensor fusion topics are invited such as distributed methods, sensor/actuator networks, bio-based systems, evolutionary methods, biomedical applications and many more.

21. REFERENCES I.P.Howard, Human Visual Orientation chichester, U.K: wiley,1982 I.P.Howard, Human Visual Orientation chichester, U.K: wiley,1982 B.K.P.Horn, Robot vision cambridge, MA:MIT Press 1986 B.K.P.Horn, Robot vision cambridge, MA:MIT Press 1986 M.J.Magee and J.K. Aggarwal,''using multi sensor images to derive the structure of 3-d objects -A review''. M.J.Magee and J.K. Aggarwal,''using multi sensor images to derive the structure of 3-d objects -A review''. M.J.Magee, B.A Boyter,C,Chien and J.K Aggarwal,IEEE Trans Pattern anal. Machine Intell. Vol. PAM I-7,no.6,1985 M.J.Magee, B.A Boyter,C,Chien and J.K Aggarwal,IEEE Trans Pattern anal. Machine Intell. Vol. PAM I-7,no.6,1985