Multi-Sensor Data Fusion H.B. Mitchell UNCLASSIFIED

Multi-Sensor Data Fusion Three-hour tutorial on multi-sensor data fusion The tutorial is closely based on a selection of material taken from the book: Multi-Sensor Data Fusion: An Introduction by H.B. Mitchell published by Springer-Verlag (2007) 1. Introduction 2. Sensors 3. Common Representational Format 4. Spatial, Temporal, Semantic Alignment 5. Robust Statistics 6. Ensemble Learning UNCLASSIFIED

Introduction UNCLASSIFIED

Multi-Sensor Data Fusion Data fusion: Theory and Techniques which combine sensor data into a common representational format. Aim is to improve the quality of information. Data fusion is analogous to the manner in which humans and animals improve their chances of survival by exploiting their Man-Made Fusion System EO, IR, Radar A priori information and/or historical information Bayesian inference, fuzzy logic, Dempster-Shaefer Multiple senses Experience Ability to reason UNCLASSIFIED

Multi-Disciplinary Subject Multi-sensor data fusion brings together many different techniques and applications Medical Imaging Remote Sensing Surveillance Data Mining Computer Vision Stereo Imaging Bayesian networks Signal Processing Statistical Estimation Tracking Algorithms Classification Algorithms Invariant Subspaces MCMC Genetic Algorithms Bagging, Boosting Computing Power Fusion Techniques Applications UNCLASSIFIED

Complementary Fusion After. Toet. Natural color mapping for multiand nightvision imagery. Information Fusion (2003) UNCLASSIFIED

Pan Sharpening Panochromatic image Multi-spectral image Pan-sharpened image UNCLASSIFIED

Colorization After. Toet. Natural color mapping for multiband nightvision iamgery. Information Fusion (2003) UNCLASSIFIED

Sensors UNCLASSIFIED

Sensors Sensors are devices which interact directly with environment Sensors are the source of all input data. Often use smart sensors which Transform sensor signal to standardized digital format Calibrates sensor signal Transmits sensor signal via standardized interfaces. Amp FilterA/D  sensor element transmitter/ receiver UNCLASSIFIED

Fusion Node Build a Data Fusion System as a distributed assembly of fusion nodes Fusion Ext. InformAux Inform. Input Data Output Data UNCLASSIFIED

Logical Sensor Logical sensor is any device which functions as a source of inform. for a multi-sensor data fusion node S 1 : Physical sensor F 1: Virtual Sensor: A fusion node whose output is fed into another fusion node S3S3 S2S2 S1S1 F1F1 F2F2 “virtual” sensor UNCLASSIFIED

Sensor Errors Sensors only give an estimate of the measured physical property Nature of errors often determine the preferred fusion algorithm No Bias. Concatenate measurements into one vector then track with Kalman Filter Bias. Separately track with each set of measurements, then fuse tracks. UNCLASSIFIED

Common Representational Format UNCLASSIFIED