Multi-Sensor Image Fusion (MSIF) Team Members: Phu Kieu, Keenan Knaur Faculty Advisor: Dr. Eun-Young (Elaine) Kang Northrop Grumman Liaison: Richard Gilmore Department of Computer Science College of Engineering, Computer Science, and Technology California State University, Los Angeles GOES-12 (West) Resolution: 2500x912 Scan Time: 5 min. GOES-11 (East) Resolution: 2000x850 Scan Time: 15 min. MSIF aims to fuse two images of the same scene from two space sensors. MSIF takes two space sensor images, as well as per pixel latitude and longitude information, and calculates the altitudes and velocities of clouds. SW development environment  Microsoft Visual C++  OpenCV Libraries: An open source library of programming functions for real time computer vision. 1.Input Data Preprocessing : The routine extracts the visual channel from the GOES satellite dataset 2.Equirectangular Map Projection: Both images are transformed into an equirectangular map project, which rectifies the images for matching algorithms. 3.Cloud Detection: K-means clustering and connected component analysis are used to detect clouds. 4.Matching and Disparity Extraction: Matching algorithms are used to find the disparity of the clouds. 5.Altitude and Velocity Calculation: The disparity is used to calculate the altitude of the cloud assuming that no movement has taken place between two images. 6.Data Visualization: A disparity map and a velocity-altitude graph are displayed for each cloud. Rectifies both images to a common coordinate system Results in equal distances between latitude and longitude lines Makes matching image features easier K-means clustering identifies cloud pixels based on intensity. Connected component analysis (CCA) identifies individual clouds Accuracy in the neighborhood of 85-90% Finds SHM feature vector per cloud Best match is when Euclidian distance of two vectors is minimum Finds bounding box of a cloud, and compares pixel intensities Best match is when MSD is at a minimum IntroductionAlgorithm Description 2. Equirectangular Projection West (Overlap Region)East (Overlap Region) 3. Cloud Identification West 4. Matching and Disparity Extraction Shape Histogram MatchingMean Squared Difference (MSD) 5. Altitude and Velocity Calculation 6. Data Visualization Altitude from Disparity Velocity from Altitude Construct vectors from satellites (S1, S2) to their ground intersect (G1 and G2) of the cloud Compute the intersection of two points (or the midpoint of the shortest line segment connecting the two vectors) Derive the altitude of the cloud from the intersection Alter blue to red (assume altitude) Construct vectors from satellites to red location Find the intersection of vectors with earth plane Compute distance of the two intersections and subtract from original disparity—this is the cloud motion Divide motion by time difference Clouds do not normally travel faster than 200 km/h. The possible altitudes of the cloud is narrowed down with this information. 1. Input Data ? ? Disparity Map West Intensities are the disparity magnitudes of clouds. The brighter, the larger.