1. Yuping Lin and Gérard Medioni

2.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to Map registration ▪ Interleaving image to image and image to map ▪ Partial local mosaic  Synchronization of multiple video streams  Conclusion

3.  Input:  Multiple UAV video streams  Position of moving objects in each video stream  Goal: Synchronize using a common moving object

4.  Register UAV streams to a global reference image (a map), then  Synchronize the streams using the unique path of a common moving object on the map

5.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to Map registration ▪ Interleaving image to image and image to map ▪ Partial local mosaic  Synchronization of multiple video streams  Conclusion

6.  Input:  Global reference image (Map)  UAV stream  The homography of the first frame of the UAV stream to the map

7. ISSUES  UAV images and the map are different in terms of viewpoints, sensors and time of capture  Direct matching is difficult APPROACH  Given the homography of the first UAV frame to the map,  Two step registration  Consecutive UAV image registration, then  UAV to Map registration

8.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to Map registration ▪ Interleaving image to image and image to map ▪ Partial local mosaic  Synchronization of multiple video streams  Conclusion

9.  Method:  extract features in each frame  Establish feature correspondences between consecutive images  estimate the transformation

10. ISSUES  Features should be descriptive for matching and sufficient to give good transform estimation  Feature matching  Transform estimation APPROACH  SIFT feature extraction  128 dimension feature descriptor  Avg. 2000 features in each image  Nearest neighbor matching  Avg. 1000 matches in each pair of images  RANSAC  Avg. 600 inliers in each pair of images

11.  Illustration

16.  Problem: error is accumulated

17.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to Map registration ▪ Interleaving image to image and image to map ▪ Partial local mosaic  Synchronization of multiple video streams  Conclusion

18.  Method:  Perform local search for correspondences between the UAV image and the map

19. ISSUES  UAV images are very different from the map, SIFT features cannot always match APPROACHES  Sample points in the map  For each point, locally search for the most similar image patch in the UAV image  Use Mutual Information as similarity measurement

20.  Illustration

25.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to Map registration ▪ Interleaving image to image and image to map ▪ Partial local mosaic  Synchronization of multiple video streams  Conclusion

26.  Method:  Perform consecutive UAV image registration and UAV to Map registration iteratively ▪ Consecutive UAV image registration produce good initials for UAV to Map registration ▪ Register the partial local mosaic to the map

27. ISSUES  Correspondences in a single frame are not enough Registration is unstable APPROACH  Multiple frames in a time window forms a partial local mosaic which spans a larger region and provides more correspondences  More robust  Smooth transition

28. ISSUES  Correspondences in a single frame are not enough Registration is unstable APPROACH  Multiple frames in a time window forms a partial local mosaic which spans a larger region and provides more correspondences  More robust  Smooth transition

29.  Result Register single frameRegister partial local mosaic

30.  Illustration

34.  Result

35.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to Map registration ▪ Interleaving image to image and image to map ▪ Partial local mosaic  Synchronization of multiple video streams  Conclusion

36.  Input: UAV image sequences of different views, different frame rates, but capture the same area and overlap in time  An moving object on the ground plane which serves as a “clock” to synchronize the sequences

37.  The moving object should generate a single path on the map  Use sequence alignment algorithm to synchronize the UAV streams

39.  Two steps to register an UAV image to the map  Register each frame to its previous frame to derive an initial estimate  Register UAV image to the map to derive  Limitations  Initial estimate should be given  Unable to recover from a bad estimate