Face Detection and Notification System Conclusion and Reference The Rise of Project ICU Team GTG 24in X 36in David Tag, Eric Gu, Jason Guo What’s ICU? Object Avoidance ICU is an integrated mobile solution aimed to assist the blind community with all aspects in life. For the initial stage, it has two main areas of focus, one is for self-navigation, the other is for smart assistance. For self-navigation, we believe it is necessary to have a complete solution that can intelligently lead a blind person from point A to point B, in both indoor and outdoor scenario. Google maps has proven the convenience a public map can bring, and our vision is to have a public library of all the public indoor spaces, so that indoor navigation can be possible in the future. For smart assistance, we identified the major challenges blind community faces on a daily basis, so we hope to integrate social-aware functions and object recognition with the app so that revolutionary change can be brought to the blind’s life. With the Tango tablet’s capability of depth sensing, we immediately had the thoughts of replying on the depth map to implement the obstacle avoidance function. At the very first, we had a very simple solution of checking if the average depth is < 1.5m, if it is, then we determine the area is obstructed. However, based on this design we couldn’t implement directional assistance. Thus, we decide to change the design by implementing the average X-Y position of everything within the view; however, that could not solve the case when there are obstacles on each side of the screen, since the field of view for tango is very wide, this can brings lots of issues. Thus, we decided to divide the area into three regions, left, center and right, but based on the average X-Y position it is hard to determine what can be considered as an obstacle. Thus, we made the decision of counting the point cloud objects in each region, and figure out if there is obstacle. We give directional advice through a decision tree. Face Detection and Notification System Conclusion and Reference With our vision in mind, we decide to implement the most important functions for each of the main areas of focus, which is – Obstacle Avoidance for self-navigation and Face Detection for Smart Assistance. As the Professor suggested, OpenCV is a great library that we can utilize for computer vision and it has support for face detection, which we redeemed as the most important feature for the smart assistance side of the functions. However, at first we ran into the problem of Tango tablet does not allow direct access to camera video data, so we had to render feed on-screen, and take some periodic screenshots, then feeding these information into the OpenCV functions. Face detection is done by Cascade Classification, which detects and classifies objects by applying several stages in sequence, and determining whether the stages are passed. OpenCV comes with two types of pre-trained facial cascade classifiers, HAAR and LBP. We tried HAAR at first, but the speed of detection was super slow, which would take a few seconds to analyze one picture. We think this is too slow for an actual use case, thus we decided to switch to LBP, which is less accurate, but has a detection speed a lot faster than the HAAR classifier. Even though LBP can throws up many false positives, it is still marginally better than HAAR in this case. In terms of notification or warning system, we had both voice warning and vibration warning. At first we only had the voice warning, using TTS functions saying ”There is an object ahead of you, try turning left/right”, but after thinking about different use case, for example, if a user is in a busy street with high noise level, it might be hard to pick up all the warnings. Thus we also decided to add the vibration so that different pattern can signal different warning, and this way can work really well for the blind community since they are shown to be more responsive to vibration than normal people. Overall, we think that we did a good job conceiving ICU and having a set of plans to execute, but that we could have improved more on execution if we had the tablet for longer time, because there are more interesting functions we could have completed. We also thought that despite some of the technical limitations of the Tango tablet, we believed that the Project Tango is a great start on exploring the possibilities of advanced sensing and AR. Even though currently there is not big enough of a community developing on Tango, but as more of the VR/ AR projects come, the number of developers will rise. Reference: http://stackoverflow.com/questions/10913682/how-to-capture-and-save-an-image-using-custom-camera-in-android http://docs.opencv.org/2.4.13.2/modules/refman.html http://stackoverflow.com/questions/28402718/dark-google-tango-camera-surface-when-using-depth-information https://developers.google.com/tango/apis/java/ https://developers.google.com/tango/overview/depth-perception https://developers.google.com/tango/apis/java/java-support-tutorial Acknowledgements This project was completed as part of CS 234/334 Mobile Computing (Winter 2017), taught by Prof. Andrew A Chien with TA support by Gushu Li and Ryan Wu. We gratefully acknowledge the generous support of Samsung in providing GearVR equipment.