Virtual Image Peephole By Kyle Patience Supervisor: Reg Dodds Co Supervisor: Mehrdad Ghaziasgar

Quick Recap Development of an interface that does the following: A very large virtual screen is imagined to exist, The mobile phone screen is used as a "peephole" into that interface.

Overview Design Decisions and System Changes Implementation Finding and Tracking Features User Interface Tools

Design Decisions and System Changes PREVIOSLYCURRENTLY No NDK Uses NDK Use Android camera API Uses device’s camera natively Didn’t use sensor readings Uses both accelerometer and gyroscope Display was modifies camera frames 3D interface

NDK Implementation Get framesProcess frames Get sensor readings Output

Finding and Tracking Features The Shi-Tomasi Corner Detector Based on the Harris Corner Detector. Tries to find little patches of image that generate a large variation when moved around. Ultimately finds small corners in a frame. Lucas & Kanade Method Technique which can provide an estimate of the movement of certain features in successive images of a scene. Algorithm makes a "best guess" of the displacement of a neighbourhood by looking at changes in pixel intensity. These intensities are known through intensity gradients of the image in that neighbourhood.

User Interface using camera frames Moving the device without tilt will move the interface

User Interface using gyroscope Tilting phone to the left Tilting phone to the right

User Interface using accelerometer Tilting phone upwards Tilting phone downwards

Tools PlatformWindows 8.1 x64 ApplicationsAndroid Studio SDKAndroid SDK OpenCV Android SDK Android NDK LibrariesOpenCV jPCT LanguagesJava C++

Project Plan Term 1 Learn OpenCV Learn Android Term 2 Learn NDK Build Android Prototype Capture Footage Term 3 Process Frames Process Accelerometer and Gyroscope Readings Create 3D Interface Intergrade all components Term 4 Implementing Tuning Testing

References Ali, S. I., Jain, S., Lal, B., & Sharma, N. (2012). A framework for modelling and designing of intelligent and adaptive interfaces for human computer interaction. International Journal of Applied Information Systems (IJAIS) Volume. Anuar, A., Saipullah, K. M., Ismail, N. A., & Soo, Y. (2011). OpenCV based real-time video processing using android smartphone. International Journal of Computer Technology and Electronics Engineering (IJCTEE), 1(3). Shi, J., & Tomasi, C. (1994, June). Good features to track. In Computer Vision and Pattern Recognition, Proceedings CVPR'94., 1994 IEEE Computer Society Conference on (pp ). IEEE. Tomasi, C., & Kanade, T. (1991). Detection and tracking of point features. Pittsburgh: School of Computer Science, Carnegie Mellon Univ..

Questions?