Progress Presentation IRALAR Breanna Heidenburg -- Michael Lenisa -- Daniel Wentzel Advisor: Dr. Malinowski

 Project Overview  Application Layout & Progress ◦ Image Processing ◦ Calibration ◦ Clicking ◦ Communication ◦ User Interface  Overall Progress

What is our project?

 Track a user’s eye ◦ Small form factor camera  Use input to control cursor ◦ Where you look is where the cursor is ◦ Need to calibrate for every user  Create user interface

How will we achieve our goal?

 Hardware ◦ Camera  QuickCam Pro for Notebooks ◦ Polarizer  Tiffen 25mm polarizing filter  Removes glare from eye reflections ◦ Lighting  Diffuse LEDs  Not sure if they are necessary, will find out when we have the display

 New Image processing methods: ◦ Blob Recognition ◦ Classic Image Processing Techniques  Contrast stretching  Thresholding  Adaptive Windowing

 Sample Recognitions: Difficult Low Light False Positive

 Why do we need to calibrate? ◦ Map the user’s eye to a point on the screen ◦ Everyone’s eyes are different ◦ Account for various positions of the camera  What does our data look like? ◦ Somewhat unreliable ◦ Problem:  Movements while taking data

 X-Axis values

 Y-Axis values

 What does the data tell us? ◦ A 2D mapping is not good enough ◦ Horizontal eye movement is closer to linear than vertical eye movement ◦ Vertical eye movement depends on both coordinates of data

 How do we solve the problem? ◦ Multiple Variable Linear Regression – Least Squares Y = B 0 + B 1 x 1 + … + B k x k ◦ Uses matrix algebra to obtain a coefficient matrix B[] = (X’X) -1 X’Y

 Results ◦ pixelX = *eyeX – 2.9*eyeY ◦ pixelY = *eyeX – 5.6*eyeY ◦ Error = very high (need more than 4 random data points) B[] = (X’X) -1 X’Y

 How do we click? ◦ Monitor eye movements ◦ Identify pauses ◦ When eye position is within a small percentage for a certain amount of time ◦ Set to 5 frames (~200 mS) ◦ Generally, it takes 230 mS for a hand to click a mouse.

 Why is it important? ◦ Allows multiple processes to communicate ◦ Allows relay of time sensitive information  UDP vs. TCP ◦ UDP -> ‘Connectionless’ ◦ TCP -> ‘Connection oriented’  Multi-threading ◦ Necessary for running multiple pieces of code in a single process

 Multi-threading ◦ Issues with public variable usage ◦ Solutions we are using:  Critical Section  Raises thread priority (does not allow for interrupts)  Mutual Exclusion  Creates a variable lock  Using variable first requires ‘unlocking’ it

 Achieved via a combination of Hardware and Software  Software: WPF & C# ◦ XAML (eXtensible Application Markup Language) ◦ Very similar to XML ◦ Uses ‘code-behind’ in a similar style to javascript  ‘code-behind’ is in C#

 Framework is in place for all pages  Navigation between pages is complete  Core operating portions are complete

 Hardware ◦ LitEye LE500 has been ordered (will arrive early-mid March) ◦ SVGA resolution (800x600) ◦ Transparent

How far along are we?

 HMD has arrived (today) ◦ Have not had much time to work with it ◦ We aren’t 100% sure how we will physically mount the camera to the display ◦ Aren’t sure if extra lighting will be necessary

 Image Processing ◦ Still working on some recognition improvements  Calibration ◦ Minor adjustments left to be made  Communication ◦ Minor adjustments left to be made  User Interface ◦ Only essential parts are included at the moment Still ironing out details on what will be included in final version