1. Multi-Touch Navigation Engine Presented by Team Extra Touch: Chris Jones Shuopeng Yuan Nathan Wiedeback

2. Multi-Touch Navigation Engine Background System Design Accomplishments 2

3. Background AIS offers single-touch Multi-touch is the “cool new thing” –iPhone, MacBook –MS Surface Mitch Butzer had good ideas… 3

4. Project Specifications 4 Microcontroller-based Written in C 2 - 6 independent control channels Defined at compile time Recognize defined patterns e.g., finger but not palm

5. 5 System Flow

6. Serial Input 6

7. Serial Stream Parser 74 byte “packets” from the screen Processed to get 32 sensor values, etc. Subtract the lowest value from each 7 (Little-endian) Average: 0x488B = 18,571 First sensor: 0x4886 = 18,566

8. Interpolator 8

9. Red dots: intersections of sensors (16 x 16) Green dot `P`: interpolated point Set a threshold and discretize at the end Each pixel -> 1 or 0 9 Image source: Wikipedia

10. Phantom Filter 10

11. Phantom Filter 11 One touchTwo touches Phantom touches (unfiltered) Intersections -> phantom touches Compare touches to last frame for filtering

12. Pattern Recognizer 12

13. Pattern Recognizer: Flowchart 13 Data In : Frame of Data Resolution: 240 * 180

14. Blob Classification and Checking Engine 14 Input Example: Resolution 60*20

15. Blob edge finding algorithm 000001000 000011000 000011100 000011100 000011100 000001100 000001000 000000000 000001000 000011000 000011100 000011100 000011100 000001100 000001000 000000000 000001000 000011000 000011100 000011100 000011100 000001100 000001000 000000000 First edgeLast edge

16. 16 Blob Classification and Checking Engine Input Example: Resolution 60*20 -- two blobs found

17. Voronoi Skeleton Algorithm 17 Ideal caseActual case

18. 18 Voronoi Skeleton with Test Data

19. Slope Algorithm Linear skeleton should have the same angle at each point. (a1 = a2 = a3 = a4) 19 Linear -> accept and find centerNonlinear -> reject

20. Channel Assignment 20

21. 21 Channel Frames – Overall Look 111000333 111000333 111000333 000000333 222220333 222220333 222220333 222220333 222220333 222220333 111102222 111102222 111102222 111102222 111102222 111102222 111102222 111102222 111102222 111102222 Current FormPotential Form

22. 22 Current Setup Channel Frames 111100000 111100000 111100000 111100000 111100000 111100000 111100000 111100000 111100000 111100000 000002222 000002222 000002222 000002222 000002222 000002222 000002222 000002222 000002222 000002222 Channel 1 FrameChannel 2 Frame

23. Channel Assignment – How it Works 00000 00110 11010 11000 11000 Channel 1 Frame 00111 00001 00001 00111 00111 Channel 2 Frame Channel 3 Frame 11000 11000 00100 00000 00000 Output to One Touch per Channel Limiter: Coordinates 3,3 Channel 3 Input: Coordinate Data (3,3) touched We have a match at Channel 3! 23

24. 24 One Touch per Channel Limiting 5 4 1 3 2 1 1 1 12345 Channel A – 3 touches 5 4 1 3 1 2 1 1 1 12345 Channel A -Phase 1 Reduction Channel A - Phase 2 Reduction 5 4 1 3 2 1 11 1 12345

25. 25 Output Data X0 Y0 X1 Y1 X2 Y2 X3 Y3 C0 C1 C2 C3 Altered Coordinate Array Channel Array

26. Accomplishments Translate sensor wires into 16x16 matrix Interpolate Recognize finger touches, reject other shapes Assign to either of two channels 26

27. Challenges Challenge Phantom touches “[40K] ought to be enough for anybody!” Need “tap points” for intermediate data Solution Compare to previous frame Low resolution (240 x 180 ?) for proof-of-concept Make each stage’s data available at the output 27

28. Future Serial interrupt issues Phantom filtering implementation –Would also enable more shapes Different touchscreen technology? 28

29. Questions?

30. Channel Assignment – Text Overview Data In: Center coordinates (X,Y) Processing: Data gets transformed into a 2d array, and the compared against each channel frame to determine its channel. Channel frames are defined before compile. Data Out: (X,Y) coordinates with channel data attached. Details: Aiming for 2 channels initially Should be simple to add channels in the final iteration.

31. Channel Assignment: Flow Chart 31

32. Summary Major design: “Proof-of-concept” –240 x 180 x 1-bit “color” –Simple phantom filtering –Accept a single finger per channel, reject other cases –Two channels 32