Multi-Touch Navigation Engine Presented by Team Extra Touch: Chris Jones Shuopeng Yuan Nathan Wiedeback Detailed Design Review 1

THE PLAN 2

Make it interesting… Challenge Phantom touches “[40K] ought to be enough for anybody!” Need “tap points” for intermediate data We’re not really programmers… Solution Compare to previous frame Low resolution (240 x 180) for proof-of-concept Make each stage’s data available at the output Get help! 3

Serial Input 4

Serial Stream Parser 74 byte “packets” from the screen Interrupt-driven I/O on ARM -- until an entire packet received Then process it to get 32 sensor values, average, etc. Normalize by subtracting the lowest value from each Variations of ~ (Little-endian) Average: 0x488B = 18,571 First sensor: 0x4886 = 18,566

Interpolation 6

Interpolator Input: 32 x 2-byte sensor levels Output: “Image” frame -- n x m x 1 bit pixel array Proof-of-concept: 240 x 180 (memory constraints) Find intersection points of sensors (16 x 16) Interpolate linearly based on distance – first in X, then in Y (bilinear) Set a threshold and discretize at the end Each pixel ends up as a 1 or a 0 7 Image source: Wikipedia

Phantom Filtering 8

Phantom Filter Input: Frame Output: Filtered frame ( 240 x 180 x 1 bit) Intersections method -> phantom touches Remember what was touched first – i.e., save and compare to the last frame Proof of concept -- only the simple case covered 9

Pattern Recognition 10

Pattern Recognizer: Flowchart 11

Input: Whole frame (60*20) 12

First step: Blob classification 13

(Visio files provided separately) 14

Output: the outline of each single blob 15

The X-Y Coordinates in separate arrays xy xy xy

Voronoi Skeleton Algorithm 1 17

The block diagram 18

Voronoi Skeleton Algorithm 2 (The ideal case) 19

Voronoi Skeleton Algorithm (The actual result)

Shape recognition 21

Slope algorithm If a skeleton is linear, according to geometry, angle a1 = a2 = a3 = a4 Therefore, only linear shaped skeleton can pass this filter. To the touch panel application, it means only the side of a palm instead of a palm shape can pass. 22

Channel Assignment 23

Channel Assignment Data In: Center coordinates (X,Y) Processing: Data gets transformed into a 2d array, and the compared against each channel frame to determine its channel. Channel frames are defined before compile. Data Out: (X,Y) coordinates with channel data attached. Details: Aiming for 2 channels initially Should be simple to add channels in the final iteration. 24

Channel Assignment – Flow Chart 25

Channel Assignment - Visual Representation Channel 0 Frame Channel 1 Frame Channel 2 Frame Output to PC: Coordinates 3,3 Channel 2 Input: Coordinate Data (3,3) touched We have a match at Channel 2! 26

Questions? 27