Presentation is loading. Please wait.

Presentation is loading. Please wait.

UI Issues, Neural Nets, RTS

Similar presentations


Presentation on theme: "UI Issues, Neural Nets, RTS"— Presentation transcript:

1 UI Issues, Neural Nets, RTS
CSE 490RA January 27, 2005

2 Lecture outline Leftovers from UI Lecture Neural Networks RTS

3 Uses of the stylus Direct writing Abstract writing Pointing Selecting
Gesture Direct manipulation Control

4 Mode issues in pen computing
Adding modes to the pen Barrel button Secondary button with non-dominant hand Eraser tip Pressure Explicit mode buttons Area based modes (writing area, gesture area, control area, etc.) Cursor feedback Importance of visual cues for informing user Errors in crossing mode boundaries

5 Control Widgets design for stylus use

6 Flow Menu Use movement through octants for control information Move
Item 100% 100% Text Item Highlight Zoom 66.6% 200% 66.6% 200% Shape 50% 75.0 400% 50% 100.0 400% 25% 800% 25% 800% Custom Custom

7 Interaction with direct manipulation
Move Item Highlight Zoom

8 CrossY: Crossing based UI
Specify operations by drawing through

9 Gestures Commands issued with a single stroke
May be drawn or invisible Support from SDK Register gestures to be recognized UI Issues Similar to keyboard short cuts Speed up for experts Hard to learn / remember

10 Gestures Ambiguity Robustness Distinction between gestures
Distinction between gesture and other ink Robustness Handling misrecognized gestures False positive False negative Gesture initiated actions should be undoable

11 Neural Networks Fundamentals for Handwriting Reco Lecture (Jay Pittman) Recognition algorithm Learning based recognition algorithm

12 General considerations for learning algorithms
Training sets Collection Evaluation Training cost Time and space Algorithm cost Robustness to error

13 Neural networks Perceptrons
Motivated by considerations of the brain

14 Single layer neural networks
Bias weights Threshold activation function Step function Sigmoid function: 1/(1 + e-x)

15 What you can do with single layer networks
Any linearly separable dataset can be recognized with a single layer neural network

16 Gradient descent algorithm
Choose initial weights While not at optimum Compute derivative Move along derivative It can be proved this converges

17 However, single layer networks are very limited

18 Multilayer networks with hidden nodes
Can recognized much wider range of data set The gradient descent algorithm generalizes to this case

19 Real Time Stylus Allow for user computation on the ink thread

20 Architecture (Overview)
RealTime Event Sink RealTime Event Sink RealTime Event Sink RealTimeStylus Ink Collecting Object queue storage Pen Service InkCollector “Inking” Thread UI Thread

21 Substroke operations Examples Rendering Custom Inking Multiple Ink
Distributed Ink Rendering Dynamic – draw on ink packet Static – draw on paint event

22 Custom Inking Create plugin to listen for packets Registor for Packets
Draw triangle on each packet

23 public void Packets(RealTimeStylus sender, PacketsData data){
for (int i = 0; i < data.Count; i += data.PacketPropertyCount){ Point point = new Point(data[i], data[i+1]); // Packet data always has x, y // followed by the rest Point convertedPoint = new Point(); // We need to convert to Pixels... convertedPoint.X = (int)Math.Round((float)point.X * (float)myGraphics.DpiX/2540.0F); convertedPoint.Y = (int)Math.Round((float)point.Y * (float)myGraphics.DpiY/2540.0F); if (this.pointCount == 0) this.firstPoint = convertedPoint; if (this.pointCount > 1){ Color color = pointCount % 2 == 0 ? Color.Red : Color.Yellow; Point[] triangle = new Point[3]; triangle[0] = this.firstPoint; triangle[1] = this.previousPoint; triangle[2] = convertedPoint; myGraphics.FillPolygon(new SolidBrush(color), triangle); } this.pointCount++; this.previousPoint = convertedPoint;

24 Distributed Ink Capture ink packets on machine 1
Send packets to machine 2 Reconstruct ink stroke by inserting packets

25 Sample application

26 Distributed Ink Sender: Receiver:
Collect packets in packet, when count is above threshold, send message Receiver: Insert packets as custom data Render custom data as it is received Assemble custom data as an ink stroke


Download ppt "UI Issues, Neural Nets, RTS"

Similar presentations


Ads by Google