Download presentation
Presentation is loading. Please wait.
Published byAnnabel Ward Modified over 9 years ago
1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | From a certain point of view Eye tracking with Java Gerrit Grunwald Java Technology Evangelist Oracle Simon Ritter Head of Java Technology Evangelism Oracle
2
Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 2
3
Human Machine Interaction Eye tracking TheEyeTribe Java and Eye tracking Demos 1 2 3 4 3 5
4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Human Machine Interaction Eye Tracking TheEyeTribe Java and Eye tracking Demos 1 2 3 4 4 5
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. How It All Started
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Progress Was Made…
7
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Multi-Touch Has Become Popular
8
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Gaming Has Driven Several Interfaces
9
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Gestures Rather Than Touch…
10
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Interactions Without Hands
11
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Human Machine Interaction Eye Tracking TheEyeTribe Java and Eye tracking Demos 2 1 3 4 11 5
12
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. “Eye tracking is the process of measuring either the point of gaze or the motion of an eye relative to the head.” – Wikipedia 12
13
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Of the stimuli that reach the brain are visual 80 % 13 The eyes are the windows to the soul
14
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Eyes Give Clues When Answering Questions Visually Constructe d Visually Remembered Auditory Constructed Auditory Remembered Feeling/Kinesthetic Internal Dialog This is for a right-handed person. Positions reversed if left-handed
15
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Where to use it ? 15
16
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 16 Eye gaze correction for video conferencing
17
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 17 Minimize dangers in air traffic displays
18
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 18 Developing Video Games and Graphics
19
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 19 Marketing and E-Commerce website development
20
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 20 Improved computer-human interaction
21
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. What can it do ? 21
22
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Determine whether a user is looking at the screen – Paying attention to course, video, etc If a user is reading or scanning – Speed of movement of the eyes over text Intensity of user’s attention – How long they look ata aprticular thing Whether the user is searching for specific information Capabilities
23
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Eye attached tracking Electric potential measurement Optical tracking 23 Types of Eye Trackers
24
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Special contact lense with embedded mirror or sensor (magnetic search coils) Very precise Allows measurement in horizontal, vertical and torsion directions Expensive 24 Eye attached tracking
25
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Measures electric potential field of eyes Can measure eye movement even if eyes are closed Wearable method – also used in sleep research Relatively poor accuracy in gaze direction detection Expensive 25 Electric potential measurement
26
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Measures reflection of IR light from the front of the cornea Contactless method Good for fast and slow eye movements Inexpensive 26 Optical tracking
27
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Optical Eye Tracking Principle 27
28
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 28 Optical Eye Tracking Principle IR-Light emission Eyeball Cornea Reflected IR-Light Eye Tracker
29
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Human Machine Interaction Eye Tracking TheEyeTribe Java and Eye tracking Demos 2 1 3 4 29 5
30
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. TheEyeTribe 30
31
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Sampling rate Accuracy Latency Operating range Tracking area Screen sizes Support for Connection 31 TheEyeTribe 30 Hz and 60 Hz 0.5° - 1° < 20ms at 60 Hz 45 cm - 75 cm 40 cm x 30 cm at 65 cm up to 24" C++, C# and Java USB 3.0
32
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 32 TheEyeTribe APPLICATION SDK EyeTribe SERVER NETWORK HW TRACKER Open API External applications (e.g. Java program) Language spec. wrappers (C++, Java etc.) The eye tracking process. Initializes device(s), frameworks, servers etc. No GUI - output goes through API
33
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Human Machine Interaction Eye Tracking TheEyeTribe Java and Eye tracking Demos 1 4 33 5 23
34
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 34 import com.theeyetribe.client.*; public enum EyeTracker implements IGazeListener { INSTANCE; private GazeData gazeData; … private EyeTracker { init(); initBuffers(); initEyeTracker(); timer.start(); } private void init() {…}; private void initBuffers() {…}; } Java and Eye Tracking Java package
35
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 35 import com.theeyetribe.client.*; public enum EyeTracker implements IGazeListener { INSTANCE; private GazeData gazeData; … private EyeTracker { init(); initBuffers(); initEyeTracker(); timer.start(); } private void init() {…}; private void initBuffers() {…}; Java and Eye Tracking implement Interface
36
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 36 import com.theeyetribe.client.*; public enum EyeTracker implements IGazeListener { INSTANCE; private GazeData gazeData; … private EyeTracker { init(); initBuffers(); initEyeTracker(); timer.start(); } private void init() {…}; private void initBuffers() {…}; } Java and Eye Tracking initialize EyeTracker
37
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 37 private void initEyeTracker() { GazeManager.getInstance().activate(GazeManager.ApiVersion.VERSION_1_0, GazeManager.ClientMode.PUSH); } Java and Eye Tracking initialize EyeTracker
38
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 38 public void start() { if (running.get()) return; running.set(true); GazeManager.getInstance().addGazeListener(this); timer.start(); } Java and Eye Tracking start EyeTracker
39
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 39 @Override public void onGazeUpdate(final GazeData GAZE_DATA) { gazeData = GAZE_DATA; xBuffer.remove(); yBuffer.remove(); xBuffer.addLast(gazeData.smoothedCoordinates.x); yBuffer.addLast(gazeData.smoothedCoordinates.y); xAvg = xBuffer.stream().mapToDouble((x) -> x).average(); yAvg = yBuffer.stream().mapToDouble((y) -> y).average(); } Java and Eye Tracking get Gaze data
40
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 40 public void stop() { GazeManager.getInstance().removeGazeListener(EyeTracker.this); GazeManager.getInstance().deactivate(); } Java and Eye Tracking stop EyeTracker
41
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. What data do you get ? 41
42
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Gaze Data delivered in frames… 42
43
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Tracking state State of the eye and gaze tracking process. (e.g. Person in field of view) Fixation If the tracked gaze is currently fixated, the frame will contain that information Gaze Coordinates The point on screen (pixels) that the user is currently looking at. (Needs calibration) Pupil coordinates The position of a tracked persons pupil relative to the Tracker sensor 43 Gaze data
44
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Problem… 44
45
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Problem…SPEED 45
46
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 46 timer = new AnimationTimer() { @Override public void handle(final long now) { if (now > lastTimerCall + UPDATE_INTERVAL) { if (null == gazeData) return; focusPoint.set(new Point2D(xAvg.getAsDouble(), yAvg.getAsDouble())); eventCounter++; leftEyeOpen.set(gazeData.leftEye.pupilSize > 0); rightEyeOpen.set(gazeData.rightEye.pupilSize > 0); state.set(gazeData.state); lastTimerCall = NOW; } }; } Problem…SPEED use AnimationTimer to update data in intervals
47
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Eye tracking provides a useful and different way to interact with a UI – Control is hard, tracking a user’s interaction is easier Many potential applications – Helping the physically challenged – Reducing potential problems in critical systems www.eyetribe.com dev.eyetribe.com/java Conclusions And Resources
48
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Human Machine Interaction Eye Tracking TheEyeTribe Java and Eye tracking Demos 1 4 48 5 23
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.