1. New Interaction Techniques Department of Computer Sciences University of Tampere, Finland September – December, 2003 Grigori Evreinov www.cs.uta.fi/~grse/ Dwell Time Selection & more…

2. TAUCHI MMIG G. Evreinov 01_26 10.10.2003 Cursor  Object Inter-actions Dwell Time Time X Y demo1 [1]

3. TAUCHI MMIG G. Evreinov 02_26 10.10.2003 MagicCursor 2000: Dwell Selection Clicking Solution, http://www.gstsdesigns.com/madentec/MagicCursor.htm http://www.gstsdesigns.com/madentec/MagicCursor.htm Gus! Dwell Cursor, http://www.gusinc.com/dwell.html The Smart-Nav AT package (Dwell clicking 0.2-2.0 s), http://www.naturalpoint.com/prod/d_dwell.htm http://www.naturalpoint.com/prod/d_dwell.htm Dwell Time

4. TAUCHI MMIG G. Evreinov 03_26 10.10.2003 Gaze Control [1] http://www.metrovision.fr/mv-vi-notice-us.htmlhttp://www.metrovision.fr/mv-vi-notice-us.html [2] http://www.delta7.asso.fr/Deltavision%202001/ecrire3.htmlhttp://www.delta7.asso.fr/Deltavision%202001/ecrire3.html Dwell Time

5. TAUCHI MMIG G. Evreinov 04_26 10.10.2003 [31] http://www.metrovision.fr/mv-vi-notice-us.htmlhttp://www.metrovision.fr/mv-vi-notice-us.html LC Technologies, Inc. The Eye-Gaze System Makers Dwell Time

6. TAUCHI MMIG G. Evreinov 05_26 10.10.2003 Dwell Time [9] Myers, B.A., et al. Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002. [12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI 2001. Laser Control

7. TAUCHI MMIG G. Evreinov 06_26 10.10.2003 Dwell Time Natural Point trackIR TM http://www.naturalpoint.com, http://www.headtracker.com/ http://www.naturalpoint.com http://www.headtracker.com/ Marsden, R. 100% Hands-free Computer Access- Madentec’s 2000 Series http://www.madentec.com/products/comaccess/2000/whitepaper.html http://www.csun.edu/cod/conf/2000/proceedings/0185Marsden.htm demo2 demo3 Head Control

8. TAUCHI MMIG G. Evreinov 07_26 10.10.2003 Usability-testing software for… txtEntry Eye-Gaze Dwell Time

9. Clear Data TAUCHI MMIG G. Evreinov 08_26 10.10.2003 TWords.txt /phrases GridData1: test words/chars entered text /chars time per char, ms GridData2: char per word /phrase num. of entered words num. of strokes /clicks per word /phrase time per word/phrase, s lblTestSymbol txtText1 On-screen Keyboard Break test Test initialization Ctrl+K=>move keys SetSigns SetCharacters BackSp Statistics() Rtime, ms s (st.dev), ms Errors wpm TestTime, s lblSave_Click() GridData2_Click() to save column GridData1_Click() to save column txtPersonData comments… fraData WordsLoading SetTest lblOpen_Click() SetData Trial start Timer1 Timer2 Lesson6  txtEntry_EyeGaze3 SpotActivate lblDwDelay lblDwell Ctrl+L=>EditCapts. Up/Down Timer3 Timer4 Ctrl+W=>Dwell MouseInputSymbDec HideBorders Dwell Time

10. Clear Data TAUCHI MMIG G. Evreinov 09_26 10.10.2003 lblTestSymbol txtText1 On-screen Keyboard Break test Test initialization Ctrl+K=>move keys SetSigns SetCharacters fraData SetData Trial start Lesson6  txtEntry_EyeGaze3 SpotActivate lblDwDelay lblDwell Ctrl+L=>EditCapts. Up/Down tmrDwDelay tmrDwell Ctrl+W=>Dwell MouseInputSymbDec HideBorders Dwell Time

11. TAUCHI MMIG G. Evreinov 10_26 10.10.2003 Dwell Time Adaptive Dwelling according to Windows Interface Design Guide *, the action associated with the control is initiated when the mouse button is released if the pointer is dragged to another location, the control will return to its original state and undesirable click will be stopped for instance, the user can put cursor over icon or command button and choose it by clicking the mouse sometimes, clicks are undesirable or impossible (laser pointer, gaze / head / ‘brain’ control…) to provide productivity and easy access during communication with computer it is necessary a balance between flexibility of the dialogue structure, adaptability on each level of the interface, cognitive abilities of the person and limitations used interaction techniques semantic, syntactic, lexical * The Windows Interface. An Application Design Guide. Microsoft Press, Redmond, Washington, 1992.

12. TAUCHI MMIG G. Evreinov 11_26 10.10.2003 Dwell Time the time is one of the most important critical parameters of the system feedback loop a normal time of simple visual-motor control task in accomplishing with the help of intact muscles of the finger is about 150-250 ms the time is progressively increased up to 1000 ms and more, if an additional semantic analysis or physical limitations take place to prevent wrong selection external (dwell) timer, mental and motor activities of the user should strictly be coordinated the feedback cues (highlighted areas) could play a role of strobe-signals to stimulate user behavior and to support, to stabilize temporal framework for rhythmically-alternating cognitive and motor activities if the system could individually and dynamically turn, dwell mode could be more flexible

13. TAUCHI MMIG G. Evreinov 12_26 10.10.2003 Dwell Time how we could know what temporal window satisfies of user requirements and when it should be changed? menu pointing could be considered as temporal process or stimulus- dependent goal-directed behavior of the user; herewith, a behavior model includes a sequence of actions both on the user side – cognitive processing and motor actions, and procedures are within interface – highlighting or other verification about the state of a particular alternative the physical parameters of feedback signals may either facilitate synchronization of the interactive process or hinder performance in both cases, physical stimulus starts the motor reaction that could be measured based on real-time analysis the visual-motor reaction time, we could predict or optimize dwell interval *. * Bourhis, G., Pino, P. Temporal analysis of the use of an augmentative communication device. AAATE’03. IOS Press, Netherlands (2003), 294-298.

14. T threshold selection highlighting the menu item TAUCHI MMIG G. Evreinov 13_26 10.10.2003 Dwell Time the temporal diagram of the algorithm for measuring the user performance through visual-motor reaction time and correction of the dwell interval T 0 – the first variable interval, T 1 – the second variable interval and T 2 = T 1 T thresh. - dynamical threshold T thresh. = T 0 + T 1 Time T dwell RT i T0T0 T2T2 T1T1 selection

15. TAUCHI MMIG G. Evreinov 14_26 10.10.2003 Dwell Time after each highlighting the menu item, we can record a time of user reply in a kind of selection time besides that, we can count an average magnitude for AveRT on each 5 realizations now, if (AveRT < T thresh. – dT) then we may decrease T 0 on dT too that is, a new scan interval will equal T dwell = (T 0 + 2  T 1 ) – dT or T dwell = (T 0 – dT) + 2  T 1 demo4

16. TAUCHI MMIG G. Evreinov 15_26 10.10.2003 Dwell Time changing scan interval can occurred with equal probability in plus and minus, if user reactions are symmetrical regarding dynamical threshold T thresh. if user changed typing speed, each user replies AveRT < T thresh. and T dwell are permanently decreased due to decreasing T 0, therefore the time to change T thresh. in a kind of criterion could be index of some number of dT for instance, if T 0 have changed on ((-3)  dT), the magnitude of T thresh. may also be decreased, at least, on one half of this magnitude then, a new scan interval will equal T dwell = (T 0 – 3  dT) + 2  (T 1 – 3  dT/2) after changing dynamical threshold, probability of user replies in a field T 2 will be higher and AveRT > T thresh. this situation will lead, or not, to increasing T 0

17. TAUCHI MMIG G. Evreinov 16_26 10.10.2003 Dwell Time Usability-testing software for… Adaptive dwelling

18. TAUCHI MMIG G. Evreinov 17_26 10.10.2003 Dwell Time Shape1Shape2 Shape9Shape10 Menu Save Open ShapePos Data Timer1 lblGrid1 fraData Timer2 lblDwell lblThresh lblDelta lblWait txtTest comments… chkColor chkBorder chkSound chkTactile lblNAve chkAve chkSAve lblIndexC chkCThresh T0 changing T1 changing lblGrid2, 3 Clear lblDrawGraph lblError chkCursor chkSpots picGraph1picGraph2

19. TAUCHI MMIG G. Evreinov 18_26 10.10.2003 Dwell Time bPos = False kThresh = CInt(t2 - t1) t1 = GetTickCount Timer1.Enabled = True Timer1.Interval = tDwell Yes No Yes Exit Private Sub Form_MouseMove(…) Xtmp1 = CInt(X) Ytmp1 = CInt(Y) cursor is inside Shape1 bShape1 = True No Yes bThresh = True Yes t2 = GetTickCount t1 > 0 Yes reply's time or (T0 + T1) Shape1.BorderColor = &H80& Change Border Color =1 Yes bShape1 = True bShape2 = True bShape2 = False reset BorderColor reset FillColor Yes Timer1.Enabled = 0, Int. = 0 Timer2.Enabled = 0, Int. = 0 ImmWebControl1.StopEffect bShape1 = False reset BorderColor, reset FillColor If bThresh = True Then ChangeDwell If bThresh = True Then ChangeDwell

20. chkSAve.Value = 0 And chkAve.Value = 0 TAUCHI MMIG G. Evreinov 19_26 10.10.2003 Dwell Time Private Sub ChangeDwell() kThresh > 100 or tDwell/2 Yes No ChangeDwell ArrThresh(nClick) = tThresh ArrCThresh(nClick) = tThresh ArrScan(nClick) = tDwell t1 = 0: t2 = 0 ArrThresh(nClick) = kThresh nClick < (nAve + 1) Yes No ArrDTmp(nClick) = kThresh aveThresh = tThresh Shift of array and count of the sliding aveThresh aveThresh < min lower limit = tDelta + 20 Yes No aveThresh = min aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta chkSAve.Value = 1 chkAve.Value = 1 aveThresh = tThresh

21. Line1.BorderColor = &HFF00FF Line1.Y2 = picGraph1.ScaleHeight - tThresh Line1.Y1 = Line1.Y2 TAUCHI MMIG G. Evreinov 20_26 10.10.2003 Dwell Time aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta tDwell = tDwell - tDelta sDelta = sDelta - tDelta ArrScan(nClick) = tScan tDwell = tDwell + tDelta sDelta = sDelta + tDelta sDelta > tIndex  tDelta sDelta < (-1)  tIndex  tDelta new current value tThresh = tThresh + CInt(sDelta / 2) sDelta = 0 tThresh = tThresh + CInt(sDelta / 2) sDelta = 0 tThresh < 50 tThresh = 50 lower limit ArrCThresh(nClick) = tThresh new current value Private Sub ChangeDwell() tDwell < 50 tDwell = 50 lower limit

22. TAUCHI MMIG G. Evreinov 21_26 10.10.2003 Dwell Time after some kind of calibration the proposed algorithm will automatically keep dwell interval near convenient magnitude with given speed or increment dT adaptive temporal interval could be useful for many applications, which require of the periodic time correction in dependence on user performance, for instance, in the systems (head / eye / finger tracking) using dwell time to simulate mouse actions etc.

23. TAUCHI MMIG G. Evreinov 22_26 10.10.2003 Dwell Time T-Adaptive Unit form transfers buttons' events to the TextBox therefore you can use KeyPreview or directly txtControl TextBox

24. TAUCHI MMIG G. Evreinov 23_26 10.10.2003 Dwell Time Private Sub ChangeScan() rTime > 100 or tScan/2 Yes No ChangeScan ArrThresh(nClick) = tThresh ArrCThresh(nClick) = tThresh ArrScan(nClick) = tScan t1 = 0: t2 = 0 ArrThresh(nClick) = rTime - tScan nClick < (nAve + 1) Yes No ArrDTmp(nClick) = rTime - tScan aveThresh = tThresh Shift of array and count of the sliding aveThresh aveThresh < min lower limit Yes No aveThresh = min aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta T0T0

25. TAUCHI MMIG G. Evreinov p 24_26 10.10.2003 Dwell Time aveThresh < tThresh - tDelta aveThresh > tThresh + tDelta tScan = tScan - tDelta sDelta = sDelta - tDelta ArrScan(nClick) = tScan tScan = tScan + tDelta sDelta = sDelta + tDelta sDelta > tIndex  tDelta sDelta < (-1)  tIndex  tDelta new current value tThresh = tThresh + CInt(sDelta / 2) sDelta = 0 tThresh = tThresh + CInt(sDelta / 2) sDelta = 0 tThresh < 50 tThresh = 50 lower limit ArrCThresh(nClick) = tThresh new current value Private Sub ChangeScan() tScan < 100 tScan = 100 lower limit

26. TAUCHI MMIG G. Evreinov p 25_26 10.10.2003 References [1] Accot, J., Zhai, Sh. More than dotting the i’s — Foundations for crossing-based interfaces, CHI 2002, April 20-25, 2002, Minneapolis, Minnesota, USA. / AccotZhai2002.pdf [2] Zhai, Sh., Morimoto, C., Ihde, S. Manual And Gaze Input Cascaded (MAGIC) Pointing. In Proc. CHI’99: ACM Conference on Human Factors in Computing Systems. 246-253, Pittsburgh, 15-20 May1999. / magic.pdf [3] Jacob, R.J.K. Eye Movement-Based Interaction Techniques and the Elements of Next-Generation, Non- WIMP User Interfaces, http://www.roetting.de/eyes-tea/history/020919/jacob.htmlhttp://www.roetting.de/eyes-tea/history/020919/jacob.html [4] Jacob, R.J.K. What You Look At Is What You Get: Eye Movement-based Interaction Techniques. CHI’90. / EyeMovBasedInteraction.pdf [5] Jacob, R.J.K. Eye Movement-based Human-Computer Interaction Techniques: Toward Non-Command Interfaces. / EyeMovementBased.pdf [6] Shell, J.S., Vertegaal, R., Skaburskis, A.W. EyePliances: Attention-Seeking Devices that Respond to Visual Attention. CHI2003 / EyePliances.pdf [7] Hyrskykari, A. Gaze Control as an Input Device. / gazecontrol.pdf [8] Aoki, H., Itoh, K., Sumitomo, N. and Hansen, J.P. Usability of Gaze Interaction Compared to Mouse and Head-Tracking in Typing Japanese Texts on a Restricted On-Screen Keyboard for Disabled People. / GazeInteraction_iea2003-aoki.pdf [9] Myers, B.A., Bhatnagar, R., Nichols, J., Choon Hong Peck, Kong, D., Miller, R. and Long, A.Chr. Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002, April 20-25, 2002, Minneapolis, Minnesota, USA. / InteractingAtDistance.pdf [10] Cheng, K., Pulo, K. Direct Interaction with Large-Scale Display Systems using Infrared Laser Tracking Devices. Australasian Symposium on Information Visualisation, Adelaide, 2003. / DirectInteractionLargeScaleDisplays.pdf Dwell Time

27. TAUCHI MMIG G. Evreinov p 26_26 10.10.2003 [11] Hwang, F. Partitioning Cursor Movements in “Point and Click” Tasks. ACM 1-58113-630-7/03/0004. CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida, USA. / PartitioningCursor Movements.pdf [12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI2001, SIGCHI’01, March 31-April 4, 2001, Seattle, WA, USA. / LaserPointerInteraction.pdf [13] Steriadis, C.E. and Constantinou, Ph. Designing Human-Computer Interfaces for Quadriplegic People. ACM Transactions on Computer-Human Interaction, Vol. 10, No. 2, June 2003. / Interafce_QuadriplegicPeople.pdf [14] Lankford, C. Effective Eye-gaze Input Into Windows tm. Eye Tracking Research & Applications Symposium 2000 Palm Beach Gardens, FL, USA. / EffectiveEyeGazeInput.pdf [15] Bates, R. Have Patience with Your Eye Mouse! Eye-Gaze Interaction with Computers Can Work. pp. 33-37. / EyeMouseGazeInteraction.pdf [16] Ware, C., Mikaelian, H.H. An Evaluation of an Eye Tracker as a Device for Computer Input. CHI + GI 1987 / EvaluationEyeTracker_Input.pdf [17] Hinckley, K., Pausch, R., Goblel, J.C. and Kassell, N.F. A Survey of Design Issues in Spatial Input. / Spatial_InputSurvey.pdf [18] Edwards, G. A Tool for Creating Eye-aware Applications that Adapt to Changes in User Behavior. / EyeAwareApplications_Adapt.txt [19] Youngblut, C., Johnson, R.E. et al. Review of Virtual Environment Interface Technology, Institute for Defence Analyses, available at http://www.hitl.washington.edu/scivw/scivw-ftp/publications/IDA-pdf/http://www.hitl.washington.edu/scivw/scivw-ftp/publications/IDA-pdf/ Dwell Time