1. Dwell Time Selection & more…
New Interaction Techniques
Dwell Time Selection & more…
Grigori Evreinov
Department of Computer Sciences University of Tampere, Finland
Department of Computer Sciences University of Tampere, Finland
September – December, 2003

2. Cursor Object Inter-actions
Dwell Time
Cursor Object Inter-actions Y
[1] X
Time
Time
demo1
TAUCHI MMIG G. Evreinov _

3. Dwell Time Gus! Dwell Cursor,
The Smart-Nav AT package (Dwell clicking s),
MagicCursor 2000: Dwell Selection Clicking Solution,
TAUCHI MMIG G. Evreinov _

4. Gaze Control Dwell Time
[1]
[2]
TAUCHI MMIG G. Evreinov _

5. Dwell Time LC Technologies, Inc. The Eye-Gaze System Makers
[31]
TAUCHI MMIG G. Evreinov _

6. Laser Control Dwell Time
[9] Myers, B.A., et al. Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002.
[12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI 2001.
TAUCHI MMIG G. Evreinov _

7. Head Control Dwell Time
Natural Point trackIRTM ,
demo2
demo3
Marsden, R. 100% Hands-free Computer Access- Madentec’s 2000 Series
TAUCHI MMIG G. Evreinov _

8. txtEntry Eye-Gaze Usability-testing software for… Dwell Time
TAUCHI MMIG G. Evreinov _

9. Ctrl+L=>EditCapts.
Lesson6 txtEntry_EyeGaze3
Dwell Time
TWords.txt /phrases
txtPersonData comments…
WordsLoading
GridData1:
test words/chars
entered text /chars
time per char, ms
fraData
lblSave_Click()
Test initialization
SetTest
lblOpen_Click()
lblTestSymbol
GridData1_Click()
to save column
lblDwDelay
Up/Down
txtText1
SetData
GridData2:
char per word /phrase
num. of entered words
num. of strokes /clicks per word /phrase
time per word/phrase, s
lblDwell
Statistics()
Rtime, ms
s (st.dev), ms
Errors
wpm
TestTime, s
SpotActivate
Ctrl+K=>move keys
Ctrl+L=>EditCapts.
On-screen Keyboard
Ctrl+W=>Dwell
SetSigns
GridData2_Click()
to save column
Trial start
SetCharacters
Break test
BackSp
MouseInput
SymbDec
Timer1
Clear Data
Timer3
Timer4
HideBorders
Timer2
TAUCHI MMIG G. Evreinov _

10. Ctrl+L=>EditCapts.
Lesson6 txtEntry_EyeGaze3
Dwell Time
lblDwDelay
fraData
Up/Down
Test initialization
lblDwell
lblTestSymbol
txtText1
SetData
SpotActivate
Ctrl+K=>move keys
Ctrl+L=>EditCapts.
On-screen Keyboard
Ctrl+W=>Dwell
SetSigns
Trial start
SetCharacters
MouseInput
SymbDec
Break test
Clear Data
tmrDwDelay
tmrDwell
HideBorders
TAUCHI MMIG G. Evreinov _

11. Adaptive Dwelling Dwell Time
according to Windows Interface Design Guide*, the action associated with the control is initiated when the mouse button is released
if the pointer is dragged to another location, the control will return to its original state and undesirable click will be stopped
for instance, the user can put cursor over icon or command button and choose it by clicking the mouse
sometimes, clicks are undesirable or impossible (laser pointer, gaze / head / ‘brain’ control…)
to provide productivity and easy access during communication with computer it is necessary a balance between flexibility of the dialogue structure, adaptability on each level of the interface, cognitive abilities of the person and limitations used interaction techniques
semantic, syntactic, lexical
* The Windows Interface. An Application Design Guide. Microsoft Press, Redmond, Washington, 1992.
TAUCHI MMIG G. Evreinov _

12. Dwell Time
the time is one of the most important critical parameters of the system feedback loop
a normal time of simple visual-motor control task in accomplishing with the help of intact muscles of the finger is about ms
the time is progressively increased up to 1000 ms and more, if an additional semantic analysis or physical limitations take place
to prevent wrong selection external (dwell) timer, mental and motor activities of the user should strictly be coordinated
the feedback cues (highlighted areas) could play a role of strobe-signals to stimulate user behavior and to support, to stabilize temporal framework for rhythmically-alternating cognitive and motor activities
if the system could individually and dynamically turn, dwell mode could be more flexible
TAUCHI MMIG G. Evreinov _

13. Dwell Time
how we could know what temporal window satisfies of user requirements and when it should be changed?
menu pointing could be considered as temporal process or stimulus-dependent goal-directed behavior of the user; herewith, a behavior model includes a sequence of actions both on the user side – cognitive processing and motor actions, and procedures are within interface – highlighting or other verification about the state of a particular alternative
the physical parameters of feedback signals may either facilitate synchronization of the interactive process or hinder performance
in both cases, physical stimulus starts the motor reaction that could be measured
based on real-time analysis the visual-motor reaction time, we could predict or optimize dwell interval*.
* Bourhis, G., Pino, P. Temporal analysis of the use of an augmentative communication device. AAATE’03. IOS Press, Netherlands (2003),
TAUCHI MMIG G. Evreinov _

14. highlighting the menu item
Dwell Time
selection
highlighting the menu item
selection
selection
Tthreshold
RTi
Time T0 T1 T2
Tdwell
the temporal diagram of the algorithm for measuring the user performance through visual-motor reaction time and correction of the dwell interval
T0 – the first variable interval, T1 – the second variable interval and T2 = T1
Tthresh. - dynamical threshold Tthresh. = T0 + T1
TAUCHI MMIG G. Evreinov _

15. Dwell Time
after each highlighting the menu item, we can record a time of user reply in a kind of selection time
besides that, we can count an average magnitude for AveRT on each 5 realizations
now, if (AveRT < Tthresh. – dT) then we may decrease T0 on dT too
that is, a new scan interval will equal
Tdwell = (T0 + 2  T1) – dT or Tdwell = (T0 – dT) + 2  T1
demo4
TAUCHI MMIG G. Evreinov _

16. Dwell Time
changing scan interval can occurred with equal probability in plus and minus, if user reactions are symmetrical regarding dynamical threshold Tthresh.
if user changed typing speed, each user replies
AveRT < Tthresh. and Tdwell
are permanently decreased due to decreasing T0, therefore the time to change Tthresh.
in a kind of criterion could be index of some number of dT
for instance, if T0 have changed on ((-3)  dT), the magnitude of Tthresh. may also be decreased, at least, on one half of this magnitude
then, a new scan interval will equal
Tdwell = (T0 – 3  dT) + 2  (T1 – 3  dT/2)
after changing dynamical threshold, probability of user replies in a field T2 will be higher and AveRT > Tthresh.
this situation will lead, or not, to increasing T0
TAUCHI MMIG G. Evreinov _

17. Adaptive dwelling Usability-testing software for… Dwell Time
TAUCHI MMIG G. Evreinov _

18. Dwell Time Data fraData ShapePos txtTest comments… lblGrid2, 3 Clear
Open
lblDwell
chkColor
T0 changing
T1 changing
Save
Menu
lblThresh
chkBorder
lblNAve
lblIndexC
lblGrid1
lblDelta
chkSound
chkAve
chkCThresh
lblWait
chkTactile
chkSAve
Shape1
Shape2
lblError
chkCursor
chkSpots
lblDrawGraph
picGraph1
Shape9
Shape10
Timer1
picGraph2
Timer2
TAUCHI MMIG G. Evreinov _

19. Dwell Time Private Sub Form_MouseMove(…) bPos = False
Xtmp1 = CInt(X) Ytmp1 = CInt(Y)
Yes
cursor is inside Shape1
Timer1.Enabled = 0, Int. = 0
Timer2.Enabled = 0, Int. = 0
ImmWebControl1.StopEffect
bShape1 = False
reset BorderColor, reset FillColor No
Yes
bThresh = True
bShape1 = True
Yes
Yes No
t2 = GetTickCount
bShape1 = True
bShape2 = True
t1 > 0
Change Border Color =1
Yes
Yes
Yes
bShape2 = False
reset BorderColor
reset FillColor
kThresh = CInt(t2 - t1)
Shape1.BorderColor = &H80&
reply's time or (T0 + T1)
t1 = GetTickCount
Timer1.Enabled = True
Timer1.Interval = tDwell
If bThresh = True Then
ChangeDwell
If bThresh = True Then
ChangeDwell
Exit
TAUCHI MMIG G. Evreinov _

20. Dwell Time Private Sub ChangeDwell() ChangeDwell
kThresh > 100 or tDwell/2
ArrThresh(nClick) = kThresh
Yes
ChangeDwell
nClick < (nAve + 1) No
Yes
ArrDTmp(nClick) = kThresh
aveThresh = tThresh
ArrThresh(nClick) = tThresh
ArrCThresh(nClick) = tThresh
ArrScan(nClick) = tDwell
t1 = 0: t2 = 0 No
Shift of array and count of the sliding aveThresh
chkSAve.Value = 1
aveThresh < min
lower limit = tDelta + 20
chkAve.Value = 1
Yes
aveThresh = min
chkSAve.Value = 0 And chkAve.Value = 0 No
aveThresh < tThresh - tDelta
aveThresh = tThresh
aveThresh > tThresh + tDelta
TAUCHI MMIG G. Evreinov _

21. Dwell Time Private Sub ChangeDwell() aveThresh < tThresh - tDelta
tDwell = tDwell - tDelta
sDelta = sDelta - tDelta
tDwell = tDwell + tDelta
sDelta = sDelta + tDelta
tDwell < 50
lower limit
tDwell = 50
ArrScan(nClick) = tScan
new current value
sDelta > tIndex  tDelta
sDelta < (-1)  tIndex  tDelta
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh < 50
tThresh = 50
lower limit
ArrCThresh(nClick) = tThresh
new current value
Line1.BorderColor = &HFF00FF
Line1.Y2 = picGraph1.ScaleHeight - tThresh
Line1.Y1 = Line1.Y2
TAUCHI MMIG G. Evreinov _

22. Dwell Time
after some kind of calibration the proposed algorithm will automatically keep dwell interval near convenient magnitude with given speed or increment dT
adaptive temporal interval could be useful for many applications, which require of the periodic time correction in dependence on user performance, for instance, in the systems (head / eye / finger tracking) using dwell time to simulate mouse actions etc.
TAUCHI MMIG G. Evreinov _

23. T-Adaptive Unit Dwell Time
form transfers buttons' events to the TextBox therefore you can use KeyPreview or
directly txtControl TextBox
TAUCHI MMIG G. Evreinov _

24. Dwell Time Private Sub ChangeScan() ChangeScan nClick < (nAve + 1)
rTime > 100 or tScan/2
ArrThresh(nClick) = rTime - tScan T0
Yes
ChangeScan
nClick < (nAve + 1) No
Yes
ArrDTmp(nClick) = rTime - tScan
aveThresh = tThresh
ArrThresh(nClick) = tThresh
ArrCThresh(nClick) = tThresh
ArrScan(nClick) = tScan
t1 = 0: t2 = 0 No
Shift of array and count of the sliding aveThresh
aveThresh < min
lower limit
Yes
aveThresh = min No
aveThresh < tThresh - tDelta
aveThresh > tThresh + tDelta
TAUCHI MMIG G. Evreinov _

25. Dwell Time Private Sub ChangeScan() aveThresh < tThresh - tDelta
tScan = tScan - tDelta
sDelta = sDelta - tDelta
tScan = tScan + tDelta
sDelta = sDelta + tDelta
tScan < 100
lower limit
tScan = 100
ArrScan(nClick) = tScan
new current value
sDelta > tIndex  tDelta
sDelta < (-1)  tIndex  tDelta
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh = tThresh + CInt(sDelta / 2)
sDelta = 0
tThresh < 50
tThresh = 50
lower limit
ArrCThresh(nClick) = tThresh
new current value
TAUCHI MMIG G. Evreinov p 24_

26. Dwell Time
References
[1] Accot, J., Zhai, Sh. More than dotting the i’s — Foundations for crossing-based interfaces, CHI 2002, April 20-25, 2002, Minneapolis, Minnesota, USA. / AccotZhai2002.pdf
[2] Zhai, Sh., Morimoto, C., Ihde, S. Manual And Gaze Input Cascaded (MAGIC) Pointing. In Proc. CHI’99: ACM Conference on Human Factors in Computing Systems , Pittsburgh, May1999. / magic.pdf
[3] Jacob, R.J.K. Eye Movement-Based Interaction Techniques and the Elements of Next-Generation, Non-WIMP User Interfaces,
[4] Jacob, R.J.K. What You Look At Is What You Get: Eye Movement-based Interaction Techniques. CHI’90. / EyeMovBasedInteraction.pdf
[5] Jacob, R.J.K. Eye Movement-based Human-Computer Interaction Techniques: Toward Non-Command Interfaces. / EyeMovementBased.pdf
[6] Shell, J.S., Vertegaal, R., Skaburskis, A.W. EyePliances: Attention-Seeking Devices that Respond to Visual Attention. CHI2003 / EyePliances.pdf
[7] Hyrskykari, A. Gaze Control as an Input Device. / gazecontrol.pdf
[8] Aoki, H., Itoh, K., Sumitomo, N. and Hansen, J.P. Usability of Gaze Interaction Compared to Mouse and Head-Tracking in Typing Japanese Texts on a Restricted On-Screen Keyboard for Disabled People. / GazeInteraction_iea2003-aoki.pdf
[9] Myers, B.A., Bhatnagar, R., Nichols, J., Choon Hong Peck, Kong, D., Miller, R. and Long, A.Chr. Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices. CHI 2002, April 20-25, 2002, Minneapolis, Minnesota, USA. / InteractingAtDistance.pdf
[10] Cheng, K., Pulo, K. Direct Interaction with Large-Scale Display Systems using Infrared Laser Tracking Devices. Australasian Symposium on Information Visualisation, Adelaide, / DirectInteractionLargeScaleDisplays.pdf
TAUCHI MMIG G. Evreinov p 25_

27. Dwell Time
[11] Hwang, F. Partitioning Cursor Movements in “Point and Click” Tasks. ACM /03/0004. CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida, USA. / PartitioningCursor Movements.pdf
[12] Olsen Jr., D.R. and Nielsen, T. Laser Pointer Interaction. CHI2001, SIGCHI’01, March 31-April 4, 2001, Seattle, WA, USA. / LaserPointerInteraction.pdf
[13] Steriadis, C.E. and Constantinou, Ph. Designing Human-Computer Interfaces for Quadriplegic People. ACM Transactions on Computer-Human Interaction, Vol. 10, No. 2, June / Interafce_QuadriplegicPeople.pdf
[14] Lankford, C. Effective Eye-gaze Input Into Windowstm. Eye Tracking Research & Applications Symposium 2000 Palm Beach Gardens, FL, USA. / EffectiveEyeGazeInput.pdf
[15] Bates, R. Have Patience with Your Eye Mouse! Eye-Gaze Interaction with Computers Can Work. pp / EyeMouseGazeInteraction.pdf
[16] Ware, C., Mikaelian, H.H. An Evaluation of an Eye Tracker as a Device for Computer Input. CHI + GI 1987 / EvaluationEyeTracker_Input.pdf
[17] Hinckley, K., Pausch, R., Goblel, J.C. and Kassell, N.F. A Survey of Design Issues in Spatial Input. / Spatial_InputSurvey.pdf
[18] Edwards, G. A Tool for Creating Eye-aware Applications that Adapt to Changes in User Behavior. / EyeAwareApplications_Adapt.txt
[19] Youngblut, C., Johnson, R.E. et al. Review of Virtual Environment Interface Technology, Institute for Defence Analyses, available at
TAUCHI MMIG G. Evreinov p 26_