New Interaction Techniques Department of Computer Sciences University of Tampere, Finland September – December, 2003 Grigori Evreinov Visualization for Temporal events

TAUCHI MMIG G. Evreinov 01_ … Temporal Events a sequential appearance of events, entities, or their properties leads to the time perception [1, 2] in-/decrement of some parameter with time can be transformed into the spatial pattern of discrete or analogous signals spatial distribution of these signals can be proportional, or non- proportional, to the initial temporal intervals – depends on conventions in any case, spatially-distributed signals will be perceived like a sequence of similar or different events which are stimuli for human perception there are two main metaphors for time presentation: Time-line and Clock Face /Dial both “absolute” time imaging and perception have difference /error between real temporal parameters of the event, presented value and perceived moments: onset /start, offset /stop and duration of some interval  Time perception … …

TAUCHI MMIG G. Evreinov 02_ Temporal Events time is the most important dimension in music its representation and manipulation is crucial for the expressiveness of a composition environment [3] therefore imaging for temporal events was especially developed for Musical Instrument Digital Interface (MIDI) [4]

TAUCHI MMIG G. Evreinov 03_ Temporal Events Peter Hanappe [3] gives the following list of some of the issues in time representation concerning music events and objects: the description of timing information the construction of composite temporal objects the use of composite objects as basic elements the definition of temporal relations between objects the manipulation and transformation of composite objects the interactive, dynamic control of a composition the relations between continuous time and event time time and hierarchy in musical structures the manipulation of continuous time functions the relation between composed time and performance time the representation of temporal information during runtime the relation between ``out-of-time'' structures and ``in-time'' realization

[7] TAUCHI MMIG G. Evreinov 04_ Temporal Events there are many other examples when user is in time-dependent situation and must evaluate short intervals to make a decision, change own behavior or interaction strategy the systems, based on head /eye /finger tracking, use temporal window to simulate mouse actions herewith, spatial or temporal (sound) time imaging can influence onto the experimental data due to attention & perception features [5, 6, 7] [8]

TAUCHI MMIG G. Evreinov 05_ Temporal Events [8] [10] “an attention shift can be easily elicited by a stimulus appearing in the periphery (e.g., Theeuwes, 1991) an attention shift takes time (e.g., Müller & Findlay, 1988) a phenomenal representation of a stimulus is not available before the end of an attention shift (e.g., Van der Heijden, 1992)” (cit. on [7])

TAUCHI MMIG G. Evreinov 06_ Temporal Events to be an effective, visualization first should concentrate user attention on the task, target or whatever noise is interference of signals with similar parameters that distract and not allow to accomplish the task effectively

TAUCHI MMIG G. Evreinov 07_ Temporal Events [11] [12] [13]

TAUCHI MMIG G. Evreinov 08_ Temporal Events the task is how to synchronize the user behavior regarding temporal window (dwell time, scan interval or whatever) not only to help in visual search, but the time imaging should aid the user in planning the motor actions and with a decision making Usability-testing software for… Time Indicator 1

TAUCHI MMIG G. Evreinov 09_ Temporal Events the special test showed that visualization of the temporal process sometimes does not add useful information for user but distracts and throws out of time while an average error rate was inessential increased, latent period and dispersion of reactions increased up to 30% ERA DHL GWV Time, ms 600 ERA DHL GWV

TAUCHI MMIG G. Evreinov 10_ Shape1 Shape2 Shape4 String1 Timer1 lblTestSymb fraData Timer2 lblScan lblRTime lblTimeS lblMode txtTest comments… lblGrid1 lblError2 lblMode1 lblComm Clear lblDrawGraph lblError lblGrid2 picGraph1 Shape3 Shape5 Shape6 String2 String3 Timer3 CommPort OpenSave Statistics() Rtime, ms s (st.dev), ms BtnInput Grid DrawError DrawGraph1 ShapeSize to change ind. Clear TestStop EditTools Temporal Events delay scan interval

TAUCHI MMIG G. Evreinov 11_ Temporal Events Time Indicator 2 the task is how to present a temporal window (dwell time) in eye-gaze interface adhering to the principle of minimal distraction

TAUCHI MMIG G. Evreinov 12_ Temporal Events

TAUCHI MMIG G. Evreinov 13_ Temporal Events shpKey1 shpKey2 Shape1 Arrow1 Shape2 imgAnim fraData txtTest comments… OpenSave chkTrack1 chkTrack2 optArrow optCircle chkCircleV lblSzCircle lblSizeInd lblTime lblStart lblVisCursor lblTestTime lblGraph lblAnim lblInterval lblPause ArrowPos() DrawTrack() DwStart() DwStop() EditControls() PictureRefresh() Timer2 Delay (150ms) Timer1 time per segment Timer3 Animation of user tracks Form_MouseMove()

References [1] The Experience and Perception of Time. Stanford Encyclopedia of Philosophy, [2] William James (1890) The Principles of Psychology. Chapter XV[J. of Speculative Philosophy, vol. XX. p. 374] The perception of time. Cit. on [3] Hanappe, P. Design and implementation of an integrated environment for music composition and synthesis. PhD Theses, 1999, available at: [4] MIDI Timing Concepts, [5] Perception of Space and Time. Task dependent timing of perceptual events. Project site on [6] Thornton, I.M.: Exploring the onset repulsion effect. Investigative Ophthalmology & Visual Science 41, # 4221 (2000), [7] Müsseler, J. & Aschersleben, G. (1998). Localising the first position of a moving stimulus: The Fröhlich effect and an attention-shifting explanation. Perception & Psychophysics, 60, , muenchen.mpg.de/CA/PEOPLE/MUJO/POSTER/Poster_Chicago.pdfhttp:// muenchen.mpg.de/CA/PEOPLE/MUJO/POSTER/Poster_Chicago.pdf [8] Stork, S., Müsseler, J. & Neggers, B. (2001). The influence of eye movements on the perceived vanishing point of a moving stimulus. Poster präsentiert auf der 4. Tübinger Wahrnehmungskonferenz, Tübingen, März [9] Wu Jing-Ling, Nobuki, O. and Sakai, Y. Human Characteristic on Phase Discrimination between Visual and Auditory Sine-Wave Stimuli. Available at: Temporal Events TAUCHI MMIG G. Evreinov 14_

[10] Müsseler, J., Stork, S., & Kerzel, D. (2000) Vergleich von Lokalisationsfehlern am Anfang, in der Mitte und am Ende einer Bewegung (Fröhlich Effekt, Flash-Lag Effekt und Repräsentationales Momentum) Poster präsentiert auf der 41. Meeting der Psychonomic Society, New Orleans, November 2000, [11] Lankford, C Effective Eye-Gaze Input Into Windows, in Proceedings of the Eye Tracking Research & Applications Symposium 2000 (ETRA'00), Palm Beach Gardens, FL, ACM Press, [12] Hansen, J.P., Johansen, A.S., Hansen, D.W., Kenji Itoh and Satoru Mashino. Language Technology in a Predictive, Restricted On-screen Keyboard with Ambiguous Layout for Severely Disabled People. Paper submitted to the EACL 2003 Workshop on Language Modeling for Text Entry Methods, April 13, 2003, Budapest, Hungary. [13] Majaranta, P., MacKenzie, I. S., Aula, A., & Räihä, K.-J. (2003). Metrics for text entry research: Auditory and visual feedback during eye typing. Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2003, pp New York: ACM. [14] Fabre, J., Howard, S. and Smith, R. Designing time at the user interface. Behaviour & Information Technology, 2000, Vol. 19(6), [15] Carlis, J.-V. and Konston, J.-A. Interactive visualization of serial periodic data. In Proc. Of the ACM Conference on User Interface Software and Technology (UIST), San Francisco, CA, [16] Mizobuchi, S., Wanobe, E. How long is a “long” key press? In Proceedings of INTERACT’01, Michitake Hirose (Ed.), IOS Press, IFIP, 2001, pp Temporal Events TAUCHI MMIG G. Evreinov 15_