1 Design Process for Auditory Interfaces Sylvain Daudé and Laurence Nigay HCI team of the CLIPS-IMAG Lab. (Grenoble, France)

2 Purpose zAuditory display = interlinking location zUnified design framework for auditory interfaces HCI Psycho-acoustics DSP Information visualization Computer music Sonification

3 Motivations zDesign framework yexisting for data visualization... y... but not for data sonification zDouble interest of a design framework yclassification of existing results and systems yhelp to the auditory interfaces designer

4 Visualization process [Chi 1998] Data Bienvenue à IHM2002 Documents set Analytical abstraction : Data view Data transformation Documents list Bienvenue à IHM2002 Visualization abstraction : Abstract visual space Visualization transformation Visual realization transformation View : Space view

5 Visualization process [Chi 1998] Data Analytical abstraction : Data view Data transformation Documents list Bienvenue à IHM2002 Visualization abstraction : Abstract visual space Visualization transformation 2D space Bienvenue à IHM2002 View : Space view Visual realization transformation

6 Visualization process [Chi 1998] Data Analytical abstraction : Data view Data transformation Visualization abstraction : Abstract visual space Visualization transformation 2D space Bienvenue à IHM2002 Visual realization transformation Deformed view View : Space view

7 Sonification process Data Picture Analytical abstraction : Data view Data transformation Pixels columns Sonic rendering Aud. display transformation Sonification transformation Sonic abstraction : Abstract sound space...

8 Sonification process Data Analytical abstraction : Data view Data transformation Sonic rendering Aud. display transformation Sonification transformation Gray level = volume Ordinate = pitch Sonic abstraction : Abstract sound space Sounds sequence...

9 Sonification process Data Analytical abstraction : Data view Data transformation Sonic rendering Aud. display transformation Sonification transformation Sonic abstraction : Abstract sound space Sounds sequence...

10 Data transformation z Common with visualization z Dependent on ythe data ythe data semantics ythe user task yex : AROMA Data Analytical abstraction : Data view Sonic abstraction : Abstract sound space Sonic rendering Data transformation Sonification transformation Aud. display transformation

11 Sonification transformation : abstract sound space Data Analytical abstraction : Data view Sonic abstraction : Abstract sound space Sonic rendering Data transformation Sonification transformation Aud. display transformation z Abstract sound space : y= euclidean space y= 1 point if no spatialization yabstract sound = xduration xduring which acoustical cues vary z Two sonification levels y1 data element -> 1 abstract sound yCoordination of the abstract sounds

12 Sonification transformation : abstract sound space Data Analytical abstraction : Data view Sonic abstraction : Abstract sound space Sonic rendering Data transformation Sonification transformation Aud. display transformation z Abstract sound space : y= euclidean space centered on user y= 1 point if no spatialization yabstract sound = xduration xacoustical cues varying in time z Two sonification levels y1 data element -> 1 abstract sound yCoordination of the abstract sounds

13 Sonification transformation : representing a single element z3 independent mappings Data elementAbstract sound - semantics - structure - value - semantics - structure - value 1 2 3

14 Sonification transformation : representing a single element z3 independent mappings zSpecial cue : spatial position zSemantics = independent from semantics of sound yAnalogical spatialization ? xAnalogy with a phenomenon : Sonic : cathedral Non-sonic : Diary in the Sky Appointment at 3 pm Data elementAbstract sound - semantics - structure - value - semantics - structure - value 1 2 3

15 Sonification transformation : representing a set of data zTwo sonification levels y1 single data element  1 sound y1 set of data  1 set of sounds zCoordination of the sounds of the sound set yin time yin space Same place Different places SimultaneousAlternate Pixels column Stereo sound Diary in the Sky Speech synthesis Space Time

16 Auditory display transformation: physical device z 4 steps transformation : (i) chosing a view point (ii) sound space -> device parameters (iii) device parameters -> sound signal (iv) sound signal -> physical sound Data Analytical abstraction : Data view Sonic abstraction : Abstract sound space Sonic rendering Data transformation Sonification transformation Aud. display transformation

17 Auditory display transformation: physical device z (iv) Physical device : ysound precision ydegree of spatiality ygeometrical distribution of the loudspeakers ydegree of sharing yliberty of movement ydegree of immersion z Ex : Soundbeam Neckset Data Analytical abstraction : Data view Sonic abstraction : Abstract sound space Sonic rendering Data transformation Sonification transformation Aud. display transformation

18 Contributions a. classification z Example : the « earcons category » :

19 Contributions b. help for the designer z Identification of design steps z Identification of design parameters for each step : yguide for the design yfoundation for design heuristics ycontext-aware sonifications

20 Perspectives z Design heuristics z Links with a software architecture z Platform for sonification design based on this process

21 Thank you for your attention… z… Any question ?

22