A UDITORY D ISPLAYS AND S ONIFICATION Ryan McGee MAT 200C.

Slides:

Advertisements

Similar presentations

Paul Lunn BSc(Hons) MSc PCPD FHEA MIET Supervised by Dr A Hunt ( Department of Electronics, The University of York)

Advertisements

Geber Ramalho Computação Musical: taxonomia Proposta por Stephen Pope.

Reflections Diffraction Diffusion Sound Observations Report AUD202 Audio and Acoustics Theory.

Chapter 7 Principles of Analog Synthesis and Voltage Control Contents Understanding Musical Sound Electronic Sound Generation Voltage Control Fundamentals.

Basic Spectrogram Lab 8. Spectrograms §Spectrograph: Produces visible patterns of acoustic energy called spectrograms §Spectrographic Analysis: l Acoustic.

Some from Heim Chap 13 Prepared By Beryl Plimmer 2011 Sound The sound files are not linked – most should work on windows 7 machines.

R OLE OF O BJECT I DENTIFICATION IN S ONIFICATION S YSTEM FOR V ISUALLY I MPAIRED Presented By, Ranjan Bangalore Seetharama.

Sonification in Theoretical Physics Katharina Vogt Prof. W. Plessas Prof. R. Höldrich Todtmoos, 12th september 2007.

Speech perception Relating features of hearing to the perception of speech.

SPPA 6010 Advanced Speech Science 1 The Source-Filter Theory: The Sound Source.

1 Interspeech Synthesis of Singing Challenge, Aug 28, 2007 Formant-based Synthesis of Singing Sten Ternström and Johan Sundberg KTH Music Acoustics, Speech.

Digital audio and computer music COS 116: 2/26/2008.

Multi-Modal Interface for A Real-Time CFD Solver Maryia Kazakevich, Pierre Boulanger, Walter Bischof and Manuel Garcia Nov 4, 2006.

Cyclic input of characters through a single digital button without visual feedback Yang Xiaoqing New Interaction Techniques Dept.

Grasping Graphs by Ear: Sonification of Interaction with Hidden Graphs Leena Vesterinen Department of Computer Sciences University of Tampere Finland.

Multimodal Interfaces in a Ubiquitous Computing Environment 3 rd UK-Ubinet Workshop —————— 9 th – 11 th February 2005 —————— Fausto. J. Sainz Salces, Dr.

So nification of the facial expressio ns New Interaction Techniques Department of Computer Science University of Tampere 27 February 2003 Guo Zhiguo Ioulia.

Music Processing Roger B. Dannenberg. Overview  Music Representation  MIDI and Synthesizers  Synthesis Techniques  Music Understanding.

Harmonics and Overtones Waveforms / Wave Interaction Phase Concepts / Comb Filtering Beat Frequencies / Noise AUD202 Audio and Acoustics Theory.

Human Psychoacoustics shows ‘tuning’ for frequencies of speech If a tree falls in the forest and no one is there to hear it, will it make a sound?

Emotional Intelligence and Agents – Survey and Possible Applications Mirjana Ivanovic, Milos Radovanovic, Zoran Budimac, Dejan Mitrovic, Vladimir Kurbalija,

Digital audio and computer music COS 116, Spring 2012 Guest lecture: Rebecca Fiebrink.

A Full Frequency Masking Vocoder for Legal Eavesdropping Conversation Recording R. F. B. Sotero Filho, H. M. de Oliveira (qPGOM), R. Campello de Souza.

CS Spring 2011 CS 414 – Multimedia Systems Design Lecture 2 –Auditory Perception and Digital Audio Klara Nahrstedt Spring 2011.

Introduction to Interactive Media 10: Audio in Interactive Digital Media.

Some from Heim Chap 13 Sound The sound files are not included– most should work on windows 7 machines you tube.

DIGITAL WATERMARKING OF AUDIO SIGNALS USING A PSYCHOACOUSTIC AUDITORY MODEL AND SPREAD SPECTRUM THEORY * By: Ricardo A. Garcia *Research done at: University.

DIGITAL WATERMARKING OF AUDIO SIGNALS USING A PSYCHOACOUSTIC AUDITORY MODEL AND SPREAD SPECTRUM THEORY By: Ricardo A. Garcia University of Miami School.

Audio Scene Analysis and Music Cognitive Elements of Music Listening

Synthesis advanced techniques. Other modules Synthesis would be fairly dull if we were limited to mixing together and filtering a few standard waveforms.

Multimedia Specification Design and Production 2013 / Semester 2 / week 3 Lecturer: Dr. Nikos Gazepidis

Biofeedback: What and Why?

1 Auditory, tactile, and vestibular sensory systems n Perceptually relevant characteristics of sound n The receptor system: The ear n Basic sensory characteristics.

Copyright 2004 Ken Greenebaum Introduction to Interactive Sound Synthesis Lecture 11: Modulation Ken Greenebaum.

10/13/98Acoustical Society of America1 Synchronization of musical sound and visual images: Issues of empirical and practical significance in multimedia.

Supervisor: Dr. Boaz Rafaely Student: Limor Eger Dept. of Electrical and Computer Engineering, Ben-Gurion University Goal Directional analysis of sound.

MULTIMEDIA INPUT / OUTPUT TECHNOLOGIES INTRODUCTION 6/1/ A.Aruna, Assistant Professor, Faculty of Information Technology.

Digital audio and computer music COS 116, Spring 2010 Adam Finkelstein Slides and demo thanks to Rebecca Fiebrink.

School of Informatics CG087 Time-based Multimedia Assets IntroductionDr Paul Vickers1 Introduction to time-based multimedia assets Dr Paul Vickers Dr.

School of Informatics School Research Conference 2003 Noise and other stuffDr Paul Vickers1 Noise, mobility, accessibility, and other stuff Paul Vickers.

The Physics of Music PHYS 140 Instructor: Deva O’Neil.

1 Human Computer Interaction Week 5 Interaction Devices and Input-Output.

Technician License Course Chapter 2 Lesson Plan Module 3 – Modulation and Bandwidth.

Harmonics. Each instrument has a mixture of harmonics at varying intensities Principle of superposition Periodics- Repeating patterns of waveforms.

School of Informatics CG087 Time-based Multimedia Assets Auditory displayDr Paul Vickers1 Sound as a communication medium Auditory display, sonification,

Performance Comparison of Speaker and Emotion Recognition

Vocal Tract & Lip Shape Estimation By MS Shah & Vikash Sethia Supervisor: Prof. PC Pandey EE Dept, IIT Bombay AIM-2003, EE Dept, IIT Bombay, 27 th June,

CHAPTER 4 COMPLEX STIMULI. Types of Sounds So far we’ve talked a lot about sine waves =periodic =energy at one frequency But, not all sounds are like.

Sound and Its System. What is sound? basically a waveform of energy that is produced by some form of a mechanical vibration (ex: a tuning fork), and has.

Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Lecturer: Jim Warren Lecture 15 Based on Chapter 13: Speech and Hearing.

Introduction to psycho-acoustics: Some basic auditory attributes For audio demonstrations, click on any loudspeaker icons you see....

Oscillator & Oscilloscope, Hearing Includes Lab 1 and Lab 3.

Lucent Technologies - Proprietary 1 Interactive Pattern Discovery with Mirage Mirage uses exploratory visualization, intuitive graphical operations to.

C HAPTER 10 D ATA D ISPLAY T ABLES, G RAPHS, M APS, V ISUALIZATIONS.

A BRIEF OVERVIEW THE AUDITORY PERSPECTIVE OF INTERFACES.

Framework for Modeling the Cognitive Process Spatial and Temporal Considerations in a Signal-Based Approach Paul Yaworsky AFRL/IFSB 10th ICCRTS June.

9 th Grade Science Chapter 12c.  Definition: ◦ Sounds used in regular pattern that is pleasing to hear  Every musical instrument has: ◦ Natural frequency:

Audio Scene Analysis and Music Cognitive Elements of Music Listening Kevin D. Donohue Databeam Professor Electrical and Computer Engineering University.

Spectral subtraction algorithm and optimize Wanfeng Zou 7/3/2014.

Measurement and Instrumentation

Music Music is a “ pleasant ” sound (longitudinal) wave. The pitch is the frequency of the wave. The loudness is the amplitude of the wave. Music is made.

1 Design Process for Auditory Interfaces Sylvain Daudé and Laurence Nigay HCI team of the CLIPS-IMAG Lab. (Grenoble, France)

Earcons, Auditory Icons and spearcons

Information-Theoretic Listening

Review Session 3: Sensation and Perception

Standing Waves Destructive interference Constructive interference

Auditory Interfaces map to event in computer sound new mail mail sent

Data Exploration Using Sonification

Speech Perception (acoustic cues)

Presentation transcript:

A UDITORY D ISPLAYS AND S ONIFICATION Ryan McGee MAT 200C

A UDITORY D ISPLAY A system that uses sound to convey computer data. Geiger Counter Allosphere

A UDITORY D ISPLAYS Mini-Allosphere – The US Airforce’s ALF

WHY? Adding sonifications to data visualization will be like adding sound to silent films.

L IMITATIONS Interfere with other auditory communication (speech or other AD output) Humans are better at remembering abstract visual patterns and shapes than abstract sounds. Musicians tend to do better though.

S ONIFICATION The processes and algorithms that generate the sound output of auditory displays “the transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication of interpretation”

S ONIFICATION

S OUNDS IN S ONIFICATION Auditory Icons Earcons Spearcons (new)

T YPES OF S ONIFICATION Iconic Direct Conversion (Audification) Musical Parameter-Mapping Model-Based Interactivity with biofeedback or parameter adjustment.

M APPING D ATA TO S OUND Not well defined Few formal guidelines Mainly an issue of psychology => Difficult Considering Aesthetics => More difficult What does data sound like?

P ARAMETERS OF S OUND 8 fundamental characteristics of a waveform by JoAnn Kuchera-Morin: frequency, amplitude, phase, envelope, spectrum, shape, velocity, wavelength Synthesis parameters: FM, AM, Additive, Grainular, etc Musical Parameters: tempo, tuning, etc Spatial Parameters: sound position in space

A PPLICTIONS Process Monitoring Exploratory Data Analysis Art/Music Human-Computer Interfaces Artificial Vision: vOICe => artificial synesthesia

D ISCUSSION : I S IT M USIC ? Music - “an artistic form of auditory communication incorporating instrumental or vocal tones in a structured and continuous manner” Is sonification a valid form of music? When can we call it music? Do we need to define “pleasing music”

R EFERENCES Bruce N. Walker and Gregory Kramer, Ecological Psychoacoustics and Auditory Displays: Hearing, Grouping, and Meaning Making Kramer, G. (1994a). An introduction to auditory display. In G. Kramer (Ed.), Auditory display: Sonification, audMA: Addison Wesley. Thomas Hermann, TAXONOMY AND DEFINITIONS FOR SONIFICATION AND AUDITORY DISPLAY (Proceedings of ICAD 2008) Tilman Dingler1, Jeffrey Lindsay2, Bruce N. Walker, LEARNABILTIY OF SOUND CUES FOR ENVIRONMENTAL FEATURES: AUDITORY ICONS, EARCONS, SPEARCONS, AND SPEECH, Proceedings of the 14th International Conference on Auditory Display, Paris, France, June 24-27, 2008 Thomas Hermann’s research on Sonification, Data Mining & Ambient Intelligence,