November 4, 2000SMPC Conference1 Technology and Film-Music (Multi-modal) Research Dr. Scott D. Lipscomb Institute for Music Research University of Texas at San Antonio Interdisciplinary Perspectives on Film Music

November 4, 2000SMPC Conference2 Organization of Presentation Types of audio & visual stimuli used in film music research Methods of A-V stimulus presentation Future directions & possibilities

November 4, 2000SMPC Conference3 Sine Tones

November 4, 2000SMPC Conference4 The Phonograph

November 4, 2000SMPC Conference5 Research Studies Tannenbaum (1956); dramatic presentation (live, studio tape, or recording of live performance) w/phonograph “accompaniment”

November 4, 2000SMPC Conference6 Videocassette Recorder (VHS)

November 4, 2000SMPC Conference7 Research Studies Thompson, Russo, & Sinclair (1994); investigation of perceived “closure” Experiment 3 Excerpts from Clue (1985) w/newly-composed MIDI scores Bolivar, Cohen, & Fentress (1994); “congruency” between audio & visual Experiment 1, 2, & 3 Videos showing “friendly” or “aggressive” social interactions between wolves w/audio selected from commercially-used excerpts for broadcast Marshall & Cohen (1988)

November 4, 2000SMPC Conference8 Research Studies Lipscomb & Kendall (1994); reliability of composer intent & subject SD ratings Excerpts from Star Trek IV: The Voyage Home with intended music & “other” music from same film Bullerjahn & Güldenring (1994); qualitative content analysis Excerpts from “The Joker” (film school project) with newly-composed music by three German composers

November 4, 2000SMPC Conference9 Research Studies Krumhansl & Schenck (1997); can dance reflect structural & expressive qualities of music? VHS presentation of filmed ballet performance (single camera) w/Mozart’s Divertimento No. 15 in Bb major (K. 287) Iwamiya (1994); audio-visual interactions Laserdisc excerpts transferred to VHS in “matched” and “mismatched” conditions Boltz, et al (late 80s & 90s); filmed narrative & memory for events/time estimates

November 4, 2000SMPC Conference10 Laserdisc Player

November 4, 2000SMPC Conference11 Research Studies Lipscomb (1995); synchronization of audio & visual Laserdisc excerpts accompanied by digital audio files

November 4, 2000SMPC Conference12 DVD Player

November 4, 2000SMPC Conference13 Incredible Potential for Future Investigations Substantial increase in data capacity DVD-R to create “compilation” of excerpts from a variety of resources

November 4, 2000SMPC Conference14 Microcomputer

November 4, 2000SMPC Conference15 Stimulus Presentation Now you see it …

November 4, 2000SMPC Conference16 Television Monitor

November 4, 2000SMPC Conference17 Large-Screen TV

November 4, 2000SMPC Conference18 Data Projector

November 4, 2000SMPC Conference19 Viewing DVD source as Projected Image

November 4, 2000SMPC Conference20 Computer Control Complete Randomization in Stimulus Presentation

November 4, 2000SMPC Conference21 Interfaces VHS Interface (RS-232) Laserdisc Interface (serial)

November 4, 2000SMPC Conference22 Research Studies Thompson, Russo, & Sinclair (1994); investigation of perceived “closure” Experiment 1 3-D animation w/MIDI audio Experiment 2 Original film footage digitized into Quicktime format w/MIDI audio

November 4, 2000SMPC Conference23 Research Studies Sirius & Clarke (1994); effect of music on perceived meaning of images Original 3-D animations w/newly-composed MIDI audio in various “styles” Romantic, sci-fi, comic, Spanish, chase, disco, thriller, western

November 4, 2000SMPC Conference24 Research Studies Lipscomb (1995); synchronization of audio & visual Laserdisc excerpts accompanied by digital audio files Allowed complete randomization of stimuli Datteri (1998); influence of audio on ambiguous visual stimulus black & white vertical bars and sine signal, either single tone or scale

November 4, 2000SMPC Conference25 Music Experiment Development System (MEDS) Dr. Roger A. Kendall University of California, Los Angeles

November 4, 2000SMPC Conference26 MEDS’ Control Panel

November 4, 2000SMPC Conference27 MEDS’ Experiment Module

November 4, 2000SMPC Conference28 Stimulus PlayList

November 4, 2000SMPC Conference29 Stimulus Creation Synthesize complex signals MIDI data

November 4, 2000SMPC Conference30 Data Analysis A closer look at the stimuli

November 4, 2000SMPC Conference31 Fast Fourier Transform

November 4, 2000SMPC Conference32 FFT Displays

November 4, 2000SMPC Conference33 RMS Calculation

November 4, 2000SMPC Conference34 RMS Display

November 4, 2000SMPC Conference35 MEDS Data Editor

November 4, 2000SMPC Conference36 Statistical Analysis Exporting Data

November 4, 2000SMPC Conference37 SPSS, Systat, Sygraph, etc.

November 4, 2000SMPC Conference38 Descriptive Stats

November 4, 2000SMPC Conference39 Graphic Representation

November 4, 2000SMPC Conference40 Cluster Analysis

November 4, 2000SMPC Conference41 Multidimensional Scaling

November 4, 2000SMPC Conference42 Future Possibilities The Internet as a Research Tool

November 4, 2000SMPC Conference43 Difficulties in A-V Presentation Full-screen, full-motion A-V presentation is processor-intensive Especially w/16-bit, 44.1 KHz stereo audio A-V synchronization can be problematic Even with streaming technologies, internet presentation of A-V stimuli is unreliable over the internet Quality varies radically at the receiving end May be due to a number of unpredictable factors

November 4, 2000SMPC Conference44 Broadband Connections Modem28.8/56 kbps ISDN128 kbps DSL500+ kbps T kbps T3??

November 4, 2000SMPC Conference45 Streaming Technologies Apple Quicktime Real Media MPEG video Macromedia Shockwave & Flash Microsoft Media Technologies Media 100 iFINISH Avid ePublisher others

November 4, 2000SMPC Conference46 Future Experiment? Subject pool world-wide Rather than freshmen taking General Psych Stimuli viewed over broadband internet connection full-screen, full-motion streaming video with (relatively) hi- fidelity sound Subject responses are fed directly into a database at the researcher’s location Data are analyzed & interpreted Results published in a peer-reviewed ejournal Complete data set is made available online

November 4, 2000SMPC Conference47 Potential Advantages Greater generalizeability of results Larger “N” Less heterogeneous groups Facilitates cross-cultural research Fosters collaborative research and confirmation of data analysis

November 4, 2000SMPC Conference48 Contact Info Dr. Scott D. Lipscomb Institute for Music Research The University of Texas at San Antonio

November 4, 2000SMPC Conference49 Model of Film Music Perception (original, 1994)