1. November 4, 2000SMPC Conference1 Technology and Film-Music (Multi-modal) Research Dr. Scott D. Lipscomb Institute for Music Research University of Texas at San Antonio Interdisciplinary Perspectives on Film Music

2. November 4, 2000SMPC Conference2 Organization of Presentation Types of audio & visual stimuli used in film music research Methods of A-V stimulus presentation Future directions & possibilities

3. November 4, 2000SMPC Conference3 Sine Tones

4. November 4, 2000SMPC Conference4 The Phonograph

5. November 4, 2000SMPC Conference5 Research Studies Tannenbaum (1956); dramatic presentation (live, studio tape, or recording of live performance) w/phonograph “accompaniment”

6. November 4, 2000SMPC Conference6 Videocassette Recorder (VHS)

7. November 4, 2000SMPC Conference7 Research Studies Thompson, Russo, & Sinclair (1994); investigation of perceived “closure” Experiment 3 Excerpts from Clue (1985) w/newly-composed MIDI scores Bolivar, Cohen, & Fentress (1994); “congruency” between audio & visual Experiment 1, 2, & 3 Videos showing “friendly” or “aggressive” social interactions between wolves w/audio selected from commercially-used excerpts for broadcast Marshall & Cohen (1988)

8. November 4, 2000SMPC Conference8 Research Studies Lipscomb & Kendall (1994); reliability of composer intent & subject SD ratings Excerpts from Star Trek IV: The Voyage Home with intended music & “other” music from same film Bullerjahn & Güldenring (1994); qualitative content analysis Excerpts from “The Joker” (film school project) with newly-composed music by three German composers

9. November 4, 2000SMPC Conference9 Research Studies Krumhansl & Schenck (1997); can dance reflect structural & expressive qualities of music? VHS presentation of filmed ballet performance (single camera) w/Mozart’s Divertimento No. 15 in Bb major (K. 287) Iwamiya (1994); audio-visual interactions Laserdisc excerpts transferred to VHS in “matched” and “mismatched” conditions Boltz, et al (late 80s & 90s); filmed narrative & memory for events/time estimates

10. November 4, 2000SMPC Conference10 Laserdisc Player

11. November 4, 2000SMPC Conference11 Research Studies Lipscomb (1995); synchronization of audio & visual Laserdisc excerpts accompanied by digital audio files

12. November 4, 2000SMPC Conference12 DVD Player

13. November 4, 2000SMPC Conference13 Incredible Potential for Future Investigations Substantial increase in data capacity DVD-R to create “compilation” of excerpts from a variety of resources

14. November 4, 2000SMPC Conference14 Microcomputer

15. November 4, 2000SMPC Conference15 Stimulus Presentation Now you see it …

16. November 4, 2000SMPC Conference16 Television Monitor

17. November 4, 2000SMPC Conference17 Large-Screen TV

18. November 4, 2000SMPC Conference18 Data Projector

19. November 4, 2000SMPC Conference19 Viewing DVD source as Projected Image

20. November 4, 2000SMPC Conference20 Computer Control Complete Randomization in Stimulus Presentation

21. November 4, 2000SMPC Conference21 Interfaces VHS Interface (RS-232) Laserdisc Interface (serial)

22. November 4, 2000SMPC Conference22 Research Studies Thompson, Russo, & Sinclair (1994); investigation of perceived “closure” Experiment 1 3-D animation w/MIDI audio Experiment 2 Original film footage digitized into Quicktime format w/MIDI audio

23. November 4, 2000SMPC Conference23 Research Studies Sirius & Clarke (1994); effect of music on perceived meaning of images Original 3-D animations w/newly-composed MIDI audio in various “styles” Romantic, sci-fi, comic, Spanish, chase, disco, thriller, western

24. November 4, 2000SMPC Conference24 Research Studies Lipscomb (1995); synchronization of audio & visual Laserdisc excerpts accompanied by digital audio files Allowed complete randomization of stimuli Datteri (1998); influence of audio on ambiguous visual stimulus black & white vertical bars and sine signal, either single tone or scale

25. November 4, 2000SMPC Conference25 Music Experiment Development System (MEDS) Dr. Roger A. Kendall University of California, Los Angeles

26. November 4, 2000SMPC Conference26 MEDS’ Control Panel

27. November 4, 2000SMPC Conference27 MEDS’ Experiment Module

28. November 4, 2000SMPC Conference28 Stimulus PlayList

29. November 4, 2000SMPC Conference29 Stimulus Creation Synthesize complex signals MIDI data

30. November 4, 2000SMPC Conference30 Data Analysis A closer look at the stimuli

31. November 4, 2000SMPC Conference31 Fast Fourier Transform

32. November 4, 2000SMPC Conference32 FFT Displays

33. November 4, 2000SMPC Conference33 RMS Calculation

34. November 4, 2000SMPC Conference34 RMS Display

35. November 4, 2000SMPC Conference35 MEDS Data Editor

36. November 4, 2000SMPC Conference36 Statistical Analysis Exporting Data

37. November 4, 2000SMPC Conference37 SPSS, Systat, Sygraph, etc.

38. November 4, 2000SMPC Conference38 Descriptive Stats

39. November 4, 2000SMPC Conference39 Graphic Representation

40. November 4, 2000SMPC Conference40 Cluster Analysis

41. November 4, 2000SMPC Conference41 Multidimensional Scaling

42. November 4, 2000SMPC Conference42 Future Possibilities The Internet as a Research Tool

43. November 4, 2000SMPC Conference43 Difficulties in A-V Presentation Full-screen, full-motion A-V presentation is processor-intensive Especially w/16-bit, 44.1 KHz stereo audio A-V synchronization can be problematic Even with streaming technologies, internet presentation of A-V stimuli is unreliable over the internet Quality varies radically at the receiving end May be due to a number of unpredictable factors

44. November 4, 2000SMPC Conference44 Broadband Connections Modem28.8/56 kbps ISDN128 kbps DSL500+ kbps T11300+ kbps T3??

45. November 4, 2000SMPC Conference45 Streaming Technologies Apple Quicktime Real Media MPEG video Macromedia Shockwave & Flash Microsoft Media Technologies Media 100 iFINISH Avid ePublisher others

46. November 4, 2000SMPC Conference46 Future Experiment? Subject pool world-wide Rather than freshmen taking General Psych Stimuli viewed over broadband internet connection full-screen, full-motion streaming video with (relatively) hi- fidelity sound Subject responses are fed directly into a database at the researcher’s location Data are analyzed & interpreted Results published in a peer-reviewed ejournal Complete data set is made available online

47. November 4, 2000SMPC Conference47 Potential Advantages Greater generalizeability of results Larger “N” Less heterogeneous groups Facilitates cross-cultural research Fosters collaborative research and confirmation of data analysis

48. November 4, 2000SMPC Conference48 Contact Info Dr. Scott D. Lipscomb Institute for Music Research The University of Texas at San Antonio lipscomb@utsa.edu lipscomb@utsa.edu http://imr.utsa.edu/lipscomb/

49. November 4, 2000SMPC Conference49 Model of Film Music Perception (original, 1994)