1. Robert Wechsler Workshop/Roundtable McGill University, Feb.29-Mar.2, 2012 MotionComposer - the search for coherence in movement and sound www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

2. Mercredi 1 – 6 workshop Jeudi 1 – 6 workshop Vendredi 2 – 3 presentation 3 – 5 roundtable Überblick www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

3. introductions demonstration 3 talks - 40 min ea. discussion after each 20 – 40 min. project design and execution project presentation, de-briefing Überblick - Workshop (both days) www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

4. who am i? interactive performance motion tracking mapping and synaesthesia the motioncomposer: an intuitive expression device for persons with disabilities (Bauhaus Universität) Überblick – 3 talks over 2 days. www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

5. dancer, choreographer, developer - New York, Nürnberg, Stuttgart, Weimar - Palindrome Dance Company - interactive since 1975 - developed EyeCon software with Frieder Weiss 1995-2005 - teach motion tracking - MotionComposer project director, Bauhaus University me www.palindrome.de/ppt Part 1: me

6. influences New York City in the 1980's www.palindrome.de/ppt Part 1: me

7. John Cage Merce Cunningham

9. Interactive what activity response activity /response www.palindrome.de/ppt Part 2: Interactive Performance

10. interactive performance is nothing new Cave drawings, Spain ca. 4000 BC

11. Dionysis Thr. Athens - 300 BC mechanical technology was used to empower performers

12. Louis XIV - sun king and megalomanic

13. Louis loved mechanical devices, toys, remote acting devices (automation)

14. He also loved cannons, the ultimate amplifier of gesture.

15. Interactive devices were used by New York performance artists in the 60's and 70's Robert Rauschenberg - "Pelican" 1963 Laurie Anderson - "Walking Solo for Tape Bow Violin" 1979

16. Interactive Performance today Interaction Design Gesture Sonification Motion Capture Virtual Environments Immersive Environments Haptic responders Diversified Performance (using internet) and and and www.palindrome.de/ppt Part 2: Interactive Performance

17. Part 3: Motion Tracking

18. motion tracking vs. motion capture motion capture –highly accurate, expensive (ca. 1m €) –generally not realtime –used for data collection (research), making realistic human or animal animations (films, games, etc.) motion tracking –less data, less equipment, less cost (ca. 1k-2k €) –concerned with motion qualities like dynamic, direction of motion –real time www.palindrome.de/ppt Part 3: Motion Tracking

19. motion capture –tracks location of fixed positions on body with reflective markers –12-24 cameras, each lens is ringed with infrared lights www.palindrome.de/ppt Part 3: Motion Tracking

20. Vicon is a leading company in motion capture motion capture

24. –performances with interactive animated characters: "No DNA" (http://www.nodna.com)http://www.nodna.com –dance pieces on stage – Motione (2003-5) Arizona State U. (http://ame.asu.edu/motione)http://ame.asu.edu/motione (Trisha Brown, Kaiser, et.al.) "22" (Bill T. Jones, Kaiser, et.al.) – Luc Vanier et al. at the interactive performance facility U of Wisconsin at Milwaukee (http://www.isl.uiuc.edu/Publications/final%20dance1.pdf)http://www.isl.uiuc.edu/Publications/final%20dance1.pdf real-time mo-capture www.palindrome.de/ppt Part 3: Motion Tracking

25. motion tracking sensors body-oriented environment-oriented www.palindrome.de/ppt Part 3: Motion Tracking

26. motion tracking sensors body-oriented –body-worn electrodes EEG ECG EMG Touching –joint bend-sensors –accelerometers, other body-worn devices environment-oriented –video camera-based (eg. eyecon) –infrared camera-based –floor sensors, wall sensors, etc www.palindrome.de/ppt Part 3: Motion Tracking

27. Part 4: Mapping and Synaesthesia

28. Synaesthesia Kandinsky Dostoyovsky www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

29. Dance + Music = ?

30. Where, how do they connect? How can we build up this connection? What is Mapping? movement sound www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

31. movement sound www.palindrome.de/ppt Part 4: Mapping and Synaesthesia What possibilities does the technology allow?

32. analogue Digital sensor device d-to-a conversion a-to-d conversion processing output device e.g. movement e.g. sound analogue

33. What is Mapping? movement sound makes www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

34. What is Mapping? dance music makes www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

35. What is Mapping? dancemusic www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

36. What is Mapping? dancemusic www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

37. What is Mapping? dancemusic correlation?

38. What is Mapping? turns off correlation turns on iterate toggle lower raise complex continuous variables dancemusic

39. What is Mapping? turns off iterate toggle lower raise correlation turns on dance music complex continuous variables

40. What is Mapping? correlationdancemusic

41. What is Mapping? Number of mappings (M) = number of Input parameters (I) x number of Relationships (R) x number of Output parameters (O) M = I x R x O www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

42. There are thousands of possibilities. Most are intelligible. few are palpable. www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

43. Coherent Mapping -- movement parameters

44. spatial / temporal

45. Coherent Mapping -- movement parameters / spatial

48. Coherent Mapping -- music parameters / time and timing 1 discrete events

49. 1 discrete events Accents Stops Starts Other instaneous changes www.palindrome.de/ppt Part 4: Mapping and Synaesthesia Coherent Mapping -- music parameters / time and timing

50. 2 Tags www.palindrome.de/ppt Part 4: Mapping and Synaesthesia Coherent Mapping -- music parameters / time and timing

51. 3 Modulation www.palindrome.de/ppt Part 4: Mapping and Synaesthesia Coherent Mapping -- music parameters / time and timing

52. 4 Repetition, sequences and timing patterns A, B, A, B,... (bi-modal) A, B, C, D, E, F,... (scales and melodies) A, A, A, A, A, A,A,... (rhythms and randomness) www.palindrome.de/ppt Part 4: Mapping and Synaesthesia Coherent Mapping -- music parameters / time and timing

53. 5 Multi-dimensional „qualitative“ assocations many evocative qualities arise through combinations of aural and kinesthetic experience. www.palindrome.de/ppt Part 4: Mapping and Synaesthesia Coherent Mapping – time and space

54. dynamic activity, stillness, flow (direction) sensing background substraction, „3d cams“ position, shape intelligemt systems body part and gesture recognition Motion Tracking technology - approaches

55. Motion Tracking - Software MT: EyeCon, Eyesweb, MotionComposer, MAX/Jitter (et al. Cyclops), Isadora, VNS, Icarus.. Vicon (etc.) Sound: EyeCon, Kontakt, Ableton (Max live),... DSP: Csound, Max/msp, Pure Data, Supercollider

56. Hardware Computers Cameras Framegrabbers Motion Tracking - Hardware

57. thank you. questions? Join MotionComposer... www.motioncomposer.com robert@motioncomposer.com www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt

58. thank you. questions? ideas? robert@palindrome.de www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt

59. thank you. questions? ideas? robert@palindrome.de www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt

60. thank you. questions? ideas? robert@palindrome.de www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt

61. thank you. questions? ideas? robert@palindrome.de www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt

62. thank you. questions? ideas? robert@palindrome.de www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt