1. ELECTRONIC CONDUCTING SYSTEM Kenzo Abrahams Supervisor: Mehrdad Ghaziasgar Co-supervisor: James Connan Assisted by: Diego Mushfieldt

2. Overview  Introduction  High Level Design  Implementation  Testing and Results  Demo  References  Questions and Answers

3. Introduction  Interactive Conducting System  Tracks hands using webcam  Real-time alterations depending on hand gestures  Change volume  Change tempo

4. High Level Design  The solution can be broken up into 3 parts  Input  Image processing  Adjust music

5. Implementation Extract F rames Detect face HSV Region of interest Extract Hue component

6. Implementation Draw histogram Track right hand (control tempo) Track both hands (change volume)

7. Testing and Results  Three tests were performed  Response Time Test  Performance Test  Robustness Test

8. Response Time Test  Each gesture is played for 20 seconds iteration Number of noticeable delays Gesture 24Gesture 34Gesture 44 1222 2213 3112 Average212

9. Performance Testing  Each section of the algorithm was timed

10. Robustness Test  15 users of different ethnicity  Check number of times the system loses tracking User number Number of losses BlackColouredWhite Total 1001 2111 3000 4000 5100 2125

11. Demo  Demo Overview  Track right hand  Change the tempo of music  Track left hand  Decrease volume of symphony  Increase volume of one ensemble  Show that all the gestures work

12. References  O'Niel, D L. (2008). Music Theater Jobs. Available: http://www.ehow.com/list_6547349_music- theater-jobs.html. Last accessed 29th March 2012.  Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M Learning OpenCV. United States of America: O’Reilly. p31-87.  Gibson, J. (2009). RTcmix Tutorial. Available: http://iub.edu/~emusic/509/rtcmixtut.htm. Last accessed 10th Sep 2012.  Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C.: The USB Virtual Maestro: an Interactive Conducting System, p.250-255, NIME2009 (2009).  BORCHERS, J., HADJAKOS, A., AND M¨UHLH¨AUSER, M. MICON (2006) A music stand for interactive conducting. Proceedings of the 2006 conference on New interfaces for musical expression, p254– 259.  Viola, P. Jones, M. (2001). Rapid Object Detection using a Boosted Cascade of Simple Features. Conference on computer vision and pattern matching 2001.  Lansky, A. (2005). An Open-Source, Digital Signal Processing and Sound Synthesis Language. Available: http://rtcmix.org/. Last accessed 27th May 2012.  Nokia. (2009). Qt. Available: http://qt.nokia.com/products. Last accessed 10th Sep 2012.

13. Questions and Answers