ELECTRONIC CONDUCTING SYSTEM Kenzo Abrahams Supervisor: Mehrdad Ghaziasgar Co-supervisor: James Connan Assisted by: Diego Mushfieldt

Overview  Introduction  High Level Design  Implementation  Testing and Results  Demo  References  Questions and Answers

Introduction  Interactive Conducting System  Tracks hands using webcam  Real-time alterations depending on hand gestures  Change volume  Change tempo

High Level Design  The solution can be broken up into 3 parts  Input  Image processing  Adjust music

Implementation Extract F rames Detect face HSV Region of interest Extract Hue component

Implementation Draw histogram Track right hand (control tempo) Track both hands (change volume)

Testing and Results  Three tests were performed  Response Time Test  Performance Test  Robustness Test

Response Time Test  Each gesture is played for 20 seconds iteration Number of noticeable delays Gesture 24Gesture 34Gesture Average212

Performance Testing  Each section of the algorithm was timed

Robustness Test  15 users of different ethnicity  Check number of times the system loses tracking User number Number of losses BlackColouredWhite Total

Demo  Demo Overview  Track right hand  Change the tempo of music  Track left hand  Decrease volume of symphony  Increase volume of one ensemble  Show that all the gestures work

References  O'Niel, D L. (2008). Music Theater Jobs. Available: theater-jobs.html. Last accessed 29th March  Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M Learning OpenCV. United States of America: O’Reilly. p  Gibson, J. (2009). RTcmix Tutorial. Available: Last accessed 10th Sep  Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C.: The USB Virtual Maestro: an Interactive Conducting System, p , NIME2009 (2009).  BORCHERS, J., HADJAKOS, A., AND M¨UHLH¨AUSER, M. MICON (2006) A music stand for interactive conducting. Proceedings of the 2006 conference on New interfaces for musical expression, p254– 259.  Viola, P. Jones, M. (2001). Rapid Object Detection using a Boosted Cascade of Simple Features. Conference on computer vision and pattern matching  Lansky, A. (2005). An Open-Source, Digital Signal Processing and Sound Synthesis Language. Available: Last accessed 27th May  Nokia. (2009). Qt. Available: Last accessed 10th Sep 2012.

Questions and Answers