By: Kenzo Abrahams Supervisor: Mehrdad Ghaziasgar Co-supervisor: James Connan Mentored by: Diego Mushfieldt

Overview Introduction Design Decisions and System Changes Implementation Tools Used Project Plan Demo

Introduction Interactive Conducting System Tracks hands using webcam Real-time alterations depending on hand gestures Change volume Change tempo

Design Decisions and System changes Use Haar classifiers to detect the face Dynamically check for skin colour Track left hand only when needed

Implementation Detect face and block it out

Implementation Convert frame into HSV colour space

Implementation set hand as region of interest

Implementation HSV image split up into separate components

Implementation Create histogram of skin colour

Implementation Track the right hand (CAMshift)

Implementation Play music

Implementation Choose ensemble or entire orchestra

Implementation Track left hand

Tools Used Ubuntu Qt Creator with Qt 4.7 OpenCV 2.4 Rtcmix 4.0.1

Demo Demo Overview Track right hand Change the tempo of music Track left hand Decrease volume of symphony Increase volume of one ensemble

Project Plan GoalDue Date Learn how to use OpenCV and its tools. Elicit the requirements and define a designers interpretation of the problem. Completed From the users requirements design a prototype for the system Completed Construct the system with all its functionality present Completed Test the system and deployment Term 4

References O'Niel, D L. (2008). Music Theater Jobs. Available: Last accessed 29th March Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M Learning OpenCV. United States of America: O’Reilly. p Gibson, J. (2009). RTcmix Tutorial. Available: Last accessed 10th Sep Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C.: The USB Virtual Maestro: an Interactive Conducting System, p , NIME2009 (2009). BORCHERS, J., HADJAKOS, A., AND M¨UHLH¨AUSER, M. MICON (2006) A music stand for interactive conducting. Proceedings of the 2006 conference on New interfaces for musical expression, p254–259. Viola, P. Jones, M. (2001). Rapid Object Detection using a Boosted Cascade of Simple Features. Conference on computer vision and pattern matching Lansky, A. (2005). An Open-Source, Digital Signal Processing and Sound Synthesis Language. Available: Last accessed 27th May Nokia. (2009). Qt. Available: Last accessed 10th Sep 2012.

Question and Answers