Electronic Conducting System Kenzo Abrahams Supervisor: Mehrdad Ghaziasgar Co-supervisor: James Connon Mentored by: Diego Mushfieldt

Overview Introduction User Interface Specification High Level Design Low level Design Demo

Introduction Interactive Conducting System Tracks hands using webcam Real-time alterations depending on hand gestures ◦ Change volume ◦ Change tempo

User Interface Specification Graphic User Interface (GUI) User interacts using webcam and mouse

User Interface Specification

High Level Design The solution can be broken up into 3 parts ◦ Input ◦ Image processing ◦ Adjust music

Low Level Design

Input ◦ The frames need to be acquired from the webcam ◦ cvQueryFrame(capture)

Low Level Design Image Processing ◦ Convert copies of frames to HSV colour space ◦ cvCvtColor(frame, img_hsv, CV_BGR2HSV)

Low Level Design Image Processing ◦ Skin segmentation ◦ Use of a predefined method ◦ Detect if pixels in the rectangle fall within a certain range

Low Level Design Image Processing ◦ Set hand as region of interest ◦ cvSetImageROI(pHueImg, pHandRect )

Low Level Design Image Processing ◦ Perform Camshift cvCamShift( pProbImg, prevHandRect, cvTermCriteria( CV_TERMCRIT_EPS | CV_TERMCRIT_ITER, 10, 1 ), &components, &HandBox )

Low Level Design Adjust Music ◦ Library RTcmix is used to produce the music ◦ Can be imbedded into C++ code ◦ load("WAVETABLE") ◦ Wave = maketable("wave", 1000, "tri") ◦ WAVETABLE(start time, duration, amp, frequency, pan, wave)

Demo Overview of demonstration ◦ Track the right hand ◦ Perform the gestures that are going to be used ◦ Track the left hand ◦ Perform simple gestures such as raising and lowering hand ◦ Move each hand separately

Project Plan GoalDue Date Learn how to use OpenCV and its tools. Elicit the requirements and define a designers interpretation of the problem. Completed From the users requirements design a prototype for the system Completed Construct the system with all its functionality present Term 3 Test the system and deployment Term 4

References O'Niel, D L. (2008). Music Theater Jobs. Available: Last accessed 29th March Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M Learning OpenCV. United States of America: O’Reilly. pp Ivanciuc, O. (2005). SVM - Support Vector Machines. Available: Last accessed 29th March Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C. (2009). The USB Virtual Maestro: an Interactive Conducting System, pp , NIME2009 Borchers, J., Hadjakos, A., M¨uhlh¨auser, M. (2006), MICON a music stand for interactive conducting. Proceedings of the 2006 conference on New interfaces for musical expression, pp254–259.

Questions and Answers