1. Ryan C. Bergsmith Ross Kelly Kevin Warne Sponsor: Steve Peralta Motion Music Controller

2. Project Description Utilize Microsoft’s XBOX Kinect SDK to develop a MIDI controller with the Kinect sensor XBOX Kinect – motion/voice sensing device originally developed for the XBOX 360 Kinect SDK - provides Kinect capabilities to developers to build applications with C++, C#, or Visual Basic by using Microsoft Visual Studio 2010 MIDI controller – (Musical Instrument Digital Interface) Enables electronic musical instruments, computers and other electronic equipment to communicate and synchronize with each other

3. Project Goal Use the existing SDK in conjunction with our own application to perform the essential functions of the MMC Positional Tracking Voice Recognition GUI Interaction Utilize a third party Digital Artist Workshop (DAW) to provide recording/looping functionality Provide the open source community with a framework for using the XBOX Kinect as a MIDI controller Develop a new and exciting method of creating music

4. Good Project Because… Opportunity to learn/understand the software development life cycle Develop software around hardware constraints Makes use of newly released SDK for XBOX Kinect Develop a Windows application using C#/.NET Familiarization with the industry standard MIDI protocol Will serve as a useful reference for open source community Unique musical creation environment

5. Sponsor Steve Peralta Provide programming consultation Track/report team progress Host regular meetings Aid in the resolution of encountered problems

6. Roles and Responsibilities Ross Kelly – GUI development and design, MIDI interfacing, interfacing with Skeletal Recognition System Ryan Bergsmith – Skeletal Tracking System development and design, MIDI interfacing Kevin Warne – Voice Recognition System development and MIDI interfacing

7. Voice Recognition System To Dos Develop a library of necessary voice commands to utilize our application Design command heirarchy Test/validate correctness Deliverable Voice command interface which provides menu navigation, instrument page selection, and DAW interaction

8. Positional Tracking System To Dos Develop positional indication software Add velocity recognition and instrument specific behaviors Design interaction scheme with other modules Test/validate correctness Deliverable Positional tracking/feedback system which generates MIDI signals triggered by body movements

9. Graphical User Interface To Dos Design a reference instrument page Interface with other modules Test/validate correctness Deliverable GUI which displays instrument pages, provides visual feedback of interactions, and supplies a text based menu of voice commands and their functionality

10. MIDI Interaction To Dos Configure the Kinect as a MIDI device Create a library of MIDI signals Test/validate correctness Deliverable Software module which provides correct interaction between the XBOX Kinect and the DAW

11. Summary of Deliverables Voice command interface which provides menu navigation, instrument page selection, and DAW interaction Positional tracking/feedback system which generates MIDI signals triggered by body movements GUI which displays instrument pages, provides visual feedback of interactions, and supplies a text based menu of voice commands and their functionality Software module which provides correct interaction between the XBOX Kinect and the DAW Software requirements specification, Gantt chart, non technical user manual, final report, and a Windows application

12. Challenges and Concerns Latency issues in the SDK Industry standard for a MIDI device is 10ms! Third party DAW interfacing How to implement advertised functionality Implementing a C# based MIDI protocol Staying up to date with SDK releases

13. Milestones Implement proof of concept software Generate a desired sound using the MIDI protocol Develop a fully functional instrument page Record/loop/playback a musical composition Develop beta software for testing and public feedback Present final release

14. Plan of Action Through 2011 Finish all proof of concept testing Fully understand MIDI protocol/interfacing Develop Alpha software release