“To investigate the benefits of AR in relation to music and its appropriateness in the music realm.” By Justin Samartzis

 What is currently available?  What can AR offer to music?  The AR Piano Tutor  Issues  Future Development  AR in the music realm

 Sheet Music and Books  Online Resources ◦ Websites  ◦ Social Networks & Programs  YouTube and Garage Band  Lessons recorded by other musicians

 Synthesia

 Increase Engagement  Increase Visual Accuracy

 Microsoft Visual Studio 2010  Microsoft XNA Game Studio 4.0 ◦ A programming environment that allows you to use Visual Studio 2010 to create games for windows- based computers  GoblinXNA v4.1 ◦ A platform for research on 3D user interfaces ◦ Developed by Columbia University ◦ Written in C# ◦ Comes with completed tutorials

 Investigated the use of Open Source Computer Vision – OpenCV ◦ Scan video ◦ Detect patterns and curves  GoblinXNA provides an OpenCV wrapper ◦ Only provides edge detection ◦ Limited functions

 Uses the ALVAR library ◦ Developed by VTT Technical Research Centre of Finland ◦ Allows the use of AR tags

 Set-up of GoblinXNA proved to be difficult ◦ Missing DLL libraries ◦ Well documented however unique issues occur  AR Tags need to be displayed on a solid background, preferably white  Graphics shudder when holding web-camera  Don’t know when the user is playing the note  You can only use one finger to play

 Integration with OpenCV ◦ Edge Detection  Integration with NAudio ◦ Open Source.NET audio and MIDI Library ◦ Detect when user is playing ◦ Introduce more instruments  Improvement in graphics ◦ Only basic spheres used ◦ Possibility of adding effects

 Benefits ◦ Increased user engagement ◦ Accurate interaction with an instrument  Appropriateness ◦ As a tutor for beginners  Perfectly acceptable ◦ For professional use  questionable

 Any Questions?