DJ Spatial Tracking and Gesture Recognition for Audio Effects and Mixing Andrew Hamblin, Evan Leong, and Theo Wiersema Dr. Jose Sanchez Bradley University.

Slides:



Advertisements
Similar presentations
By: Russ Butler ECE4220 Spring 2012 Dr. DeSouza May 2, 2012.
Advertisements

Lecture 51 The Telephone System. Lecture 52 The Telephone System The modern telephone system draws from these Electrical Engineering subdisciplines: Signal.
SWE 423: Multimedia Systems Chapter 3: Audio Technology (2)
Design & Prototyping of Hybrid Electric Vehicle Electronic Control Unit Dinçer Mehmet BAHAR Energy Institute 2008,Gebze.
Autonomous Helicopter: James Lyden Harris Okazaki EE 496 A project to create a system that would allow a remote- controlled helicopter to fly without user.
Home Area Networking for the Smart Grid Erik Cates David Green Daniel Han Justin Michela ECE 4007 Koblasz/Maxwell Fall 2010 In Conjunction With: Unmesh.
MotoHawk Training Model-Based Design of Embedded Systems.
Jason CrayJoseph Mundackal Michael WarscoRyan Sherlock.
Design Review Team Digital Burnout Senior Design Fall 2011 Analog Gauge w/ Digital Display.
1 Electrical and Computer Engineering Cam Proctor Ramsey Khudairi Tom Gilbert Chad Young Project Orion TekBand.
Photolithography Machine Control System Ben Conrad and Mark Edwards Projects in Computer Engineering II December 9, 2003.
Vex 1.0 © 2006 Carnegie Mellon Robotics Academy Inc. Test Bed Assembly.
Design of a Control Workstation for Controller Algorithm Testing Aaron Mahaffey Dave Tastsides Dr. Dempsey.
Chapter 2: Fundamentals of Data and Signals. 2 Objectives After reading this chapter, you should be able to: Distinguish between data and signals, and.
Input/Output and Communication
Motor Control of an Oscillating Pendulum Nick Myers and Chirag Patel March 9, 2004 Advised by: Dr. James Irwin and Mr. Jose Sanchez Bradley University.
Wireless Digital Audio Transmission Senior Design Project Summer 2002 E. Donnelly, K. Khan, S. Owen D. Choy Dr. H. Lam.
ECE 353 Introduction to Microprocessor Systems Michael G. Morrow, P.E. Week 13.
Dynamic Traffic Light Timing Tony Faillaci John Gilroy Ben Hughes Justin Porter Zach Zientek.
Computerized Train Control System by: Shawn Lord Christian Thompson.
Critical Design Review 27 February 2007 Black Box Car System (BBCS) ctrl + z: Benjamin Baker, Lisa Furnish, Chris Klepac, Benjamin Mauser, Zachary Miers.
Chapter 2 Basic Communication Theory Basic Communications Theory w Understand the basic transmission theory, and figure out the maximum data rate. w.
Oppenheimer Technologies Rick King Jonathan Creekmore.
Group Members: Brad Cox Kevin Burkett Tera Cline Arthur Perkins CS10 Battery Management System.
LSU 10/22/2004Serial I/O1 Programming Unit, Lecture 5.
Jonathan Yitzchaki David Michaeli Instructor: Ina Rivkin Spring
Digital Electronics. Digital circuits work on the basis of a transistor being used as a switch. Consider a light switch, a transistor can be considered.
Virtual Imaging Peripheral for Enhanced Reality Aaron Garrett, Ryan Hannah, Justin Huffaker, Brendon McCool.
1 SERIAL PORT INTERFACE FOR MICROCONTROLLER EMBEDDED INTO INTEGRATED POWER METER Mr. Borisav Jovanović, Prof.dr Predrag Petković, Prof.dr. Milunka Damnjanović,
DLS Digital Controller Tony Dobbing Head of Power Supplies Group.
Gesture Recognition Interface Device
ECE 353 Introduction to Microprocessor Systems Michael Schulte Week 13.
ECE 477 Design Review Team 4  Fall 2010 (L to R) Andy Sydelko, Chris Cadawallader, Mike Wiliams, Craig Pilcher.
Stanley – RC Car.
Computer Communication & Networks Lecture # 05 Physical Layer: Signals & Digital Transmission Nadeem Majeed Choudhary
Autonomous Helicopter James LydenEE 496Harris Okazaki.
MIDI Musical Instrument Digital Interface. MIDI A data communications protocol that describes a means for music systems and related equipment to exchange.
By: Eric Backman Advisor: Dr. Malinowski.  Introduction  Goals  Project Overview and Changes  Work Completed  Updated Schedule.
ATtiny23131 A SEMINAR ON AVR MICROCONTROLLER ATtiny2313.
Microprocessor based Design for Biomedical Applications MBE 3 – MDBA XI : Project Outlooks.
Logic Analyzer ECE-4220 Real-Time Embedded Systems Final Project Dallas Fletchall.
1 Electrical and Computer Engineering Guitar Virtuos Justin D’Alessando (EE) Jacob Dionne (CSE) Adam Montalbano (CSE) Jeffrey Newton (EE) Team Kelly Comprehensive.
DJ Spatial Tracking and Gesture Recognition for Audio Effects and Mixing Andrew Hamblin, Evan Leong, and Theo Wiersema Dr. José Sanchez Bradley University.
ECE 4007 L01 DK6 1 FAST: Fully Autonomous Sentry Turret Patrick Croom, Kevin Neas, Anthony Ogidi, Joleon Pettway ECE 4007 Dr. David Keezer.
ECE 477 Design Review Team 5  Spring 2010 Fred Grandlienard Andrew Gregor Kevin Mohr Ryan DeFord.
© 2008, Renesas Technology America, Inc., All Rights Reserved 1 Course Introduction Purpose  This course provides an introduction to the peripheral functions.
Lab Environment and Miniproject Assignment Spring 2009 ECE554 Digital Engineering Laboratory.
Final Year Project Eoin Culhane. MIDI Guitar Guitar with 6 outputs 1 output for each string Each individual string output will be converted to MIDI.
Voice Controlled Home Automation System Group 13 Zhe Gong Hongchuan Li.
MIDI Controller By: Ronal Inga Jeyner Cespedes. Musical Instrument Digital Interface is a computer language that allows communication with other devices.
Smart Streetlight Proof of Concept Group 3 03/26/16 Tucker Russ Thor Cutler Brandon Berry Anthony Giordano.
Refrigerator Diagnostics Group #14 Jacob Belica Bradley Snyder Darwin Walters.
Visible Light Photon Counter Integrator Group 48: Katie Nguyen, Austin Jin ECE445 Spring 2016 May 1, 2016.
Lecture Notes / PPT UNIT III
Timothy Kritzler and Joseph Mintun Sponsor: Martin Engineering, Illinois Advisors: Dr. Malinowski and Dr. Ahn Bradley University Electrical and Computer.
Launchpad with MIDI TAHA SHASHTARI MAHMOUD AL-SHAKA’A.
Drum Tutor Lite Yuanheng Yan Xun Yu Zhen Qin ECE 445 Senior Project Group 43 December 3,2015.
Emotional Intelligence Vivian Tseng, Matt Palmer, Jonathan Fouk Group #41.
Jeremy Sandoval University of Washington May 14, 2013
Application Case Study Christmas Lights Controller
SUBMITTED BY EDGEFX TEAM
Instrumented Walker Skyler Bullington Tommy Frankenberger Larson Stacy
Nicholas Reich Ethan Plummer R. Alex Kibler
Chapter A - The Raspberry Pi Computer
Group 4 Alix Krahn Denis Lachance Adam Thomsen
Group 4 Alix Krahn Denis Lachance Adam Thomsen
ENGR 1 Presentation Thomas Matthews.
Requirements Definition
ECE Computer Engineering Design Project
Presentation transcript:

DJ Spatial Tracking and Gesture Recognition for Audio Effects and Mixing Andrew Hamblin, Evan Leong, and Theo Wiersema Dr. Jose Sanchez Bradley University ECE March 1, 2016 Project Progress #teamhozai

Objective Glove for disc jockeys (DJ) Gestural control of music effects 2 Fig. 1. System diagram

Goals Glove with tri-color light-emitting diodes (LED) Acquire and recognize gesture Seamless communication Real-time dynamic effects 3

Motivation Passion for music Unique product idea 4

Significance Disconnect for DJ Complexity of DJ equipment Lack of natural connection 5 Fig. 2. Numark MIXTRACK DJ board [1]

System Block Diagram Fig. 3. System block diagram 6

Top-Level State Diagram Fig. 4. Top-level state diagram 7

Glove State Diagram Fig. 5. Glove state diagram 8

Andrew Hamblin: Progress to Date 9 Fig. 6. Andrew Gantt chart

MIDI Communication Acquire and recognize Musical Instrument Digital Interface (MIDI) signals from Raspberry Pi 10 Fig. 7. System visual diagram

MIDI Communication: MIDI Signals MIDI signals [2] 3 bytes Describes command Status byte Op-code Channel Data bytes Command descriptor 11 Fig. 8. MIDI signal breakdown

MIDI Communication: Protocol Universal asynchronous receiver transmitter (UART) communication - Raspberry Pi to computer 31,250 bits/sec baud rate ‘0’ start bit, ‘1’ stop bit Universal Serial Bus (USB) header packet byte cable number code index number 12

MIDI Communication: Hairless MIDI 13 Fig. 9. Hairless MIDI serial bridge [3]

MIDI Communication: USB MIDI Converter Bidirectional USB- MIDI conversion MIDI in & MIDI out capability General Purpose Input/Output (GPIO) to MIDI circuit Requires 31,250 bits/sec 14 Fig. 10. Neewer USB MIDI converter [4]

MIDI Communication: Results UART communication - Raspberry Pi to computer SerialTools - serial port monitor [5] Information transmitted and received Hairless MIDI - MIDI to Serial bridge Errors Faulty MIDI command recognition USB MIDI Converter Fails to recognize MIDI input 31,250 bits/sec not Raspberry Pi standard 15

MIDI Communication: Next Step Continue testing Hairless MIDI solution Achieve MIDI command recognition Route MIDI signals to Mixxx DJ software. 16 Fig. 11. Numark MIXTRACK DJ board

Andrew Hamblin: Upcoming Work 17 Fig. 12. Andrew Gantt chart

Theo Wiersema: Progress to Date 18 Fig. 13. Theo Gantt chart

Gesture Recognition: Goals Establish communication between Pixy and Raspberry Pi Filter noise data from Pixy Train hidden Markov model (HMM) 19 Fig. 14. System visual diagram

Gesture Recognition: Pixy Communication Serial peripheral interface (SPI) communication [6] Libraries [7][8] WiringPi pixy_rpi 20 Fig. 15. System visual diagram

Gesture Recognition: Pixy Data Extraction Object data saved every 50 ms Object data width height x position y position 21

Pixy Object Data 22 Fig. 16. Example of frame from Pixy

Gesture Recognition: Weighting Object Data Object < 5% of average object area Removed completely Object > 200% of average object area Weight doubled 23

Gesture Recognition: Training the HMM Required for initialization Adjusts the A, B, and matrices Steps Perform gestures Save object data Perform forward and backward algorithm [9] 24

Gesture Recognition: Example Gestures 25 Fig. 17. System visual diagram

Gesture Recognition: Debugging Heavily weighted first state: all gestures produce same state sequence Recursive multiplication of probabilities < 1 Logarithmic scaling 26

Theo Wiersema: Upcoming Work 27 Fig. 18. Theo Gantt chart

Evan Leong: Progress to Date 28 Fig. 19. Evan Gantt chart

Glove System: Goals Complete circuitry Mount circuitry to glove Test glove system 29 Fig. 20. System visual diagram

Glove System: Considerations Stranded wire vs solid gauge wire Velcro for securing components Switch, button, and battery pack placement 30

Glove System: Soldered Circuit Board 31 Fig. 21. System visual diagram

Glove System: Circuit Box 32 Fig. 22. Circuit box base design

Glove System: Circuit Box 33 Fig. 23. Circuit box cover design

Glove System: LEDs 34 Fig. 24. Switching between color modes

Glove System: Integration 35 Fig. 25. Glove on the hand

Evan Leong: Upcoming Work 36 Fig. 25. Evan Gantt chart

Summary MIDI communication Gesture recognition Glove development 37 Fig. 26. System diagram

DJ Spatial Tracking and Gesture Recognition for Audio Effects and Mixing Andrew Hamblin, Evan Leong, and Theo Wiersema Dr. Jose Sanchez Bradley University ECE March 1, 2016 Project Progress

References [1] [2] [3] [4] Shipping.jpg_350x350.jpg [5] [6] [7] [8] [9] 39

Fig. 27. Gantt Chart 40

MIDI Mapping: JavaScript Defines audio effect functions Reads data from XML file Relays command to Mixxx DJ software 41

Op-Amp and Loading Effect Loading Effect R LOAD < R IN Large voltage drop across R IN Small voltage drop across R LOAD Less load power Solution Unity gain buffer op-amp 42 Fig. 28. Loading effect example Fig. 29. Unity gain follower

Transistor Circuit: Design Considerations Set base current value Calculate collector current Calculate resistor values 43 Fig. 30. Blue lead transistor circuit

Transistor Circuitry: Calculations Base I B = 5.00 mA V Batt = 6.0 V R B = 1060 Ω = 5 Collector I C = 25.0 mA V PWM = 3.3 V R B = 29.6 Ω 44 Fig. 31. Blue LED example with values

Glove Design Conceptual design Placement of LEDs Intuitive button location Sheaths to direct light Battery pack placement 45

Angle Quantization Divided among “bins” [10] Angles are rounded to the nearest bin 46 Fig. 32. Quantized angle bins

Training with Test Gestures Fig. 33. Test gestures 47

Angles of Test Gesture 48 Fig. 34. Gesture angles

Quantized Gesture Angles Fig. 35. Quantized gesture angles 49

Forward Algorithm Equations Initialize (1) Run through gesture forwards (2) 50

Backward Algorithm Equations Initialize (3) Run through gesture backwards (4) 51

Initialize delta (5) Recursive delta (6) Viterbi Algorithm Equations 52

Introduction to HMM 53 Fig. 36. Gumball example Dwight

Introduction to HMM (Cont.) 54 Fig. 37. Gumball machine emission probabilities

Introduction to HMM Cont. 55 Fig. 38. Gumball state diagram

Introduction to HMM (Cont.) 56 Fig. 39. Gumball example

Introduction to HMM Cont. 57 Fig. 40. Possible outcomes [9][10][11]

How do Gumballs Relate to HMM? Gumballs → observations Gumballs on conveyor belt → observation sequence Gumball machines → states Succession of machines dropping gumballs → sequence of states Food → result of observation 58

How HMM Relates to Gesture Recognition Observations → angles Sequence of observations → trajectory of glove States → hidden, abstract representation of angles Sequence of states → abstract representation of gesture Result of observation → audio effect applied 59

Old References [1] [2] [3] Shipping/ htmlhttp:// Shipping/ html [4] [1] [2] [3] midi-learn.jpghttp://static1.squarespace.com/static/ e4b00907bc18522b/t/51d1ba1fe4b025acba75a0c3/ /mixxx- midi-learn.jpg [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] 60