Presented By Motion Capture Group: Azadeh Jamalian Ata Naemi Sa'ed Abu-Alhaija Sunghoon Ivan Lee SensIT Technology

Overview  Introduction  System Overview  Hardware unit  Image Processing  Modeling and Animation  Results  Financing  Applications and Marketing  Future Missions  Question Period  Demonstration Welcome to our Presentation!

Introduction Study dance movementsMedicineRobotic Sports Our Mission: Study dance movements

System Overview Image Processing Unit Animation Unit Real time User friendly Easy to customize for different applications Compatible with other systems Low cost

System Overview Image Processing Unit Animation Unit

Hardware Block Diagram Microcontroller PIC16LF84A 4 Sel 4-16 Decoder Turn on One LED Turn on LED’s for 10 frames/sec Timing issues Components from local vendors and engineering department

Hardware Components  Microcontroller (PIC16LF84A) - Assembly Language - Watch Dog Timer (WDT) - Low power (1mA), Sleep mode (3.5μA)  Regulator (LM317) - Regulate 3.3V power supply off a 9V battery - Power off operation as well as staple WDT  Infrared LED’s (KIE-7305) - 1.6V, draw a current of 2mA - Each LED is on for 50 msec

Test Models  Angle Sensitivity  Problem solving as project progressed  Use high efficient IR LED’s

System Overview Image Processing Unit Animation Unit

Software Components  Low Level Design - Capture Control Unit (CCU) - Image Processing Unit (IPU)

Capture Control Unit (CCU)  Three main tasks 1. Adjusting Phase Difference 2. Capturing Image 3. Convert Raw Image into RGB

Image Processing Unit (IPU)  Calculate position of IR light source in the frame  Placing film in front of the web camera - Film absorbs all visible light - Only IR lights penetrate the film

IPU- Challenges  Synchronization with Hardware  Distorted Images and Virtual Images

Synchronization Frequency of IR sensors and frame rate can be identical IR Freq. Frame Rate t t TbTb TbTb

Frequency of IR sensors and frame rate can be identical IR Freq. Frame Rate t t TbTb TbTb T phase Synchronization

Unexpected Images  Distorted Images & Virtual Images  Solutions: - Intensity (changing threshold value) - Width & Height

System Overview Image Processing Unit Animation Unit

Data Flow Diagram Captured Frames Calculated Coordinates of the LED’s X and Y values Animated Model

Animation Unit (AU)  Software: -C++ (OpenGL)  Main Tasks: -Reads the coordinates from an external file -Assign the coordinates of each joint and record them in a file -Model 2D motion in real time

AU- Challenges  Synchronization with IPU - Timing - Start Point - Missing Points  Simulation and Modeling - Coordinate Systems - Matrix Transformation and Rotation

Simulation Algorithm- Cont. It’s not that easy to think smart!

AU- User Interface

Results  Frame Rate: 10 frames/sec  Max Motion Frequency: 2Hz  Max Distance from camera: 1m  Precision: 100% (Clear angle of sight)  Accuracy: ± 10%

Finance EquipmentEstimated Cost Sensors/markers + interface-electronics$400 Sources + interface-electronics$400 Computer interface-electronics$100 User Interface$80 Cables$20 Total Cost$ Available Funding : $400 from EUSS Actual Cost: Estimated Cost: EquipmentEstimated Cost Sensors/markers + interface-electronics$60 Sources + interface-electronics$80 Computer interface-electronics0 User Interface$60 Cables$10 Total Cost$200

Supplementary Application SensIT Mouse: Tracks the motion of finger

Marketing  Different applications: - Art - Robotic - Medicine - Sports - Game Industry  Competitors: - Electronic Arts Canada - C4 Digital Entertainment - PhoeniX Technologies Incorporated

Future Improvements Known Limitations: - Captures the motion only in 2D - Line of sight - Angle of sight - Limited range of monitoring - Interference from other IR sources - Frequency limitation of object motion Improvements: - Capture the motion in 3D - Multiple marking - Enhance the speed

What We Learned “If you’re not sure what makes the problem Guess what causes it, Make it worse, Try to solve it, If it didn’t work, try another source!”

Conclusion  Our system: - Research based project - Proof of concept to design a low cost motion capture system - Initial step to designing a 3D motion capture system  In future: - Unifying music and dance - Future research opportunities - Future Coop positions

Acknowledgements Special Thanks to: Dr. Lucky One Mr. Steve Whitmore Dr. Pavaneh Dr. Patrick Leung Dr. Ivan Bajic Dr. Payandeh Mr. Vinay Kotamraju Mr. Amir Masoud Niroumand Mr. Brad Oldham Thanks a ton!

Questions ?

Points to make here Camera Recognizes the IR as a white light True white color pixel in 24-bit RGB gives [R,G,B] = [255,255,255] Only Use one of R,G,B matrices IPU- Cont.

IPU Algorithm

Simulation Algorithm - One LED blinks twice indicating the start point - An array to hold the coordinates  Useful for applications that track the motion of a specific object  However, not smart enough to detect the position of various joints in real time