Computer Vision: Parallelize or Paralyze Team Purple Threads CSE Capstone 2012 April 2012

Abstract Purple ThreadsPurple Threads Project DescriptionProject Description MotivationMotivation System OverviewSystem Overview First StepsFirst Steps Target Drone PlatformTarget Drone Platform Turret SystemTurret System Final StatusFinal Status

Purple Threads Top Row: Duc Tran, Aviral Shrivastava(Sponsor), Gabriel Silva Bottom Row: Nicholas Moulin, Craig Hartmann, Anthony Russo, Nadim Hoque

Project Description This project is a real world system of robots that visually demonstrates the benefits of parallelization for computer vision applicationsThis project is a real world system of robots that visually demonstrates the benefits of parallelization for computer vision applications

Motivation Computer vision systems have a wide variety of applications, but are very processor intensiveComputer vision systems have a wide variety of applications, but are very processor intensive Parallelization allows the implementation of more advanced computer vision techniques by removing the bottleneck on the processorParallelization allows the implementation of more advanced computer vision techniques by removing the bottleneck on the processor

System Overview Project consists of two different robotsProject consists of two different robots –Target Drone Platform Detect & track projectiles Avoid projectiles –Turret System Detect & track the target Hit target with foam darts

First Steps Requirement ElicitationRequirement Elicitation –High Level –Low Level Hardware/Software SpecificationHardware/Software Specification Risk ManagementRisk Management Project TimelineProject Timeline BudgetBudget Configuring Development EnvironmentConfiguring Development Environment

Target Drone Platform HardwareHardware –Traxxas Slash VXL* –ArduPilot Mega w/ IMU Shield –ION Intel Atom Motherboard* –Microsoft Xbox 360 Kinect Sensor* *Other names and brands may be claimed as the property of others

Target Drone Platform SoftwareSoftware –EMGU CV Open CV Wrapper (C#) –ArduRover Code for ArduPilot –Microsoft Robotics Developer Studio 4* –Kinect for Windows SDK v1.0* *Other names and brands may be claimed as the property of others

Target Drone Platform ImplementationImplementation –Assemble Target Drone Hardware Components Power System –Setup Wireless Access to Drone –Configure Drone Software Stack –Implemented Platform Services through RDS

Target Drone Platform ImplementationImplementation –Obtain RGB & Depth Image Data From Kinect* –Projectile Detection Color Threshold –Projectile Tracking –Collision Avoidance Algorithm *Other names and brands may be claimed as the property of others

Target Drone Platform SetbacksSetbacks –Foam dart too small to track accurately –Hardware too heavy for original shocks –Depth frame coordinates are offset from RGB frame coordinates with no translation function –Dr. Shrivastava crashed the car into a tree!

Turret System HardwareHardware –USB Missile Turret –Microsoft Xbox360 Kinect Sensor* –Arduino Uno –Servos SoftwareSoftware –OpenCV –Libfreenect –Ubuntu 10.04* *Other names and brands may be claimed as the property of others

Turret System ImplementationImplementation –Obtain RGB/Depth Images from Kinect* Sensor –Object Detection Color Threshold Cascade Classification –C USB Turret Driver –Program Arduino to Control Servos –C NetDuino Driver *Other names and brands may be claimed as the property of others

Turret System ImplementationImplementation –Equations Created to Track Target Depth Disparity to Depth(ft) –d_feet = (disparity-824.8)/15.2 –d_feet = (0.1236*tan((disparity/2842.5) ))* Pixels per Foot depending on Depth –Pixels/Foot = *log(d_feet)

Turret System SetbacksSetbacks –Original turret configuration not precise enough –Unable to implement libfreenect driver on PS3* –Turret caught on fire –Turret mounting unstable *Other names and brands may be claimed as the property of others

Turret System Turret Tracking and Hitting a Moving CarTurret Tracking and Hitting a Moving Car

Turret System Effect of Parallelization on Turret SystemEffect of Parallelization on Turret System

Final Status

Questions ?

Backup Slides

Turret System ImplementationImplementation –Equations To Track Target Horizontal Rotation Vertical Rotation