CSE 424 Final Presentation Team Members: Edward Andert Shang Wang Michael Vetrano Thomas Barry Roger Dolan Eric Barber Sponsor: Aviral Shrivastava

Overview Scope Schedule Tower Technology and Design Car Technology and Design Video Tower Results and Lessons Car Results and Lessons Conclusion

Two completely autonomous systems working in parallel Turret aims at target and decides when to fire Car follows a person and implements obstacle avoidance Multiple uses and implementations Military applications Autonomous systems and vehicles Image recognition Children's toys

Car and turret remain autonomously controlled Car detects objects in its path Car responds to overrides from controller and voice Turret fires at circular objects with easily defined center point Turret is more responsive and accurate with pan and tilt system Easily demonstrated with crowd appeal Very interactive Marketable

Schedule vs. Execution - Semester 2 JanFebMarApr Week Stage 2 Optimizations Interface with sensors Implement software Code Documentation Demonstrations Developers Guide Website Building Red - Car Team; Green - Tower Team; Blue - Both Teams; Yellow - Difference (Additional Time Spent); Aqua – Difference (Late Start);

Kinect Camera information used to calculate current position of target Depth information used to calculate time to target Arduino-USB interface to servos Changes position of turret Dream Cheeky Turret Fire command sent through USB OpenCV and OpenKinect libraries used for image processing Filter image color Noise Reduction Detect edges Find Contour

Optimize existing tracking algorithm Enhance robustness Add motion prediction Improve performance Create a more stable tower Kinect needs less interference from moving servos Revamp old turret design for better stability

Re-use hardware from previous team Traxxas RC car Atom Board Microsoft Kinect Arduino Mega (Custom DIYdrones model) Use Microsoft software for ease of interfacing with hardware Windows 7 OS Visual Studio for development (C#) Microsoft Kinect with Kinect SDK

Start from scratch on software Progressively add autonomous functionality to the car Create a fail safe to override autonomous controls Make car follow humans Build in human recognition using Kinect SDK Demonstrate autonomous tracking with human interaction Use voice commands to change car behavior from user Eliminate need for any controller/remote

Recognizing and tracking objects Able to recognize an object in a certain color Using shape analysis mechanism to filter the background noise and stabilize the turret Able to shoot the object accurately and quickly Enabling auto-shoot and manual–shooting Physics calculation Predict the car movement

Linux development Computer vision technique and OpenCV programming Version control Module testing Iteration in software development

Implemented autonomous car Recognizes the skeletal structure of a person Locks onto that person while in view Follows the person Feedback loop Depth plane recognition for object detection Voice recognition of commands (stop, start)

Autonomous solutions can be unpredictable at times Fail safe was necessary Feedback loop important for monitoring car functions Initialization detrimental to startup Inconsistent servos require calibration and algorithmic correction

We have created an autonomous car and turret pair The turret tracks and predicts object motion and shoots at the target The car recognizes and follows a human with the ability to take voice commands and detect objects The team overcame many obstacles and learned valuable lessons in cyber-physical systems and project management