NETWORK RC CAR© James Crosetto BS (Computer Science and Computer Engineering) Jeremy Ellison BS (Computer Engineering) Seth Schwiethale BS (Computer Science)

Member Bio’s James Crosetto Seth Schwiethale Jeremy Ellison Computer Science and Computer Engineering Pizza Guru Seth Schwiethale Computer Science Music Diversity Contributor Jeremy Ellison Computer Engineering Anti-Coldplay

Presentation Outline Objectives Requirements Design and Implementation Goal Setting Innovate & Design Requirements Necessary Equipment Design and Implementation Code Review Future Development Where to go from here

Objectives Goal Setting Innovative development Project Overview Functional Objectives Learning Objectives Innovative development Radio Frequency vs. Internet Computer vs. IP Camera Goal Setting Innovative Development Design Development Research Products

Objectives Goal Setting Project Overview Expanding the range of a remote control car Control car over network Be able to have first person view of car’s location Build something cool Goal Setting Innovative Development Design Development Research Products

Objectives Goal Setting Functional Objectives Establish connection between RC car and driver’s computer Get real time visual feed Control RC car User friendly GUI Goal Setting Innovative Development Design Development Research Products

Objectives Goal Setting Learning Objectives Understand Wireless Communication Efficient methods of sending and receiving data Embedded Systems Microprocessors Assembly Language Linux / C R/C car design and functionality Servos Pulse Width Modulation Goal Setting Innovative Development Design Development Research Products

Objectives Innovative Development Radio Frequency 350-400ft Line-of-sight Object interference Internet Network Advanced Range Non-line-of-sight Goal Setting Innovative Development Design Development Research Products

Objectives Animated Design Progression Goal Setting Innovative Development Design Development Research Products

Requirements Necessary Components Car Camera Microprocessor User’s PC

Requirements The Car Goal Setting Innovative Development Design Development Research Products

Requirements The Car 3 Leads: Ground Vcc Pulse width modulation Goal Setting Innovative Development Design Development Research Products

Requirements The Car Goal Setting Innovative Development Design Development Research Products

Requirements The Car Square Pulse wave of 1.0-2.0ms repeats every ~20ms Width of pulse determines the position of the servo with 1.5ms as the normal center The amplitude of the pulse is from the reference level to the Vcc Vcc = 4.6-6.0V Goal Setting Innovative Development Design Development Research Products

Requirements IP Camera Power: 5.1 V DC, max 3.5 W Alarm output (motion, audio, external) Open API for software integration CPU, video processing and compression; Ram: 32MB Flash: 8MB Goal Setting Innovative Development Design Development Research Products

Requirements Microprocessor Receives and translates signals from the transistor output of the IP camera Sends translated signals to steering box and speed control Programmed with C and Assembly DragonFly12 Goal Setting Innovative Development Design Development Research Products

Research and Development product research – end user R&D Human Resources Purchasing Production Management Sales Research and Development product research – end user View video stream from camera Send commands to the car Goal Setting Innovative Development Design Development Research Products

Unexpected outcome of design… Swarmed with applications to be our advisor:

Project Costs: Camera Little Caesars Traveling Costs One time Purchase: $275 Little Caesars Hot-N-Ready: $600 annual cost Traveling Costs Seminar in Hawaii - $800/member Still pending department approval…

Implementation

PARTYING? How to accomplish our goals?

NO!!!

STUDYING?

YES!

Implementation 3 Major Lines of Communication PC  Camera Camera  Microprocessor Microprocessor  Car

Implementation Animated Diagram 1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  Camera Camera  Microprocessor Microprocessor  Car

Design: PC to Camera Software requirements to communicate Server to take commands (camera side) Client to give commands (user pc side) GUI to show video feed Reusability PC  Camera Camera  Microprocessor Microprocessor  Car

Server Background ARTPEC processor running Linux 2.6.18 on a armv4tl architecture Server written in C Compiled using an x86 to ARM cross-compiler PC  Camera Camera  Microprocessor Microprocessor  Car

Crashing Ubuntu V8.10 v8.04 v7.10 v7.04 v6.10 v6.06 LTS

Server Implementation TCP/IP Socket can be thought of as a file Basic Process create socket bind to port listen on port for connections accept connection handle input PC  Camera Camera  Microprocessor Microprocessor  Car

Server code

Server implementation

Server implementation

Client Background Java for cross-platform TCP/IP Socket to communicate w/ Server Incorporate into a GUI PC  Camera Camera  Microprocessor Microprocessor  Car

Video Feed Background Using MJPEG stream from Camera’s webserver Get the feed Put it in a component of the GUI PC  Camera Camera  Microprocessor Microprocessor  Car

Motion JPEG Stream Stream looks like: Content-Type: image/jpeg Content-Length: <image size> <JPEG image data> --<boundary> PC  Camera Camera  Microprocessor Microprocessor  Car

Making it usable Parse each JPEG out of the stream Update GUI when JPEG ready Continuously Paint image to screen PC  Camera Camera  Microprocessor Microprocessor  Car

Video Feed Implementation StreamParser splits up Axis’ MJPEG stream into JPEGs strips out headers and looks for start of jpeg happens to be: 0xFF 0xD8 constantly gets byte arrays (segments) when –boundary found, jpeg is ready AxisCamera extends JComponent can be added directly to GUI Listener of StreamParser Runs as a Thread from GUI calling StreamParser PC  Camera Camera  Microprocessor Microprocessor  Car

GUI Cockpit Using awt and swing is the Client, connects directly to Server starts a Thread of AxisCamera to get images adds an AxisCamera to it’s JFrame puts these 2 connections together to operate as 1 Using awt and swing has the actual controls PC  Camera Camera  Microprocessor Microprocessor  Car

Relations PC  Camera Camera  Microprocessor Microprocessor  Car

Reuse between different IP cameras Control Side Server only need to change (if necessary) output toggle command (talk about this later) Video Feed Side Since MJPEG Streams are not standardized vary from camera to camera StreamParser and “AxisCamera” should work for any camera giving a MJPEG Stream as long as you know the boundary PC  Camera Camera  Microprocessor Microprocessor  Car

Implementation Animated Diagram 1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  Camera Camera  Microprocessor Microprocessor  Car

Implementation Code Review – Part II Using a microprocessor Initial coding/design on Dragon12 development board (MC9S12DP256) has a bus speed of 24MHz has registers for a Pulse Width Modulator and an Enhanced Capture Timer Transferred to DragonFly12 (on the car) Bus speed 48MHz Fewer ports PC  Camera Camera  Microprocessor Microprocessor  Car

Design: Camera to Microprocessor Camera’s alarm output can be activated about every 10 ms Initially 10-120 ms speed and 130-240 ms steering in increments of 10 ms (alternating) Problems encountered: Camera couldn’t be activated this fast when using the camera’s program for activating the output Slowed to around 0.1 second when streaming video Very inconsistent signal

Design: Camera to Microprocessor Changed to four bit signal in the form XXXX Still takes on the order of 250 ms to send signal Solved using Open Office word processor, Google, and beagle Speed Steering

Design: Camera to Microprocessor Solution: Wrote program to test input/output files (GPIOx) to find correct output file Using ioctl system call, output can be activated in less than 1 ms Number of signals sent (0-14) determine speed and direction of car Longer signal sent to mark end of signal sequence

# pulses Steering Speed 1 -1 2 3 4 5 6 7 8 9 10 11 12 13 14

Design: Camera to Microprocessor Needed End of signal

Implementation Code Review – Part II

Implementation Amplifier PC  Camera Camera  Microprocessor Microprocessor  Car

Implementation Animated Diagram 1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  Camera Camera  Microprocessor Microprocessor  Car

Implementation

Design: Microprocessor to Car Problems putting code onto the DragonFly12 Can’t download code from Codewarrior to DragonFly12 Has to be converted to format suitable for DragonFly12 Used EmbeddedGNU connects to D-Bug12 on Dragon12 Dragon12 connected to DragonFly12 PC  Camera Camera  Microprocessor Microprocessor  Car

Design: Microprocessor to Car EmbeddedGNU

DragonFly12 Test Setup

Future Development What could we do if we had more time? Put everything on a separate car with reverse (maybe need to explain why we’re using this car with no reverse in the first place) With above ability, apply backtracking idea? (application of a stack storing commands of steering and compliments of speed) Wireless strength monitor?

Sales and Marketing You can place order for yours TODAY! $700…. Next year capstone students? Cash and Visa No Checks w/o valid drivers license

Special Thanks George Hauser Tosh Kakar Wayne Chu Frank Wornle Ph.D., University of Rochester Tosh Kakar Ph.D., Washington State University Wayne Chu Frank Wornle Cory Stevens Make-a-wish foundation Little Caesars Delivery guys who deliver straight to Morken 212a Ourselves