1. ECE477 Project Minotaur Software Narrative Jon Roose

2. Project Overview Home Security Robot Interaction through a Webserver Robot driven manually through house and spots intruders Relays video to web interface using kinnect

3. Hardware Hierarchy

4. Software Hierarchy

5. Benefits of Model Reliance upon open source projects Ease of access, available to HTML5 browsers Emphasis on x86 architecture Modularity Scalability

6. Software Hierarchy - Web Interface

7. Apache Web Server – Serves interface to user’s web browser HTML5 and JavaScript WebSockets – Opens efficient communication with C&C Server – Real time commands, removes HTTP reliance RDP Video Stream – Implemented and configured through FFServer – Connects via an embedded video object

8. Software Hierarchy - C&C Server

9. Software Hierarchy - C&C Gateway C&C Server (Written in Python) – Mediator for command and control signals – Communicates on internal LAN with robot – Simple websocket interface with User’s browser – Leverages TCP/IP LAN interface to communicate with robot via WiFi FFServer (Open source application) – Accepts video from Atom board and serves to user, manages RDP video stream

10. Software Hierarchy - Atom Board

11. Software Hierarchy – Atom Board Intelligent Drive System (C++) – Navigates room and avoids obstacles – Accepts commands on a TCP/IP socket from C&C – Interprets sensor data from Kinect and PIC24 – Converts video into YUV encoding for FFMpeg FFMpeg (Open source application) – Accepts data via stdin from the Intelligent Drive system and packages it in an MPEG video format

12. IDS Software Architecture Thread 1 - 30 FPS event loop – Transcodes and Outputs video to FFMpeg – Polls Kinect Video and Depth Sensor – Polls network socket for C&C Commands Thread 2 - Queue driven event loop – Implements room mapping and human detection – Timers add standard events to queue – Flexible queue design allows adjusting task frequency – Communicates with PIC24 over RS232

13. Software Hierarchy - Microcontroller

14. Software Hierarchy - MicroController Low Level Peripheral Controller (C) – Implements fail-safe obstacle avoidance – Accepts commands via 5 byte protocol over RS232 – Controls motors and polls IR sensors – Flag driven event loop Libfreenect & OpenNI (Open source) – Library for communicating with Kinect – Retrieves depth sensor and video data – Recognizes human form

15. Debugging Hardware/MicroController – Standard Reset pushbutton and debug LED – RJ11 allows for in-circuit debugging via ICD3 – Headers for port pins broken out – Printing strings over SCI Atom Board – SSH directly into board – Sending metrics and debugging data to C&C – Sending bitmaps of room map