Richard Landau Sarah Patten Jacob Stewart BROBOT THE STUDY COMPANION GROUP 25

 BroBot assists in easing the minds of students longing for a worry-free break.  Watches study materials while the user is away from the table.  Sends updated images and alerts to user through an Android app on user’s phone.  Will provide real-time updates and will notify user if something is stolen. PROJECT OVERVIEW

 Detect theft of an object.  Alert user when a theft occurs.  Be reliable and simple to use.  Rechargeable or easily replaceable battery  Be lightweight, portable, and affordable. GOALS AND OBJECTIVES

Specification WeightLess than 3 lbs DimensionsLess than 6inX6inX6in when closed Communication RangeAt least 250 ft Battery Life while in operation At least 2 hours Image Comparison Rate2/sec Refresh App PictureEvery 15 seconds DESIGN SPECIFICATIONS

OVERALL BLOCK DIAGRAM

 All of the computations are on the microcontroller side  A good amount of serial communication protocols (SPI, I2C, UART)  Low power modes including sleep mode  Special features with digital cameras (?)  Good community backing  Cheap development board with all the pin outs on the microcontroller PROCESSOR CONSIDERATIONS

TM4C123GH6PMSTM32F407VGATSAM4S Flash Memory256 KB1 MB SRAM32 KB192KB128 KB GPIOs Max Speed80 MHz168 MHz120 MHz Package64 LQFP100 LQFP64 LQFP Dev Price (USD) Board plusesNot too many output pins but has good community behind it Comes with many MEMS sensors and all output pins Comes with 2GB external flash and all output pins ARM MICROCONTROLLERS

 3.3V Voltage  Low power modes  Digital Camera Interface(DCMI)  Can take 8 bit parallel data from a camera  Good amount of SRAM  Fast operation speed  A lot of serial communication lines  Good community backing it with pre-written libraries for most features STM32F407VG

MT9D111LinkSprite JPEG Color Camera Control CommunicationOwn protocolUART Data Lines8 Bit ParallelUART Voltage3.1V5V Current (Max Operation)350 mA100mA Max Resolution1600x x480 Cost CAMERAS

 Cheap and small  Adjustable output  Adjustable resolution  Adjustable quality on the JPEGs  Parallel data  Control on shutter speed, and auto focus  Sleep modes  Has own serial protocol  Interfacing parallel data might be tricky MT9D111

 Local shipping  Higher voltage needed  Data is serial making it easier to interface but slower overall  Smaller resolution  Will be used mostly for testing, or used if we have troubles with the MT9D111  Bought since we are under budget LINKSPRITE JPEG COLOR CAMERA

 Wi-Fi direct and Bluetooth are comparable  Wi-Fi direct has longer range than Bluetooth  However, Bluetooth has more support and is easier to implement  Bluetooth is also more common in everyday life so we wanted experience with it BLUETOOTH VS WIFI-DIRECT

 Chose RN41 due to low cost, high range, and black box functionality  WT41 strong contender but expensive and drains battery quickly Bluetooth Module Comparison Name TI LMX9830 BC04RN41BLE112WT41 Connection Type UART Range2 – 10m1-15m1 – 100m2-10m1-800m Low PowerYes No Price$13.46$9.96$21.70$21.00$33.00

 All components need 3.3V  Around 500mA when everything is going  Power source needs to be small and lightweight  Rechargeable or easily replicable power source  One switching regulator  3 AA Batteries held in a battery holder POWER SYSTEM OVERVIEW

 Exact platform hasn’t been determined  Bluetooth antenna needs to be protruding  Small and lightweight  Expandable/ Contractible casing for easy traveling BROBOT’S PLATFORM

 Use computer vision techniques.  Compare 2 images from different points in time  Determine if there has been a significant amount of change ITEM WATCHING OVERVIEW

 Store an “original” image  Every ½ a second, take a new picture  Compare new picture with “original”  If the images differ by too much, sound alarm  Every 15 seconds replace “original” with the newest picture taken ALGORITHM EXPLANATION

PSEUDOCODE

 Android application’s main focus is to alert the user if something has been stolen  Picture of BroBot’s current view will be refreshed for the user to view at any time  App will vibrate to alert the user if something is taken APP SUMMARY

 Android represents a bigger market share  Does not require a Mac to code  Good APIs to interface with Bluetooth ANDROID VS IOS

APP DESIGN  Initial screen shown upon startup  If connection fails, user will see an error  If it succeeds…

 User will be taken to screen showing BroBot’s field of vision  When his sight is aligned properly, press the button to start the item watching software  This button will be replaced with a “Stop Watching” button, or a “Silence Alarm” button if it is tripped falsely.

BLUETOOTH CONNECTION  Once Bluetooth connection is established, app opens an input thread and output thread  Output thread stays idle and waits for signals to be sent to BroBot  Input thread constantly looks for new pictures or information from BroBot  Bluetooth’s input stream buffer reads in an array of bytes  Combine these bytes and restore the original picture

 Bluetooth module to terminal communication  Items are bought  Item watching algorithm done  App GUI is done SUCCESSES!

 Camera shipping  App Bluetooth Connection  Converting byte array into viewable JPEG  Enough time for testing for Item Watching  Finalize power DIFFICULTIES

ItemPrice Platform$20.00 STM Discovery Development Board$14.90 RN-41 Bluetooth Module$ Megapixel camera$18.99 Printed Circuit Boards$75.00 Report Binding$15.00 Batteries$20 PVC supplies$20.00 Total$ ORIGINAL BUDGET

ItemPrice STM Discovery Development Board$0.00(Previously Owned) RN-41 Bluetooth Module$29.89 MT9D111 Camera$18.99 Backup Camera$44.95 Report Binding$5 Total$98.83 CURRENT SPENDING Still $ in our budget Need: Platform Battery PCB

 App communicating with module by first week of February  Bluetooth Module communicating with ARM processor by mid February  Camera communications finished by end of February  Item watching program tested and working on a desktop by end of February  PCB designed and finished by early March  Full app development finished by mid March  Item watching program testing during March, finished by end of March  Full prototype testing during mid March and finished by early April PLANS FOR SUCCESS

PROGRESS CHART

 Siren implementation  Video stream OTHER CONSIDERATIONS

QUESTIONS?