1. Sudo Chef Dan Abbate Kyle Despins Tom Murphy Eli Siskind Advisor: Professor Tessier

2. Current State of the Kitchen Difficulties of following instructions across one or more recipes. Picking a recipe involves multiple cross references to your inventory of ingredients.

3. What is Sudo Chef? Home Automation + Recipe Chooser and Ingredients Tracker

4. System Overview Bluetooth Hub 123 456 789 0

5. Block Diagram (Before) Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Stove controller Oven Controller Microwave Controller Bluetooth Hub Image Processing Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Mobile Camera (Barcode Capture) Recipe API Kitchen Actions Dan Tom Eli Kyle

6. Block Diagram (After) Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle

7. MDR Deliverables Digitally controlled hot plate Standard handshake via Bluetooth between tablet and microcontroller hub Bar code module up and running GUI prototype and basic recipe parsing

8. Recipe Fetch, Parsing, and Guidance Eli Siskind

9. Context Recipe Database Bluetooth Hub 123 456 789 0

10. Getting Recipes Need a way to search for recipes based on ingredients User must be able to view recipes and select one A parsed recipe must be converted step by step actions performed by the Sudo Chef system

11. Implementation

12. Recipe Fetching Yummly API Search Request Recipe Request Ingredients Yummly URL Parsing Module Recipe Results String External Site Recipe Fetch Recipe URL Recipe Metadata String HTML File Recipe Results Recipe ID Client Side

13. Optimization Asynchronous HTTP Get threads for all requests Thumbnails transformed in separate threads

14. Recipe Guidance Steps Instructions Push notifications Voice commands Appliance Actions Preheat oven Set hot plate temperature Feedback Request Check if meat is cooked

15. MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle

16. Barcode Scanner Kyle Despins

17. Context Bluetooth Hub 123 456 789 0 Barcode Decoding Product Lookup Barcode

18. Barcode Purpose Easiest way for user to put ingredients into inventory

19. How do you get a product from a barcode?

20. Barcode Process Barcode Capture Processing Grayscale Gaussian ZXing Format image Decode call UPC Query HTTP GET Parse JSON Ingredient Database Raw Image Smoothed Image UPC Integer Product Object

21. Gaussian Smoothing Collins, Robert Penn State

22. MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle

23. Going Forward Switch focus to Voice Processing Optimization and Integration of Barcode scanner CDR: Barcode integration Prototype voice processing

24. Bluetooth Handshake Tom Murphy

25. Context Bluetooth Hub 123 456 789 0

26. Why Bluetooth ? Need to be able to connect application to appliances 123 456 789 0

27. Android App to create connection from phone to Bluetooth module Bluetooth Hub implemented Atmega32 with Uart connected to HC-05 Bluetooth module

28. Android App Main Activity used to discover possible Bluetooth devices SendData activity used to send data from device to chip

29. Bluetooth Module HC-05 blinks constant ready light when on and searching for device to connect to HC-05 indicates connected when blinking light is stalled and not constant

30. MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle

31. CDR Deliverables Data being sent from Android application and forwarded to corrected corresponding device (i.e hot plate or convection oven)

32. Hotplate Controller Dan Abbate

33. Context Bluetooth Hub 123 456 789 0

34. Hot Plate Automation USART Signal Pulse Width Modulation LPF Control Signal

35. Microcontroller Setup Input: USART Output to solid state relay: PWM Non-inverted, Fast PWM mode, with no clock prescaling Sent through low pass filter (Cutoff 4.8 Hz) USART SignalController Action PWM Duty Cycle SSR Input Voltage 0x00Turned Off0%0v 0x01Low43%1.8v 0x02High100%5v 0x03Send Current Status to App -- Functions

36. MDR Progress Ingredients Tracking Recipe Assistance Voice Recognition Appliance Devices Microwave controller Stove Controller Oven Controller Bluetooth Hub UPC lookup Ingredients Database Interpret Results Audio input Mobile Bluetooth Adaptor Voice Processing Recipe helper mobile interface Thermometer Camera system controller Yumlee API Kitchen Actions Dan Tom Eli Kyle

37. Plan for CDR Digitally controlled Toaster Oven Begin interfacing with microwave

38. MDR Deliverables Digitally controlled hot plate Standard handshake via Bluetooth between tablet and microcontroller hub Bar code module up and running GUI prototype and basic recipe parsing

39. CDR Deliverables Digitally controlled convection oven Data being sent from Android application and forwarded to corrected corresponding device (i.e hot plate or convection oven) Barcode module full integration and prototype voice processing Full recipe parsing, communication with hardware via Bluetooth, inventory tracking coupled with barcode module

40. Demonstration

41. Communication Bluetooth Protocol

42. UPC-A format

43. Sphinx Voice Recognition Use grammar language model: Java Speech Grammar Format Statistical Language Model

44. Yummly Two API calls Search recipes Get recipe Supports ANDing or ORing terms Returns JSON objects with multiple fields Ingredients Allergens Source and attribution information Prep time Number of servings

45. Specifications Keep track of bought food using bar codes Suggest recipes containing ingredients in inventory while they are fresh Parse 100% of recipes, and recognize 95% of recipe terms Respond to natural voice commands Guide user through recipes by preheating oven, heating hot plate, microwaving food, and alerting users with push notifications and voice instructions Always shut off appliances when not in use, including when mobile app fails

46. Budget

47. Bit No76543210 NameFOC0WGM00COM01COM00WGM01CS02CS01CS00 Initial Val01101001 TCCR0 Register: http://www.atmel.com/Images/doc2503.pdf

48. PWM Modes

49. 16 Mhz Baud Rate UBRR (dec) UBRR (hex) % of error 30033320x0D040.0 60016660x06820.0 12008320x03400.0 24004160x01A00.0 48002070x00CF0.2 96001030x00670.2 14400680x00440.6 19200510x00330.2 28800340x00220.8 38400250x00190.2 57600160x00102.1 76800120x000C0.2 11520080x00083.7 23040030x00037.8 25000030x00030.0 http://www.wormfood.net/avrbaudcalc.ph p?postbitrate=19200&postclock=16

50. http://www.grainger.com/product/OMRON-Relay-5ZH25