Networked Embedded Systems EE107 Fall 2017 Lecture 1 Introduction Networked Embedded Systems Sachin Katti

EE107: Networked Embedded Systems Goal: Provide a hands on introduction to the design of networked embedded systems Class will focus broadly on how to design a fully functional embedded system as well as use it for the function its designed

Logistics Instructors: Sachin Katti, Pengyu Zhang TAs: Colleen Josephson, TBD Lectures: Mon 4:30-5:20 Code review session: Wed 4:30-5:20 Lab sessions: Fri: 4:30-5:20 Office hours: TBD Web: http://ee107.stanford.edu https://piazza.com/stanford/fall2017/ee107/home

Lectures & Assignments Lecture slides will be posted on the web Four assignments Due every two weeks Assignments will be based on programming Important to do them to keep pace in class Everyone needs to work on his/her own assignment

Project & Exams Final project: build an embedded vision and audio sensing system that transmits vision and audio data to the cloud which hosts AI/ML applications. Done in groups of 2 or 3 Project presentation Dec 4 and Dec 6 One exam in class Exam: Nov 15 in class

Grading Logistics Overall Class Grade Exam: 30% Four Assignments: 40% End quarter project: 30%

What is a networked embedded system? Computing systems are everywhere Most of us think of “desktop” computers PC’s Laptops Mainframes Servers But there’s another type of computing system Far more common...

Networked Embedded Systems Networked embedded computing systems Computing systems embedded within things/gadgets connected to the network Hard to call out any one thing/gadget, nearly everything now has a chip and a radio in it Billions of units produced yearly, versus millions of desktop/laptop units

Typical Internal Structure

Examples

ICam: vision and speech interfaces for smart home Motivation of building ICam 38 minutes for continuous face detection Connected to a wall Can we design a system that lasts for months on two AA batteries?

ICam: vision and speech interfaces for smart home Let us work on math… 5000mAh/(50mA+300mA+500mA+800mA)=3hr AA alkaline battery x 2 = 5000mAh Power Consumption Camera 50mA WiFi 300mA Codec 500mA GPU processing 800mA

ICam: vision and speech interfaces for smart home Let us work on math… 5000mAh/(50mA+300mA+500mA+800mA)=3hr AA alkaline battery x 2 = 5000mAh Power Consumption Camera 50mA WiFi 300mA Codec 500mA GPU processing 800mA What is the power budget of a system that lasts for months on two AA batteries? 5000mAh/(60days x 24 hours)=3.47mA

ICam: vision and speech interfaces for smart home Let us work on math… AA alkaline battery x 2 = 5000mAh Power Consumption Camera 50mA WiFi 300mA Codec 500mA GPU processing 800mA Sub mA image sensor? Low power radio? Compute and process somewhere else? HM01B0 image sensor (600uA) Backscatter radio Cloud processing

Why is radio power consumption high?

What is backscatter? Signal reflection consumes much less power compared to active radio transmission

How does backscatter work?

How does backscatter work?

How does backscatter work?

ICam: vision and speech interfaces for smart home UART Envelope detector DAC MCU Microphone FPGA Backscatter Image sensor

ICam: vision and speech interfaces for smart home Hardware MCU that collects vision data from an image sensor and speech data from a microphone A low-power backscatter radio board that wirelessly transmits data while consuming 2mA Software in the cloud AI and machine learning applications using the vision and speech data collected by the hardware

What will you take away from this project? General knowledge of how an embedded system works GPIO, Clocks, Interrupts, DMA, ADC, SPI, I2C, UART Low-level firmware development in C Device drivers Digital signal processing Hardware testing (and building) Reading circuit layout and schematics Analog circuit measurement (Oscilloscope, Multimeter) Digital logic observation (Digital Logic Analyzer) Surface mount soldering (I strongly suggest it)

Potential Projects Your ideas are GREAT! Who are you? Propose and implement a cool application Propose and implement something to make the system faster, more reliable or more power-efficient Who are you? Real time reliable face detection and recognition on the laptop life casting What did you say? Real time reliable speech recognition on the laptop Secure and smart camera Security camera - motion detection Amazon Go using our camera running on top of battery with an energy budget

Potential Projects Run faster Low-power SD card data storage Design 11Mbps backscatter radio on top of 802.11b WiFi packets Low-power SD card data storage Design and implement SDIO on an FPGA to allow energy efficient data storage Toward battery-free Design a more energy efficient architecture to stream data from the camera to a laptop. Beat Stanford SNSG architecture Compression is quite power-intensive. Explore an all-analog or analog-assisted compression technique to try and save power More is better Implement a system that can run 2+ cameras simultaneously Bonus: use these cameras to do 3D reconstruction