1. Effects of Limiting Numerical Precision on Neural Networks
An Empirical Study on Deep Learning Accelerators

2. Deep Learning Applications Neural Network
Compute-intensive embedded projects like
Drones,
Autonomous robotic systems,
Mobile medical imaging, and
Intelligent Video Analytics (IVA). OEMs, independent developers,
Neural Network
Training
Inference

3. Problem Statement
(New age) Neural Network Architects are having a hard time
Training
Time
Compute Resources
Hyperparameter search …
Inference
Power Budget
Accuracy

4. Problem Statement
(New age) Neural Network Architects are having a hard time
Training
Time
Compute Resources
Hyperparameter search …
Inference
Power Budget
Accuracy

5. Lecture: Tools to Explore Accelerators
We saw Minerva In Class …
Topics: the Minerva tool to explore the design space,
prune/quantize, lower voltages
Project prep

6. Mine-rva Keras Aladdin Madonna
Overview
Keras
Aladdin
Madonna

7. Motivation

8. Project Proposal
MADONNA : A tool for Measurement & Assistance in Design Of NN to its Architect
Measurement
Nvidia jetson TK-1
Assistance
FPTuner
Ristretto

9. Numeric Precision

10. Assistance

11. Measurement

12. Architecting a good Deep Learning Applicaiton
Low power
User defined accuracy
Best possible within power budget
MAUD
Our project, Framework for NN architect
Principle: Measure As yoU Design

13. Hardware Jetson Embedded Platform Measurement hardware Topology
NVIDIA Jetson with GPU-accelerated parallel processing.
Leading embedded visual computing platform.
It features high-performance, low-energy computing for deep learning and computer vision
Ideal for compute-intensive embedded projects like
drones,
autonomous robotic systems,
mobile medical imaging, and
Intelligent Video Analytics (IVA). OEMs, independent developers,
Makers and hobbyists can use the NVIDIA Jetson TX1 to explore the future of embedded computing.
Measurement hardware
Yokogawa  wt310
The WT300E series digital power analyzer
Provides extremely low current measurement capability down to 50 micro-Amps,
This instrument is ideal for engineers performing stand-by power measurements.
Topology
The head/gateway node is mir.cs.utah.edu 
This is the large tower computer on the floor.
mir is then connected to the switch on the table.
This switch is then connected to the nvidia jetson tk-1 boards (mir01, mir02,.... mir16).

14. nVIDIA Tegra, Jetson K1 Board

15. GPU based Accelerator Tegra K1 GPU NVIDIA® Kepler™ Architecture
TEGRA K1 PROCESSOR SPECIFICATIONS
- See more at:
GPU based Accelerator
 Tegra K1
GPU
NVIDIA® Kepler™ Architecture
192 NVIDIA CUDA® Cores
CPU
CPU Cores and Architecture
NVIDIA 4-Plus-1™ Quad-Core ARM Cortex-A15 "r3"
Max Clock Speed
2.3 GHz
Memory
Memory Type
DDR3L and LPDDR3
Max Memory Size
8 GB (with 40-bit address extension)
Display
LCD
3840x2160
HDMI
4K (UltraHD, 4096x2160)
Package
Package Size/Type
23x23 FCBGA 16x16 S-FCCSP 15x15 FC PoP
Process
28 nm

16. Workflow Design

17. Software Measurement Assistance eServer.exe (Backend)
eNergy.py (Frontend)
Assistance
Caffe
FPTuner
Ristretto

18. Caffe: make –j 4 all
10 W
8690 Joules

19. Caffe: make clean
5 W
34 Joules

20. Performance Metrics (No free lunch)
DOUBLE
I :12: caffe.cpp:275]
Batch 49, loss = I :12: caffe.cpp:280]
Loss: I :12: caffe.cpp:292]
accuracy = I :12: caffe.cpp:292]
loss = (* 1 = loss)
SINGLE
I :02: caffe.cpp:275]
Batch 49, loss = I :02: caffe.cpp:280]
Loss: I :02: caffe.cpp:292]
accuracy = I :02: caffe.cpp:292]
loss = (* 1 = loss)

21. 34K Joules 346 Joules 12K Joules 244 Joules6W
10 W
34K Joules
346 Joules
Power Metrics
10W
5.5 W
12K Joules
244 Joules
CIFAR 10

22. 11K Joules 62 Joules 4K Joules 45 Joules
8 W
4.5W
11K Joules
62 Joules
8 W
4 W
4K Joules
45 Joules
LENET - MNIST

23. Next Steps Measurement Assistance Complete measurement studies
CaffeNet
ImageNet
Study and Report Impact of Precision on energy consumption
Assistance
Attempt implementing fixed point support in Ristretto / native Caffe
Resume FPTuner addition in workFlow, aiming for automation