1. Course Summary Joseph E. Gonzalez
Co-director of the RISE Lab

2. Recap: Reflecting on the Class

3. Covered Many Topics Problem Formulation Autonomous Vehicles
Neural Architectures
ML for Networking
Deep Learning Frameworks
Dynamic Networks
Reinforcement Learning
Model Compression
Distributed Learning
Prediction Serving
DNN Compilers
Natural Language Processing
Hyperparameter Search
Explainability
Secure Learning
Learned Data Structures
Scheduling for Training

4. ? Covered Many Topics Learning Systems Problem Formulation
Deep Learning Frameworks ?
Neural Architectures
Distributed Learning
Reinforcement Learning
DNN Compilers
Distributed Dataflow Systems
Hyperparameter Search
Secure Learning
Actors
Dynamic Networks
Learned Data Structures
Real-time Systems
Tail at scale and discussion on tail latency
Large scale data management
Active learning
Why the GPU
Overview of GPU Architectures
Brainwave ISCA
Autonomous Vehicles
Model Compression
Architecture of a Compiler
ML for Networking
Vector Accelerators
Natural Language Processing
Prediction Serving
Explainability
Scheduling for Training

5. Big Ideas and Results Emerging trends in AI research
Tradeoff between statistical and computation efficiency
Role of systems (simplification, scale) in “AI Revolution”
New applications of AI techniques to systems problems
New problem domains (e.g., autonomous driving)

6. Big Ideas in ML Research
Generalization
What is being “learned”?
Inductive Biases and Representations
What assumptions about domain enable efficient learning?
Efficiency (Data and Computation)
How much data and time are needed to learn?
Details: Objectives/Models/Algorithms

7. Big Ideas in Systems Research
Problem Framing
Identifying the right problem and solution requirements
Abstraction & Managing Complexity
Reducing complex problems into smaller parts
Tradeoffs
Understanding fundamental constraints
Details: System design and Implementation

8. What is AI-Systems Research?
Should be both good AI and Systems research
Provides insights to both communities
Leverages understanding of both domains
Examples:
Studies tradeoff between statistical and computational efficiency
Identify essential abstractions in DNN design
Leverages framing of indexes to exploit overfitting
Do we need another venue?
ICML, NIPS, ICLR, UAI, AAAI, ICRA, CVPR, ICCV, ECCV, SOSP, OSDI, NSDI, EuroSys, SIGMOD, VLDB, ASPLOS, SOCC  SysML?
1) an ML paper that builds a new distributed learning algorithm without discussing interaction with systems (This is just modern ML work) 2) a systems paper that builds a system for scheduling ML jobs that would be the same without the ML-ness of those jobs. (This is just systems work which has applications to ML.) 3) a software framework that is just software without any insight or design ideas. (Just because you wrote software does not make it SysML research.)

9. Logistics Poster Session next Wednesday 5/8/19 from 9:00 to 11:00
Final Reports due: Monday 5/13/19 at 11:59PM
8 pages in Google Docs
link to and