1. LF Deep Learning Foundation Technical Advisory Council Meeting
January 17, 2019

2. Recording of Calls
This is a reminder that we have decided to record TAC meetings and store them on the TAC Wiki.

3. Antitrust Policy Notice
Linux Foundation meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.
Examples of types of actions that are prohibited at Linux Foundation meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Undergone LLP, which provides legal counsel to the Linux Foundation.

4. Dial-in Information
Join from PC, Mac, Linux, iOS or Android:
Or iPhone one-tap:
US: ,, # or ,, #
Or Telephone:
Dial(for higher quality, dial a number based on your current location):
US: or or (Toll Free) or (Toll Free)
Meeting ID:
International numbers available:

5. Agenda Roll Call Project Contribution Proposal Review & Discussion:
Pyro
Next Meeting Information
Discussion / Q&A
11/16/18

6. TAC Member Directory Member TAC Contact TAC Email Amdocs Ofer Hermoni*
AT&T
Jack Murray
Baidu
Xin Pan
Ericsson
Nimisha Radia
Huawei
Yinglong Xia
Nokia
Pantelis Monogioudis
Tech Mahindra
Manish Kampani
Tencent
XiaoLong Zhu
ZTE
Wei Meng
Acumos AI Project
*TAC Chairperson

7. TAC Overview
LF Deep Learning Foundation is an umbrella project of The Linux Foundation with the mission of supporting AI, ML and DL open source projects.
LF DL currently has three projects and we are accepting contributions of additional projects.
The TAC is a committee within LF DL that is responsible for assisting in the coordination and communication of projects within LF DL. It is not a technical oversight body, all technical oversight is handled by the individual projects themselves according to their own governance.
Since the March 2018 launch of LF DL, activity of the TAC has included:
approving a project lifecycle for projects of LF DL (incubation, graduation, archive stages);
reviewing and then approving contribution proposals for the Angel and EDL projects;
starting work on a landscape analysis diagram for open source AI projects overall; and
identifying the area of model interchange formats for exploration by the TAC.

8. TAC Wiki
We have established a Wiki for the TAC which can be found here (login to lists.deeplearning.org required):

9. Project Contribution Proposal Review & Discussion: Pyro
Project Pyro
Deep universal probabilistic programming with Python and PyTorch
(As of Jan 14th: 50 contributors, 4870 stars)
Presenter:
Fritz Obermeyer, Pyro Engineering Lead, Uber AI Labs

10. Pyro: Deep Universal Probabilistic Programming
Fritz Obermeyer, Pyro engineering lead, Uber AI Labs

11. ML Problems at Uber
Uber does not have one machine learning problem, it has thousands.

12. ML Solutions in Practice

13. ML Solutions in Practice

14. Probabilistic programming
Build custom solutions that combine the best of all worlds: - probabilistic models (handling uncertainty) - high-level programming languages (means of abstraction) - scalable deep learning (learning from large data)

15. Example application: semi-supervised learning
Suppose we want to classify handwritten digits...
but we only have a few labeled examples

16. Pyro: Python with two extra primitives
x = pyro.sample(“x”, Bernoulli(0.5)) assert isinstance(x, torch.Tensor) pyro.sample(“data”, Normal(0., 1.), obs=data) theta = pyro.param(“theta”, torch.ones(100), constraint=positive)
return value of pyro.sample(...) is a torch.Tensor (not an RV object).
pyro.sample(..., obs=...) acts as a loss term.
pyro.param(...) persists across function calls.

17. Pyro models generate data
style
digit
digit
decoder = ... # nn def model(image=None): pyro.module("decoder", decoder) style = pyro.sample("style", Normal(zeros(20), ones(20))) digit = pyro.sample("digit", Categorical(ones(10) * 0.1)) image = pyro.sample("image", Bernoulli(decoder(style, digit)), obs=image) return image
image
Semi-supervised Learning with Deep Generative Models,
Diederik P Kingma, Danilo Jimenez Rezende, Shakir Mohamed, Max Welling

18. Approximate inference as optimization
We can do inference by learning a model of the posterior, called a guide:
Variational Bayes and Beyond: Bayesian Inference for Big Data, Broderick, T.

19. Pyro guides infer latent variables
style
digit
digit
def model(image=None): ... def guide(image): pyro.module("encoder_digit", encoder_digit) pyro.module("encoder_style", encoder_style) digit = pyro.sample("digit", Categorical(encoder_digit(image))) loc, scale = encoder_style(image, digit) style = pyro.sample("style", Normal(loc, scale)) return digit, style
image
go into detail

20. Pyro SVI trains the guide
style
digit
digit
def model(image): ... def guide(image): ... svi = SVI(model, guide, optim.Adam(), TraceEnum_ELBO()) for epoch in range(10): for batch in partition(data): svi.step(batch)
image
infer.step() updates parameters registered with pyro.param() statement.
T=this uses monte carlo estimate of a loss function (elbo).

21. The guide now serves as a classifier
style
digit
digit
def model(image): ... def guide(image): ... # ...train... digit, style = guide(image)
image

22. Results
style
digit
digit
image

23. Pyro Pyro stack Generative models Custom inference components
Inference building blocks (e.g. loss functions, MCMC kernels)
Pyro
Runtime
Ops, distributions
Effect handlers
Candidate #1
PyTorch 1.0: JIT, autograd, distributions, tensor math
Python
C++
CUDA

24. Research advances must interact correctly
Subsampling
Reparameterization
Sticking the Landing
SVI
Neural Baselines
RSVI
ADVI
Discrete Enumeration
Each of these are papers and months to years of work. They all have to interact with each other
Message Passing
Rao- Blackwellization
DiCE

25. Documentation catalyzes community growth
Design interfaces to be easy to document
Document internals, e.g. minipyro.py
Follow Python & PyTorch idioms
Tutorials, examples, reference, forum, tech reports
Doctests and usage examples
Allow researchers to hack Pyro

26. Community
goals: automated inference

27. Community We collaborate on PyTorch Distributions Open Design Process
- university collaborators around the world
- contribute code to other OSS

28. Community pyro.contrib.oed pyro.contrib.tracking
goals: automated inference

29. Community
goals: automated inference

30. Results at Uber and beyond
Pyro is used by large companies like Siemens, IBM, and Uber, and startups like Noodle.AI
Pyro is widely used in research institutions including Harvard, MIT, Stanford, Oxford, Cambridge, and The Broad Institute
Pyro solves a range of Uber problems, spanning: sensor fusion, time series forecasting, ads campaign optimization, and data augmentation for deep image understanding
Joint Mapping and Calibration via Differentiable Sensor Fusion, Chen et al. (2018)

31. 2019H1 Roadmap
Engineering foundations: - Improve support for deployment/serving with PyTorch Pay down tech debt and simplify user interfaces - OSS citizenship: push code upstream to PyTorch and GPyTorch
Supporting applications, at Uber and beyond: - Optimal experimental design - Sensor fusion & perception - Time series segmentation
Core PPL research: - Automating inference patterns - Improving inference algorithm scalability and performance

32. For documentation, examples, and more: https://pyro.ai
Thank you!
For documentation, examples, and more:
Alternate end slide design

33. TAC Vote on Project Proposal
›Proposed Resolution:
The TAC approves the Pyro Project as an incubation project of the LF Deep Learning Foundation.

34. TAC Vote on Project Proposal
RESOLVED:
The TAC approves the Pyro Project as an incubation project of the LF Deep Learning Foundation.

35. Next Meeting Information
Upcoming Meetings
January 31, 2019: Invited Presentation: Pachyderm
February 14, 2019: Regular TAC Call (No invited speaker)

36. Questions / Discussion

37. Legal Notices
The Linux Foundation, The Linux Foundation logos, and other marks that may be used herein are owned by The Linux Foundation or its affiliated entities, and are subject to The Linux Foundation’s Trademark Usage Policy at as may be modified from time to time.
Linux is a registered trademark of Linus Torvalds. Please see the Linux Mark Institute’s trademark usage page at for details regarding use of this trademark.
Some marks that may be used herein are owned by projects operating as separately incorporated entities managed by The Linux Foundation, and have their own trademarks, policies and usage guidelines.
TWITTER, TWEET, RETWEET and the Twitter logo are trademarks of Twitter, Inc. or its affiliates.
Facebook and the “f” logo are trademarks of Facebook or its affiliates.
LinkedIn, the LinkedIn logo, the IN logo and InMail are registered trademarks or trademarks of LinkedIn Corporation and its affiliates in the United States and/or other countries.
YouTube and the YouTube icon are trademarks of YouTube or its affiliates.
All other trademarks are the property of their respective owners. Use of such marks herein does not represent affiliation with or authorization, sponsorship or approval by such owners unless otherwise expressly specified.
The Linux Foundation is subject to other policies, including without limitation its Privacy Policy at and its Antitrust Policy at each as may be modified from time to time. More information about The Linux Foundation’s policies is available at
Please with any questions about The Linux Foundation’s policies or the notices set forth on this slide.
The Linux Foundation Internal Use Only
1/3/18