1. Reflections on GDPR Article 15
On meaningful understanding of the logic of automated decision-making
Eric Horvitz
@erichorvitz
Berkeley Center for Law & Technology
Privacy Law Forum: Silicon Valley
March 24, 2017

2. General Data Protection Regulation
(EU) 2016/679, 25 May 2018 
Article 15 (1)(h):
"existence of automated decision-making…and …meaningful information about the logic involved
Many questions about interpreting Article 15.

3. Multiple aspects of AI decision-making pipeline.

4. Data  Predictions  Decisions
Training cases
Dataset
Active sensing & learning
Predictive model
ML algorithm
Data  Predictions  Decisions
Predictions
Decision model
Decisions

5. Data  Predictions  Decisions
Training cases
Dataset
Predictive model
ML algorithm
Data  Predictions  Decisions
Predictions
Decision model
Decisions

6. Logic of processing meaningful information about the logic involved
Personal
Data
Training
Data
Algorithm
Predictions
Decisions
Use
meaningful information about the logic involved

7. Logic of processing Personal Data Training Data Algorithm Predictions
What, when?
Training
Data
Algorithm
Predictions
Decisions
Use
What, who, when?
Accuracy?
Procedure used?
Validation?
Accuracy?
Decision policy?
Values, cost/benefit?
Fully automated?
Decision support?

8. Localization & Personalization
Training
Domain
Personal
Data
Training
Data
Algorithm
Predictions
Decisions
Use
Validation?
Accuracy?

10. Localization & Personalization
Machine learning “contact lens” for children
March 2017

11. Moving into High-Stakes Arenas

12. Localization & Personalization
Readmissions Manager

13. Site A Site B Site C Site-specific evidence
Site-specific pts, prevalencies
Site covariate dependencies
Site C

14. Site A Site B Site C Site-specific evidence
Site-specific pts, prevalencies
Site covariate dependencies
Hospital A
Community hospital: 180 beds, 10,000 admissions/year.
Hospital B
Acute care teaching hospital: 250 beds, 15, 000 inpatients/year.
Hospital C
Major teaching & research hospital: 900 beds, 40,000 inpatient/year.
Same city

15. Challenges of Localization & Personalization
Transfer
Learning
Local
Training
Training
Domain
Personal
Data
Training
Data
Algorithm
Predictions
Decisions
Use
Validation?
Accuracy?

16. On Decisions & Values Training Domain Personal Data Training Data
Algorithm
Predictions
Decisions
Decisions
Use
Validation?
Accuracy?

17. Narrowing Focus Interpretability & explainability
Personal
Data
Training
Data
Algorithm
Predictions
Decisions
Use
meaningful information about the logic involved
Interpretability & explainability

18. Meaningful Information about ML Procedure?
M. Zeiler, R. Fergus Visualizing and Understanding Convolutional Networks, Arxiv (2013)

20. Interpretability & Explanation of Machine Learning
Rich & open-area of research
One approach: Enable end users to understand contribution of individual features
What influence does changing observations x have if other values are not changed?
Medical applications
Scientific understanding
Model engineering

21. Interpretability-Power Tradeoff
Logistic regression
Neural networks
Promising space

22. Interpretability & Explanation
Medical applications
Scientific understanding
Model engineering

23. Insights—and Errors
AI Journal 1996

24. Inspection & Troubleshooting
Cooper, et al. 1996

25. Inspection & Troubleshooting
(!)

26. Having our Cake and…
Caruana, et al. 2015

27. Directions Research on understandability & explanation
Best practices, norms, standards on comprehension
What aspects of AI pipelines?
How much detail is “meaningful” understanding?
For whom? when? What uses & contexts?