Reflections on GDPR Article 15 On meaningful understanding of the logic of automated decision-making Eric Horvitz @erichorvitz Berkeley Center for Law & Technology Privacy Law Forum: Silicon Valley March 24, 2017 horvitz@microsoft.com

General Data Protection Regulation (EU) 2016/679, 25 May 2018  Article 15 (1)(h): "existence of automated decision-making…and …meaningful information about the logic involved Many questions about interpreting Article 15.

Multiple aspects of AI decision-making pipeline.

Data  Predictions  Decisions Training cases Dataset Active sensing & learning Predictive model ML algorithm Data  Predictions  Decisions Predictions Decision model Decisions

Data  Predictions  Decisions Training cases Dataset Predictive model ML algorithm Data  Predictions  Decisions Predictions Decision model Decisions

Logic of processing meaningful information about the logic involved Personal Data Training Data Algorithm Predictions Decisions Use meaningful information about the logic involved

Logic of processing Personal Data Training Data Algorithm Predictions What, when? Training Data Algorithm Predictions Decisions Use What, who, when? Accuracy? Procedure used? Validation? Accuracy? Decision policy? Values, cost/benefit? Fully automated? Decision support?

Localization & Personalization Training Domain Personal Data Training Data Algorithm Predictions Decisions Use Validation? Accuracy?

Localization & Personalization Machine learning “contact lens” for children March 2017

Moving into High-Stakes Arenas

Localization & Personalization Readmissions Manager

Site A Site B Site C Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Site C

Site A Site B Site C Site-specific evidence Site-specific pts, prevalencies Site covariate dependencies Hospital A Community hospital: 180 beds, 10,000 admissions/year. Hospital B Acute care teaching hospital: 250 beds, 15, 000 inpatients/year. Hospital C Major teaching & research hospital: 900 beds, 40,000 inpatient/year. Same city

Challenges of Localization & Personalization Transfer Learning Local Training Training Domain Personal Data Training Data Algorithm Predictions Decisions Use Validation? Accuracy?

On Decisions & Values Training Domain Personal Data Training Data Algorithm Predictions Decisions Decisions Use Validation? Accuracy?

Narrowing Focus Interpretability & explainability Personal Data Training Data Algorithm Predictions Decisions Use meaningful information about the logic involved Interpretability & explainability

Meaningful Information about ML Procedure? M. Zeiler, R. Fergus Visualizing and Understanding Convolutional Networks, Arxiv (2013)

Interpretability & Explanation of Machine Learning Rich & open-area of research One approach: Enable end users to understand contribution of individual features What influence does changing observations x have if other values are not changed? Medical applications Scientific understanding Model engineering

Interpretability-Power Tradeoff Logistic regression Neural networks Promising space

Interpretability & Explanation Medical applications Scientific understanding Model engineering

Insights—and Errors AI Journal 1996

Inspection & Troubleshooting Cooper, et al. 1996

Inspection & Troubleshooting (!)

Having our Cake and… Caruana, et al. 2015

Directions Research on understandability & explanation Best practices, norms, standards on comprehension What aspects of AI pipelines? How much detail is “meaningful” understanding? For whom? when? What uses & contexts?