Preventing Information Leaks with Jeeves Jean Yang, MIT April 8, 2015

Wearable devices Data, Data Everywhere Jean Yang / Jeeves2 Social media Electronic health records Online courses

Jean Yang / Jeeves3 All Kinds of People Are Writing All Kinds of Code Open source lines of code Journalists Medical researchers Social scientists Children

Jean Yang / Jeeves4 Even Trained Developers Leak Information

Why Aren’t Existing Approaches Enough? Jean Yang / Jeeves5 Exploit Patch But leaves system builders a step behind. Defensive protection But people are still showing the data wrong. Encrypting Data

My Approach: Privacy by Construction Jean Yang / Jeeves6 Factor out privacy to reduce opportunity for leaks. Programmer specifies high-level policies about how sensitive data can be used. Rest of program is policy-agnostic. System manages policies automatically.

Social Calendar Example Jean Yang / Jeeves7 Alice and Bob throw a surprise party for Carol.

Even Seemingly Simple Policies Have Subtleties Jean Yang / Jeeves8 Guests Carol Strangers Surprise party for Carol at Chuck E. Cheese. Pizza with Alice/Bob. Private event at Chuck E. Cheese. Policy: Must be guest. Policies can depend on sensitive values and other policies. Policy: Only visible to hosts until finalized. Problem:

Enforcing Policies Can Leak Information! Jean Yang / Jeeves9 Guests Surprise party at Chuck E. Cheese. Policy: Only visible to hosts until finalized. Policy: Must be guest. Guest list finalized Guests can’t see event Guests can see event Subtle mistake: check for policy 1 neglects dependency on policy 2. Problem arises when programmers trusted to get dependencies right. 1 2

Policies Are Intertwined Across the Code Jean Yang / Jeeves10 “What is the most popular location among friends 7pm Tuesday?” Update to event subscribers Track information flow through derived values. Track where derived values flow. Problem:

Jean Yang / Jeeves11 “Policy Spaghetti” in Real Systems Code from HotCRP conference management system Highlighted: conditional permissions checks everywhere.

Jean Yang / Jeeves12 Programming model. Mathematical guarantees. Implementations, web framework, and case studies. Automatic Enforcement with Jeeves The well-intentioned programmer writes same code no matter what policies are.

Jeeves Factors Out Policies Jean Yang / Jeeves13 Centralized policies. Policy-agnostic program. Runtime differentiates behavior. Model ViewController

Background. Jean Yang / Jeeves14 Jeeves programming model. How Jeeves works. Theoretical guarantees. Implementation & evaluation. This Talk [PLAS ’13] [draft] [POPL ’12] Background.

1.Specifying policies. 2.Specifying differentiated behavior. 3.Factoring out policies from other code. 4.Enforcing policies that depend on sensitive values. Automatically Managing Information Flow Jean Yang / Jeeves15 Only Jeeves supports 3 and 4. Jeeves supports more expressive policies than prior work.

Jean Yang / Jeeves16 Information flow controls check that only allowed flows occur [HiStar, Flume, Resin, LIO/Hails, Jif/Sif, flow locks, F*, and more]. Related Work: Checking Information Flow == true == false if viewer == : elif viewer != : error Programmer tags data. System prevents leaks by raising error. Must explicitly define and tag alternate behavior. My previous work.

Jean Yang / Jeeves17 == Multi-execution approaches differentiate behavior based on viewer [secure multi-execution (Devriese et al); faceted execution (Austin and Flanagan)]. true false Computing with Multiple Values Related Work: Must encode policies as tags. “Policy spaghetti.” Policies can leak information.

Background. Jean Yang / Jeeves18 Jeeves programming model How Jeeves works. Theoretical guarantees. Implementation & evaluation. This Talk Jeeves programming model. Supporting expressive policies that can depend on sensitive values. 2 Factoring out information flow. 1 Challenges

Policy-Agnostic Programming in Jeeves Jean Yang / Jeeves19 Application Code Separate from policies. policies Sensitive values encapsulate multiple behaviors. Policies describe rules for how values may flow to output contexts.

.guests [ ] Jeeves Supports Expressive Policies def isNotCarol(oc): return oc != Jean Yang / Jeeves20 Output context can be of arbitrary type. def isGuest(oc): return oc in.guests A policy is an arbitrary function that takes the output context and returns a Boolean value. Policies can depend on sensitive values.

21Jean Yang / Jeeves == true false print { } true false Jeeves Programming Model Programmer writes policy- agnostic programs. Runtime propagates values and policies. Runtime produces differentiated output based on the viewer. Programmer specifies policies and facets

The Jeeves Programming Model Well-defined runtime semantics for policy-agnostic programming with information flow policies. Can be implemented standalone or embedded as a library. Has been adapted across runtimes in web frameworks. Jean Yang / Jeeves22

Background. Jean Yang / Jeeves23 Jeeves programming model How Jeeves works. Theoretical guarantees. Implementation & evaluation. This Talk How Jeeves works. Handling policies with dependencies. 1 Enforcing policies when state changes. 2 Challenges

24Jean Yang / Jeeves if == : x += 1 return x x = 0 print { } 10 Jeeves Execution Model Runtime propagates values and policies. Runtime solves for values to show based on policies and viewer. 2 1 Runtime simulates simultaneous multiple executions.

Using Policies to Produce Outputs Jean Yang / Jeeves25 print { } 0 ( != ) ? 1 : 0 policy( ) Jeeves uses policies to defacet appropriately. 1 0 def isNotCarol(oc): return oc !=

Jean Yang / Jeeves26 print { } ( == ) ? : policy( ) def isMaybeCarol(oc): return oc == But What About Dependencies? Possible solutions: ( == ) ? : Jeeves runtime will pick the secret value if allowed. Need to find a fixed point!

Jean Yang / Jeeves27 Using Constraints to Handle Dependencies LabelPolicy a def isGuest(oc): return oc in.guests policy( ) print { } a Constraints contain only Boolean variables. Always a consistent assignment. Evaluated with respect to state at time of output.

Handling Indirect Flows Jean Yang / Jeeves28 if == : a x += 1 true false a if : x += 1 x = x old +1 x old a Labels follow values through all computations, including conditionals and assignments.

Outputting to Sinks Jean Yang / Jeeves29 Evaluate output context and expression to print. Retrieve labels and policies. Evaluate policies applied to the output context. Defacet using satisfying policy assignment. Jeeves Runtime Semantics: Statement evaluation Expression evaluation Output context Result Policies

Background. Jean Yang / Jeeves30 Jeeves programming model How Jeeves works. Theoretical guarantees. Implementation & evaluation. This Talk Theoretical guarantees. Proving enforcement with respect to our semantics. 2 Defining what it means to enforce policies. 1 Challenges

Non-Interference Jean Yang / Jeeves31 a if == : x += 1 print { } a a if == : x += 1 Jeeves Challenge: Compute labels from program—may have dependencies on secret values! Secret values should not affect public output.

Jean Yang / Jeeves32 a if == : x += 1 print { } a a if == : x += 1 Jeeves Theorem: All executions where a must be public produce equivalent outputs. Jeeves Non-Interference and Policy Compliance Can’t tell apart secret values that require a to be public.

Jeeves Policy Compliance Theorem Suppose we have Jean Yang / Jeeves33 Need to account for policies that depend on sensitive values!

Background. Jean Yang / Jeeves34 Jeeves programming model How Jeeves works. Theoretical guarantees. Implementation & evaluation. This Talk Implementation and evaluation. Adapting the approach for web applications. 1 Scaling the approach. 2 Challenges

Facets in the Application and Database Jean Yang / Jeeves35 Application Queries select * from Users where location = SQL Database Application All data SQL Database select * from Users Database queries can leak information! Impractical and potentially slow! Solution: Facet the database. Implemented as an object- relational mapping supporting uniform facet representation.

Jeeves runtime Jacqueline Web Framework Jean Yang / Jeeves36 Application Viewer Propagate policies. Attach policies. Defacet values based on viewer. Programmer is responsible Framework is responsible

Jean Yang / Jeeves37 Django-like data schema for describing fields. Policy for ‘location’ field. Helper functions for policy include queries. Public value for ‘location’ field.

Jean Yang / Jeeves38 DEPLOYED FOR PLOOC 2014! Conference management system Course managerMedical records (HIPAA) Implemented in Jeeves Questions 1.How does faceted execution scale? 2.How does faceting the database affect queries?

Demo Jean Yang / Jeeves39

CMS Running Times Jean Yang / Jeeves40 Database size does not affect query speed. Showing sensitive data grows linearly. Tests from Amazon AWS machine via HTTP requests from another machine.

Evaluation Lessons Faceted execution is expensive. Policy-agnostic programming does not always need faceted execution. Many opportunities for optimization in web frameworks. Jean Yang / Jeeves41

Policy-Agnostic Programming in Jeeves Jean Yang / Jeeves42 Design of a policy- agnostic programming language [POPL ‘12] Semantics and guarantees [PLAS ’13] Implementation, web framework, and case studies [in submission] == Other functionality Policies Sensitive values

Jean Yang / Jeeves43 F*: type-checking security properties. [ICFP ’11; JFP ‘13] Ask Reeves: verified privacy policies. [patent filed] Verified Type- checked Verve: a provably type-safe OS. [PLDI ’10 (Best Paper); CACM ‘10] My Interests in Provable Guarantees Lessons Difficult to write secure and private programs. Even more difficult to prove them correct. Easier if we can get privacy by construction.

Jean Yang / Jeeves44 The Future of Policy- Agnostic Programming Factor out policies from other code. Rely on compiler and runtime to differentiate behavior.

Programmability. –Languages for different kinds of policies. –Tools for verifying and visualizing policies. Scaling. –Policy-agnostic semantics with minimal faceting. –Building foundations with policy management in mind. Jean Yang / Jeeves45 Making Policy-Agnostic Programming Work Web frameworks DatabasesWeb browsers

Parting Thoughts By reducing opportunity for programmer error, we can eliminate whole classes of privacy leaks. Jean Yang / Jeeves46