Bootstrapping Privacy Compliance in Big Data System Shayak Sen, Saikat Guha et al Carnegie Mellon University Microsoft Research Presenter: Cheng Li

We have your everything Your bank account Your mobile Your social network Your shopping account

We will keep it as a secret

This is how we work Legal team craft privacy policy Privacy Champion interprets policy Developer writes code Audit Team verifies compliance

Life could be much easier encode refine code analysis

Outline Introduction LEGALEASE ◦ Goal ◦ Syntax ◦ Domain-Specific Attribute ◦ Formal Semantics ◦ Properties GROK Validation Discussion Conclusion

LEGALEASE Goal ◦ Usability: Policy clauses are structured very similarly to clauses in English language policy. ◦ Expressivity: Clauses are built around an attribute abstraction that allows the language to evolve as policy evolves. ◦ Compositional Reasoning: LEGALEASE provides meaningful syntactic restrictions to allow compositional reasoning.

Outline Introduction LEGALEASE ◦ Goal ◦ Syntax ◦ Domain-Specific Attribute ◦ Formal Semantics ◦ Properties GROK Validation Discussion Conclusion

LEGALEASE Syntax Domain-Specific attributes are defined in concept lattice L EGLEASE Policies are checked at each node in the data dependency graph. Each node is labeled with attr’s name and set of values. ALLOW: permits node labeled with subset of values. DENY: forbids node labeled with sets that overlaps the attribute values.

LEGALEASE Example ◦ Full IP address will not be used for advertising. IP address may be used for detecting abuse. In such cases it will not be combined with account information. ◦ DENY DataType IPAddress UseForPurpose Advertising EXCEPT ALLOW DataType IPAddress:Truncated ALLOW DataType IPAddress UseForPurpose AbuseDetect EXCEPT DENY DataType IPAddress, AccountInfo

Outline Introduction LEGALEASE ◦ Goal ◦ Syntax ◦ Domain-Specific Attribute ◦ Formal Semantics ◦ Properties GROK Validation Discussion Conclusion

LEGALEASE Domain-specific Attribute ◦ Attribute values are organized as a concept lattice. ◦ Advantages of concept lattice:  Abstracts away semantics.  The lattice structure allows users to concisely define sets of elements through their least upper bound.  The lattice structure allows us to statically check the policy for certain classes of errors.

LEGALEASE Attribute define in the implementation ◦ InStore attribute: encode certain policies around collection and storage of data.

LEGALEASE Attribute define in the implementation ◦ UseForPurpose attribute: Encode the data usage.

LEGALEASE Attribute define in the implementation ◦ AccessByRole attribute: For encoding internal access-control based policies.

LEGALEASE Attribute define in the implementation ◦ DataType attribute:  Policy datatypes: types of data

LEGALEASE Attribute define in the implementation ◦ DataType attribute:  Policy datatypes: Category of data types  Limited typestate: A limited way of tracking history.

LEGALEASE Attribute define in the implementation ◦ DataType attribute:  Combining policy datatypes and typestates:  t:s where t is policy datatypes and s is typestates.

Outline Introduction LEGALEASE ◦ Goal ◦ Syntax ◦ Domain-Specific Attribute ◦ Formal Semantics ◦ Properties GROK Validation Discussion Conclusion

LEGALEASE Formal Semantics ◦ Notions:  T – a vector of sets of latice elements.  T x – the value of attribute x in T.  T G – Graph node.  T C – Policy clause vector.

LEGALEASE Formal Semantics ◦ where is ALLOW T C applies to a graph node T G if T G ⊑ T C ◦ is for each x, DENY T C applies to T G if

LEGALEASE Formal Semantics ◦ A graph node is allowed by an ALLOW clause if and only if the clause applies and is allowed by each exception.

LEGALEASE Formal Semantics ◦ A graph node is denied by an DENY clause if and only if the clause applies and is denied by each exception.

Outline Introduction LEGALEASE ◦ Goal ◦ Syntax ◦ Domain-Specific Attribute ◦ Formal Semantics ◦ Properties GROK Validation Discussion Conclusion

LEGALEASE Properties ◦ Totality: C should either allow T or deny it. ◦ Unicity: C cannot allow T and deny T at the same time. ◦ Monotonicity: If C 1 C 2, then for any T G, C 1 allows T G implies that C 2 allows T G and C 2 ;C 2 denies T G implies C 1 denies T G.

Outline Introduction LEGALEASE GROK Validation Discussion Conclusion

GROK GROK System Nodes are labeled with attribute Confidence value Different granularity

GROK Data Flow Edges and Labeling Nodes ◦ Log Analysis: Use log to bootstrap the coarse- grained data flow graph  Label file nodes with InStore attribute, entity nodes with AccessByRole attribute. (high confidence)  Label UseForPurpose attribute for each job. (low confidence)

Log Analysis

GROK Data Flow Edges and Labeling Nodes ◦ Syntactic Analysis: Label Datatype attr by syntactically analyzing the source code of the job that read or wrote data. (low confidence)

Syntactic Analysis

GROK Data Flow Edges and Labeling Nodes ◦ Semantic Analysis: Refine file nodes to a collection of column nodes. Refine job nodes to a sub-graph of nodes.

Semantic Analysis

GROK Data Flow Analysis ◦ Copy DataType attribute of one node to all nodes that data flows to. ◦ Join two attributes that has the same confidence value. ◦ If data flow through UDF(user defined function), check whether typestate has been modified. If it does, assign low confidence value.

GROK Verifying Labels ◦ Attributes verified by developers are assigned with high confidence value. low = IPAddress low confidence attribute related source file related low confidence attribute low = IPAddress low = UserAgent … source file reverse mapping Contact the developer with highest- ranking source file

GROK Implementation GRO K static semantic analyzer data flow analyzer processes individual jobs from the cluster log into the nodes and edges in data dependency graph without attr collates all the graph node, syntactic analysis and conservative data flow analysis, augmented with attrs.

Outline Introduction LEGALEASE GROK Validation Discussion Conclusion

Validation Scale ◦ 100 day period, 77 thousand jobs each day, submitted by over 7 thousand entities in over 300 functional units. ◦ 1.1 million unique lines of code, 21% changes on a day-to-day basis.

Validation Coverage simulate syntactic analyses on real-world DDG add dataflow analysis add manual verification

Validation Usability ◦ Online survey ◦ 12 participants from Microsoft privacy champions. ◦ Majority of participants were able to use LEGALEASE to code policy clauses

Validation Expressiveness

Outline Introduction LEGALEASE GROK Validation Discussion Conclusion

Discussion Expressiveness: LEGALEASE cannot express policies based on first-order temporal-logic. However, LEGALEASE is enough to express privacy policies. Infer sensitive data: Unless explicitly labeled, GROK cannot detect inference from non- sensitive data to sensitive data. Precision: Major source of precision comes from overly conservative treatment of UDF.

Discussion False Negatives: The authors are unable to characterize the exact nature of false negatives in the system due to lack of ground truth. Assurance: The system can not guarantee the result in face of adversarial developers’ behavior.

Outline Introduction LEGALEASE GROK Validation Discussion Conclusion

Conclusion Automated privacy compliance checking ◦ LEGALEASE: stating privacy policies as a form of restrictions on information flows. ◦ GROK: data inventory that maps low level data types in code to high level policy concepts. Evaluation results show that ◦ LEGALEASE is expressive enough to capture real-world privacy policies. ◦ GROK could bootstrap labeling the graph with LEGALEASE at massive scale.