Quantification of Integrity Michael Clarkson and Fred B. Schneider Cornell University MIT Systems Security and Cryptography Discussion Group October 15, 2010

Goal Information-theoretic Quantification of programs impact on Integrity of Information 2 [Denning 1982]

What is Integrity? Common Criteria: Protection of assets from unauthorized modification Biba (1977): Guarantee that a subsystem will perform as it was intended Isolation necessary for protection from subversion Dual to confidentiality Databases: Constraints that relations must satisfy Provenance of data Utility of anonymized data …no universal definition 3

Our Notions of Integrity 4 Starting PointCorruption Measure Taint analysisContamination Program correctness Suppression Corruption: damage to integrity

Our Notions of Integrity 5 Starting PointCorruption Measure Taint analysisContamination Program correctness Suppression Corruption: damage to integrity Contamination: bad information present in output Suppression: good information lost from output …distinct, but interact

Contamination Goal: model taint analysis 6 Program User Attacker User Attacker trusted untrusted

Contamination Goal: model taint analysis Untrusted input contaminates trusted output 7 Program User Attacker User Attacker trusted untrusted

Contamination u contaminates o 8 o:=(t,u)

Contamination u contaminates o (Cant u be filtered from o ?) 9 o:=(t,u)

Quantification of Contamination Use information theory: information is surprise X, Y, Z: distributions I (X,Y): mutual information between X and Y (in bits) I (X,Y | Z): conditional mutual information 10

Quantification of Contamination 11 Program User Attacker User Attacker trusted untrusted

Quantification of Contamination 12 Program User Attacker User Attacker trusted untrusted U in T in T out

Quantification of Contamination Contamination = I (U in,T out | T in ) 13 Program User Attacker User Attacker trusted untrusted U in T in T out [Newsome et al. 2009] Dual of [Clark et al. 2005, 2007]

Example of Contamination 14 o:=(t,u) Contamination = I (U, O | T) = k bits if U is uniform on [0,2 k -1]

Our Notions of Integrity 15 Starting PointCorruption Measure Taint analysisContamination Program correctness Suppression Corruption: damage to integrity Contamination: bad information present in output Suppression: good information lost from output

Program Suppression Goal: model program (in)correctness 16 Sender Receiver correct Specification (Specification must be deterministic)

Program Suppression Goal: model program (in)correctness 17 Implementation Sender Attacker Receiver Attacker trusted untrusted Specification Sender Receiver correct real

Program Suppression Goal: model program (in)correctness 18 Implementation Sender Attacker Receiver Attacker trusted untrusted Sender Receiver Implementation might suppress information about correct output from real output real correct Specification

Example of Program Suppression 19 for (i=0; i<m; i++) { s := s + a[i]; } for (i=0; i<m; i++) { s := s + a[i]; } Spec. a[0..m-1]: trusted

Example of Program Suppression 20 for (i=0; i<m; i++) { s := s + a[i]; } for (i=0; i<m; i++) { s := s + a[i]; } for (i=1; i<m; i++) { s := s + a[i]; } for (i=1; i<m; i++) { s := s + a[i]; } Spec. Impl. 1 Suppressiona[0] missing No contamination a[0..m-1]: trusted

Example of Program Suppression 21 for (i=0; i<m; i++) { s := s + a[i]; } for (i=0; i<m; i++) { s := s + a[i]; } for (i=1; i<m; i++) { s := s + a[i]; } for (i=1; i<m; i++) { s := s + a[i]; } for (i=0; i<=m; i++) { s := s + a[i]; } for (i=0; i<=m; i++) { s := s + a[i]; } Spec. Impl. 1Impl. 2 Suppressiona[0] missing No contamination Suppressiona[m] added Contamination a[0..m-1]: trusted a[m]: untrusted

Suppression vs. Contamination 22 * * Attacker * * Contaminatio n Suppression output := input

Suppression vs. Contamination 23 n:=rnd(); o:=t xor n n:=rnd(); o:=t xor n o:=t xor u o:=(t,u) t suppressed by noise no contamination t suppressed by u o contaminated by u no suppression o := t

Quantification of Program Suppression 24 Implementation Sender Attacker Receiver Attacker trusted untrusted Specification Sender Receiver

Quantification of Program Suppression 25 Implementation Sender Attacker Receiver Attacker trusted untrusted Specification In Spec Sender Receiver

Quantification of Program Suppression 26 Implementation Sender Attacker Receiver Attacker trusted untrusted Specification U in T in Impl In Spec Sender Receiver

Quantification of Program Suppression 27 Implementation Sender Attacker Receiver Attacker trusted untrusted Specification U in T in Program transmission = I (Spec, Impl) In Spec Impl Sender Receiver

Quantification of Program Suppression H (X): entropy (uncertainty) of X H (X|Y): conditional entropy of X given Y Program Transmission = I (Spec, Impl) = H (Spec) H (Spec | Impl) 28

Quantification of Program Suppression H (X): entropy (uncertainty) of X H (X|Y): conditional entropy of X given Y Program Transmission = I (Spec, Impl) = H (Spec) H (Spec | Impl) 29 Total info to learn about Spec

Quantification of Program Suppression H (X): entropy (uncertainty) of X H (X|Y): conditional entropy of X given Y Program Transmission = I (Spec, Impl) = H (Spec) H (Spec | Impl) 30 Info actually learned about Spec by observing Impl Total info to learn about Spec

Quantification of Program Suppression H (X): entropy (uncertainty) of X H (X|Y): conditional entropy of X given Y Program Transmission = I (Spec, Impl) = H (Spec) H (Spec | Impl) 31 Info actually learned about Spec by observing Impl Total info to learn about Spec Info NOT learned about Spec by observing Impl

Quantification of Program Suppression H (X): entropy (uncertainty) of X H (X|Y): conditional entropy of X given Y Program Transmission = I (Spec, Impl) = H (Spec) H (Spec | Impl) Program Suppression = H (Spec | Impl) 32

Example of Program Suppression 33 for (i=0; i<m; i++) { s := s + a[i]; } for (i=0; i<m; i++) { s := s + a[i]; } for (i=1; i<m; i++) { s := s + a[i]; } for (i=1; i<m; i++) { s := s + a[i]; } for (i=0; i<=m; i++) { s := s + a[i]; } for (i=0; i<=m; i++) { s := s + a[i]; } Spec. Impl. 1Impl. 2 Suppression = H (A)Suppression H (A) A = distribution of individual array elements

Echo Specification 34 output := input Sender Receiver trusted T in

Echo Specification 35 Implementation Sender Attacker Receiver Attacker trusted untrusted output := input Sender Receiver trusted T in T out U in

Echo Specification 36 Implementation Sender Attacker Receiver Attacker trusted untrusted output := input Sender Receiver trusted T in T out U in Simplifies to information-theoretic model of channels, with attacker

Channel Suppression 37 Channel transmission = I (T in,T out ) Channel suppression = H (T in | T out ) (T out depends on U in ) Channel Sender Attacker Receiver Attacker trusted untrusted T in T out U in

Belief-based Metrics What if users/receivers distribution on unobservable inputs is wrong? Belief-based information flow [Clarkson et al. 2005] Belief-based generalizes information- theoretic: On single executions, the same In expectation, the same …if users/receivers distribution is correct 38

Suppression and Confidentiality Declassifier: program that reveals (leaks) some information; suppresses rest Leakage: [Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005, 2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009] Thm. Leakage + Suppression is a constant What isnt leaked is suppressed 39

Database Privacy Statistical database anonymizes query results: …sacrifices utility for privacys sake 40 Anonymizer User Databas e User query response anonymized response

Database Privacy Statistical database anonymizes query results: …sacrifices utility for privacys sake …suppresses to avoid leakage 41 Anonymizer User Databas e User query response anonymized response anon. resp. := resp.

Database Privacy Statistical database anonymizes query results: …sacrifices utility for privacys sake …suppresses to avoid leakage …sacrifices integrity for confidentialitys sake 42 Anonymizer User Databas e User query response anonymized response

K-anonymity DB. Every individual must be anonymous within set of size k [Sweeney 2002] Iflow. Every output corresponds to k inputs …no bound on leakage or suppression 43

L-diversity DB. Every individuals sensitive information should appear to have L (roughly) equally likely values [Machanavajjhala et al. 2007] DB. Entropy L-diversity: H (anon. block) log L [Øhrn and Ohno-Machado 1999, Machanavajjhala et al. 2007] Iflow. H (T in | t out ) log L …implies suppression log L 44

Differential Privacy DB. [Dwork et al. 2006, Dwork 2006] No individual loses privacy by including data in database Anonymized data set reveals no information about an individual beyond what other individuals already reveal Iflow. Output reveals almost no information about individual input beyond what other inputs already reveal …implies almost all information about individual suppressed …quite similar to noninterference 45

Summary Measures of information corruption: Contamination (generalizes taint analysis, dual to leakage) Suppression (generalizes program correctness, no dual) Application: database privacy (model anonymizers; relate utility and privacy; security conditions) 46

More Integrity Measures Attacker- and program-controlled suppression Granularity: Average over all executions Single executions Sequences of executions …interaction of attacker with program Applications: Error-correcting codes, malware 47

Quantification of Integrity Michael Clarkson and Fred B. Schneider Cornell University MIT Systems Security and Cryptography Discussion Group October 15, 2010