Li Xiong CS573 Data Privacy and Security Privacy Preserving Data Mining – Secure multiparty computation and random response techniques

Outline Privacy preserving two-party decision tree mining using SMC protocols (Lindell & Pinkas ’00) Primitive SMC protocols – Secure sum – Secure union (encryption based) – Secure max (probabilistic random response based) – Secure union (probabilistic and randomization based) Secure data mining using sub protocols Random response for privacy preserving data mining or data sanitization

Random response protocols Multi-round probabilistic protocols Randomization probability associated with each round Random response with randomization probability

Multiple rounds Randomization Probability at round r : – Pr(r) = Local algorithm at round r and node i: 4 Max Protocol – multi-round random response g i-1 (r)>=vig i-1 (r)<vi g i (r)g i-1 (r)w/ prob Pr: rand [g i-1 (r), v i ) w/ prob 1-Pr: v i i g i-1 (r)g i (r) vivi

5 Max Protocol - Illustration Start D2D2 D3D3 D2D2 D4D

6 Min/Max Protocol - Correctness Precision bound: – Converges with r – Smaller p0 and d provides faster convergence

7 Min/Max Protocol - Cost Communication cost – single round: O(n) – Minimum # of rounds given precision guarantee (1-e):

8 Min/Max Protocol - Security Probability/confidence based metric: P(C|IR,R) – Different types of exposures based on claim Data value: v i =a Data ownership: Vi contains a – Change of beliefs P(C|IR,R) – P(C|R) P(C|IR, R) / P(C|R) Relationship to privacy in anonymization – Change of beliefs P(C|D*, BR) – P(C|BR) Absolute Privacy Provable Exposure

9 Min/Max Protocol – Security (Analysis) Upper bound for average expected change of beliefs: max r 1/2 r-1 * (1-P 0 *d r-1 ) Larger p0 and d provides better privacy

10 Loss of privacy decreases with increasing number of nodes Probabilistic protocol achieves better privacy (close to 0) When n is large, anonymous protocol is actually okay! Min/Max Protocol – Security (Experiments)

Union Commutative encryption based approach – Number of rounds: 2 rounds – Each round: encryption and decryption Multi-round random-response approach?

Vector Each database has a boolean vector of the data items Union vector is a logical OR of all vectors b1b1 b2b2 bLbL … p1p … p2p … pcpc OR … = VGVG … Privacy Preserving Indexing of Documents on the Network, Bawa, 2003

Group Vector Protocol … … vG’vG’ … vG’vG’ r=1, P ex =1/2, P in =1/2 P ex =1/2 r, P in =1-P ex for(i=1; i<L; i++) if (V s [i]=1 and V G ’[i]=0) Set V G ’[i]=1 with prob. P in if (V s [i]=0 and V G ’[i]=1) Set V G ’[i]=0 with prob. P ex Processing of V G ’ at p s of round r … v1v … v2v … vcvc r=2, P ex =1/4, P in =3/ … vG’vG’ … vG’vG’ … vG’vG’ … vG’vG’ … vG’vG’ p1p1 p2p2 pcpc

Random Shares based Secure Union Phase 1: random item addition – Multiple rounds with permutated ring – Each node sends a random share of its item set and a random share of a random item set Phase 2: random item removal – Each node subtracts its random items set 14

Random Shares based Secure Union - Analysis Item exposure attack – An adversary makes a claim C on a particular item a node i contributes to the final result (C: vi in xi) Set exposure attack – An adversary makes a claim C on the whole set of items a node i contributes to the final union result X (C: xi = ai). Change of beliefs (posterior probability and prior probability) – P(C|IR,X) - P(C|X) – P(C|IR,X)/P(C|X) 15

Exposure Risk – Set Exposure Disclosure decreases with increasing number of generated random items and increasing number of participating nodes Set exposure risk is or close to 0 for probabilistic and crypto approach 16

Exposure Risk – Risk Exposure Item exposure risk decreases with increasing number of generated random items and participating nodes Item exposure risk for probabilistic approach is quite high 17

Cost Comparison Commutative protocol and anonymous communication protocol efficient but sensitive to union size Probabilistic protocol efficient but sensitive to domain size Estimated runtime for the general circuit-based protocol implemented by FairplayMP framework is 15 days, 127 days and 1.4 years for the domain sizes tested 18

Open issues Tradeoff between accuracy, efficiency, and security How to quantify security How to design adjustable protocols Can we generalize the random-response algorithms and randomization algorithms for operators based on their properties Operators: sum, union, max, min … Properties: commutative, associative, invertible, randomizable

Secure Sum Secure Comparison Secure Union Secure Logarithm Secure Poly. Evaluation Association Rule Mining Decision Trees EM Clustering Naïve Bayes Classifier Data Mining on Horizontally Partitioned Data Specific Secure Tools

Secure Comparison Secure Set Intersection Secure Dot Product Secure Logarithm Secure Poly. Evaluation Association Rule Mining Decision Trees K-means Clustering Naïve Bayes Classifier Outlier Detection Data Mining on Vertically Partitioned Data Specific Secure Tools

Summary of SMC Based PPDDM Mainly used for distributed data mining. Efficient/specific cryptographic solutions for many distributed data mining problems are developed. Random response or randomization based protocols offer tradeoff between accuracy, efficiency, and security Mainly semi-honest assumption(i.e. parties follow the protocols)

Ongoing research New models that can trade-off better between efficiency and security Game theoretic / incentive issues in PPDM

Outline Privacy preserving two-party decision tree mining using SMC protocols (Lindell & Pinkas ’00) Primitive SMC protocols – Secure sum – Secure union (encryption based) – Secure max (probabilistic random response based) – Secure union (probabilistic and randomization based) Secure data mining using sub protocols Random response for privacy preserving data mining or data collection

Data Collection Model Data cannot be shared directly because of privacy concern

Randomized Response Do you smoke? Head Tail No Yes The true answer is “Yes” Biased coin:

Randomized Response Multiple attributes encoded in bits Head Tail False answer !E: 001 True answer E: 110 Biased coin: Using Randomized Response Techniques for Privacy-Preserving Data Mining, Du, 2003

Generalization for Multi-Valued Categorical Data True Value: S i S i S i+1 S i+2 S i+3 q1 q2 q3 q4 M

A Generalization RR Matrices [Warner 65], [R.Agrawal 05], [S. Agrawal 05] RR Matrix can be arbitrary Can we find optimal RR matrices? OptRR:Optimizing Randomized Response Schemes for Privacy-Preserving Data Mining, Huang, 2008

What is an optimal matrix? Which of the following is better?

What is an optimal matrix? Which of the following is better? Privacy: M 2 is better Utility: M 1 is better So, what is an optimal matrix?

Optimal RR Matrix An RR matrix M is optimal if no other RR matrix’s privacy and utility are both better than M (i, e, no other matrix dominates M). – Privacy Quantification – Utility Quantification A number of privacy and utility metrics have been proposed. – Privacy: how accurately one can estimate individual info. – Utility: how accurately we can estimate aggregate info.

Optimization Methods Approach 1: Weighted sum: w 1 Privacy + w 2 Utility Approach 2 – Fix Privacy, find M with the optimal Utility. – Fix Utility, find M with the optimal Privacy. – Challenge: Difficult to generate M with a fixed privacy or utility. Proposed Approach: Multi-Objective Optimization

Optimization algorithm Evolutionary Multi-Objective Optimization (EMOO) The algorithm – Start with a set of initial RR matrices – Repeat the following steps in each iteration Mating: selecting two RR matrices in the pool Crossover: exchanging several columns between the two RR matrices Mutation: change some values in a RR matrix Meet the privacy bound: filtering the resultant matrices Evaluate the fitness value for the new RR matrices. Note : the fitness values is defined in terms of privacy and utility metrics

Illustration

Output of Optimization Privacy Utility Worse Better M1M1 M2M2 M4M4 M3M3 M5M5 M7M7 M6M6 M8M8 The optimal set is often plotted in the objective space as Pareto front.

For First attribute of Adult data

Summary Privacy preserving data mining – Secure multi-party computation protocols – Random response techniques for computation and data collection Knowledge sensitive data mining