1. Differential Privacy on Linked Data: Theory and Implementation Yotam Aron

2. Table of Contents Introduction Differential Privacy for Linked Data SPIM implementation Evaluation

3. Contributions Theory on how to apply differential privacy to linked data. Experimental implementation of differential privacy on linked data. Overall privacy module for SPARQL queries.

4. Introduction

5. Overview: Why Privacy Risk? Statistical data can leak privacy. Mosaic Theory: Different data sources harmful when combined. Examples: Netflix Prize Data set GIC Medical Data set AOL Data logs Linked data has added ontologies and meta-data, making it even more vulnerable. Linked data has added ontologies and meta-data, making it even more vulnerable

6. Current Solutions Accountability: Privacy Ontologies Privacy Policies and Laws Problems: Requires agreement among parties. Does not actually prevent breaches, just a deterent. Heterogeneous

7. Current Solutions (Cont’d) Anonymization Delete “private” data K – anonymity (Strong Privacy Guarantee) K – anonymity Problems Deletion provides no strong guarantees Must be carried out for every data set What data should be anonymized? High computational cost (k-anonimity is np-hard)(k-anonimity is np-hard)

8. Differential Privacy

10. How Achieved? Add noise to result. Simplest: Add Laplace noise

11. Laplace Noise Parameters

12. Other Benefit of Laplace Noise

13. Benefits of Differential Privacy Strong Privacy Guarantee Mechanism-Based, so don’t have to mess with data. Independent of data set’s structure. Works well with for statistical analysis algorithms.

14. Problems with Differential Privacy Potentially poor performance Complexity (especially for non-linear functions) Noise Only works with statistical data (though this has fixes) How to calculate sensitivity of arbitrary query?

15. Differential Privacy for Linked Data

16. Differential Privacy and Linked Data Want same privacy guarantees for linked data without, but no “records.” What should be “unit of difference”? One triple All URIs related to person’s URI All links going out from person’s URI

17. Differential Privacy and Linked Data Want same privacy guarantees for linked data without, but no “records.” What should be “unit of difference”? One triple All URIs related to person’s URI All links going out from person’s URI

18. Differential Privacy and Linked Data Want same privacy guarantees for linked data without, but no “records.” What should be “unit of difference”? One triple All URIs related to person’s URI All links going out from person’s URI

19. Differential Privacy and Linked Data Want same privacy guarantees for linked data without, but no “records.” What should be “unit of difference”? One triple All URIs related to person’s URI All links going out from person’s URI

20. “Records” for Linked Data Reduce links in graph to attributes Idea: Identify individual contributions from a single individual to total answer. Find contribution that affects answer most.

21. “Records” for Linked Data Reduce links in graph to attributes, makes it a record. P1P2 Knows PersonKnows P1P2

22. “Records” for Linked Data Repeated attributes and null values allowed P1P2 Knows P3P4 Loves Knows

23. “Records” for Linked Data Repeated attributes and null values allowed (not good RDBMS form but makes definitions easier) PersonKnows Loves P1P2NullP4 P3P2P4Null

24. Query Sensitivity in Practice Need to find triples that “belong” to a person. Idea: Identify individual contributions from a single individual to total answer. Find contribution that affects answer most. Done using sorting and limiting functions in SPARQL

25. Example COUNT of places visited P1 P2 MA S2 S3 State of Residence S1 Visited

26. Example COUNT of places visited P1 P2 MA S2 S3 State of Residence S1 Visited

27. Example COUNT of places visited P1 P2 MA S2 S3 State of Residence S1 Visited Answer: Sensitivity of 2

28. Using SPARQL Query: (COUNT(?s) as ?num_places_visited) WHERE{ ?p :visited ?s }

29. Using SPARQL Sensitivity Calculation Query (Ideally): SELECT ?p (COUNT(ABS(?s)) as ?num_places_visited) WHERE{ ?p :visited ?s; ?p foaf:name ?n } GROUP BY ?p ORDER BY ?num_places_visited LIMIT 1

30. In reality… LIMIT, ORDER BY, GROUP BY doesn’t work together in 4store… For now: Don’t use LIMIT and get top answers manually. I.e. Simulate using these in python Would like to keep it on sparql-side ideally so there is less transmitted data (e.g. on large data sets)

31. (Side rant) 4store limitations Many operations not supported in unison E.g. cannot always filter and use “order by” for some reason Severely limits the types of queries I could use to test. May be desirable to work with a different triplestore that is more up-to-date (ARQ). Didn’t because wanted to keep code in python. Also had already written all code for 4store

32. Problems with this Approach Need to identify “people” in graph. Assume, for example, that URI with a foaf:name is a person and use its triples in privacy calculations. Imposes some constraints on linked data format for this to work. For future work, maybe there’s a way to automatically identify private data, maybe by using ontologies. Complexity is tied to speed of performing query over large data set.

33. …and on the Plus Side Model for sensitivity calculation can be expanded to arbitrary statistical functions. e.g. dot products, distance functions, etc. Relatively simple to implement using SPARQL 1.1

34. Differential Privacy Protocol Differential Privacy Module Client SPARQL Endpoint Scenario: Client wishes to make standard SPARQL 1.1 statistical query. Client has Ɛ “budget” of overall accuracy for all queries.

35. Differential Privacy Protocol Differential Privacy Module Client SPARQL Endpoint Step 1: Query and epsilon value sent to the endpoint and intercepted by the enforcement module. Query, Ɛ > 0

36. Differential Privacy Protocol Differential Privacy Module Client SPARQL Endpoint Step 2: The sensitivity of the query is calculated using a re-written, related query. Sens Query

37. Differential Privacy Protocol Differential Privacy Module Client SPARQL Endpoint Step 3: Actual query sent. Query

38. Differential Privacy Protocol Differential Privacy Module Client SPARQL Endpoint Step 4: Result with Laplace noise sent over. Result and Noise

39. Design of Privacy System

40. SPARQL Privacy Insurance Module i.e. SPIM Use authentication, AIR, and differential privacy in one system. Authentication to manage Ɛ-budgets. AIR to control flow of information and non-statistical data. Differential privacy for statistics. Goal: Provide a module that can integrate into SPARQL 1.1 endpoints and provide privacy.

41. Design Triplestore User Data Privacy Policies SPIM Main Process AIR Reasoner Differential Privacy Module HTTP Server OpenID Authentication

42. HTTP Server and Authentication HTTP Server: Django server that handles http requests. OpenID Authentication: Django module. HTTP Server OpenID Authentication

43. SPIM Main Process Controls flow of information. First checks user’s budget, then uses AIR, then performs final differentially-private query. SPIM Main Process

44. AIR Reasoner Performs access control by translating SPARQL queries to n3 and checking against policies. Can potentially perform more complicated operations (e.g. check user credentials) Privacy Policies AIR Reasoner

45. Differential Privacy Works as discussed in previous slides. Contains users and their Ɛ- values. Differential Privacy Module User Data

46. Evaluation

47. Three things to evaluate: Correctness of operation Correctness of differential privacy Runtime Used a anonymized clinical database as the test data and added fake names, social security numbers, and addresses.

48. Correctness of Operation Can the system do what we want? Authentication provides access control AIR restricts information and types of queries Differential privacy gives strong privacy guarantees. Can we do better?

49. Use Case Used in Thesis Clinical database data protection HIPAA: Federal protection of private information fields, such as name and social security number, for patients. 3 users Alice: Works in CDC, needs unhindered access Bob: Researcher that needs access to private fields (e.g. addresses) Charlie: Amateur researcher to whom HIPAA should apply Assumptions: Django is secure enough to handle “clever attacks” Users do not collude, so can allocate individual epsilon values.

50. Use Case Solution Overview What should happen: Dynamically apply different AIR policies at runtime. Give different epsilon-budgets. How allocated: Alice: No AIR Policy, no noise. Bob: Give access to addresses but hide all other private information fields. Epsilon budget: E1 Charlie: Hide all private information fields in accordance with HIPAA Epsilon budget: E2

51. Use Case Solution Overview Alice: No AIR Policy Bob: Give access to addresses but hide all other private information fields. Epsilon budget: E1 Charlie: Hide all private information fields in accordance with HIPAA Epsilon budget: E2

52. Example: A Clinical Database Client Accesses triplestore via HTTP server. OpenID Authentication verifies user has access to data. Finds epsilon value, HTTP Server OpenID Authentication

53. Example: A Clinical Database AIR reasoner checks incoming queries for HIPAA violations. Privacy policies contain HIPAA rules. Privacy Policies AIR Reasoner

54. Example: A Clinical Database Differential Privacy applied to statistical queries. Statistical result + noise returned to client. Differential Privacy Module

55. Correctness of Differential Privacy Need to test how much noise is added. Too much noise = poor results. Too little noise = no guarantee. Test: Run queries and look at sensitivity calculated vs. actual sensitivity.

56. How to test sensitivity? Ideally: Test noise calculation is correct Test that noise makes data still useful (e.g. by applying machine learning algorithms). Fort his project, just tested former Machine learning APIs not as prevalent for linked data. What results to compare to?

57. Test suite 10 queries for each operation (COUNT, SUM, AVG, MIN, MAX) 10 different WHERE CLAUSES Test: Sensitivity calculated from original query Remove each personal URI using “MINUS” keyword and see which removal is most sensitive

58. Example for Sens Test Query: PREFIX rdf: PREFIX rdfs: PREFIX foaf: PREFIX mimic: SELECT (SUM(?o) as ?aggr) WHERE{ ?s foaf:name ?n. ?s mimic:event ?e. ?e mimic:m1 "Insulin". ?e mimic:v1 ?o. FILTER(isNumeric(?o)) }

59. Example for Sens Test Sensitivity query: PREFIX rdf: PREFIX rdfs: PREFIX foaf: PREFIX mimic: SELECT (SUM(?o) as ?aggr) WHERE{ ?s foaf:name ?n. ?s mimic:event ?e. ?e mimic:m1 "Insulin". ?e mimic:v1 ?o. FILTER(isNumeric(?o)) MINUS {?s foaf:name "%s"} } % (name)

60. Results Query 6 - Error

61. Runtime Queries were also tested for runtime. Bigger WHERE clauses More keywords Extra overhead of doing the calculations.

62. Results Query 6 - Runtime

63. Interpretation Sensitivity calculation time on-par with query time Might not be good for big data Find ways to reduce sensitivity calculation time? AVG does not do so well… Approximation yields too much noise vs. trying all possibilities Runs ~4x slower than simple querying Solution 1: Look at all data manually (large data transfer) Solution 2: Can we use NOISY_SUM / NOISY_COUNT instead?

64. Conclusion

65. Contributions Theory on how to apply differential privacy to linked data. Experimental implementation of differential privacy. Verification that it is applied correctly. Overall privacy module for SPARQL queries. Limited but a good start Other: Updated sparql to n3 translation to Sparql version 1.1 Expanded upon IARPA project to create policies against statistical queries.

66. Shortcomings and Future Work Triplestores need some structure for this to work Personal information must be explicitly defined in triples. Is there a way to automatically detect what triples would constitute private information? Complexity Lots of noise for sparse data. Can divide data into disjoint sets to reduce noise like PINQ does Use localized sensitivity measures? Third party software problems Would this work better using a different Triplestore implementation?

67. Other work Other implementations: PINQ Airavat PDDP Some of the Theoretical Work Out There Differential privacy paper Exponential Mechanism Noise Calculation Differential Privacy and Machine Learning

68. Appendix: Results Q1, Q2 Q2ErrorQuery_TimeSens_Calc_Time COUNT00.0158231260.011798859 SUM00.0102989670.01198101 AVG868.83790.0103349690.04432416 MAX00.0106458660.012124062 MIN00.0105249880.012120962

69. Appendix: Results Q3, Q4 Q3ErrorQuery_TimeSens_Calc_Time COUNT00.0079278950.00800705 SUM00.0075299740.007997036 AVG375.82530.007630110.030416012 MAX00.0074510570.008117914 MIN00.0075120930.008100986 Q4ErrorQuery_TimeSens_Calc_Time COUNT00.010487080.012546062 SUM00.011237860.012809038 AVG860.910.0112869740.048202038 MAX00.011457920.01297307 MIN00.0113921170.012881041

70. Appendix: Results Q5, Q6 Q5ErrorQuery_TimeSens_Calc_Time COUNT00.080810070.098078012 SUM00.0856788160.097680092 AVG115099.50.0872709750.373119116 MAX00.0849039550.097922087 MIN00.0832138060.098366022 Q6ErrorQuery_TimeSens_Calc_Time COUNT00.1366059780.153807878 SUM00.1399950980.155878067 AVG115118.40.1398811340.616436958 MAX00.1483600140.160467148 MIN00.1446359160.158998966

71. Appendix: Results Q7, Q8 Q7ErrorQuery_TimeSens_Calc_Time COUNT00.0061001780.004678965 SUM00.0042600630.004747868 AVG00.0042839050.017117977 MAX00.0041031840.004703999 MIN00.0041880610.004717112 Q8ErrorQuery_TimeSens_Calc_Time COUNT00.0021829610.002643108 SUM00.0020921230.002592087 AVG00.0020759110.002662182 MAX00.002079010.002576113 MIN00.0020489690.002597094

72. Appendix: Results Q9, Q10 Q9ErrorQuery_TimeSens_Calc_Time COUNT00.0049209590.010298014 SUM00.0048220160.010312796 AVG0.000370.0049099920.024574041 MAX00.0048432350.01032114 MIN00.0048930640.010319948 Q10ErrorQuery_TimeSens_Calc_Time COUNT00.0123658180.014447212 SUM00.0130660530.014631987 AVG860.910.0131669040.056000948 MAX00.0133540630.014893055 MIN00.0133290290.014914989