Building an Encrypted and Searchable Audit Log Brent Waters Dirk Balfanz Glenn Durfee D.K. Smetters
Audit Logs Employed on most server systems Web logs Database logs Provide invaluable access to past activity Hold users accountable for their actions Diagnostics
Desirable Characteristics Tamper Resistant Verifiable Can check that entries are present and have not been altered Data Access Control Entries may be sensitive to individuals or log owner Searchability Search for log on specific criteria e.g keyword search
Desirable Characteristics Tamper Resistant Verifiable Can check that entries are present and have not been altered Data Access Control Entries may be sensitive to individuals or log owner Searchability Search for log on specific criteria e.g keyword search
An Audit Log for a Database System user: Alice Smith keyword: cars keyword: make keyword: ford time: 2003/08/26 23:34:24 authentication clock keyword extraction “select * from cars where make=‘ford’” database audit record creation keywords for audit record log storage (untrusted)
Requirements Data Access Control Searchability Entries must be encrypted on untrusted storage Forward security in case auditing device becomes compromised asymmetric encryption Limit scope of data released to that of the search Searchability Be able to efficiently retrieve entries based on certain criteria We focus on keyword search
A Simple Solution Encrypt all entries with a public key Auditor downloads all entries, then decrypts them, then performs the search
A Simple Solution Encrypt all entries with a public key Auditor downloads all entries, then decrypts them, then performs the search Disadvantages Auditor sees all entries and regardless of what search criteria was All entries must be transmitted from server
Delegating Search Capabilities The investigator requests a capability to search for all entries that were made by the user Alice. “user: Alice Smith” 1 capability for search master secret investigator audit escrow agent The investigator submits the capability to the audit log and receives only entries that the capability matches. capability for search 2 audit record audit record … audit record investigator audit log
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans Auditing Device
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans Auditing Device Encrypted Data Keywords must not be in the clear!
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans master secret audit escrow agent Auditing Device Encrypted Data
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans master secret audit escrow agent Auditing Device Encrypted Data Honda Search Capability
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans master secret audit escrow agent Auditing Device Encrypted Data Honda Search Capability
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans master secret audit escrow agent Auditing Device Encrypted Data Honda Search Capability No information is learned
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans master secret audit escrow agent Auditing Device Encrypted Data
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans master secret audit escrow agent Auditing Device Encrypted Data Alice Search Capability
Searching on Asymmetrically Encrypted Data Document Keywords Alice Ford Loans master secret audit escrow agent Auditing Device Embed decryption in search Encrypted Data Alice Search Capability Document Keywords Alice Ford Loans
Identity Based Encryption (IBE) Public Key is simply a string e.g. bob@parc.com Private Key given from master secret holder(s) Removes need for distribution of public key certificates We use scheme of Boneh and Franklin (2001)
Using IBE to Search on Asymmetrically Encrypted Data Auditing Device Keywords Alice Ford Loans Document
Using IBE to Search on Asymmetrically Encrypted Data Auditing Device Keywords Alice Ford Loans Document Document K
Using IBE to Search on Asymmetrically Encrypted Data Auditing Device Keywords Alice Ford Loans Document FLAG | K “Alice” Document K
Using IBE to Search on Asymmetrically Encrypted Data Auditing Device Keywords Alice Ford Loans Document FLAG | K “Ford” FLAG | K “Alice” Document K
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Auditing Device Keywords Alice Ford Loans Document FLAG | K “Ford” FLAG | K “Alice” Document K
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Auditing Device Keywords Alice Ford Loans Document FLAG | K “Ford” FLAG | K “Alice” Document K FLAG used to test K to decrypt on match
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Auditing Device Keywords Alice Ford Loans Document FLAG | K “Ford” FLAG | K “Alice” Document K FLAG used to test K to decrypt on match Key-privacy propertykeywords kept private
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Auditing Device Keywords Alice Ford Loans Document FLAG | K “Ford” FLAG | K “Alice” Document K FLAG used to test K to decrypt on match Key-privacy propertykeywords kept private “Pairing” operation per keyword
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Alice Search Capability FLAG | K “Ford” FLAG | K “Alice” Document K
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Alice Search Capability FLAG | K “Ford” FLAG | K “Alice” Document K Attempt IBE decryption on each part Test for presence of FLAG
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” 011010… Alice Search Capability FLAG | K “Ford” FLAG | K “Alice” Document K Attempt IBE decryption on each part Test for presence of FLAG
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Alice Search Capability FLAG | K “Ford” 0011100… FLAG | K “Alice” Document K Attempt IBE decryption on each part Test for presence of FLAG
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Alice Search Capability FLAG | K “Ford” FLAG | K “Alice” FLAG | K Document K Attempt IBE decryption on each part Test for presence of FLAG
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Alice Search Capability FLAG | K “Ford” FLAG | K “Alice” FLAG | K Document K Attempt IBE decryption on each part Test for presence of FLAG On match use K to decrypt document Document
Using IBE to Search on Asymmetrically Encrypted Data FLAG | K “Loans” Alice Search Capability FLAG | K “Ford” FLAG | K “Alice” FLAG | K Document K Attempt IBE decryption on each part Test for presence of FLAG On match use K to decrypt document Pairing per keyword in document Document
Scoping of Keywords We want to type keywords e.g. Capability to search on entries about “Alice” vs. those made by “Alice” Solution: Prefix keywords with type “user:Alice” “kw:Alice”
Performance Encryption Search/Decryption One pairing per keyword in document One exponentiation per keyword Search/Decryption One pairing per keyword per document
Optimizations Cache pairings of frequently used keywords eg. ê(“user:Alice”,sP) Only need a pairing per new keyword on encryption In limit exponentiation per keyword is dominant cost
Optimizations Cache pairings of frequently used keywords eg. ê(“user:Alice”,sP) Only need a pairing per new keyword on encryption In limit exponentiation per keyword is dominant cost Reuse randomness for IBE encryption within one document Okay since cannot use same public key per document In decryption only one pairing per document Save storage in log
Indexing Incremental update of an index on untrusted storage is insecure
Indexing Incremental update of an index on untrusted storage is insecure Keywords Alice Ford Loans Document Index
Indexing Incremental update of an index on untrusted storage is insecure Keywords Alice Ford Loans Document Index
Indexing Incremental update of an index on untrusted storage is insecure Keywords Alice Loans Washington Document Index
Indexing Incremental update of an index on untrusted storage is insecure Keywords Alice Loans Washington Document Index
Indexing Build local index on auditing device and flush out to storage
Indexing Build local index on auditing device and flush out to storage FLAG | … “Sam” FLAG | K | K’’ “Alice” Document K Document K’ Document K’’
Indexing Longer index is held in auditing device more information leaked on device compromise
Implementation Implemented a logging system for MySQL database queries Goal to protect individual’s privacy Used Stanford IBE library Pairing cost ~80ms on current machines
Related Work Searching on Encrypted Data Boneh, Crescenzo, Ostrovsky and Persiano (2003) Song, Wagner and Perrig (2000) Goh (2003) Identity Based Encryption Boneh and Franklin (2001)
Conclusion Tension between data access control and searchability in audit logs Asymmetric scheme for searching on encrypted data Explored optimizations for practical systems
Searching on Asymmetrically Encrypted Data audit escrow agent master secret Keywords Alice Ford Loans Document Auditing Device Encrypted Data Bob Search Capability Keywords Alice Ford Loans Document
Using IBE to Search on Asymmetrically Encrypted Data Document K FLAG | K “Alice” “Ford” “Loans” Auditing Device Document Keywords Alice Ford Loans FLAG used to test K to decrypt on match Key-privacy propertykeywords kept private “Pairing” operation per keyword
Using IBE to Search on Asymmetrically Encrypted Data Document K FLAG | K “Alice” “Ford” “Loans” Alice Search Capability Attempt IBE decryption on each part Test for presence of FLAG On match use K to decrypt document Pairing per test