Digital Forensics Dr. Bhavani Thuraisingham The University of Texas at Dallas Detection and Analysis of Database Tampering November 2012

Outline: Review of Two Papers l Richard T. Snodgrass, Stanley Yao and Christian Collberg, "Tamper Detection in Audit Logs," In Proceedings of the International Conference on Very Large Databases, Toronto, Canada, August–September 2004, pp. 504– Tamper Detection in Audit Logs l Did the problem occur? (e.g. similar to intrusion detection) l Kyri Pavlou and Richard T. Snodgrass, "Forensic Analysis of Database Tampering," in Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD), pages , Chicago, June, l Who caused the problem (e.g., similar to digital forensics analysis)

"Tamper Detection in Audit Logs“ Overview of Paper l Emphasize the fact that audit logs be correct and verifiable - Required now by several US Federal laws (e.g. Sarbanes-Oxley, HIPAA, etc.) l Review of existing audit log techniques l Presentation of their basic idea (converting the audit log to a transaction time database with periodic validation and notarization) l Give some performance enhancements (e.g. opportunistic hashing, linked hashing) l Performance graphs and final summary

Transaction Time Database l A subset of “Temporal Databases” - l A temporal database is a database that tracks, among other things, two different time parameters: valid-time and transaction-time. - Valid time denotes the time period during which a fact is true with respect to the real world (i.e. “real” time) - Transaction time refers to the time period during which a fact is stored in the database. l Bitemporal data combines both Valid and Transaction Time.

Transaction Time Database l Records and retains the history of its content. [1] - All past states are retained and can be reconstructed from the information in the DB. l Past state reconstruction enabled by the append only property: [1] - All new information is added only - No information is ever deleted. l In addition, the transaction time component must be auditable. That is, - An audit log is maintained - Can be examined later by a validator l Ultimate goal is to have enough information to both: - detect a bad event - determine exactly when, how, and by whom it occurred.

Transaction Time Database l Transaction time table contains all the columns a normal database table might have, with two extra fields: Start and Stop. - START:tracks when the data item was added to the database (transaction time) - STOP:tracks different states of the row (tuple) l Example operations that maintain history: - Deletion: STOP marked deleted, but row is retained - Modification: Deletion of old value; insertion of new l Invisible to user; maintained by DBMS. l Extra fields are carried for each tuple (row).

"Tamper Detection in Audit Logs“ Main Steps of Basic Algorithm l On each modification of a tuple, the DBMS: - Gets a timestamp for the modification - Computes a cryptographically strong one-way hash of the (new) data and the time stamp together. - Sends that value to a trusted notarization service, which sends back a unique Notary ID based on that value. - The Notary ID is then stored with the tuple. l If the data or timestamp are modified, the ID will be inconsistent with the new tuple (i.e. detected when rehashed and re-notarized). - Holds even if intruder has access to the hash function. He can calculate a new hash, but it won’t match the ID. l It is very important that the ID cannot be calculated from the data in the database (i.e. must be calculated by an independent and trusted source): - This prevents an intruder from changing the database and then recalculating the ID.

" Tamper Detection in Audit Logs“ Validation l An independent and trusted audit log validation service can then be used to verify the integrity of the DB. l For each tuple (basic algorithm), the validation service will rehash the data and time-stamp, recalculate the ID, and compare. Called a “Validation Event” (VE). Inconsistencies are reported as an “Corruption Event” (CE). l Modern systems can update thousands of tuples per second, leading to time efficiency problems. l Optimizations seek to minimize the time spent calculating hashes and interacting with the notarization service - Opportunistic hashing: Reduce the interactions with the notary to one per transaction, rather than to one per tuple. - Linked hashing l Final commit hash done at midnight each day. Reduces the interactions with the notary to one per day. creates a “hash chain” that can be used in later analysis

Hashing Functions Verifying the accuracy of the copy l A hashing function can be used to generate a “digest” specific for each file. l The digest is usually a hexadecimal number that is, with a high probability, unique for each file. l A hashing function is secure if, for a given algorithm, it is computationally infeasible - to find a message that corresponds to a given message digest, or - to find two different messages that produce the same message digest (i.e. “collision”) l In general, any change to a message will, with a very high probability, result in a different message digest. - Failure called a “collision” l MD5 Hash Function - Most commonly used (although it has been shown to have flaws (i.e. collisions)) - developed by Ronald Rivest, produces a 32 character (16 digit) hex number.

Hashing Functions Reduce work load l Hashing functions can be used to cut down on the number of files that have to be analyzed. l Databases of known hash results are maintained (e.g. KFF – “Known File Filter” in FTK) l Can be used to identify “Known Bad” files - Hacking tools - Training manuals - Contraband photographs l Ignore “Known Good” files - Microsoft Windows files - Standard application files - Standard build files (corporate server deployments)

"Tamper Detection in Audit Logs“ Summary of main points l Main contributions of first paper: - the DBMS can maintain a transaction-time audit log in the background - transactions can be cryptographically hashed to generate a secure, one-way hash of the transaction - the transaction hash can be notarized by an independent and trusted service to generate a unique and secure ID value. - optimizations that reduce the overhead

Details: Tamper Detection in Audit Logs l A variety of federal laws (e.g., Code of Federal Regulations for the Food and Drug Administration, Sarbanes-Oxley Act, Health Insurance Portability and Accountability Act, Canada's PIPEDA) and standards (e.g., Orange Book for security) mandate audit logs. l The correctness of the auditing records themselves is critical. l Cannot make the assumption that the audit records are secure and tamperproof l Therefore need a mechanism to validate the audit records – these records are created/modified during transaction execution

Current Audit Log Techniques l The traditional way to protect logging data from tampering is to write it to an append-only device, such as a Write Once Read Multiple (WORM) optical drive or a continuous-feed printer. l The security of such schemes assumes, however, that the computing site will not be compromised. l If this is a possible attack scenario the logging data can be sent to a remote site over the network, so called remote logging. l Log replication can be used to send the data to several hosts to require the attacker to physically compromise several sites. l Schneier and Kelsey describe a secure audit log system using hash functions and encryption but does not work for databases.

Database Audit Log l Authors show how a database can be protected by having the DBMS maintain the audit log in the background and by using cryptographic techniques to ensure that any alteration of prior entries in this audit log can be detected. l This approach thus applies to any application that uses a DBMS. l As an example, assume that we are a pharmacological research firm that develops new drugs, providing data to the FDA for approval of those drugs. As part of this effort we have a relational table, Administer, that records what drugs were administered to which patients during a drug trial. l 62 FR requires a computer-generated, time-stamped audit trail.

Database Audit Log l Authors define the table as follows in MySQL - CREATE TABLE Administer (...) AS TRANSACTIONTIME TYPE = BDB AUDITABLE = 1 l The first line - which also species the columns and primary and foreign key constraints for the table is supported by the conventional MySQL release, as is the third line, which specifies that the Berkeley DB storage system be used. l The second line specifies that this Administer table includes transaction-time support (this is an open-source extension that we have implemented). l A transaction-time database records the history of its content. All past states are retained and can be reconstituted from the information in the database. This is ensured through the append only property of a transaction-time database: modifications only add information; no information is ever deleted. It is this basic property that the authors exploit to validate the audit log (in fact, the table is the audit log).

Database Audit Log l The last line species that this transaction-time table be auditable, that is, that the system take additional steps so that an audit log in maintained and so that later a separate audit log validator can examine the database and state conclusively whether the audit log has been compromised. l The authors have implemented support for auditable tables and independent validation in MySQL and Berkeley DB. l It is important to emphasize that the applications that update and query this table need not be aware of the last three lines of the CREATE TABLE statement. l Behind the scenes, transparently to the application, the DBMS creates an audit log and ensures auditability.

Threat Model l Authors assume that Trusted Computing Base (TCB) consisting of correctly booted and functioning hardware and a correctly installed operating system and DBMS. That is, the TCB is correctly functioning until such a time T when a penetration occurs. l Similarly, until time T the DBMS is created, maintained, and operated in a secure manner, and all network communication is performed through secure channels (such as SSL), ensuring the correctness of the internal state of the DBMS. l Since the DBMS is a transaction-time database, all previous states can be reconstructed. l A penetration by an adversary “Bob” can take many forms. An intruder (or an insider) who gains physical access to the DBMS server will have full freedom to corrupt any database file, including data, timestamps, and audit logs stored in tuples.

The Approach l The user application performs transactions on the database, which insert, delete, and update the rows of current state. Behind the scenes, the DBMS retains for each tuple hidden Start and Stop times, recording when each change occurred. The DBMS ensures that only the current state of the table is accessible to the application, with the rest of the table serving as the audit log. l Alternatively, the table itself could be viewed by the application as the audit log. In that case, the application only makes insertions to the audited table; these insertions are associated with a monotonically increasing Start time. l The approach and implementation support both usages. l The basic idea is to store a check field" in each tuple. This check field cannot be computed directly from the data (and timestamps) of the tuple, because then Bob could simply recompute the check field after he has altered the tuple. l Indeed, if needed he could replay all of the transactions, making whatever changes he wanted to the data or the timestamps.

The Approach l Authors use a digital notarization service that, when provided with a digital document, provides a notary ID1. l Later, during audit log validation, the notarization service can ascertain, when presented with supposedly unaltered document and the notary ID, whether that document was notarized, and if so, when. l On each modication of a tuple, the DBMS obtains a timestamp, computes a cryptographically strong oneway hash function of the (new) data in the tuple and the timestamp, and sends that hash value, as a digital document, to the notarization service, obtaining a notary ID. The DBMS stores that ID in the tuple. l Later, Bob gets access to the database. If Bob changes the data or a timestamp, the ID will now be inconsistent with the rest of the tuple. l Bob cannot manipulate the data or timestamp so that the ID remains valid, because the hash function is one-way.

The Approach l Note that this holds even when Bob has access to the hash function itself. Bob can instead compute a new hash value for the altered tuple, but that hash value won't match the one that was notarized. l If an independent audit log validation service was provided with the database that service could, for each tuple, hash the data and the timestamp, provide it with the ID to the notarization service, which will check the notarization time with the stored timestamp. l There is a timing issue in that the database obtains a timestamp first and later notarizes the tuple. The notarization time will be slightly later than the timestamp. Also, there will be many tuples with identical timestamps, as they were all modfiied within a single transaction. l The validation service would then report whether the database and the audit log are consistent. If not, either or both had been compromised.

The Approach l The system is secure until Bob gets access, at which point he has access to everything: the DBMS, the operating system, the hardware, and the data in the database. l It is assumed that the notarization and validation services remain in the trusted computing base. This can be done by making them geographically and perhaps organizationally separate from the DBMS and the database. l The basic mechanism provides correct tamper detection. If Bob modifies even a single byte of the data or its timestamp, the independent validator will detect a mismatch with the notarized document, thereby detecting the tampering. l Bob could simply re-execute the transactions, making whatever changes he wanted, and then replace the original database with his altered one. However, the notarized documents would not match in time. l Avoiding tamper detection comes down to inverting the cryptographically- strong one-way hash function.

Performance Improvements l Different types of hash function l Different size storage for tuples l Minimize interactions l Simulation studies l Details discussed in paper

“Forensic Analysis of Database Tampering” Overview of Paper l Motivates the need for forensic analysis (e.g. legal requirements, need to determine who, what, and when). l Reviews the contributions of first paper l Defines the “Corruption Diagram” and gives an example l Gives details of and comparisons between the four algorithms discussed l Describes the notion of “forensic strength” l Related work, summary

From:

Forensic Analysis of Database Tampering l Audit log tampering technique provides exactly one bit of information: has the audit log been tampered? l Authors introduce a schematic representation termed a “corruption diagram” for analyzing an intrusion. l They then consider how additional validation steps provide a sequence of bits that can dramatically narrow down the “when” and “where.” l They examine the corruption diagram for this initial approach; this diagram is central in all of our further analyses. We characterize the “forensic strength” of this algorithm, defined as the reduction in area of the uncertainty region in the corruption diagram.

Forensic Analysis of Database Tampering l Authors look at the more complex case in which the timestamp of the data item is corrupted, along with the data. l Such an action by the intruder turns out to greatly decrease the forensic strength. Along the way, we identify some configurations that turn out not to improve the forensic strength, thus helping to cull the most appropriate alternatives. l They then consider computing and notarizing additional sequence of hash values. For each successively more powerful forensic analysis algorithm, we provide a formal/diagrammatic analysis of its forensic strength. The above-mentioned algorithm is the monochromatic algorithm; they also consider the RGB algorithm and the polychromatic algorithm. This last algorithm can efficiently extract a good deal of information concerning a corruption event.

Forensic Analysis of Database Tampering l To explain forensic analysis, authors introduce the Corruption l Diagram, which is a graphical representation of CE(s) in terms of the temporal-spatial dimensions of a database. They have found these diagrams to be very helpful in understanding and communicating the many forensic algorithms Consider the simplest case. During validation, they have detected a corruption event. Though they don’t know it (yet), assume that this corruption event is a single-locus CE. Furthermore, assume that the CE just altered the data of a tuple; no timestamps were changed.

Definitions l Corruption event (or CE), which is any event that corrupts the data and compromises the database. The corruption event could be due to an intrusion, some kind of human intervention, a bug in the software (be it the DBMS or the file system or somewhere in the operating system), or a hardware failure, either in the processor or on the disk. l There exists a one-to-one correspondence between a CE and its corruption time (tc), which is the actual time instant (in seconds) at which a CE has occurred. l The CE was detected during a validation of the audit log by the Notarization Service (NS), termed a validation event (or VE). A validation can be scheduled (that is, is periodic) or could be an ad hoc VE. The time (instant) at which a VE occurred is termed the time of validation event, and is denoted by tv. l Tampering is indicated by a validation failure, in which the validation service returns false for the particular query of a hash value and a notarization time.

Definitions l What is desired is a validation success, in which the NS returns true, stating that everything is OK: the data has not been tampered. l The validator compares the hash value it computes over the data with the hash value that was previously notarized. A notarization event (or NE) is the notarization of a document (specifically, a hash value) by the notarization service. l As with validation, notarization can be scheduled (is periodic) or can be an ad hoc notarization event. Each NE has an associated notarization time (tn), which is a time instant. l Forensic analysis involves temporal detection, the determination of the corruption time, tc. Forensic analysis also involves spatial detection, the determination of “where,” that is, the location in the database of the data altered in a CE. l The finest granularity of the corruption data locus would be an explicit attribute of a tuple, or a particular timestamp attribute. We term this data that has been corrupted the corruption locus data (lc).

Definitions l Forensic analysis involves temporal detection, the determination of the corruption time, tc. Forensic analysis also involves spatial detection, the determination of “where,” that is, the location in the database of the data altered in a CE. l The finest granularity of the corruption data locus would be an explicit attribute of a tuple, or a particular timestamp attribute. We term this data that has been corrupted the corruption locus data (lc).

“Forensic Analysis of Database Tampering” Basic Definitions l Corruption Event (CE) - any event that corrupts data or compromises the database l Corruption Time (t c ) l Notarization Event (NE) (t n ) - Notarization Interval (I N ) l Validation Event (VE) (t v ) - Validation Interval (I V ) l Corruption locus data (l c ) - the data that have been corrupted l Locus time (t l ) - the time when the locus data (l c ) were stored

Basic Definitions l A “Corruption Diagram” is used to perform the analysis. l Terminology (based on temporal database): - x axis is the “transaction time” (“where” axis) - y axis is the actual (i.e. “valid”) time (“when” axis) - action axis – 45 degree line relating the transaction time and valid time - t fvf – time of “first validation failure” – time when corruption of log first detected by a VE l See example, p112 of paper (data only event) l Once corruption has been detected, the forensic analyzer begins working. l Objective: to define the corruption region, i.e. the bounds on the “where” and “when” of the CE, as narrowly as possible. l The paper presents four algorithms for doing this: - Trivial Forensic Analysis algorithm - Monochromatic Forensic Analysis algorithm - RGB (Red-Green-Blue) algorithm - Polychromatic algorithm

“Forensic Analysis of Database Tampering” Monochromatic Forensic Analysis l Let’s use the “Monochromatic Forensic Analysis” algorithm to define the Corruption Region: - the analyzer rehashes the log from the beginning to determine the time of “most recent validation success” (t rvs ). This is the “Lower Spatial Bound” = LSB. - “Upper Spatial Bound” (USB)= LSB + I N - “Lower Temporal Bound” (LTB)= t FVF – I V - “Upper Temporal Bound” (UTB)= t FVF l Time of Occurrence: - Retroactive CE: locus time (t l ) occurs before the second to last validation event (VE) - Introactive CE: t l occurs after the next to last VE l Type of Corruption: - data only - backdating:a timestamp is changed to indicate a time earlier than the tuple time - Postdating timestamp changed to indicate a later time

“Forensic Analysis of Database Tampering” Classification of Corruption Events l This leads to 6 different types of Corruption Events: l Each of the four algorithms handles these events differently l See Fig 4 for an example for postdating and backdating CE’s. X Retroactive Introactive Data only Backdating Postdating

“Forensic Analysis of Database Tampering” Algorithms l Rest of the paper describes in detail the four algorithms: - Trivial Forensic Analysis algorithm l when t FVF detected, return entire upper triangle - Monochromatic Forensic Analysis algorithm l calculate LSB, USB, LTB, and UTB as in example - RGB (Red-Green-Blue) algorithm l localizes Corruption Region more tightly by re-hashing selected portions of the database (instead of the entire hash chain) - Polychromatic algorithm l adds additional hash chains to reduce the size of the corruption region to one day l The forensic strength of an algorithm is determined by: - the effort or work of the analysis, i.e. the effort it takes to calculate t c, t l, and t p - the region area - the uncertainty

The Approach l To explain forensic analysis, authors introduce the Corruption Diagram, which is a graphical representation of CE(s) in terms of the temporal-spatial dimensions of a database. They have found these diagrams to be very helpful in understanding and communicating the many forensic algorithms l Once the corruption has been detected, a forensics analyzer determines the corruption region, the bounds on where and when of the corruption using the corruption diagrams and the locus. l From this validation there is 1 bit information and that is validation failure. l Need to go to other sources of information to learn about the failure l Backup copy of the database is one such source

The Approach l One approach is to compare tuple by tuple the backup with the current content of the database to determine the CE, (where is it?), also look at the time. Form where and when then the SSO could determine who. l Paper then describes more sophisticated algorithms, monochromatic, RGB and Polychromatic l Paper shows thorough a formal forensic strength comparison that each successive algorithm adds extra work in the form of main-memory processing, but that the resulting additional precision in the obtained information more than counterbalances this extra work l The polychromatic algorithm in particular is able to determine the “where”, the “when,” and the “to when” of a tampering quite precisely l Essentially the approach is to use corruption diagrams as a way of visualizing corruption events and forensic analysis algorithms. showing through

References l Richard T. Snodgrass, Stanley Yao and Christian Collberg, "Tamper Detection in Audit Logs," In Proceedings of the International Conference on Very Large Databases, Toronto, Canada, August–September 2004, pp. 504–515. l Kyri Pavlou and Richard T. Snodgrass, "Forensic Analysis of Database Tampering," in Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD), pages , Chicago, June, l Additional paper for reading: Kyri Pavlou and Richard. T. Snodgrass, "The Pre-images of Bitwise AND Functions in Forensic Analysis,'' TimeCenter TR 87, October, l

Reading for Lecture November 2, 2011 l l XIRAF – XML-based indexing and querying for digital forensics l Selective and intelligent imaging using digital evidence bags l l Detecting false captioning using common-sense reasoning