Accountable Virtual Machines Andreas Haeberlen, Paarijaat Aditya, Rodrigo Rodrigues, Peter Druschel OSDI 2010 Presenter: Lili Sun 2019/2/24

Outline What is accountable virtual machine (AVM) and why do we need it? What AVM can do and how? How to evaluate its performance? What’s the advantages and disadvantages of AVM?

The concept of AVM and Goals Motivation X X Services Providers X Users

The Concept of AVM and Goals AVM provides strong accountability. It provides users with the capacity to audit the execution of a software system by obtaining a log of the execution, and comparing it to a known-good execution. Goals Detection Evidence

Accountable Virtual Machine AVM approach Bob installs an AVMM and runs the software S AVMM maintains a tamper-evident log and records nondeterministic events Alice receives a message from M ——authenticator Alice periodically audits M If a fault is detected, give the evidence (MR, S, log, am) AVM Software S AVMM replay a tamper-evident log nondeterministic events verify a1 a2 m1 a1 m2 a2

Accountable Virtual Machine Users do not have to check the entire log AVMM can enable users do spot check spot check can save time. an incorrect state transition in an unchecked segment will not be detected. AVMM offers two guarantees Completeness: if the machine is faulty, the audit of M will report a fault and produce the evidence Accuracy: if a machine is not faulty, no audit of M will report a fault

Accountable Virtual Machine AVM can be extended to multiple parties. such as symmetric multi-party scenario or asymmetric multi-party scenario. collect authenticators from other users prevent using the network problem distribute the evidence to other users

AVMM Design The tamper-evident log It is structured as a hash chain, ei := (si, ti, ci, hi) When a user sends a message to M, the user signs the message with her own private key. When M sends a message to the user, the AVMM attaches an authenticator which includes a signature with M’s private key. AVM Software S AVMM logs the signatures and messages removes signatures m1 s1 m1 a1 Acknowledgment Acknowledgment

AVMM Design Accountable Virtual Machine Monitor (AVMM) Recording nondeterministic inputs Detecting inconsistencies Checking snapshot AVM Software S AVMM Nondeterministic inputs logs the signatures and messages m1 s1 m1 a1 Acknowledgment Acknowledgment

Recompute the hash tree AVMM Design Auditing and replay Verify the log's integrity Verify a snapshot Verify the execution syntactic check semantic check AVM Software S AVMM verify Download a snapshot Recompute the hash tree verify Download log: Lij Log segment (ei,…ej) ai aj mi ai mj aj

AVMM Design Syntactic Check Semantic check Determines whether the log itself is well-formed Including the cryptographic signature in the message and the acknowledgement, the sequence of the messages Fast (6.9 seconds) Semantic check Determines whether the information in the log corresponds to a correct execution of MR Instantiates a VM, and initializes with the snapshot Reads Lij, and replays the inputs, and check the outputs Verify the snapshot hashes in Lij against that of the replayed execution Take as long as the application (1,977 seconds)

Application: Cheat Detection in Games The three cheats that are used in Counterstrike are as follows: aimbot, a cheat that works by feeding the game with forged inputs; wallhack, a cheat that violate secrecy; unlimited ammunition, cheats that rely on modifying local in-memory state. AVMs are effective against two specific classes of cheats cheats that need to be installed along with the game; cheats that make the network-visible behavior of the cheater’s machine inconsistent with any correct execution.

Evaluation Prototype Implementation VMM: VMware workstation Extended the VMM to record extra information Adapted code from PeerReview, a system that provide accountability Audit tool implements syntactic check and semantic check If one of them fails, the log and the authenticators will be given to a third party as the evidences.

Evaluation Experiment Setup Five different configurations Three workstations, each for one player Each CPU has four cores and two hyperthreads per core The machines are connected to switch via 1Gbps Ethernet links Five different configurations Barehw, the game runs directly on the hardware, without virtualization vmware-norec, adds the virtual machine monitor without modifications vmware-rec, adds the logging for deterministic replay avmm-nosig, uses AVMM implementation without signatures avmm-rsa768, is the full system as described.

Evaluation Log sizes and contents Figure 3 shows the growth of the AVMM log Figure 4 shows the average log growth rate about the content 8MB/minute or 2.47MB/minute after compression 30% 27% 14% 70% 59%

Evaluation Network traffic AVMM increases network traffic for two reasons first, it adds a cryptographic signature to each packet second, it encapsulates all packets in a TCP connection Compare bare-hw and avmm-rsa768 configuration bare-hw: 22 kbps avmm-rsa768: 215.5 kbps The per-package overhead is much higher

Evaluation Latency AVMM adds some latency to packet transmissions because of the logging and processing of authenticators In AVMM (RSA-768), both the ping and pong are acknowledged Critical threshold of latency for interactive applications is 100ms 192 μs 525 μs 621 μs 2 ms 5 ms

Evaluation CPU utilization AVMM requires additional CPU power for virtualization and for the tamper-evident log. The utilization of HT0 is below 8%, while the average utilization over 8HTs is 12.5% The overhead from the tamper-evident log is relatively low

Evaluation Frame rate The frame rate on the AVMM is 13% lower than the baseline. Generally frame rate is about 60-80 fps, and AVMM is 137fps. Recording in VMware workstation causes the average frame rate to drop 11%.

Evaluation Online auditing Online auditing can affect game performance The frame rate drops from 137fps with no audits to 104fps with 2 audits The audits can leverage the unused cores

Evaluation Spot checking the amount of data that must be transferred over the network, and the time it takes to replay the log segments chunk. The cost grows with the k, and there is an additional fixed cost per chunk for transferring the corresponding memory and disk snapshots.

Advantages and Disadvantages of AVMs AVMs are application independent. AVMs do not have to be trusted by the auditors. AVMs can produce evidence. AVMs are generic and effective against an entire class of cheats. AVMs protect the player’s privacy for anti-cheating. Disadvantages AVMs cannot detect the bug or weakness in the software S. AVMs cannot detect the correctness of inputs. AVMs face additional challenges in the cloud: auditors cannot easily replay the entire execution for lack of resources; accountable services must be able to interact with non-accountable clients it may not be practical to sign every single packet.

Discussion clues For some long-running applications, it is impossible to check the entire log, but the spot check will lose the completeness, so is there a trade-off between completeness, accuracy and effectiveness? The application in this paper is non-cloud based game and not a practical scenario, so it that sufficient to evaluate AVMs? AVMs rely on the server to record all incoming and outgoing messages and assume that all the users agree on a virtual machine in which the application is executed. However, it is not practical in existing cloud platforms, which do not provide this functionality to their clients. Because different operating systems are available for virtual machines, so how to manage the logging of AVMs in the cloud which use a large number of different operating systems?

Thank you!