1. Computer Science HyperSentry: Enabling Stealthy In-context Measurement of Hypervisor Integrity Ahmed M. Azab, Peng Ning, Zhi Wang, Xuxian Jiang North Carolina State University Xiaolan Zhang Nathan C. Skalsky IBM T.J. Watson Research Center IBM Systems & Technology Group 1

2. Computer Science Background Hypervisors are critical to virtualized platforms Can hypervisors be blindly trusted? –Two backdoors in Xen [BlackHat 2008] –Xen 3.x: 10 Secunia advisories; 17 vulnerabilities. –VM Ware ESX 3.x: 49 Secunia advisories; 362 vulnerabilities. –Existing hypervisor's code base is growing Need mechanisms to ensure hypervisor integrity –HyperSentry: Measure the hypervisor integrity at runtime 2

3. Computer Science Challenges A fundamental problem –How to measure the integrity of the highest privileged software? Hypervisor has full control of the software system –Scrubbing attacks –Tampering with the measurement agent –Tampering with the measurement results Relying on a higher privileged software goes back to the same problem 3

4. Computer Science The HyperSentry Approach HyperSentry –A generic framework to stealthily measure the integrity of a hypervisor in its context Key ideas –Allow the measurement software to gain the highest privilege temporarily –Measurement is triggered stealthily Scrubbing attacks –Isolate measurement results from the hypervisor 4

5. Computer Science Foundation of HyperSentry System Management Mode (SMM) –x86 operating mode for system management functions –SMRAM can be locked to prevent all access to it except from within the SMM Hypervisor cannot access the SMRAM once locked –System Management Interrupt (SMI) only handled by SMI handler in SMRAM SMI bypasses hypervisor’s control –Provides the isolation required for HyperSentry –Main challenges How to retrieve the needed context for hypervisor? How to attest to the measurement output? 5

6. Computer Science Foundation of HyperSentry (Cont’d) Out-of-band communication channel –Triggers a System Management Interrupt (SMI) –Out of the control of the hypervisor –Example: IPMI Uses a microcontroller on the motherboard Hard-wired to GPI chip to trigger SMI Not under the control of the Hypervisor –Main challenge How to prevent or detect hypervisor’s intervention (e.g., reprogram APIC)? 6

7. Computer Science Host (root) Mode Guest (non-root) Mode HyperSentry Architecture 7 VM Hardware Hypervisor Virtualized Platform System Management Mode Remote Verifier BMC/IMM SMI Handler Measurement Agent Trusted Components are Shaded in Green

8. Computer Science In-context Integrity Measurement Challenges –How to detect the intercepted CPU operation mode? Hypervisor or guest VM? –How to retrieve the context needed for measurement? E.g., CR3 and page table Solution –Inject a privileged instruction to force the CPU to fall back to the hypervisor mode –Run the measurement agent in the same context as the hypervisor Agent runs in a protected execution environment 8

9. Computer Science Host (root) Mode Guest (non-root) Mode System Management Mode In-context Integrity Measurement Hardware Prepare SMM fallback Hypervisor Guest VM SMI RSM Execution Path Privileged instruction PC (cache misses = 1)APIC (SMI on PC overflow) Inject privileged instruction and flush cache VM exit handler PC (cache misses = 0) Verify the measurement agent SMI The measurement agent RSM Store measurement output SMI 9

10. Computer Science Stealthy Invocation Is out-of-band invocation sufficient to achieve stealthy invocation? –Unfortunately … 10

11. Computer Science Host (root) Mode Guest (non-root) Mode A Variation of Scrubbing Attack 11 VM Hardware Hypervisor System Management Mode Remote Verifier SMI Handler Typical Scenario BMC/IMM

12. Computer Science Host (root) Mode Guest (non-root) Mode A Variation of Scrubbing Attack 12 VM Hardware Hypervisor System Management Mode Remote Verifier SMI Handler Attack Scenario BMC/IMM Compromised hypervisor cannot intercept SMIs. But what if it tries to block real SMIs and generate fake ones?

13. Computer Science Thwarting this Scrubbing Attack 13 Can we prevent the hypervisor from blocking SMIs? –Not possible with existing hardware Solution –Detecting fake SMIs generated by the (compromised) hypervisor Verifying status registers to ensure that the measurement is invoked by the out-of-band channel –Key reason: HW SMI and SW SMI are distinguishable

14. Computer Science BMC AMM Stealthy Invocation IPMI 14 CPU Core 0 Target Platform (IBM HS21XM Blade Server) Remote Verifier IO Control Hub (South Bridge) Memory Control Hub (North Bridge) GPI 0 SSH SMI_EN GPI_ROUT 0 …..0 0…….0 SMI_STS 0 ……………….0 ALT_GPI_SMI_STS ALT_GPI_SMI_EN 1 10 1 CPU Core 1 CPU Core n 1 1 - All status register are non writable - Measurement is invoked only if all other bits are 0 - A fake SMI is easily detectable 0910 SMI

15. Computer Science Attesting to the Measurement Output Challenge –Absence of a dedicated hardware for attestation The hypervisor controls the hardware most of time Solution –Providing the SMRAM with a private key –Using this key to attest to the measurement results 15

16. Computer Science Host Mode Guest Mode System Management Mode Attesting to the Measurement Output Hardware Guest VM TPM SMI handler Initialization code SMM private key SMM public key K smm K smm -1 Hypervisor Bootstrapping Remote Verifier Integrity measurement output Attestation request K smm -1 { Output|Nonce} K AIK -1 {K smm |Handler|Nonce} 16

17. Computer Science Security Analysis Stealthy Invocation –If configurations are not changed  guaranteed by hardware –If configurations change  fake SMIs are detectable Verifiable Behavior –The measurement agent is measured every time before it executes Deterministic Execution –The measurement agent possesses full control over the system In-context privileged measurement –Guarantee falling back to the hypervisor mode –The measurement agent runs in the same context as the hypervisor Attestable output –The measurement output is signed by a verifiable and protected key 17

18. Computer Science HyperSentry Evaluation IBM HS21XM blade server Measuring the Xen hypervisor –End-to-end execution time: 35 ms –Periodical measurement: Every 8 seconds: 2.4% overhead; every 16 seconds: 1.3% overhead 18

19. Computer Science Conclusion HyperSentry –A novel framework for measuring the integrity of the most privileged system software –A measurement agent for the Xen hypervisor –Low overhead Next step –Measurement agent for Linux/KVM –Verifying the hypervisor’s dynamic integrity 19

20. Computer Science Questions? amazab@ncsu.edu 20