1. Secure Out-of-band Remote Management Using Encrypted Virtual Serial Consoles in IaaS Clouds Kenichi Kourai Tatsuya Kajiwara Kyushu Institute of Technology

2. IaaS Clouds  Provide users with virtual machines (VMs)  Users manage their systems through SSH  In-band remote management is usually used  A user connects an SSH client to an SSH server running in his VM  Not usable when a user failed network configurations SSH server SSH client user VM cloud firewall

3. Virtual Serial Console (VSC)  IaaS provides out-of-band remote management  An SSH server runs in the management VM  Access a user VM via a VSC  A VSC consists of a virtual serial device and a serial driver  A user can access his VM without relying on VM's network virtual serial device serial driver SSH server SSH client management VMuser VM VSC firewall

4. Untrusted Admins in IaaS  The management VM is not always trustworthy  Honest-but-curious admins take information that they can easily look at  Malicious admins act as inside attackers  Skill-less admins cause a vulnerable management VM to be penetrated by outside attackers management VMuser VM untrusted admins outside attackers

5. Attacks against Remote Management  Attackers can eavesdrop on inputs and outputs of out-of-band remote management  By modifying an SSH server in the management VM  Attack examples  Extract passwords from inputs to login prompts  Steal displayed secrets virtual serial device serial driver SSH server SSH client management VMuser VM

6. SCCrypt  Provide an encrypted VSC to prevent information leakage  Receive encrypted inputs from the management VM  Decrypt and send them to a user VM  Receive unencrypted outputs from a user VM  Encrypt and send them to the management VM virtual serial device serial driver SSH server SSH client management VMuser VM encrypted VSC decrypt encrypt encrypted inputs/outputs

7. Where to Encrypt/Decrypt?  Virtual serial device in the management VM?  The entire management VM is untrusted  Attackers can tamper with the virtual device  Serial driver in a user VM?  Modifying the existing device driver is not desirable  Users cannot use favorite OSes virtual serial device serial driver management VM user VM en-/de- crypt?

8. Leveraging the Trusted VMM  SCCrypt encrypts/decrypts data in the virtual machine monitor (VMM)  The integrity of the VMM can be guaranteed  At boot time: remote attestation with TPM  At runtime: HyperGuard [Rutkowska+ '08], HyperCheck [Wang+ '10], and HyperSentry [Azab+ '10] with SMM virtual serial device serial driver management VM user VM VMM encrypted VSC SCCrypt trusted authority

9. How to Identify Inputs/Outputs?  The traditional VMM does not recognize a virtual serial device or a serial driver  Cannot rely on information from the untrusted device in the management VM  Cannot obtain information from the unmodified driver in a user VM untrusted virtual serial device unmodified serial driver management VM user VM VMM SCCrypt ?

10. Tracking Device States  The VMM identifies inputs/outputs without the cooperation of the device or the driver  Track the state of a virtual serial device  From the interactions between the device and the driver  Extract only inputs/outputs  Based on the knowledge of the standard of serial devices virtual serial device serial driver management VM user VM VMM SCCrypt

11. Implementation  We have implemented SCCrypt in Xen 4.1.3 and the OpenSSH 6.0p1 client  Support HVM guests  Emulate hardware as is  Support PV guests (see the paper)  Define interfaces suited for virtualization

12. Output Delivery  A serial driver writes data using OUT instruction  The VMM traps this instruction using Intel VT-x  It encrypts the written data using RC4  Only if the specified I/O port address is 3F8  A session key is exchanged for each connection virtual serial device serial driver management VM user VM VMM encrypt trap OUT SSH server SSH client

13. State Tracking  The VMM encrypts the written data if  The FIFO buffers are enabled  Neither in the divisor latch access nor loopback mode  Devisor latch access mode: set a baud rate  Loopback mode: test a device  It watches all the writes to recognize the context FIFO disabled divisor latch access mode looback mode FIFO enabled

14. Input Delivery  A serial driver reads data using IN instruction  The VMM traps this instruction  Receive input data from a virtual serial device  It decrypts input data using RC4  If the same conditions as in output delivery are satisfied virtual serial device serial driver management VM user VM VMM decrypt IN SSH server SSH client trap

15. Pending Outputs at Reconnection  An SSH client cannot decrypt pending outputs correctly at reconnection  Console outputs are stored in a virtual serial device  While an SSH client is not connected  Pending outputs are sent to the client at reconnection  Encrypted with an old session key virtual serial device management VM SSH server SSH client pending data encrypted with an old session key new session key ?

16. Re-encryption of Outputs  The VMM re-encrypts pending outputs  Restore unencrypted data with an old session key  Cannot 'decrypt' it  Reverse the encryption process in RC4  Encrypt the data with a new session key virtual serial device management VM VMM re-encrypt SSH server SSH client pending data old/new session keys !#$ A#$ AB$AB$ ABC !#$ !#C !BC!BC ABC decryptreverse

17. Experiments  We confirmed the effectiveness of SCCrypt  Response time in an SSH client  Throughput of console outputs  CPU utilization  We compared results between SCCrypt and vanilla Xen CPU: Intel Xeon E5630 Mem: 6 GB Net: Gigabit Ethernet OS: Linux 3.2.0 SSH: OpenSSH 6.0p1 CPU: Intel Core i7 870 Mem: 4 GB Net: Gigabit Ethernet VMM: Xen 4.1.3 client server vCPU: 8 Mem: 1 GB OS: Linux 3.2.0 user VM vCPU: 8 Mem: 3 GB OS: Linux 3.2.0 SSH: OpenSSH 5.9p1 management VM

18. Response Time (Input)  We measured the response time of inputs  An SSH client sent an input to an SSH server  It received an output caused by its remote echo  Result  2.9% shorter in SCCrypt  The reason was unclear

19. Throughput (Output)  We measured the throughput of outputs  We wrote large text to a VSC in a user VM  Result  5.6% higher in SCCrypt  The reason was unclear

20. CPU Utilization (Input)  We measured the CPU utilization under the keyboard auto-repeat at a client  Management VM: 0.8% point lower in SCCrypt  User VM: almost the same management VM user VM

21. CPU Utilization (Output)  We measured the CPU utilization under periodic write to a VSC from a user VM  Management VM: 1.3% points difference at most  User VM: almost the same management VM user VM

22. Related Work  FBCrypt [Egawa et al.'12]  Encrypt keyboard/mouse inputs and video outputs in out-of-band remote management using VNC  Strongly depend on VNC  Xoar [Colp et al.'11]  Run a virtual serial device in trusted Console VM  Console VM can be compromised via an SSH server  VMware vSphere  Run virtual devices and a VNC server in the VMM  The VMM can be compromised via the VNC server

23. Conclusion  SCCrypt for enabling secure out-of-band remote management in untrusted IaaS clouds  Provide encrypted VSCs  Inputs/outputs are securely decrypted/encrypted in the trusted VMM  Future work  Apply SCCrypt to other remote management systems using VSCs  E.g., web-based Ajaxterm