1. Lucian Popa Minlan Yu Steven Y. Ko UC Berkeley/ICSI Princeton Univ. Princeton Univ. Sylvia Ratnasamy Ion Stoica Intel Labs Berkeley UC Berkeley CloudPolice: Taking Access Control Out of the Network

2. Context Infrastructure as a Service virtualized clouds Traffic internal to cloud Hypervisor VM

3. Context Cloud computing requires network access control

4. Context Cloud computing requires network access control Access control policy of tenant X = what network traffic is tenant X willing to accept Tenant X Y can talk to me Tenant Y

5. Why Access Control in Clouds? (1) For isolation Policy: deny incoming traffic from any other tenant Amazonia Exbay

6. Why Access Control in Clouds? (2) For inter-tenant & tenant-provider communication Policy: allow/deny traffic from specific tenants Increasingly common in cloud environments Low latency and high bandwidth Ease of service composition Amazonia Exbay

7. Why Access Control in Clouds? (2) For inter-tenant & tenant-provider communication Policy: allow/deny traffic from specific tenants Exbay Real-time bidding advertising Amazonia

8. Why Access Control in Clouds? (2) For inter-tenant & tenant-provider communication Policy: allow/deny traffic from specific tenants Exbay Real-time bidding advertising Send information about client Amazonia Ad Network 1 Ad Network 2 Ad Networks

9. Why Access Control in Clouds? (2) For inter-tenant & tenant-provider communication Policy: allow/deny traffic from specific tenants Exbay Real-time bidding advertising Receive ad bids Amazonia Ad Network 1 Ad Network 2 Ad Networks

10. Why Access Control in Clouds? (2) For inter-tenant & tenant-provider communication Policy: allow/deny traffic from specific tenants Exbay Real-time bidding advertising Amazonia Return ad of highest bidder Ad Network 1 Ad Network 2 Ad Networks

11. Why Access Control in Clouds? (2) For inter-tenant & tenant-provider communication Policy: allow/deny traffic from specific tenants Exbay Real-time bidding advertising Amazonia Ad Network 1 Ad Network 2 Ad Networks Policy of Exbay: allow traffic from AdNetworks, deny all other traffic

12. Why Access Control in Clouds? (2) For inter-tenant & tenant-provider communication Policy: allow/deny traffic from specific tenants Other service examples: database (SimpleDB), desktop, communication (SQS), map-reduce++, Facebook, host managing, locking, etc. Exbay Amazonia Ad Network 1 Ad Network 2 Ad Networks

13. Why Access Control in Clouds? (3) For inter-tenant & tenant-provider communication Policy: weighted bandwidth allocation between tenants Exbay Amazonia Ad Network 1 Ad Network 2 Ad Networks

14. Why Access Control in Clouds? (3) For inter-tenant & tenant-provider communication Policy: weighted bandwidth allocation between tenants Exbay Amazonia Ad Network 1 Ad Network 2 Ad Networks Share bandwidth fairly among tenants regardless of #VM sources Nextbay

15. Why Access Control in Clouds? (3) For inter-tenant & tenant-provider communication Policy: weighted bandwidth allocation between tenants Exbay Amazonia Ad Network 1 Ad Network 2 Ad Networks Other example policies: Rate-limited access Allow only locally initiated connections Nighttime access only Nextbay

16. Why Access Control in Clouds? (4) DoS protection One tenant can attack another tenant Reduce bandwidth and slow down machines Attackers more powerful: higher bandwidths Barrier is lower: pay for attacking hosts (compromise credit cards instead of hosts) Exbay Ad Network 1 Ad Network 2 Ad Networks Nextbay AmazoniaX

17. Hence, the problem Want access control in clouds that Is resilient to DoS Supports rich inter-tenant policies

18. Hence, the problem Want access control in clouds that Is resilient to DoS Supports rich inter-tenant policies Scales 100k servers 10k tenants

19. Hence, the problem Want access control in clouds that Is resilient to DoS Supports rich inter-tenant policies Scales Tolerates high dynamicity 100k VMs started per day, more than one per second

20. Hence, the problem Want access control in clouds that Is resilient to DoS Supports rich inter-tenant policies Scales Tolerates high dynamicity Traditional access control mechanisms not well suited to meeting these requirements

21. Existing Access Control Cloud APIs are narrow On/off No locally initiated connections, no rate-limiting, no weighted allocation Mechanisms inherited from enterprises VLANs Firewalls

22. Existing Access Control Cloud APIs are narrow On/off No locally initiated connections, no rate-limiting, no weighted allocation Mechanisms inherited from enterprises VLANs Firewalls But clouds != enterprises

23. Clouds != Enterprises Enterprises are not multi-tenant Few DoS concerns between departments Typically simpler policies Enterprises don’t have the same dynamicity and scale 10k tenants vs. 10s departments; 1 VM/s vs. mostly static Clouds have different network designs High bisection bandwidths, multiple paths, different L2/L3 mix Many new topologies: FatTree, VL2, BCube, DCell, etc.

24. VLANS not well suited for clouds Inflexible policies Difficult to scale (cloud size & dynamicity) Limited number, spanning tree Limited network designs No L3 networks, no multiple paths, inter-VLAN through router

25. Firewalls not well suited for Clouds Offering DoS protection is difficult Must be applied at source  hard to update Inflexible policies Scale through prefix aggregation Difficult to manage 10k tenants with multiple prefixes, different scaling requirements No L3 networks

26. Recap Traditional access control is not well suited for clouds Couple access control with network operation With switching – VLANs With address assignment – Firewalls

27. Recap Traditional access control is not well suited for clouds Couple access control with network operation With switching – VLANs With address assignment – Firewalls CloudPolice takes access control out of the network

28. Outline Part 1 – Context and Motivation Access control for clouds: why and what? Limitations of traditional mechanisms Part 2 – CloudPolice Approach Operation Cloud Police

29. Goal Network Access Control for Clouds that is: 1. Independent of network topology and addressing 2. Scalable (millions hosts, high churn) 3. Flexible (on/off access, rated access, fair access) 4. Robust to (internal) DDoS attacks

30. CloudPolice Sufficient and advantageous to implement access control only within hypervisors Hypervisor VM

31. CloudPolice Sufficient and advantageous to implement access control only within hypervisors Trusted Network independent Full software programmability  flexible Close to VMs  block unwanted traffic before network and help DoS Easy deployability Hypervisor VM

32. CloudPolice Sufficient and advantageous to implement access control only within hypervisors Hypervisor VM CloudPolice Policy Model Group = set of tenant VMs with same access control policy

33. CloudPolice Sufficient and advantageous to implement access control only within hypervisors Hypervisor VM Policy = set of Rules Rule = IF Condition THEN Action CloudPolice Policy Model

34. CloudPolice Sufficient and advantageous to implement access control only within hypervisors Hypervisor VM Condition = logical expression with predicates based on: Group of sender Packet header Current time History of traffic CloudPolice Policy Model

35. CloudPolice Sufficient and advantageous to implement access control only within hypervisors Hypervisor VM Action: Allow Block Rate-limit (token bucket) CloudPolice Policy Model

36. CloudPolice Sufficient and advantageous to implement access control only within hypervisors Hypervisor VM Action: Allow Block Rate-limit (token bucket) CloudPolice Policy Model Applied per flow source VM source group

37. CloudPolice Hypervisor-based Hypervisor VM Src. VM Hypervisor VM Dst. VM

38. CloudPolice Hypervisor-based Avoid DoS and wasted resources  apply policy at source Hypervisor VM Src. VM Hypervisor VM Dst. VM

39. CloudPolice Hypervisor-based How to apply destination’s policy at the source hypervisor? Hypervisor VM Src. VM Hypervisor VM Dst. VM

40. CloudPolice Hypervisor-based Centralized policy repository? Hypervisor VM Src. VM Hypervisor VM Dst. VM

41. CloudPolice Hypervisor-based Centralized policy repository? Hypervisor VM Src. VM Allow? Hypervisor VM Dst. VM

42. CloudPolice Hypervisor-based Centralized policy repository? Centralized service requires high availability and throughput 100k servers and 10 new flows/VM/s  1M decisions/s on average! Caching can be ineffective (random patterns, malicious pollution) Centralized service can be a DoS target Hypervisor VM Src. VM Allow? Hypervisor VM Dst. VM

43. CloudPolice Hypervisor-based Decentralized Hypervisor VM Src. VM Hypervisor VM Dst. VM

44. CloudPolice Hypervisor-based Decentralized Distribute all policies to all hypervisors? Hypervisor VM Src. VM Hypervisor VM Dst. VM

45. CloudPolice Hypervisor-based Decentralized Distribute all policies to all hypervisors? Hypervisor VM Src. VM Hypervisor VM Dst. VM Allow?

46. CloudPolice Hypervisor-based Decentralized Distribute all policies to all hypervisors? Too heavyweight if network independent Full group membership required; Group updates propagated everywhere 100k new VMs/day, 100k servers  100k updates/s on average Hypervisor VM Src. VM Hypervisor VM Dst. VM

47. CloudPolice Hypervisor-based Decentralized Apply at destination and enforce at source Hypervisor VM Src. VM Hypervisor VM Dst. VM Apply destination’s policy

48. CloudPolice Hypervisor-based Decentralized Apply at destination and enforce at source Hypervisor VM Src. VM Hypervisor VM Dst. VM Enforce policy’s action

49. Inspired by Internet Research Internet solutions to DDoS Push-back filters [AIP, Pushback, AITF, StopIt] Network Capabilities [SIFF, TVA] Handle large and dynamic networks, millions of users

50. Inspired by Internet Research Internet solutions to DDoS Push-back filters [AIP, Pushback, AITF, StopIt] Network Capabilities [SIFF, TVA] Handle large and dynamic networks, millions of users More easily deployed: Clouds != Internet Clouds are controlled environments Both communication endpoints can be controlled Single administrative domain New tools: trusted software layer – Hypervisor

51. Outline Part 1 – Context and Motivation Access control for clouds: why and what? Limitations of traditional mechanisms Part 2 – CloudPolice Approach Operation Cloud Police

52. CloudPolice Hypervisor XYZ Policies for X, Y and Z CloudPolice Each hypervisor needs to know for hosted VMs: group and policy X’s group policy: IF group = A  allow IF group = B  block IF group = C & port = 80  rate-limit to 100Mbps Y’s group policy: Z’s group policy: IF … Policy could also be specified / updated by VM Installed by provider service that starts VMs

53. CloudPolice Hypervisor XYZ Filter for incoming/outgoing flows

54. CloudPolice Hypervisor XYZ ABC Start flow to C Z group CloudPolice inserts control packet containing group of Z and first packet header

55. CloudPolice Hypervisor XYZ ABC CloudPolice verifies policy of destination VM If allowed, packets are forwarded to destination VM Block/rate-limit If blocked or rate limited, send control packet to source hypervisor to block or rate-limit source (flow/VM) Z group Soft-state and timeouts handle policy invalidations and packet losses

56. Scalability CloudPolice takes the best of both worlds Centralized vs. every server stores all policies Load spread across all servers Maintaining and enforcing policies Update propagation is contained Group membership updates not propagated Policy updates propagated only to group

57. Scalability CloudPoliceCentralizedAll to all Max LoadO(1)O(N)O(1) Group updateO(1) O(N) Policy updateO(|Group|)O(1)O(N)

58. Security Analysis Sketch Attackers VMs – corrupted or paid by malicious tenants Attacks considered Violate access control policies to reach destination DoS with unauthorized traffic DoS with authorized traffic Assumptions Hypervisors not compromised

59. Security Analysis Sketch Violate access control policies to reach destination Policy distributed securely to hypervisor Control packets cannot be spoofed, only sent by hypervisors Hypervisor XYZ Fake group

60. Security Analysis Sketch Violate access control policies to reach destination DoS with unauthorized traffic Control packets block unauthorized traffic at source

61. Security Analysis Sketch Violate access control policies to reach destination DoS with unauthorized traffic Control packets block unauthorized traffic at source Attackers attempt to cause drops of control packets Block/rate-limit

62. Security Analysis Sketch Violate access control policies to reach destination DoS with unauthorized traffic Control packets block unauthorized traffic at source Attackers attempt to cause drops of control packets Retry or prioritize control packets

63. Security Analysis Sketch Violate access control policies to reach destination DoS with unauthorized traffic DoS with authorized traffic Also need performance isolation for full protection Congestion

64. Security Analysis Sketch Violate access control policies to reach destination DoS with unauthorized traffic DoS with authorized traffic Also need performance isolation for full protection CloudPolice can implement some performance isolation rate-limit Rate-limit to fair share of destination link Share access link evenly between destination VMs

65. Future Work Implement CloudPolice prototype Extend CloudPolice Policies with application-level semantics (dynamic policies) Policies based on group-wide state Beyond access control? More flexible actions, e.g., send to middlebox Performance isolation framework

66. Summary Access control in cloud computing requires new mechanisms and extended policies CloudPolice Takes advantage of trusted hypervisors Inspired by past work on Internet DDoS protection Properties Network independent Scalable Flexible Robust to (internal) DDoS attacks

67. Backup Slides

68. Related Work OpenFlow & (Onix | Difane) & OpenVSwitch Decisions not based on logical identifier (group/tenant) Onix only isolation framework OpenFlow actions designed for switches (e.g., currently can’t rate-limit) Require scaling central controller Vs. software update for CloudPolice

69. Contributions Identify that new access control mechanism is needed in clouds Pinpoint the challenges and requirements Identify that access control should be done in hypervisors Propose CloudPolice, mechanism that satisfies requirements

70. Compromise Single Hypervisor Can prevent compromised hypervisors from violating security policies 1. Security credentials associated with group identifier Cannot be sent if unknown (known only for hosted VMs) E.g., group ID has key in name 2. Prevent spoofed control packets in the network Like IP anti-spoofing in switches/routers

71. Today’s Cloud Mechanisms? Solutions not public Could be similar to our solution Could provide fewer properties API is narrow On/off between groups No locally initiated connections, no rate-limiting, no weighted allocation

72. Feasibility Working on implementing CloudPolice prototype Fast path – act on per flow state Open VSwitch and software routers [RouteBricks, PacketShader] suggest this is feasible Slow path – execute policy and install flow state 1/N of requirements for centralized repository Few hosted VMs  dominated by policy complexity Software router applications suggest if-then-else structures can be parsed fast [RBF]

73. Other Related Work VL2’s approach if it would be applied to hypervisors Centralized repository Can violate policies if IP of destination known

74. Firewalls not Suited for Clouds Not well suited against DoS Must be applied at source  hard to update Inflexible policies for clouds Scaling & network designs With no prefix aggregation Difficult to scale (100k+ entries) Needs updating on all VM starts (more than once/s) With prefix aggregation Complex to manage 10k tenants with multiple prefixes, different scaling requirements No L3 networks