1. Building a massively scalable serverless VPN using Any Source Multicast Athanasios Douitsis Dimitrios Kalogeras National Technical University of Athens

2. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Popular VPN solutions PPP over L2TP over IPSec transport mode IPSec tunnel mode OpenVPN IPSecUDPL2TPPPPIP IPSecIP TCP+SSLIP Trend: Usage of a central VPN concentrator

3. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast VPN central concentrator considerations Single point of failure Reliability impact Security impact Passage of all client traffic through the concentrator Impact on VPN concentrator resources (CPU, network) Impact on network near the VPN concentrator

4. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Alternative: P2P instead of hub topology P2P Communication through the multicast cloud No need for a central VPN server

5. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Any Source Multicast as a Shared Medium Election of a predefined common multicast group (G) as the shared medium Easy subscription of any node to the shared medium (IGMP join G) Transmissions inside the shared medium are received by all listeners Any node can also transmit messages to the shared medium G No contention issues inside G All VPN members directly connected to the L2 VPN

6. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Encapsulation of L2 packets inside UDP multicast One Ethernet frame inside each UDP packet UDP Destination = multicast group G UDP source = actual node IP address (unchanged) Ethernet Source = Host generated MAC address (some constrains apply) Ethernet destination = Destination MAC address (more on that later) UDP Ethernet

7. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Duality between Ethernet and IP multicast personality For each single node: Real global source IP address Virtual VPN source MAC address 1-1 relationship between global IP address and VPN Source MAC address Generation of VPN Source MAC address from global IP address: Just add two bytes at the front MAC uniqueness is guaranteed Example: If Source IP == 1.2.3.4, then VPN Source MAC := 0a:0a:01:02:03:04 Make sure 0a:0a doesn’t clash with real vendor

8. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Encapsulation of Ethernet inside UDP: explanation UDP: Source: 1.2.3.4 Destination: 224.1.2.3 UDP: Source: 1.2.3.4 Destination: 224.1.2.3 Ethernet: Source: 0a:0a:01:02:03:04 Destination: 0a:0a:05:06:07:08 Ethernet: Source: 0a:0a:01:02:03:04 Destination: 0a:0a:05:06:07:08 Host A: Real IP: 1.2.3.4 Virtual MAC: 0a:0a:01:02:03:04 Host B: Real IP: 5.6.7.8 Virtual MAC: 0a:0a:05:06:07:08 multicast cloud Host C: Real IP: 9.10.11.12 Virtual MAC: 0a:0a:09:10:11:12 Host A sends a packet to the VPN LAN The multicast enabled IP network takes the packet and sends it to all 224.1.2.3 subscribers Subscriber C receives the packet but is really not interested as its MAC != packet destination MAC Subscriber B receives the packet and forwards it through its networking stack

9. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Extension: Many MACs behind an IP Hiding of many VPN MAC addresses behind one single IP address 1-to-many relationship between global IP address and VPN MAC address Generation of VPN MAC address from global IP address: Again, Just add two bytes at the front Example: If Global IP == 1.2.3.4, then MAC := 0a:xx:01:02:03:04 01< xx< ff 256 MACs max behind one real IP Nice for virtualization setups

10. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Optimization: Usage of multicast only when needed Modern Ethernet Switches: MAC to port lookup table Delivery of Broadcasts (mostly ARP, ICMPv6 etc) to all ports Delivery of packets with unknown dest. MAC to all ports Delivery of packets with known dest. MAC only to corresponding port Modification of our virtual L2 VPN towards the same goal. MAC to global IP table Broadcasts (mostly ARP, ICMPv6 etc) to all G subscribers Packets with unknown dest. MAC to all G subscribers Packets with known dest. MAC only to corresponding IP using Unicast!

11. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Unicast Optimization explained UDP: Source: 1.2.3.4 Destination: 224.1.2.3 UDP: Source: 1.2.3.4 Destination: 224.1.2.3 Ethernet: Source: 0a:0a:01:02:03:04 Destination: 0a:0a:05:06:07:08 Ethernet: Source: 0a:0a:01:02:03:04 Destination: 0a:0a:05:06:07:08 Host A: Real IP: 1.2.3.4 Virtual MAC: 0a:0a:01:02:03:04 Host B: Real IP: 5.6.7.8 Virtual MAC: 0a:0a:05:06:07:08 multicast cloud Host A sends a packet to the VPN LAN The multicast enabled IP network takes the packet and sends it to all 224.1.2.3 subscribers Subscriber B receives the packet and adds the appropriate entry in it MAC-to-IP table MACIP 0a:0a:01:02:03:041.2.3.4 Subscriber B responds with a direct unicast packet to A because it knows its global IP UDP: Source: 5.6.7.8 Destination: 1.2.3.4 UDP: Source: 5.6.7.8 Destination: 1.2.3.4 Ethernet: Source: 0a:0a:05:06:07:08 Destination: 0a:0a:01:02:03:04 Ethernet: Source: 0a:0a:05:06:07:08 Destination: 0a:0a:01:02:03:04

12. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Implementation Kernel interface Usage of the versatile tun/tap driver virtual tap0 ethernet device /dev/net/tap character device User space application Reads from /dev/net/tap and writes to UDP socket Reads from UDP socket and writes to /dev/net/tap

13. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Ingress data flow Arrival of packet from network Reading of the packet from the socket (optional) Parsing of the packet and caching of MAC-to-IP pair De-capsulation of the Ethernet Frame from the UDP packet (optional) Other kinds of meddling with the de-capsulated Ethernet frame Writing of Ethernet Frame to /dev/net/tap Kernel sees an Ethernet Frame coming from i/f tap0

14. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Egress data flow Generation of an Ethernet Frame on the tap0 interface by the kernel Reading of the Ethernet Frame from the /dev/net/tap device by the user space VPN application (optional) Consultation of the MAC-to-IP cache table Encapsulation of the Ethernet Frame inside a UDP packet (optional) Other kinds of meddling with the soon-to-be-transmitted Ethernet packet Transmission of the packet either as multicast or Unicast

15. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Ingress/Egress explained multicast cloud VPN Member Host Kernel Space User Space VPN Client multicast socket tun/tap driver /dev/net/tap char device i/f tap0 VPN Member Host Kernel Space User Space VPN Client multicast socket tun/tap driver /dev/net/tap char device i/f tap0 A frame is generated read() from device send() to socket recv() from socket write() to device The frame is delivered inside tap0 Ingress Data flow Egress Data flow

16. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Security considerations Problem: Multicast Group joinable and listenable by anyone/anywhere Possible solution #1. Communications are secured at the encapsulation layer, e.g. Secure Multicast. Multicast Group Domain of Interpretation (RFC3547). Downside: group controller/key server required. But: “Normal” IPSec perfectly usable for unicast communications Possible solution #2. Communications are secured inside the VPN LAN, e.g. secure LAN. Usage of IPSec inside the VPN LAN Possible solution #3. Use secure protocols (>L3) inside the VPN LAN HTTPS, SSH, SFTP

17. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Conclusion: Benefits Resiliency: No central server needed Scalability: Solution can scale to very large number of nodes with the Unicast optimization enabled Transparency: tap0 for all intents and purposes an ordinary Ethernet interface Portability: Simple implementation easily portable to any platform.

18. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast But: Drawbacks Security provisioning somewhat tricky server required for GDOI IP Multicast required on all nodes (some networks still don’t support multicast)

19. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Extensions / Future Ideas Virtual Routers between different VPNs Physical Gateways to a VPN bridging of a real ethernet device with a tap Packet filters on tap devices Many virtual VPN members inside one physical entity Can work well with hardware virtualization

20. A. Douitsis, D.Kalogeras - Building a massively scalable serverless VPN using Any Source Multicast Thank You! Questions? {adouitsis|dkalo}@noc.ntua.gr