1. The Design and Implementation of Firewall, NAT, Traffic Shaper on FreeBSD

2. NAT, Firewall, Traffic Shaper NAT, firewall, and traffic shaper are closely related on FreeBSD. Their implementations are based on the same mechanism – packet filter. –NAT: For each packet of a connection, convert its private source IP address to a public IP address and also convert its source port number to another port number. –Firewall: Determine whether a flow’s packets should be allowed or denied. –Traffic shaper: let a flow’s packets experience artificial delay and/or artificial link bandwidth limitation. Learning this can help you do network researches

3. Packet Filter/Classifier Packet filer/classifier is needed to define a flow. A traffic flow can be defined by the combination of the following fields: –Input and output interface –Direction –Source and destination IP addresses –Source and destination port numbers –Protocol, etc. Using packet filter/classifier is also important for providing different QoS to different traffic flows. Packet filter/classifier needs to be performed at a high speed otherwise packet forwarding will be slowed down.

4. Firewall Implementation A packet filter (firewall) may have many filter rules, which are normally chained together. When a packet is received, the rule list must be scanned until a match is found and the associated action is performed. –There is a default rule which always matches packets. The action associated with it is either deny or allow. Depending on the system settings, packets can be reinjected into the firewall at the rule after the matching one for further processing.

5. Firewall in Kernel Because packet filtering/classification must be performed very fast, comparing a packet against the rules is implemented in the kernel. –We can implement a firewall daemon at the user level, however then every packet needs to be copied from the kernel to the user space and later copied into the kernel space. For performance reason, now the kernel becomes larger and messy. –Therefore, rules are copied into and stored in the kernel, and matching is performed in the kernel. –The code for performing the matching operation is placed at both ip_input() and ip_output(). This is because not only router may need the firewall mechanism, hosts may also need it.

6. In-Kernel Firewall Implementation User space Kernel space Filter NIC

7. Firewall Usage and Dynamic Rule Creation Filter rules can be created dynamically to allow more simple specification of rules. E.g., we want to allow all connections originated from my own network: –ipfw add check-state –ipfw add deny tcp from any to any established –ipfw add allow tcp from my-net to any setup keep-state

8. NAT When a NAT is used, every raw IP packet needs to pass through the NAT so that its source IP address and source port number can be modified. –The source IP address is changed to the IP address of the machine on which the NAT is running. –The NAT needs to monitor new flows (connections). –When a new flow is detected, an entry will be created in a mapping table and its index into this table will be used as the new source port number. –When a packet is received at the NAT, it will be checked with this mapping table and the reverse mapping is performed. –The checksum in the IP and TCP headers need to be fixed.

9. NAT Implementation NAT on FreeBSD is implemented as a daemon (natd) at the user level. –There is also an in-kernel implementation of NAT called ip_nat.c but it is not currently used. Thus when natd is enabled, every packet needs to be copied from the kernel to the user space and later from the user space to the kernel space. –It is not clear why at this time a user-level implementation is preferred over an in-kernel implementation.

10. User-Level NAT Implementation User space Kernel space NATD NIC

11. Traffic Shaper Sometimes when we do network researches, we need to do experiments. –E.g., study how TCP performance would change under different RTTs, link bandwidth, packet loss rates, etc. –However, it is very difficult to find a real link with such characteristics. ATT and Notel do have 100 KM cables for doing experiments. –One approach is to use a network simulator. However, the results may not be very convincing. –Another approach is to use a network emulator to act as a traffic shaper. Dummnet is the approach provided in FreeBSD to accomplish this job.

12. Pipe in Dummynet Packets of a classified/filtered flow will be directed to a pipe. In a pipe, the link bandwidth, link propagation delay and packet loss rate are implemented. –Of course, the emulated link bandwidth must be smaller than the bandwidth of the physical link. –Link bandwidth is emulated by dividing the packet length by the desired link bandwidth to calculate the packet’s transmission time. –Link delay is emulated by delaying the transmission of a packet by a timer. –A timer is used to trigger the transmission of packets. Currently, the highest resolution is only 1 ms. Thus it is difficult to emulate high-bandwidth links well.

13. Pipe Implementation Pipes currently are implemented in the kernel as queues. Clearly, to emulate desired link bandwidth, delay and packet loss rate, pipes can also be implemented in a daemon at user space. –However, to reduce the overhead of copying packets between the kernel and user space, –and to use a fine-grain and more predictable timer to trigger the transmission of packets, Sending packets smoothly at a very high speed is very difficult. In Dummnet, packets may be sent in bursts, which can change the experimental results. Be careful! –Pipes right now are implemented in the kernel.

14. In-Kernel Pipe Implementation User space Kernel space NIC Dymmynet Pipes Use heaps to sort packet transmission time.

15. Implementation Considerations We see that to implement NAT, firewall, traffic shaper, we can implement the system either in the kernel space or in the user space. The considerations that may affect our decision are that: –Whether high performance is really needed –Whether high accuracy is really needed –Which implementation is simpler –Whether the implementation makes the kernel more complicated Right now the kernel becomes very messy. If every new service is implemented in the kernel, the kernel will become too big and hard to understand.

16. Traffic Shaper Usage Configure a unidirectional link with 300 Kbps bandwidth and a queue size of 50 KB: –ipfw add pipe 1 ip from 192.168.2.0/24 to any out –ipfw pipe 1 config bw 300Kbit/s queue 50KBytes Configure a bidirectional link with 250 ms delay and 1 Mbps bandwidth in each direction: –ipfw add pipe 1 ip from any to any out –ipfw add pipe 2 ip from any to any in –ipfw pipe 1 config delay 250ms bw 1Mbit/s –ipfw pipe 2 config delay 250ms bw 1Mbit/s

17. Divert Sockets Divert sockets are similar to raw sockets, except that they can be bound to a specific divert port via the bind() system call. –The IP address in the bind() is ignored. Only the port number is significant. A divert socket bound to a divert port will receive all packets diverted to that port by the firewall mechanism in the kernel. Packets may also be written to a divert port. In this case, they re-enter kernel IP packet processing. Divert sockets are useful tools to implement user- level raw packet processing like filtering, NAT, pipes, etc.

18. Divert Sockets (Cont’d) Packets are diverted either as they are “incoming” or “outgoing.” –Incoming packets are diverted after reception on an IP interface. (In ip_input()) –Outgoing packets are diverted before next hop forwarding. (in ip_output()) The code for diverting packets is placed in both ip_input() and ip_output().

19. Reading Divert Sockets Diverted packets can be read unaltered via read() or readfrom(). –In recvfrom(), the returned port is set to the some tag supplied by the packet diverter (e.g., the ip firewall rule number if the packet diverter is the firewall) See, the action associated with a firewall filter are not limited to not only allow and deny. Other processing can also be applied to a packet. –The IP address set to the address of the interface on which the packet was received (if the packet was incoming) or INADDR_ANY (if the packet was outgoing)

20. Writing Divert Sockets Writing to a divert socket is similar to writing to a raw IP socket. The packet is injected “as is” into the normal kernel IP packet processing. Packets are written as either incoming and outgoing: –If write() or send() is used to write the packet, or if sendto() is used with a destination IP address of INADDR_ANY, then the packet is treated as if it were outgoing. (I.e., destined for a non-local address) –Otherwise, the packet is assumed to be incoming and full packet routing is done. In this case, the IP address specified must match the address of some local interface. This is used to indicate which interface the packet “arrives.”

21. Normal Usage Normally, packets read as incoming should be written as incoming. Also, packet read as outgoing should be written as outgoing. When reading and then writing back packets, you can simply pass the same socket address supplied by recvfrom() unmodified to sendto(). This simplies things The port part of the socket address passed to the sendto() contains a tag that should be meaningful to the diversion module. –In the firewall case, the tag is interpreted as the rule number after which rule processing should restart.

22. Divert Sockets Usage User space Kernel space NATD NIC Divert socket (bind on port 32000) Firewall Divert Firewall Divert Divert to (port 32000)

23. Natd on Top of ipfw To support natd on top of ipfw mechanism, the following ipfw rules can be used: –(1) /sbin/ipfw -f flush –(2) /sbin/ipfw add divert natd all from any to any via ed0 –(3) /sbin/ipfw add pass all from any to any Remember that: –In the firewall case, the tag is interpreted as the rule number after which rule processing should restart. Therefore, a packet diverted to natd by the firewall rule 2 will be passed into the kernel again and be processed by the firewall mechanism starting from rule 3.

24. Let’s trace the kernel source code and the natd source code to have a better understanding about divert sockets.