K. Salah1 Security Protocols in the Internet IPSec

K. Salah2 Security facilities in the TCP/IP protocol stack

K. Salah3 IP Security (IPSec) is a collection of protocols designed by the IETF (Internet Engineering Task Force) to provide security for a packet at the IP level. IP Security (IPSec) is a collection of protocols designed by the IETF (Internet Engineering Task Force) to provide security for a packet at the IP level. IPSec does not define the use of any specific encryption or authentication method. IPSec does not define the use of any specific encryption or authentication method. IPSec provides a framework and a mechanism; it leaves the selection of the encryption, authentication, and hashing methods to the user. IPSec provides a framework and a mechanism; it leaves the selection of the encryption, authentication, and hashing methods to the user. IPSec is still evolving, especially with present of FWs and NATs IPSec is still evolving, especially with present of FWs and NATs

K. Salah4 An SA is a crypto-protected connection An SA is a crypto-protected connection  One SA in each direction… At each end, the SA contains a key, the identity of the other party, the sequence number, and crypto parameters (DES, 3DES, MD5, SHA1, etc) At each end, the SA contains a key, the identity of the other party, the sequence number, and crypto parameters (DES, 3DES, MD5, SHA1, etc) IPSec header indicates which SA to use IPSec header indicates which SA to use Parties will maintain a database of SAs for currently- open connections Parties will maintain a database of SAs for currently- open connections  Used both to send and receive packets SA connection is uniquely defined by three elements: SA connection is uniquely defined by three elements:  32-bit security parameter index (SPI), which acts as a virtual circuit identifier (VCI) as in Frame Relay or ATM.  Type of the protocol used for security: AH or ESP or IKE IKE provides mutual authentication, establishes shared key, and creates SA  Source IP address.

K. Salah5 Two Modes of Operation IPSec operates in two different modes. Mode defines where the IPSec header is applied to the IP packet. IPSec operates in two different modes. Mode defines where the IPSec header is applied to the IP packet.  Transport mode IPSec header is added between the IP header and the rest of the packet. Most logical when IPSec is used end-to-end  Tunnel mode IPSec header is placed in front of the original IP header. The IPSec header, the preserved IP header, and the rest of the packet are treated as the payload. Can be used when IPSec is applied at intermediate point along path (e.g., for firewall-to- firewall traffic) Results in slightly longer packet Note that data may be encrypted multiple times

K. Salah6 AH Authentication Header (AH) protocol is designed to authenticate the source host and to ensure the integrity of the payload carried by the IP packet. Authentication Header (AH) protocol is designed to authenticate the source host and to ensure the integrity of the payload carried by the IP packet. The protocol calculates a message digest, using a hashing function and a symmetric key, and inserts the digest in the authentication header. The protocol calculates a message digest, using a hashing function and a symmetric key, and inserts the digest in the authentication header. The AH protocol provides source authentication and data integrity,but not privacy. The AH protocol provides source authentication and data integrity,but not privacy. This is transport AH 

K. Salah7 When an IP datagram carries an authentication header, the original value in the protocol field of the IP header is replaced by the value 51. A field inside the authentication header (next header field) defines the original value of the protocol field (the type of payload being carried by the IP datagram). When an IP datagram carries an authentication header, the original value in the protocol field of the IP header is replaced by the value 51. A field inside the authentication header (next header field) defines the original value of the protocol field (the type of payload being carried by the IP datagram). Steps for authentication header: Steps for authentication header:  AH is added to the payload with the authentication data field set to zero.  Padding may be added to make the total length even for a particular hashing algorithm  Hashing is based on total packet. For message digest, only those fields of IP header that don’t change during transmission are considered.  Authentication data are included in the authentication header  IP header is added after changing the value of protocol field to 51. Payload length: Length of AH in 4-byte multiples. Payload length: Length of AH in 4-byte multiples. SPI: plays the role of VCI SPI: plays the role of VCI Sequence number: for anti replay Sequence number: for anti replay

K. Salah8 ESP Encapsulation Security Payload (ESP) provides source authentication, privacy and integrity. Encapsulation Security Payload (ESP) provides source authentication, privacy and integrity. Value of IP protocol field is 50. Value of IP protocol field is 50. Field inside the ESP trailer (next header field) holds the original value of the protocol field of IP header. Field inside the ESP trailer (next header field) holds the original value of the protocol field of IP header. Steps Steps  ESP trailer is added to the payload  Payload and trailer or encrypted  ESP header is added  ESP header, payload and ESP trailer are used to create authenticated data.  Authenticated data are added at the end of ESP trailer.  IP header is added after changing the protocol value to 50. This is transport ESP 

K. Salah9 Why doesn’t NAT work with IPSec? Remember that the point of IPSec is not just to protect the confidentiality of the data, but also to assure the authenticity of the sender and the integrity of the data (that it hasn’t been changed in transit). The problem with NAT is obvious: NAT must change information in the packet headers in order to do its job. Remember that the point of IPSec is not just to protect the confidentiality of the data, but also to assure the authenticity of the sender and the integrity of the data (that it hasn’t been changed in transit). The problem with NAT is obvious: NAT must change information in the packet headers in order to do its job. The first problem is that NAT changes the IP address of the internal computer to that of the NAT device. The Internet Key Exchange (IKE) protocol used by IPSec embeds the sending computer’s IP address in its payload, and this embedded address doesn’t match the source address of the IKE packet (which is that of the NAT device). When these addresses don’t match, the receiving computer will drop the packet. The first problem is that NAT changes the IP address of the internal computer to that of the NAT device. The Internet Key Exchange (IKE) protocol used by IPSec embeds the sending computer’s IP address in its payload, and this embedded address doesn’t match the source address of the IKE packet (which is that of the NAT device). When these addresses don’t match, the receiving computer will drop the packet. Another problem is that TCP checksums (and optionally, UDP checksums) are used to verify the packets. The checksum is in the TCP header and it contains the IP addresses of the sending and receiving computers and the port numbers used for the communications. With normal NAT communications, this isn’t a problem because the NAT device updates the headers to show its own IP address and port in place of the sending computer’s. However, IPSec encrypts the headers with the Encapsulating Security Payload (ESP) protocol. Since the header is encrypted, NAT can’t change it. This means the checksum is invalid, so the receiving computer rejects the packet. Another problem is that TCP checksums (and optionally, UDP checksums) are used to verify the packets. The checksum is in the TCP header and it contains the IP addresses of the sending and receiving computers and the port numbers used for the communications. With normal NAT communications, this isn’t a problem because the NAT device updates the headers to show its own IP address and port in place of the sending computer’s. However, IPSec encrypts the headers with the Encapsulating Security Payload (ESP) protocol. Since the header is encrypted, NAT can’t change it. This means the checksum is invalid, so the receiving computer rejects the packet. In addition, NAT isn’t able to use the port numbers in TCP and UDP headers to multiplex packets to multiple internal computers when those headers have been encrypted by ESP In addition, NAT isn’t able to use the port numbers in TCP and UDP headers to multiplex packets to multiple internal computers when those headers have been encrypted by ESP

K. Salah10 NAT-T: How it works The IPSec working group of the IEEE has created standards for NAT-T that are defined in RFCs 3947 and NAT-T is designed to solve the problems inherent in using IPSec with NAT. The IPSec working group of the IEEE has created standards for NAT-T that are defined in RFCs 3947 and NAT-T is designed to solve the problems inherent in using IPSec with NAT. NAT-T adds a UDP header that encapsulates the ESP header (it sits between the ESP header and the outer IP header). This gives the NAT device a UDP header containing UDP ports that can be used for multiplexing IPSec data streams. NAT-T also puts the sending computer’s original IP address into a NAT-OA (Original Address) payload. This gives the receiving computer access to that information so that the source and destination IP addresses and ports can be checked and the checksum validated. This also solves the problem of the embedded source IP address not matching the source address on the packet. NAT-T adds a UDP header that encapsulates the ESP header (it sits between the ESP header and the outer IP header). This gives the NAT device a UDP header containing UDP ports that can be used for multiplexing IPSec data streams. NAT-T also puts the sending computer’s original IP address into a NAT-OA (Original Address) payload. This gives the receiving computer access to that information so that the source and destination IP addresses and ports can be checked and the checksum validated. This also solves the problem of the embedded source IP address not matching the source address on the packet. Firewall must be set up to support NAT-T Firewall must be set up to support NAT-T Note: This is a very simplified account of how NAT-T makes it possible for IPSec and NAT to work together. For more detailed information, see RFC 3947 at and RFC 3948 at Note: This is a very simplified account of how NAT-T makes it possible for IPSec and NAT to work together. For more detailed information, see RFC 3947 at and RFC 3948 at

K. Salah11 Firewall Firewall is a device (usually a router or a computer) installed between the internal network of an organization and the rest of the Internet. Firewall is a device (usually a router or a computer) installed between the internal network of an organization and the rest of the Internet. It is designed to forward some packets and filter (not forward) others. It is designed to forward some packets and filter (not forward) others. A firewall can be used to deny access to a specific host or a specific service in the organization. A firewall can be used to deny access to a specific host or a specific service in the organization.

K. Salah12 Packet-filter firewall A firewall can be used as a packet filter. It can forward or block packets based on the information in the network layer and transport layer headers: source and destination port addresses, and type of protocol (TCP or UDP). A firewall can be used as a packet filter. It can forward or block packets based on the information in the network layer and transport layer headers: source and destination port addresses, and type of protocol (TCP or UDP). Incoming packets from network are blocked. ‘*’ means any. Incoming packets from network are blocked. ‘*’ means any. Incoming packets destined for any internal TELNET server (port 23) are blocked. Incoming packets destined for any internal TELNET server (port 23) are blocked. And so on. And so on.

K. Salah13 VPN Privacy within intra-organization but still connected to global Internet. Privacy within intra-organization but still connected to global Internet. Intra-organization data are routed through the private internet; inter-organization data are routed through the global Internet. Intra-organization data are routed through the private internet; inter-organization data are routed through the global Internet.

K. Salah14 VPN Private and hybrid networks are costlier. Private and hybrid networks are costlier. Best solution is to use global Internet for both private and public communications. Best solution is to use global Internet for both private and public communications. VPN creates a network that is private but virtual. It is private but it guarantees privacy inside the organization. It is virtual because it does not use real private WANs; the network is physically public but virtually private. VPN creates a network that is private but virtual. It is private but it guarantees privacy inside the organization. It is virtual because it does not use real private WANs; the network is physically public but virtually private. VPN uses IPSec in tunnel mode to provide authentication, integrity and privacy. VPN uses IPSec in tunnel mode to provide authentication, integrity and privacy.

K. Salah15 VPN Each IP datagram destined for private use in the organization is encapsulated in another datagram. Each IP datagram destined for private use in the organization is encapsulated in another datagram. To use IPSec in the tunneling mode, the VPNs need to use two sets of addressing. To use IPSec in the tunneling mode, the VPNs need to use two sets of addressing. The public network (Internet) is responsible for carrying the packet from R1 to R2. Outsiders cannot decipher the contents of the packet or the source and destination addresses. Deciphering takes place at R2, which finds the destination address of the packet and delivers it. The public network (Internet) is responsible for carrying the packet from R1 to R2. Outsiders cannot decipher the contents of the packet or the source and destination addresses. Deciphering takes place at R2, which finds the destination address of the packet and delivers it.