Securing Network Communications Using IPSec Chapter Twelve

Exam Objectives in this Chapter:  Implement secure access between private networks. Create and implement an IPSec policy.  Configure network protocol security. Configure protocol security in a heterogeneous client computer environment. Configure protocol security by using IPSec policies.  Configure security for data transmission. Configure IPSec policy settings.

Exam Objectives in this Chapter: cont.  Plan for network protocol security. Specify the required ports and protocols for specified services. Plan an IPSec policy for secure network communications.  Plan security for data transmission. Secure data transmission between client computers to meet security requirements. Secure data transmission by using IPSec.  Troubleshoot security for data transmission. Tools might include the IP Security Monitor MMC snap-in and the Resultant Set of Policy (RSoP) MMC snap-in.

Lessons in this Chapter:  Securing Internetwork Communications  Planning an IPSec Implementation  Deploying IPSec  Troubleshooting Data Transmission Security

Before You Begin  This chapter assumes a basic understanding of TCP/IP communications, as described in Chapter 2, “Planning a TCP/IP Network Infrastructure.”  To perform the practice exercises in this chapter, you must have installed and configured Windows Server 2003 using the procedure described in “About This Book.”

Securing Internetwork Communications  Packet Filtering Packet filtering is a method for regulating the TCP/IP traffic that is permitted to reach a computer or a network, based on criteria such as IP addresses, protocols, and port numbers.

Understanding Ports and Protocols  In the packet header of each TCP/IP protocol at each layer of the OSI reference model, identifiers specify which protocol at the next layer should receive the packet.

Well-Known Port Numbers ApplicationAbbreviationProtocolPort Number File Transfer Protocol (Control)ftp-controlTCP21 File Transfer Protocol (Default Data)ftp-default dataTCP20 Telnet Simple MailtelnetTCP23 Transfer ProtocolsmtpTCP25 Domain Name Service DynamicdomainTCP/UDP53 Host Configuration Protocol (Server) Dhcps bootps UDP67 Bootstrap Protocol Server (nondynamic) Dynamic Host Configuration Protocol (Client) Bootstrap Protocol Client (nondynamic) dhcpc bootpc UDP68 World Wide Web HTTPhttpTCP80 Post Office Protocol - Version 3pop3TCP110 Simple Network Management ProtocolsnmpUDP161 Simple Network Management Protocol TrapsnmptrapUDP162

Exam Tip  Be sure to familiarize yourself with the well-known port numbers assigned to the most commonly used services in Windows Server 2003, as listed in Table 12-1.

Separate firewall products Two Advantages: First, by separating the routing and filtering functions on different systems, you are less likely to experience degraded network performance. Second, firewalls are likely to have more advanced packet filtering capabilities, such as preset filter configurations designed to protect against specific types of attacks

Packet Filtering Criteria  Creating packet filters is a matter of selecting the specific criteria you want the system to examine and specifying the values that you want to allow or deny passage.  The criteria most commonly used in packet filtering are: Port numbers Protocol identifiers IP addresses Hardware addresses

Spoofing  Once an attacker finds out the IP addresses that the filter allows access to the network, it is simple to impersonate another computer by using its IP address.

Relationship to the OSI model Physical Data-Link Network Transport Session Presentation Application Port Numbers Protocol Identifiers IP Addresses Hardware Addresses

Windows Server 2003 Packet Filtering TCP/IP Packet Filtering  Using TCP/IP Packet Filtering  Using Routing and Remote Access Service Packet Filtering Notice the limitations on page 12-8

Using Routing and Remote Access Service Packet Filtering  Creating filters based on the IP addresses, protocols, and port numbers of a packet’s source or destination  Creating filters for ICMP messages, specified by the message type and code values  Creating multiple filters of the same type Windows Server 2003 RRAS includes a packet filtering mechanism that is more capable than that of the TCP/IP client, but you can only use it when you have configured Windows Server 2003 to function as a router

Practice:  Creating Packet Filters in Routing and Remote Access Service Exercise 1: Examining the Default Routing and Remote Access Exercise 2: Creating New Packet Filters  Page 12-10

Planning an IPSec Implementation  You can store your files in encrypted form using the Encrypting File System (EFS), for example, or an individual application might be able to protect files with a password, but when you access the file over the network or send it to someone else, your computer always decrypts it first.

Evaluating Threats  There are many ways that unauthorized personnel can use this captured data against you: Compromising keys Spoofing Modifying data Attacking applications

Introducing IPSec  IPSec encrypts the information in IP datagrams by encapsulating it, so that even if the packets are captured, none of the data inside can be read.  Because IPSec operates at the network layer, as an extension to the IP protocol, it provides end-to-end encryption, meaning that the source computer encrypts the data, and it is not decrypted until it reaches its final destination

Other Protocols  Secure Sockets Layer (SSL), an application layer protocols that can encrypt only specific types of traffic.

IPSec Functions page  Key generation use a technique called the Diffie–Hellman algorithm to compute identical encryption keys.  Cryptographic checksums Uses its cryptographic keys to calculate a checksum for the data in each packet, called a hash message authentication code (HMAC), then transmits it with the data. IPSec supports two hash functions:  HMAC in combination with Message Digest 5 (MD5) and HMAC in combination with Secure Hash Algorithm-1 (SHA1.)  HMAC-SHA1 is the more secure function, partly due to SHA1’s longer key length (SHA1 uses a 160-bit key as opposed to the 128-bit key used by MD5).

IPSec Functions  Mutual authentication They must authenticate each other to establish a trust relationship IPSec can use Kerberos, digital certificates, or a preshared key for authentication.  Replay prevention IPSec prevents packet replays from being effective by assigning a sequence number to each packet. An IPSec system will not accept a packet that has an incorrect sequence number.  IP packet filtering

IPSec Protocols  IP Authentication Header When a computer uses AH to protect its transmissions, the system inserts an AH header into the IP datagram, immediately after the IP header and before the datagram’s payload. Application Data Transport Layer Protocol Header Signed IPSec AH header IP header

IPSec Protocols Next Header Payload Length Reserved Security Parameters Index Sequence Number Authentication Data Next HeaderPayload LengthReserved Security Parameters Index Sequence Number AH Header Format

IPSec Protocols  IP Encapsulating Security Payload The IP Encapsulating Security Payload (ESP) protocol is the one that actually encrypts the data in an IP datagram, preventing intruders from reading the information in packets they capture from the network. Encrypted with ESP header IPSec ESP Authentication Signed by ESP Auth trailer IPSec ESP Trailer Application Data Transport Layer Protocol header IPSec ESP header IP header

IPSec Protocols Security Parameters Index Payload Data Pad Length Next Header IP header IPSec AH header Encrypted with ESP header IPSec ESP header Transport Layer Protocol Header Application Data IPSec ESP Trailer IPSec ESP Authentication

Transport Mode and Tunnel Mode  IPSec can operate in two modes: Transport mode  you use transport mode, in which the two end systems must support IPSec Tunnel mode.  Tunnel mode is designed to provide security for wide area network (WAN) connections, and particularly virtual private network (VPN) connections, which use the Internet as a communications medium.

The tunnel mode communications Tunnel Endpoints Transit Internet work Header Tunneled Packet Transit Internet work Tunnel Packet

The tunnel mode communications  Five steps on page  The original datagram, inside the new datagram, remains unchanged. The IPSec headers are part of the outer datagram, which exists only to get the inner datagram from one router to the other. Encrypted with ESP header IPSec ESP Authentication Signed by ESP Auth trailer IPSec ESP Trailer Application Data Transport Layer Protocol Header Original IP Header IPSec ESP Header IP Header

Deploying IPSec  IPSec is based on standards published by the Internet Engineering Task Force (IETF); so all IPSec implementations conforming to those standards should be compatible.

IPSec Components  There are several components: IPSec Policy Agent Internet Key Exchange (IKE) IKE communication process  The IKE communication process proceeds in two stages. first stage  The first stage, called the Phase 1 SA, includes the negotiation of which encryption algorithm, hashing algorithm, and authentication method the systems will use. second stage  The second stage consists of the establishment of two Phase 2 SAs, one in each direction. IPSec Driver

Planning an IPSec Deployment  In actual deployment, you must consider just what network traffic you need to protect and how much protection you want to provide.  IPSec is resource intensive in two different ways. First, the addition of AH and ESP headers to each packet increases the amount of traffic on your network. Second, calculating hashes and encrypting data both require large amounts of processor time.

Working with IPSec Policies  IPSec policies flow down through the Active Directory hierarchy just like other group policy settings. When you apply an IPSec policy to a domain, for example, all the computers in the domain inherit that policy.

Using the Default IPSec Policies Client (Respond Only) Secure Server (Require Security) Server (Request Security)

Modifying IPSec Policies  Rules  IP filter lists  Filter actions

Modifying IPSec Policies  Rules  IP filter lists  Filter actions

Modifying IPSec Policies  Rules  IP filter lists  Filter actions

Exam Tip  Be sure you are familiar with the components of an IPSec policy and with the functions of each component.

Practice:  Creating an IPSec Policy Exercise 1: Creating an MMC Console and Viewing the Default Policies  Page Exercise 2: Creating a New IPSec Policy  Page 12-31

Troubleshooting Data Transmission Security  Troubleshooting Policy Mismatches Incompatible IPSec policies. It is also possible for two computers to be configured to use IPSec for a particular type of traffic, but have incompatible filter action settings, such as different authentication methods or encryption algorithms  Examine the Security logs in the Event Viewer console.

Troubleshooting Data Transmission Security Using the IP Security Monitor Snap-in  If you have IPSec policies deployed by Group Policy Objects at different levels of the Active Directory tree, the IPSec policy that is closest to the computer object is the one that takes effect.

Troubleshooting Data Transmission Security Using the Resultant Set of Policy Snap-in  You can use RSoP to view all the effective group policy settings for a computer or user, including the IPSec policies

Exam Tip  Be sure you understand the differences between the IP Security Monitor snap-in and the Resultant Set of Policy snap-in, and know when it is preferable to use each one.

Examining IPSec Traffic  Windows Server 2003 Network Monitor includes parsers for IKE, AH, and ESP traffic.  However, you cannot use Network Monitor to examine packet information that has been encrypted using ESP.

Practice:  Using Resultant Set of Policy Exercise 1: Creating a Resultant Set of Policy Console  Page Exercise 2: Performing an RSoP Scan Exercise 3: Creating a Domain IPSec Policy  Page 12-40

Summary  Case Scenario Exercise Page  Troubleshooting Lab Page  Exam Highlights Key Points Key Terms  Page 12-45