1. Chapter 6 Configuring, Monitoring & Troubleshooting IPsec

2. 6.1 Overview of IPsec Benefits of IPsec Recommended Uses of IPsec
Tools Used to Configure IPsec
What are Connection Security Rules ?

3. Benefits of IPsec
IPsec – suite of protocols that allows secure, encrypted communication between 2 computers over an unsecured network
2 goals; to protect IP packet & to defend against network attacks
IPsec secures network traffic by using encryption & data signing
An IPsec policy defines the type of traffic that Ipsec examines, how that traffic is secured & encrypted, and how IPsec peers are authenticated

4. Recommended Uses of IPsec
Authenticating & encrypting host-to-host traffic
Authenticating & encrypting traffic to servers
Layer 2 Tunneling Protocol (L2TP)/IPsec for VPN connections
Site-to-site (gateway-to-gateway) tunneling
Enforcing logical networks (server/domain isolation)

5. Tools Used to Configure IPsec
Windows Firewall with Advanced Security MMC (used for Windows Server 2008 & Windows Vista)
IP Security Policy MMC (used for mixed environments & to configure policies that apply to all Windows versions)
Netsh command-line tool

6. What are Connection Security Rules?
Connection security rules involve:
Authenticating 2 computers before they begin communications
Securing information being sent between 2 computers
Using key exchange, authentication, data integrity & data encryption (optionally)
How firewall rules & connection rules are related:
Firewall rules allow traffic through, but do not secure that traffic
Connection security rules can secure the traffic, but creating a connection security rule does not allow traffic through the firewall

7. 6.2 Configuring Connection Security Rules
Choosing a Connection Security Rule Type
What are Endpoints?
Choosing Authentication Requirements
Authentication Methods
Determining a Usage Profile

8. Choosing a Connection Security Rule Type
Description
Isolation
Restricts connections based on authentication criteria that you define
Authentication Exemption
Exempts specific computers, or a group or range of IP addresses, from being required to authenticate
Grants access to those infrastructure computers with which this computer must communicate before authentication occurs
Server-to-server
Authenticates 2 specific computers, 2 groups of computers, 2 subnets, or specific computer & a group of computers or subnet
Tunnel
Provides secure communications between 2 peer computers through tunnel endpoints (VPN or L2TP IPsec tunnels)
Custom
Enables you to create a rule with special settings

9. What are Endpoints?
Computer endpoints are the computers or the group of computers that form peers for the connection
IPsec tunnel mode protects an entire IP packet by treating it as an AH or ESP payload
ESP encrypts packets and applies a new unencryptes header to facilitate routing
ESP function in 2 modes:
Transport mode
Tunnel mode

10. ESP Transport Mode ESP Tunnel Mode IP HDR Data ESP HDR Encrypted Data
ESP TRLR
ESP Auth
IP HDR
ESP Tunnel Mode
IP HDR
Data
New IP HDR
ESP HDR
Encrypted IP Packet
ESP TRLR
ESP Auth

11. Choosing Authentication Requirements
Option
Description
Request Authentication for inbound and outbound connections
Ask that all inbound/outbound traffic be authenticated, but allow the connection if authentication fails
Require authentication for inbound connections and request authentication for outbound connections
Require inbound be authenticated or it will be blocked
Outbound can be authentication fails
Require authentication for inbound and outbound connections
Require that all inbound/outbound traffic be authenticated or the traffic will be blocked

12. Authentication Methods
Key Points
Default
Use the authentication method configured on the IPsec Settings tab
Computer & User (Kerberos V5)
You can request or require both the user & computer authenticate before communications can continue; domain membership required
Computer (Kerberos V5)
Request or require the computer to authenticate using Kerberos V5
User (Kerberos v5)
Request or require the user to authenticate using Kerberos V5; domain membership required
Computer certificate
Request or require a valid computer certificate, requires at least one CA
Only accept health certificates: request or require a valid health certificate to authenticate, requires IPsec NAP
Advanced
Configure any available method; you can specify methods for First & Second Authentication

13. Determining a Usage Profile
Security settings can change dynamically with the network location type
Windows supports 3 network types :
Domain: selected when the computer is a domain member
Private: networks trusted by the user (home or small office network)
Public: default for newly detected networks, usually the most restrictive settings are assigned because of the security risks present on public networks
The network location type is most useful on portable computers which are likely to move from network to network

14. 6.3 Configuring IPsec NAP Enforcement
IPsec Enforcement for Logical Networks
IPsec NAP Enforcement Processes
Requirements to Deploy IPsec NAP Enforcement

15. IPsec Enforcement for Logical Networks
HRA
VPN
802.1x
DHCP
NPS proxy
NAP administration server
Network policies
NAP health policies
Connection request policies
SHVs
SHAs
NAP agent NAP ECs
NAP enforcement servers
SHAs
NAP agent
NAP ECs
NPS servers
Non-compliant
NAP client
Remediation
servers
Certificate services
servers
NAP policy servers
Non-NAP
Capable client
Compliant NAP client
Secure
servers
Restricted
network
Boundary
Network
Secure Network

16. IPsec NAP Enforcement Processes
IPsec NAP Enforcement includes:
Policy validation
NAP enforcement
Network restriction
Remediation
Ongoing monitoring of compliance
VPN Server
Active Directory
IEEE 802.1x Devices
Health Registration Authority
Internet
NAP Health Policy Server
DHCP Server
Intranet
Perimeter Network
Restricted network
Remediation Server
NAP Client with limited access

17. Requirements to Deploy IPsec NAP Enforcement
Active Directory
Active Directory Certificate Services
Network Policy Server
Health Registration Authority

18. 6.4 Monitoring IPsec Activity
Tools used to Monitor IPsec
Using IP Security Monitor to Monitor Ipsec
Using Windows Firewall with Advanced Security to Monitor IPsec

19. Tools Used to Monitor IPsec
Key Points
IP Security Monitor
Used in Windows XP and higher
MMC snap-in
Administrators can monitor local and remote IPsec policy usage
IPsecmon
Only available in Windows 2000
Command-line tool
Reduced level of information available for troubleshooting
Windows Firewall with Advanced Security MMC
New in Windows Vista and Windows Server 2008
Detailed IKE tracing using Netsh
Trace file found in: systemroot\debug\oakley.log
Enabled in Windows XP and Windows through Registry modification

20. Using IP Security Monitor to Monitor IPsec
Options for using the IP Security Monitor:
Modify IPsec data refresh interval to update information in the console at a set interval
Allow DNS name resolution for IP addresses to provide additional information about computers connecting with IPsec
Computers can monitored remotely:
To enable remote management editing, the HKLM\system\currentcontrolset\services\policyagent key must have a value of 1
To Discover the Active security policy on a computer, examine the Active Policy Node in the IP Security Monitoring MMC
Main Mode Monitoring monitors initial IKE and SA:
Information about the Internet Key Exchange
Quick Mode Monitoring monitors subsequent key exchanges related to IPsec:
Information about the IPsec driver

21. Using Windows Firewall with Advanced Security to Monitor IPsec
The Windows Firewall in Windows Vista and Windows Server 2008 incorporates IPsec
Use the Connection Security Rules and Security Associations nodes to monitor IPsec connections
The Connection Security Rules and Security Associations nodes will not monitor policies defined in the IP Security Policy snap-in
Items that can be monitored include:
Security Associations
Main Mode
Quick Mode

22. 6.5 Troubleshooting IPsec
IPsec Troubleshooting Process
Troubleshooting Internet Key Exchange (IKE)
Troubleshooting IKE Negotiation Events

23. IPsec Troubleshooting Process
Stop the IPsec Policy Agent and use the ping command to verify communications 1
Verify firewall settings 2
Start the IPsec Policy Agent and use IP Security Monitor to determine if a security association exists 3
Verify that the policies are assigned 4
Review the policies and ensure they are compatible 5
Use IP Security Monitor to ensure that any changes are applied 6

24. Troubleshooting IKE ü ü ü ü
Identify connectivity issues related with IPsec and IKE ü
Identify firewall and port issues ü
View the Oakley.log file for potential issues ü
Determine Main mode exchange issues ü

25. Troubleshooting IKE Negotiation Events
Common Security Event log codes:
Success:
541 - IKE Main Mode or Quick Mode established
542 - IKE Quick Mode was deleted
543 - IKE Main Mode was deleted 
Information Log Entries:
Largely pertains to monitoring for denial of service attacks
There might not be any errors but resources will run low, which affects performance for legitimate clients
Quick Mode audit failures are denoted with 547 error message

26. End of Chapter 6