1. 1 IP Security (IPSec) Thomas Lee Chief Technologist –QA thomas.lee@qa.com

2. 2 Agenda  What is IPSec?  How does IPSec Work?  Configuring/Using IPSec  Issues  Best Practices  Resources

3. 3 What is IPSec?  Framework of open standards for ensuring private, secure communications over Internet Protocol (IP) networks  IPSec provides authenticated and encrypted traffic between hosts at the IP protocol level  Provides aggressive protection against private network and Internet attacks through end-to-end security.  Protects communication between workgroups, local area network computers, domain clients and servers, branch offices (which might be physically remote), extranets, and roving clients.  IPSec is the long-term direction for secure networking

4. 4 IPSec Objectives  To protect the contents of IP packets.  To provide a defense against network attacks through packet filtering  To enforce trusted communication based on either local or central policy  These objectives are met through the use of cryptography-based protection services, security protocols, dynamic key management and Windows Group Policy.

5. 5 Why IPSec?  IPv4 not designed with security in mind  Attacks possible with IPv4 Eavesdropping Data modification Identity spoofing (IP address spoofing) Denial-of-service attack Man-in-the-middle attack  These can be avoided by use of IPSec

6. 6 IPSec Protection  Eavesdropping The Encapsulating Security Payload (ESP) protocol in IPSec provides data confidentiality by encrypting the payload of IP packets  Data modification Cryptography-based keys, shared only by the sending and receiving computers, are use to create a cryptographic checksum for each IP packet. Modification of the data alters the checksum, which indicates to the receiving computer that the packet was modified in transit  Identity spoofing IPSec allows verification of identities without exposing that information to an attacker. Mutual authentication establishes trust between the hosts.  Man-in-the-middle attacks IPSec combines mutual authentication with shared, cryptography-based keys.  Denial-of-service attacks IPSec uses IP packet filtering allow, secure, or block traffic based on IP address ranges, IP protocols, or even specific TCP/UDP ports.

7. 7 Agenda  What is IPSec?  How does IPSec Work?  Configuring/using IPSec  Issues  Best Practices  Resources

8. 8 filters filters How Components Interacts?  Internet Key Exchange (IKE) - Identity Protect Mode – defined in RFC 2409  Phase 1 “Main Mode” establishes IKE SA – trusted channel between systems, negotiation establishes encrypted channel, mutual trust, and dynamically generates shared secret key (“master” key)  Phase 2 “Quick Mode” establishes IPSec SAs – for data protection, one SA for each direction identified by packet label (SPI), algorithms and packet formats agreed, generates shared “session” secret keys derived from “master” key NIC TCPIP Application Server or Gateway IPSecDriver IPSecPolicyAgent IKE (ISAKMP) IPSecDriver IPSecPolicyAgent NIC TCPIP Application/Serviceclient “IKE Responder” “IKE Initiator” UDP port 500 negotiation 1 IKE SA 2 IPSec SAs IP protocol 50/51

9. 9 IPSec Policy  One Active IPSec Policy Multiple IPSec Policies can be defined  Policy Consists of ISAKMP Policy IPSec rules – An IPSec policy can have many rules  IPSec Rules Filter – identifies the traffic to secure/drop/etc Filter action – drop, deny, authenticate, encrypt Authentication, encryption, etc

10. 10 IPSec Policy Components  Polling interval used to detect changes in policy  IKE parameters, such as encryption key lifetimes.  IPSec behavior for the policy  The types of traffic to which an action is applied  Permit, block, or secure  Kerberos, certificate, or preshared key  LAN, Dialup, or both

11. 11 IPSec packet filtering  Filters allow and block traffic  Filters can overlap Most specific match determines action  NO stateful inspection  Example: to open only port 80 on the IIS: From IPTo IPProtocolSrc PortDest PortAction AnyMy Internet IPAnyn/a Block AnyMy Internet IPTCPAny80Permit

12. 12 Negotiation of Protection  Require two messages Initiator to Responder : (contains proposals) Responder to Initiator: (contains a selected proposal) Details later!  Protection suites: AttributeAttribute Value Encryption algorithmDES, 3DES, Null Integrity algorithmMD5, SHA-1, Null Authentication methodKerberos, preshared key, certificate Diffie-Hellman groupGroup 1 (768-bit), Group 2 (1024-bit)

13. 13 IPSec Modes  Transport mode Used for IPSec peers doing end-to-end security Provides protection for upper-layer protocol data units (PDUs)  Tunnel mode Used by network routers to protect IP datagrams passing across insecure network Provides protection for entire IP datagrams

14. 14 Security Associations  Combination of mutually agreed security services, protection mechanisms, and cryptographic keys  ISAKMP SA  IPSec SAs One for inbound traffic One for outbound traffic  Security Parameters Index (SPI) Helps identify an SA  Creating SAs Main Mode for ISAKMP SA Quick Mode for IPSec SAs

15. 15 Agenda  What is IPSec?  How does IPSec Work? (at the packet level!)  Configuring/using IPSec  Issues  Best Practices  Resources

16. 16 Internet Key Exchange  How IPSec peers establish SAs  Combines ISAKMP and the Oakley Key Determination Protocol ISAKMP is used to identify and authenticate peers, manage SAs, and exchange key material Oakley Key Determination Protocol is used to generate secret key material for secure communications (Diffie- Hellman key exchange algorithm)

17. 17 ISAKMP Message Structure IP headerISAKMP payloads UDP message IP datagram UDP header ISAKMP header ISAKMP uses UDP source/destination port 500

18. 18 ISAKMP Header Initiator Cookie Responder Cookie Next Payload Major Version Minor Version Exchange Type Flags Message ID LengthA

19. 19 ISAKMP Payloads  SA  Proposal  Transform  Vendor ID  Nonce  Key Exchange  Notification  Delete  Identification  Hash  Certificate Request  Certificate  Signature

20. 20 IPSec Headers  IPSec Headers live inside IP datagrams and define IPSec contents  Authentication Header (AH) Provides data origin authentication, data integrity, and replay protection for the entire IP datagram  Encapsulating Security Payload (ESP) Provides data origin authentication, data integrity, replay protection, and data confidentiality for the ESP- encapsulated portion of the packet

21. 21 AH Transport Mode IPUpper layer PDU IPAH Authenticated Upper layer PDU

22. 22 AH Tunnel Mode AH Authenticated IP IP (new) Upper layer PDU

23. 23 Next Header Payload Length Reserved Security Parameters Index Sequence Number Authentication Data Payload... Authentication Header (AH)...

24. 24 ESP Transport Mode IPESP Auth Data Encrypted Authenticated IPUpper layer PDU

25. 25 ESP with AH Transport Mode IPESP Auth Encrypted Authenticated with AH IP AH Upper layer PDU Authenticated with ESP

26. 26 ESP Tunnel Mode IP (new)ESP Auth Data IP Encrypted Authenticated IPUpper layer PDU

27. 27 Security Parameters Index Sequence Number Payload Padding Padding Length Next Header Authentication Data... ESP Header and Trailer

28. 28 Internet Key Exchange  Standard that defines a mechanism to establish SAs  Combines ISAKMP and the Oakley Key Determination Protocol ISAKMP is used to identify and authenticate peers, manage SAs, and exchange key material Oakley Key Determination Protocol is used to generate secret key material for secure communications (Diffie- Hellman key exchange algorithm)

29. 29 Main Mode Negotiation  Phases of main mode negotiation: 1. Negotiation of protection suites 2. A Diffie-Hellman exchange 3. Authentication  Six ISAKMP messages 1, 2 – all authentication types 3, 4, 5, and 6 - vary by Authentication type

30. 30 Authentication in MM Negotiation  Kerberos Authentication Kerberos Tokens exchanged and validated  Certificate Authentication Certificates and signatures exchanged and validated  Preshared Key Authentication Hash payloads exchanged and validated

31. 31 Main Mode Negotiation Messages  Message 1 Sent by initiator Contains proposed security association details, vendor ID  Message 2 Sent by responder Contains acceptable SA proposal, vendor ID  These messages negotiate: Encryption (DES, 3DES) Identity Algorithm (MD5, SHA-1) Authentication Method (Kerberos, Pre-shared key, Certificate) Diffie-Hellman group (768-bit, 1024-bit, 2048-bit)

32. 32 Main Mode – Kerberos Authentication  Message 3 - Sent from initiator Contains key exchange, Nonce, initiator’s Kerberos Token, NAT Discovery information  Message 4 - Sent from responder Contains key exchange, Nonce, responder’s Kerberos Token, NAT Discovery information  Message 5 - Sent from initiator (encrypted) Contains identification (of initiator), plus hash  Message 6 - Sent from responder (encrypted) Contains identification (of responder), plus hash

33. 33 Main Mode – Certificate Authentication  Message 3 - Sent from initiator Contains key exchange, Nonce, NAT Discovery information  Message 4 - Sent from responder Contains key exchange, Nonce, NAT Discovery information Also contains certificate request (list of trusted root CAs)  Message 5 - Sent from initiator (encrypted) Contains Initiator’s Certificate, signature  Message 6 - Sent from responder (encrypted) Contains Responder’s certificate, signature

34. 34 Main Mode – Pre-Shared Key Authentication  Message 3 - Sent from initiator Contains Key Exchange, Nonce, NAT Discovery  Message 4 - Sent from responder Contains Key Exchange, Nonce, NAT Discovery  Message 5 - Sent from initiator (encrypted) Contains Identification (of initiator), hash  Message 6 - Sent from responder (encrypted) Contains Identification (of responder), hash

35. 35 Quick Mode Negotiation  Four ISAKMP messages to determine traffic to be secured and how it is secured  Initiator and responder exchange: SA payloads (how to secure traffic) Identification payloads (the traffic to secure)

36. 36 IPSec On the Wire Or Fun With NetMon! Demo

37. 37 Agenda  What is IPSec?  How does IPSec Work?  Configuring/Using IPSec  Issues  Best Practices  Resources

38. 38 Issues with IPSec  Need for machine certificates  Interoperability  Performance/Overhead  NAT traversal

39. 39 Need for Machine Certificates/Keys  IPSec is based on machine to machine communication User credentials not used You therefore need machine certificates  For Kerberos Machine is the security principal Only works for Windows 2000/2003/XP  For Certificate based authentication How to manage/deliver certificates

40. 40 Performance/Overhead  IPSec incurs three sets of overhead: Startup Wire protocol overheads Speed of encryption  Start up over-head Main Mode – 6 packets Quick Mode – 10 packets

41. 41 Wire Protocol Overhead ModeInboundOutboundTotal Bytes Normal IP Traffic2,084,03194,646,22796,730,257 AH Traffic2,700,62796,174,27598,874,902 ESP Traffic2,867,09996,941,13299,808,231 AH Overhead616,5961,528,0482,144,645 ESP Overhead783,0682,294,9053,077,974 % AH Overhead29.59%1.61%2.22% % ESP Overhead37.57%2.42%3.18%

42. 42 IPSec Hardware Acceleration  IPSec per-packet hardware acceleration for 10/100 Ethernet  Client/Svr cards retail circa $100  3Com 3CR990B-97 - 10/100 UTP 3CR990B-FX-97 – 10/100 Fiber Wire Speed IPSec Max 75 SAs supported http://www.3com.com/other/pdfs/products/en/400833.pdf - Or - http://tinyurl.com/3er3f  Intel Intel ® PRO/100 S Desktop/Server http://www.intel.com/network/connectivity/resources/doc_library/documents/ pdf/intel_ipsec_final.pdf - Or - http://tinyurl.com/37hcn

43. 43 XP IPSec Performance Improvements  Doubled number of new SAs per minute  Reliable delete handling in IKE  Doubled packet filtering speed (throughput)  Client LDAP retrieval of AD policy 5 times faster than Windows 2000  Both Intel and 3Com 32bit x86 10/100Ethernet offload support shipping in the box

44. 44 IPSec NAT-T  Network Address Translators (NATs) invalidate IPSec packet protections  IPSec NAT Traversal (NAT-T): Encapsulates ESP-protected payloads with a UDP header Defines additional Main Mode payloads to detect IPSec NAT-T-capable peers and whether either is behind a NAT Defines an additional Quick Mode payload to indicate untranslated addresses Allows ESP-protected traffic to traverse a NAT

45. 45 Agenda  What is IPSec?  How does IPSec Work?  Configuring/Using IPSec  Issues  Best Practices  Resources

46. 46 Scripting  NETSH –C IPSEC – with Windows Server 2003  Netsh IPsec  No dump command  It is included It does nothing!  Help text has few examples  Error messages generally totally unhelpful  Lots of trial and error seems to be needed!

47. 47 Best Practices  Establish an IP Security deployment plan  Avoid Pre-shared keys  Configuration of certificate requests  Script, script, script