1. CSCE 715: Network Systems Security
Chin-Tser Huang
University of South Carolina

2. Security in Network Layer
Implementing security in application layer provides flexibility in security policy and key management
Problem is need to implement security mechanism in every application individually
To reduce the overhead, implement security in network layer to provide security for all applications between selected pair of computers
02/23/2009

3. IPSec Current security standard for IP layer
Provide general security services for IP
Authentication
Confidentiality
Anti-replay
Key management
Applicable to use over LANs, across public and private WANs, and for the Internet
02/23/2009

4. Scenario of IPSec Uses
02/23/2009

5. Benefits of IPSec
Provide strong security to all traffic crossing the perimeter if installed in a firewall/router
Resistant to bypass
IPSec is below transport layer, hence transparent to applications
Can be transparent to end users
Can provide security for individual users if desired
02/23/2009

6. IP Security Architecture
Specification is quite complex
Defined in numerous RFC’s
RFC 2401/2402/2406/2408
many others, grouped by category
Two protocols
Authentication Header (AH)
Encapsulating Security Payload (ESP)
Mandatory in IPv6, optional in IPv4
02/23/2009

7. Security Association (SA)
A unidirectional relationship between sender and receiver that affords security for traffic flow
Each IPSec computer maintains a database of SA’s
Defined by 3 parameters
Security Parameters Index (SPI)
IP Destination Address
Security Protocol Identifier
02/23/2009

8. SA Parameters Sequence Number Counter Sequence Number Overflow
Anti-Replay Window
AH and ESP information
Lifetime
IPSec Protocol Mode
Path MTU
02/23/2009

9. Authentication Header (AH)
Provide support for data integrity and authentication of IP packets
end system/router can authenticate user/app
prevent address spoofing attacks
guard against replay attacks by tracking sequence numbers
Based on use of a MAC
HMAC-MD5-96 or HMAC-SHA-1-96
MAC is calculated over IP header fields that are either immutable or predictable, AH header other than authentication data, and entire upper-level protocol data
Parties must share a secret key
02/23/2009

10. Authentication Header
02/23/2009

11. Transport vs Tunnel Mode AH
Transport mode is used to authenticate IP payload and selected portion of IP header
good for host to host traffic
Tunnel mode authenticates entire IP packet and selected portion of outer IP header
good for VPNs, gateway to gateway security
02/23/2009

12. End-to-End vs End-to-Intermediate Authentication
02/23/2009

13. Scope of AH Authentication
02/23/2009

14. Encapsulating Security Payload (ESP)
Provide message content confidentiality and limited traffic flow confidentiality
Can optionally provide the same authentication services as AH
Support range of ciphers, modes, padding
DES, Triple-DES, RC5, IDEA, CAST etc
CBC most common
pad to meet blocksize, for traffic flow
02/23/2009

15. Encapsulating Security Payload
02/23/2009

16. Padding Serve several purposes expand the plaintext to required length
make Pad Length and Next Header fields aligned to 32-bit word boundary
conceal actual length of payload
02/23/2009

17. Transport vs Tunnel Mode ESP
Transport mode is used to encrypt and optionally authenticate IP data
data protected but header left in clear
can suffer from traffic analysis but is efficient
good for ESP host to host traffic
Tunnel mode encrypts entire IP packet
add new header for next hop
can counter traffic analysis
good for VPNs, gateway to gateway security
02/23/2009

18. Transport vs Tunnel Mode ESP
02/23/2009

19. Scope of ESP Encryption and Authentication
02/23/2009

20. Combining Security Associations
SAs can implement either AH or ESP, but each SA can implement only one
Some traffic flows may require services of both AH and ESP, while some other flows may require both transport and tunnel modes
To address these concerns, need to combine SAs to form a security association bundle
02/23/2009

21. Authentication plus Confidentiality
Which one first? Three approaches to consider
ESP with Authentication Option
Transport mode or tunnel mode
Authentication after encryption
Transport Adjacency
A bundle of two transport SAs, with the inner being an ESP SA and the outer being an AH SA
Transport-Tunnel Bundle
A bundle consisting of an inner AH transport SA and an outer ESP tunnel SA
Authentication before encryption
02/23/2009

22. Combining Security Associations
02/23/2009

23. Key Management Handle key generation and distribution
Typically need 2 pairs of keys
2 per direction for AH & ESP
Manual key management
sysadmin manually configures every system
Automated key management
automated system for on demand creation of keys for SA’s in large systems
Oakley and ISAKMP
02/23/2009

24. OAKLEY A key exchange protocol Based on Diffie-Hellman key exchange
Add features to address weaknesses of Diffie-Hellman
cookies to counter clogging attacks
nonces to counter replay attacks
key exchange authentication to counter man-in-the-middle attacks
Can use arithmetic in prime fields or elliptic curve fields
02/23/2009

25. Usage of Cookies Three basic requirements
Must depend on specific parties
Impossible for anyone other than issuing entity to generate cookies that will be accepted by issuing entity
Cookie generation and verification must be fast
To create a cookie, perform a fast hash over src and dst IP addresses, src and dst ports, and a locally generated secret value
02/23/2009

26. ISAKMP Internet Security Association and Key Management Protocol
Provide framework for key management
Define procedures and packet formats to establish, negotiate, modify, and delete SAs
Independent of key exchange protocol, encryption algorithm, and authentication method
02/23/2009

27. ISAKMP Header
02/23/2009

28. ISAKMP Payload
02/23/2009

29. ISAKMP Exchange
02/23/2009

30. ISAKMP Exchange
02/23/2009

31. Next Class
Denial-of-Service (DoS) attack
Hop Integrity
02/23/2009