CSCE 715:Network Systems Security

Chin-Tser Huanghuangct@cse.sc.edu

University of South Carolina

10/04/2011 2

Security in Network Layer Implementing security in application layer

provides flexibility in security policy and key management

Problem is the need to implement security mechanism in every application individually

To reduce the overhead, implement security in network layer to provide security for all applications between selected pair of computers

10/04/2011 3

IPSec Current security standard for IP layer Provide general security services for IP

Authentication Confidentiality Anti-replay Key management

Applicable to use over LANs, across public and private WANs, and for the Internet

10/04/2011 4

Scenario of IPSec Uses

10/04/2011 5

Benefits of IPSec Provide strong security to all traffic

crossing the perimeter if installed in a firewall/router

Resistant to bypass IPSec is below transport layer, hence

transparent to applications Can be transparent to end users Can provide security for individual users

if desired

10/04/2011 6

IP Security Architecture Specification is quite complex Defined in numerous RFC’s

Latest version in RFC 4301/4302/4303/4306

many others, grouped by category Two protocols

Authentication Header (AH) Encapsulating Security Payload (ESP)

Mandatory in IPv6, optional in IPv4

IP Security Architecture

10/04/2011 7

Transport Mode and Tunnel Mode

Transport mode provides protection for upper-layer protocols, namely on the payload of an IP packet

Suitable for end-to-end communication between two hosts

Tunnel mode provides protection to the entire IP packet

Entire packet plus security fields is treated as the payload of new outer IP packet with new outer IP header

Suitable when one or both ends of SA are a security gateway

10/04/2011 8

10/04/2011 9

Security Association (SA) A unidirectional relationship between

sender and receiver that affords security for traffic flow

Each IPSec computer maintains a database of SA’s

Defined by 3 parameters Security Parameters Index (SPI) IP Destination Address Security Protocol Identifier

10/04/2011 10

SA Parameters

Sequence Number Counter Sequence Number Overflow Anti-Replay Window AH and ESP information Lifetime IPSec Protocol Mode Path MTU

Security Policy Database Used to relate IP traffic to specific SAs (or no SA

if the given traffic is allowed to bypass IPsec) Each SPD entry is defined by selectors, which

are a set of IP and upper-layer protocol field values Remote IP address Local IP address Next layer protocol Name Local and remote ports

10/04/2011 11

IP Traffic Processing:Outbound Packets

10/04/2011 12

IP Traffic Processing:Inbound Packets

10/04/2011 13

10/04/2011 14

Authentication Header (AH) Provide support for data integrity and

authentication of IP packets end system/router can authenticate user/app prevent address spoofing attacks guard against replay attacks by tracking sequence

numbers Based on use of a MAC

HMAC-MD5-96 or HMAC-SHA-1-96 MAC is calculated over IP header fields that are either

immutable or predictable, AH header other than authentication data, and entire upper-level protocol data

Parties must share a secret key

10/04/2011 15

Authentication Header

10/04/2011 16

Transport vs Tunnel Mode AH Transport mode is used to authenticate

IP payload and selected portion of IP header good for host to host traffic

Tunnel mode authenticates entire IP packet and selected portion of outer IP header good for VPNs, gateway to gateway security

10/04/2011 17

Encapsulating Security Payload (ESP)

Provide message content confidentiality and limited traffic flow confidentiality

Can optionally provide the same authentication services as AH

Support range of ciphers, modes, padding DES, Triple-DES, RC5, IDEA, CAST etc CBC most common pad to meet blocksize, for traffic flow

10/04/2011 18

Encapsulating Security Payload

10/04/2011 19

Padding

Serve several purposes expand the plaintext to required

length make Pad Length and Next Header

fields aligned to 32-bit word boundary conceal actual length of payload

10/04/2011 20

Transport vs Tunnel Mode ESP Transport mode is used to encrypt and

optionally authenticate IP data data protected but header left in clear can suffer from traffic analysis but is efficient good for ESP host-to-host traffic

Tunnel mode encrypts entire IP packet add new header for next hop can counter traffic analysis good for VPNs, gateway-to-gateway security

10/04/2011 21

Transport vs Tunnel Mode ESP

10/04/2011 22

Scope of ESP Encryption and Authentication

10/04/2011 23

Combining Security Associations

SAs can implement either AH or ESP, but each SA can implement only one

Some traffic flows may require services of both AH and ESP, while some other flows may require both transport and tunnel modes

To address these concerns, need to combine SAs to form a security association bundle

10/04/2011 24

Authentication plus Confidentiality

Which one first? Three approaches to consider ESP with Authentication Option

Transport mode or tunnel mode Authentication after encryption

Transport Adjacency A bundle of two transport SAs, with the inner

being an ESP SA and the outer being an AH SA Authentication after encryption

Transport-Tunnel Bundle A bundle consisting of an inner AH transport SA

and an outer ESP tunnel SA Authentication before encryption

10/04/2011 25

Combining Security Associations

10/04/2011 26

Key Management Handle key generation and distribution Typically need 2 pairs of keys

2 per direction for AH & ESP Manual key management

sysadmin manually configures every system Automated key management

automated system for on demand creation of keys for SA’s in large systems

Oakley and ISAKMP are two essential elements IKEv2 does not use the terms Oakley and ISAKMP

but basic functionality is the same

10/04/2011 27

IKE Key Determination (OAKLEY) A key exchange protocol Based on Diffie-Hellman key exchange Add features to address weaknesses of

Diffie-Hellman cookies to counter clogging attacks nonces to counter replay attacks key exchange authentication to counter

man-in-the-middle attacks Can use arithmetic in prime fields or

elliptic curve fields

10/04/2011 28

Usage of Cookies Three basic requirements

Must depend on specific parties Impossible for anyone other than issuing entity

to generate cookies that will be accepted by issuing entity

Cookie generation and verification must be fast To create a cookie, perform a fast hash

over src and dst IP addresses, src and dst ports, and a locally generated secret value

10/04/2011 29

ISAKMP Internet Security Association and Key

Management Protocol Provide framework for key management Define procedures and packet formats

to establish, negotiate, modify, and delete SAs

Independent of key exchange protocol, encryption algorithm, and authentication method

10/04/2011 30

IKE Header

10/04/2011 31

IKE Payload

10/04/2011 32

IKE Exchange

10/04/2011 33

Next Class

Denial-of-Service (DoS) attack Hop Integrity