Chapter 3

Guide to Operating System Security

Chapter 3

Security Through Authentication and

Encryption

2 Guide to Operating System Security

Objectives

Explain encryption methods and how they are used

Describe authentication methods and how they are used

Explain and configure IP Security Discuss attacks on encryption and

authentication methods

3 Guide to Operating System Security

Encryption

Uses a secret code to disguise data Makes data unintelligible to everyone except

intended recipients Protects data from attackers using a sniffer Uses cryptography Typically involves a key and an algorithm

4 Guide to Operating System Security

Encryption Methods (Continued)

Stream cipher and block cipher Secret key Public key Hashing Data encryption standard (DES) RSA encryption

5 Guide to Operating System Security

Encryption Methods (Continued)

Pluggable authentication modules (PAMs) Microsoft Point-to-Point Encryption (MPPE) Encrypting File System (EFS) Cryptographic File System (CFS)

6 Guide to Operating System Security

Stream Cipher and Block Cipher

Stream cipher Every bit in a stream of data is encrypted

Block cipher Encrypts groupings of data in blocks Typically has specific block and key sizes

7 Guide to Operating System Security

Secret Key

Keeps encryption key secret from public access, particularly over a network connection

Uses symmetrical encryption (same key to encrypt and decrypt)

8 Guide to Operating System Security

Public Key

Uses public key and private key combination (asymmetric encryption)

Public key can be communicated over an unsecured connection

9 Guide to Operating System Security

Hashing

Uses one-way function to mix up message contents Scrambles message Associates it with a unique digital signature Enables it to be picked out of a table

Often used to create a digital signature Hashing algorithms work on only one side of a

two-way communication

10 Guide to Operating System Security

Typically Used Hashing Algorithms

Message Digest 2 (MD2) Message Digest 4 (MD4)

MS-CHAP v1 MS-CHAP in Windows Server 2003

Message Digest 5 (MD5) Secure Hash Algorithm 1 (SHA-1)

11 Guide to Operating System Security

MS-CHAP v1 or MS-CHAP Encryption

12 Guide to Operating System Security

Data Encryption Standard

Developed by IBM; refined by the National Bureau of Standards

Originally developed to use a 56-bit encryption key

New version: 3DES (Triple DES) Hashes data three times Uses a key of up to 168 bits in length

13 Guide to Operating System Security

Using DES with IPSec in Windows Server 2003

14 Guide to Operating System Security

Advanced Encryption Standard

Adopted by U.S. government to replace DES and 3DES

Employs private-key block-cipher form of encryption

Employs an algorithm called Rijndael

15 Guide to Operating System Security

RSA Encryption

Uses asymmetrical public and private keys along with an algorithm that relies on factoring large prime numbers

The algorithm uses a trapdoor function to manipulate prime numbers

More secure than DES and 3DES Used in Internet Explorer and Netscape

Navigator

16 Guide to Operating System Security

Pluggable Authentication Modules

Can be installed in UNIX or Linux OS without rewriting and recompiling existing code

Enable use of encryption techniques other than DES for passwords and communications on a network

17 Guide to Operating System Security

Microsoft Point-to-Point Encryption

Used by Microsoft operating systems for remote communications over PPP or PPTP

Uses RSA encryption Basic encryption (40-bit key) Strong encryption (56-bit key) Strongest encryption (128-bit key)

18 Guide to Operating System Security

Encrypting File System

Set by an attribute of Windows OSs that use NTFS

Protects folder/file contents on hard disk Enables user to encrypt contents of folder/file so it

can only be accessed via private key code by user who encrypted it

Employs DES for encryption Uses a registered recovery agent

19 Guide to Operating System Security

How to Configure EFS

As an advanced folder attribute By using cipher command in Command

Prompt window

20 Guide to Operating System Security

Configuring EFS as an Advanced Folder Attribute

21 Guide to Operating System Security

Cipher Command-Line Parameters

22 Guide to Operating System Security

Cryptographic File System

File system add-on available as open source software for UNIX and Linux systems

Enables encryption of disk file systems and NFS files

23 Guide to Operating System Security

Summary of Encryption Techniques (Continued)

continued…

24 Guide to Operating System Security

Summary of Encryption Techniques (Continued)

25 Guide to Operating System Security

Authentication

Process of verifying that a user is authorized to access particular resources

Typically associated with logon process Validates both user account name and

password before giving access to resources Often uses encryption techniques to protect

user names and passwords

26 Guide to Operating System Security

Authentication Methods (Continued)

Session authentication Digital certificates NT LAN Manager Kerberos Extensible Authentication Protocol (EAP) Secure Sockets Layer (SSL)

27 Guide to Operating System Security

Authentication Methods (Continued)

Transport Layer Security (TLS) Secure Shell (SSH) Security token

28 Guide to Operating System Security

Session Authentication

Ensures packets can be read in correct order Provides a way to encrypt the sequence order

to discourage attackers

29 Guide to Operating System Security

Digital Certificate

Set of unique identification information typically put at the end of the file or associated with a computer communication

Shows that the source of the file or communication is legitimate

Typically encrypted by a private key and decrypted by a public key

Issued by a certificate authority

30 Guide to Operating System Security

Digital Certificate Contents

Version Certificate serial number Signature algorithm identifier Name of issuer Validity period Subject name Subject public key information

31 Guide to Operating System Security

NT LAN Manager

Form of session authentication and challenge/response authentication compatible with all Microsoft Windows operating systems

Challenge/response authentication Hashes an account’s password Uses a secret key

32 Guide to Operating System Security

Kerberos

Employs private-key security and use of tickets that are exchanged between the client who requests logon and network services access and the server, application, or directory service that grants access

33 Guide to Operating System Security

Kerberos Configuration Options

34 Guide to Operating System Security

Extensible Authentication Protocol

Multipurpose authentication method used on networks and in remote communications

Can employ many encryption methods (DES, 3DES, public key encryption, smart cards, and certificates)

Typically provides an authentication communication between a computer and a server used to authenticate computer’s access

35 Guide to Operating System Security

Secure Sockets Layer

Service-independent; broad uses fore-commerce, HTTP, HTTPS, FTP, SMTP, and NNTP

Developed by Netscape Uses RSA public-key encryption Most commonly used form of security for

communications and transactions over the Web

36 Guide to Operating System Security

Transport Layer Security

Modeled after SSL Uses private-key symmetric data encryption

and TLS Handshake Protocol

37 Guide to Operating System Security

Secure Shell

Developed for UNIX/Linux systems to provide authentication security for TCP/IP applications, including FTP and Telnet

38 Guide to Operating System Security

Using Secure Shell

39 Guide to Operating System Security

Security Token

Physical device, often resembling a credit card or keyfob, used for authentication

Communicates with an authentication server to generate the password, using encryption for exchange of password-generating information

40 Guide to Operating System Security

Advantages of Security Token

User does not have to memorize password Value of password only lasts as long as the

communications session; new password is created next time the security token is used

41 Guide to Operating System Security

42 Guide to Operating System Security

IP Security (IPSec)

Set of IP-based secure communications and encryption standards developed by the IETF

Protect network communications through IP Elements that enable security measures

Authentication header Encapsulating Security Payload (ESP)

43 Guide to Operating System Security

IPSec Security Roles

44 Guide to Operating System Security

Authentication Header (AH)

Ensures integrity of a data transmission Ensures authentication of a packet by enabling

verification of its source

45 Guide to Operating System Security

Specific Fields in AH

Next header Payload length Reserved Security Parameter Index (SPI) Sequence number Authentication Data

46 Guide to Operating System Security

Encapsulating Security Payload (ESP)

Encrypts packet-based data Authenticates data Generally ensures security and confidentiality

of network layer information and data within packet

47 Guide to Operating System Security

Specific Fields in ESP

Security Parameter Index (SPI) Sequence number Payload data Padding Pad length Next header Authentication date

48 Guide to Operating System Security

Attacks on Encryption and Authentication

49 Guide to Operating System Security

Guidelines for Resisting Attacks

Use strong passwords Use strongest forms of authentication and

encryption permitted by OS Use longest encryption keys possible Inventory encryption and authentication

methods used by OS; close any holes Have administrators use personal accounts

with administrative privileges (rather than use administrative account directly)

50 Guide to Operating System Security

Summary

Encryption methods and how operating systems use them

How systems authenticate one another How to configure Kerberos authentication

logon security How to use IP security to keep your TCP/IP

network secure Typical methods attackers use to defeat

encryption and authentication