2: Application Layer1 School of Computing Science Simon Fraser University CMPT 371: Data Communications and Networking Chapter 2: Application Layer

2: Application Layer 1

School of Computing Science Simon Fraser University

CMPT 371: Data Communications and CMPT 371: Data Communications and NetworkingNetworking

Chapter 2: Application LayerChapter 2: Application Layer


Chapter 2: Application LayerOur goals: Understand conceptual and implementation

aspects of network application protocols

Learn about protocols by examining popular application-level protocols (HTTP and DNS)

Know how to develop network applications socket programming


Chapter 2: Roadmap

Principles of network applications Web and HTTP Domain Name System (DNS) Socket programming


Some network apps

E-mail Web Instant messaging Remote login P2P file sharing Multi-user network

games Streaming stored

video clips

Internet telephone Real-time video

conference Massive parallel

computing


What is a network app?

Programs that run on different end

systems and communicate over a

network. e.g., Web: Web server

software communicates with browser software

little software written for devices in network core network core devices do not

run user application code application on end systems

allows for rapid app development, propagation

application

transportnetworkdata linkphysical

application


application



How to create a network app? Design application architecture

how to organize the app over end systems Choose network transport service(s)

which service to use (TCP, UDP) depends on app requirements (delay, loss,

bw, …) Design app protocol

message types, format, actions, … Write code

implement the protocol


Application architectures

How to organize app over end systems

Client-server

Peer-to-peer (P2P)

Hybrid of client-server and P2P


Client-server architectureserver:

always-on host permanent IP address server farms for

scaling

clients: communicate with

server may be intermittently

connected may have dynamic IP

addresses do not communicate

directly with each other


Pure P2P architecture

no always-on server arbitrary end systems

directly communicate peers are intermittently

connected and change IP addresses

example: Gnutella

Highly scalable

But difficult to manage


Hybrid of client-server and P2PNapster

File transfer P2P File search centralized:

• Peers register content at central server• Peers query same central server to locate content

Instant messaging Chatting between two users is P2P Presence detection/location centralized:

• User registers its IP address with central server when it comes online

• User contacts central server to find IP addresses of buddies


Choosing transport services: App requirements

Data loss some apps (e.g., audio)

can tolerate some loss other apps (e.g., file

transfer, telnet) require 100% reliable data transfer

Timing some apps (e.g.,

Internet telephony, interactive games) require low delay to be “effective”

Bandwidth some apps (e.g.,

multimedia) require minimum amount of bandwidth to be “effective”

other apps (“elastic apps”) make use of whatever bandwidth they get


Requirements of common Apps

Application

file transfere-mail

Web documentsreal-time audio/video

stored audio/videointeractive gamesinstant messaging

Data loss

no lossno lossno lossloss-tolerant

loss-tolerantloss-tolerantno loss

Bandwidth

elasticelasticelasticaudio: 5kbps-1Mbpsvideo:10kbps-5Mbpssame as above few kbps upelastic

Time Sensitive

nononoyes, 100’s msec

yes, few secsyes, 100’s msecyes and no


Internet transport protocols services

TCP service: connection-oriented: setup

required between client and server processes

reliable transport between sending and receiving process

flow control: sender won’t overwhelm receiver

congestion control: throttle sender when network overloaded

does not provide: timing, minimum bandwidth guarantees

UDP service: unreliable data transfer

between sending and receiving process

does not provide: connection setup, reliability, flow control, congestion control, timing, or bandwidth guarantee

Q: why bother? Why is there a UDP?


Internet apps: application, transport protocols

Application

e-mailremote terminal access

Web file transfer

streaming multimedia

Internet telephony

Applicationlayer protocol

SMTP [RFC 2821]Telnet [RFC 854]HTTP [RFC 2616]FTP [RFC 959]proprietary(e.g. RealNetworks)proprietary(e.g., Vonage,Dialpad)

Underlyingtransport protocol

TCPTCPTCPTCPTCP or UDP

typically UDP


Design app protocol

Protocol defines … Types of messages:

request & response messages

Syntax of message types: what fields in messages & how fields are delineated

Semantics fields: meaning of information in fields

Rules for when and how processes send & respond to messages

Public-domain protocols:

defined in RFCs allows for

interoperability e.g., HTTP, SMTPProprietary

protocols: e.g., KaZaA


Writing network app code

Choose a language that supports network programming (aka socket programming) Java, C, C++, Python, …

Let us briefly discuss network programming more on this later

Note: we will talk about processes, not programs process = program running


Processes communicating

Process: program running within a host

within same host, two processes communicate using inter-process communication

processes in different hosts communicate by exchanging messages

Client process: process that initiates communication

Server process: process that waits to be contacted

Note: applications with P2P architectures have client processes & server processes


Sockets

process sends/receives messages to/from its socket

socket analogous to door sending process shoves

message out door sending process relies on

transport infrastructure on other side of door which brings message to socket at receiving process

process

TCP withbuffers,variables

socket

host orserver

process

TCP withbuffers,variables

socket

host orserver

Internet

controlledby OS

controlled byapp developer

socket is the interface (API) between application and transport layer


Addressing processes For a process to

receive messages, it must have an identifier

A host has a unique32-bit IP address

Q: does the IP address of the host on which the process runs suffice for identifying the process?

A: No, many processes can be running on same host

We use ports Process is identified by:

IP address, Transport protocol,

and Port number

Example port numbers: HTTP server: 80 (TCP) Mail server: 25 (TCP)

More on this later


Chapter 2: Roadmap

Principles of network applications Web and HTTP Domain Name System (DNS) Socket programming


Web and HTTP

First some jargon Web page consists of objects Object can be HTML file, JPEG image, Java

applet, audio file,… Web page consists of base HTML-file which

includes several referenced objects Each object is addressable by a URL Example URL:

www.someschool.edu/someDept/pic.gif

host name path name


HTTP: Hypertext Transfer Protocol Application-layer protocol for

Web Specified in

HTTP 1.0: RFC 1945 HTTP 1.1: RFC 2068

Client/server model client: browser that requests,

receives, “displays” Web objects

server: Web server sends objects in response to requests

Uses TCP on port 80 “stateless” protocol

“Cookies” are used to add some state (info about user)

PC runningExplorer

Server running

Apache Webserver

Mac runningNavigator

HTTP request

HTTP request

HTTP response

HTTP response


HTTP connections

Nonpersistent HTTP At most one object is

sent over a TCP connection

HTTP/1.0 uses nonpersistent HTTP

Persistent HTTP Multiple objects can

be sent over single TCP connection between client and server

HTTP/1.1 uses persistent connections in default mode


Response time modelingDefinition of RTT: time to

send a small packet to travel from client to server and back

Response time: one RTT to initiate TCP

connection one RTT for HTTP

request and first few bytes of HTTP response to return

file transmission timetotal = 2RTT+transmit

time

time to transmit file

initiate TCPconnection

RTT

requestfile

RTT

filereceived

time time


HTTP messages

two types of HTTP messages: request, response

HTTP request message: ASCII (human-readable format)

GET /somedir/page.html HTTP/1.1Host: www.someschool.edu User-agent: Mozilla/4.0Connection: close Accept-language:fr

(extra carriage return, line feed)

request line(GET, POST,

HEAD commands)

header lines

Carriage return, line feed

indicates end of message


HTTP request message: general format


HTTP response message

HTTP/1.1 200 OK Connection closeDate: Thu, 06 Aug 1998 12:00:15 GMT Server: Apache/1.3.0 (Unix) Last-Modified: Mon, 22 Jun 1998 …... Content-Length: 6821 Content-Type: text/html data data data data data ...

status line(protocol

status codestatus phrase)

header lines

data, e.g., requestedHTML file

See it yourself using Ethereal


HTTP response status codes

200 OK request succeeded, requested object later in this

message

301 Moved Permanently requested object moved, new location specified later in

this message (Location:)

400 Bad Request request message not understood by server

404 Not Found requested document not found on this server

505 HTTP Version Not Supported

In first line in server->client response message.A few sample codes:


Cookies: keeping “state” in HTTP

client server

usual http request msgusual http response

+Set-cookie: 1678

usual http request msg

cookie: 1678usual http response

msg

usual http request msg

cookie: 1678usual http response msg

cookie-specificaction

cookie-spectificaction

servercreates ID

1678 for user

entry in backend

database

access

acce

ss

Cookie file

amazon: 1678ebay: 8734

Cookie file

ebay: 8734

Cookie file

amazon: 1678ebay: 8734

one week later:


Cookies (cont’d)

What cookies can bring:

authorization shopping carts recommendations user session state

(Web e-mail)

Cookies and privacy: cookies permit sites to

learn a lot about you you may supply name

and e-mail to sites search engines use

redirection & cookies to learn yet more

advertising companies obtain info across sites

aside


Web caches (proxy servers)

Browser accesses web server via cache

Browser sends all HTTP requests to cache

if object in cache: cache returns object

else cache requests object from origin server, then returns object to client

client

Proxyserver

client

HTTP request

HTTP request

HTTP response

HTTP response

HTTP request

HTTP response

origin server

origin server


Web caching (cont’d)

Cache acts as both client and server Typically cache is installed by ISP (university,

company, residential ISP) Why Web caching? Reduce response time for client request Reduce traffic on an institution’s access link

reduce cost


Caching example

Assumptions average object size = 100,000 bits avg. request rate from institution’s

browsers to origin servers = 15/sec delay from institutional router to

any origin server and back to router = 2 sec (Internet delay)

Consequences utilization on LAN = 15% utilization on access link = 100% total delay = ??Internet delay + access delay + LAN

delay= 2 sec + minutes + milliseconds

Problem: Very large delay (minutes)

originservers

public Internet

institutionalnetwork 10 Mbps LAN

1.5 Mbps access link

institutionalcache


Caching example (cont’d)

Possible solution 1 increase bandwidth of

access link to, say, 10 Mbps often a costly upgrade

Consequences utilization on LAN = 15% utilization on access link =

15% Total avg delay = Internet

delay + access delay + LAN delay

= 2 sec + msecs + msecs ≈ 2 sec

originservers

public Internet


10 Mbps access link

institutionalcache


Caching example (cont’d)Possible solution 2 Install cache

suppose hit rate is 0.4

Consequence 40% requests will be satisfied

almost immediately 60% requests satisfied by origin

server utilization of access link

reduced to 60%, resulting in negligible delays (say 10 msec)

Total avg delay = Internet delay + access delay + LAN delay = 0.6 * (2 + 0.01) sec + 0.4 * msecs ≈ 1.2 sec

originservers

public Internet


1.5 Mbps access link

institutionalcache


Problem with Caching

What problem does caching introduce? Stale objects

Solution? use Time To Live (TTL) and

conditional get cache: specify date of

cached copy in HTTP requestIf-modified-since: <date>

server: response contains no object if cached copy is up-to-date: HTTP/1.0 304 Not Modified

cache server

HTTP request msgIf-modified-since:

<date>

HTTP responseHTTP/1.0

304 Not Modified

object not

modified

HTTP request msgIf-modified-since:

<date>

HTTP responseHTTP/1.0 200 OK

<data>

object modified


Chapter 2: Roadmap

Principles of network applications Web and HTTP File Transfer Protocol (FTP) Domain Name System (DNS) Socket programming


FTP: file transfer protocol

transfer file to/from remote host client/server model

client: side that initiates transfer (either to/from remote) server: remote host

ftp: RFC 959 ftp server: port 21

file transfer FTPserver

FTPuser

interface

FTPclient

local filesystem

remote filesystem

user at host


FTP: separate control, data connections

FTP client contacts FTP server at port 21, specifying TCP as transport protocol

Client obtains authorization over control connection

Client browses remote directory by sending commands over control connection

When server receives a command for a file transfer, the server opens a TCP data connection to client

After transferring one file, server closes connection

FTPclient

FTPserver

TCP control connection

port 21

TCP data connectionport 20

Server opens a second TCP data connection to transfer another file

Control connection: “out of band”

FTP server maintains “state”: current directory, earlier authentication


Chapter 2: Roadmap



DNS: Domain Name System People: many identifiers

Name: good for humans SIN, passport #: good for machines

Internet hosts, routers: two identifiers IP address (32 bit): good for routers Name: good for humans

E.g., 142.58.102.1 vs. www.sfu.ca

Problem: How to map names to IPs?

Solution: DNS, Domain Name System An Internet Directory


DNS Services Hostname to IP address translation

www.sfu.ca 142.58.102.1 Host aliasing

canonical and alias names E.g., relay1.west-coast.hotmail.com vs. hotmail.com

Mail server aliasing can use same name for mail and web servers @sfu.ca, www.sfu.ca although they are different servers

Load distribution Replicated Web servers: set of IP addresses for one

canonical name For every request, DNS returns the same set but in a

different order, clients typically use the first one in reply


DNS Architecture Distributed database

implemented in a hierarchy of many name servers No single server has all mappings, it is distributed across all

servers Application-layer protocol

host, routers, name servers communicate to resolve names (address/name translation)

Notes: core Internet function (i.e., address mapping) implemented as

application-layer protocol complexity at network’s “edge” Why distributed? Why not centralized DNS? Because centralized would:

• be single point of failure• incur huge traffic volume• be distant from many clients• require a lot of maintenance

Which means, it would not scale!


Distributed, Hierarchical Database

Root DNS servers: 13 (replicated) servers worldwide

b USC-ISI Marina del Rey, CAl ICANN Los Angeles, CA

e NASA Mt View, CAf Internet Software C. Palo Alto, CA (and 17 other locations)

i Autonomica, Stockholm (plus 3 other locations)

k RIPE London (also Amsterdam, Frankfurt)

m WIDE Tokyo

a Verisign, Dulles, VAc Cogent, Herndon, VA (also Los Angeles)d U Maryland College Park, MDg US DoD Vienna, VAh ARL Aberdeen, MDj Verisign, ( 11 locations)

Root DNS Servers

com DNS servers org DNS servers edu DNS servers

poly.eduDNS servers

umass.eduDNS servers

yahoo.comDNS servers

amazon.comDNS servers

pbs.orgDNS servers

Top-level

Authoritative


TLD and Authoritative Servers Top-level domain (TLD) servers:

responsible for com, org, net, edu, etc, and all top-level country domains uk, fr, ca, jp.

Network Solutions maintains servers for com TLD

Educause for edu TLD Authoritative DNS servers:

organization’s DNS servers, providing authoritative hostname to IP mappings for organization’s servers (e.g., Web and mail).

Can be maintained by organization or service provider


Local Name Server

Does not strictly belong to hierarchy Each ISP (residential ISP, company,

university) has one Also called “default name server”

When a host makes a DNS query, query is sent to its local DNS server Acts as a proxy, forwards query into

hierarchy


requesting hostcis.poly.edu

gaia.cs.umass.edu

root DNS server

local DNS serverdns.poly.edu

1

23

4

5

6

authoritative DNS serverdns.cs.umass.edu

78

edu TLD DNS server

Example

Host at cis.poly.edu wants IP address for gaia.cs.umass.edu

Notes: 1 is recursive query

burden on contacted server

2-7 are iterative queries

Do you see problems in this system?

A lot of traffic and long delay

Solution? Caching!


DNS: caching once (any) name server learns mapping, it

caches this mapping cache entries timeout, disappear after

some time TLD servers typically cached in local

name servers Thus root name servers not often visited


DNS records

DNS: distributed db storing resource records (RR)

Type=NS name is domain (e.g.

foo.com) value is hostname of

authoritative name server for this domain

RR format: (name, value, type, ttl)

Type=A name is hostname value is IP address

Type=CNAME name is alias name for some

“canonical” (the real) name www.ibm.com is really servereast.backup2.ibm.com value is canonical name

Type=MX value is mailserver

associated with name (sfu.ca, mail.sfu.ca, MX)


DNS protocol, messagesDNS protocol : query and reply messages, both with same message format

msg header identification: 16 bit #

for query, reply to query uses same #

flags: query or reply recursion desired recursion available reply is authoritative


DNS protocol, messages

Name, type fields for a query

RRs in responseto query

records forauthoritative servers

additional “helpful”info that may be used


Inserting records into DNS Example: just created startup “Network Utopia” Register name networkuptopia.com at a registrar

(e.g., Network Solutions) Need to provide registrar with names and IP addresses

of your authoritative name server (primary and secondary)

Registrar inserts two RRs into the com TLD server:

(networkutopia.com, dns1.networkutopia.com, NS)(dns1.networkutopia.com, 212.212.212.1, A)

Put in authoritative server: Type A record for www.networkuptopia.com, and Type MX record for @networkutopia.com


Chapter 2: Roadmap



Socket programming

Socket API introduced in BSD4.1 UNIX, 1981 explicitly created, used, released by apps client/server paradigm two types of transport service via socket API:

reliable, byte stream-oriented unreliable datagram

Goal: learn how to build client/server applications that communicate using sockets


Socket-programming using TCP

Socket: a door between application process and transport protocol (TCP or UDP)

TCP service: reliable transfer of bytes from one process to another

process

TCP withbuffers,

variables

socket

controlled byapplicationdeveloper

controlled byoperating

system

host orserver

process

TCP withbuffers,

variables

socket

controlled byapplicationdeveloper

controlled byoperatingsystem

host orserver

internet


Overview of Socket programming with TCP server process must first

be running, and creates a socket (door)

that welcomes client’s contact, then wait

client contacts server by creating local TCP socket using IP address, port number of server process

when client creates socket: client TCP establishes connection to server TCP

when contacted by client, server TCP creates new socket for server process to communicate with client allows server to talk with

multiple clients source port numbers and IPs

used to distinguish clients

TCP provides reliable, in-order transfer of bytes (“pipe”) between client and server

application viewpoint


Client/server socket interaction: TCP

wait for incomingconnection requestconnectionSocket =welcomeSocket.accept()

create socket,port=x, forincoming request:welcomeSocket =

ServerSocket()

create socket,connect to hostid, port=xclientSocket =

Socket()

closeconnectionSocket

read reply fromclientSocket

closeclientSocket

Server (running on hostid) Client

send request usingclientSocketread request from

connectionSocket

write reply toconnectionSocket

TCP connection setup


Socket programming with TCP

Example client-server app:1) client reads line from standard input

(inFromUser stream), sends to server via socket (outToServer stream)

2) server reads line from socket3) server converts line to uppercase, sends back

to client4) client reads, prints modified line from socket

(inFromServer stream)


Example: Java client (TCP)

import java.io.*; import java.net.*; class TCPClient {

public static void main(String argv[]) throws Exception { String sentence; String modifiedSentence;

BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));

Socket clientSocket = new Socket("hostname", 6789);

DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());

Createinput stream

Create client socket,

connect to server

Createoutput stream

attached to socket


Example: Java client (TCP), cont’d

BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));

sentence = inFromUser.readLine();

outToServer.writeBytes(sentence + '\n');

modifiedSentence = inFromServer.readLine();

System.out.println("FROM SERVER: " + modifiedSentence);

clientSocket.close(); } }

Createinput stream

attached to socket

Send lineto server

Read linefrom server


Example: Java server (TCP)import java.io.*; import java.net.*;

class TCPServer {

public static void main(String argv[]) throws Exception { String clientSentence; String capitalizedSentence;

ServerSocket welcomeSocket = new ServerSocket(6789); while(true) { Socket connectionSocket = welcomeSocket.accept();

BufferedReader inFromClient = new BufferedReader(new InputStreamReader(connectionSocket.getInputStream()));

Createwelcoming socket

at port 6789

Wait on welcomingsocket for contact

by client

Create inputstream, attached

to socket


Example: Java server (TCP), cont’d

DataOutputStream outToClient = new DataOutputStream(connectionSocket.getOutputStream());

clientSentence = inFromClient.readLine();

capitalizedSentence = clientSentence.toUpperCase() + '\n';

outToClient.writeBytes(capitalizedSentence); } } }

Read in linefrom socket

Create outputstream,

attached to socket

Write out lineto socket

End of while loop,loop back and wait foranother client connection

Q. Does this server handle multiple concurrent connections?

A. NO. To do so, create thread after accept() to handle new connection


Socket programming with UDP

UDP: no “connection” between client and server

no handshaking sender explicitly attaches

IP address and port of destination to each packet

server must extract IP address, port of sender from received packet

UDP: transmitted data may be received out of order, or lost

application viewpoint

UDP provides unreliable transfer of groups of bytes (“datagrams”)

between client and server


Client/server socket interaction: UDP

closeclientSocket

Server (running on hostid)

create socket,clientSocket = DatagramSocket()

Client

read reply fromclientSocket

Create datagram (hostid,port=x,data)send datagram request using clientSocket

create socket,port=x, forincoming request:serverSocket = DatagramSocket()

read request fromserverSocket

write reply toserverSocketspecifying clienthost address,port number


Example: Java client (UDP)

import java.io.*; import java.net.*; class UDPClient { public static void main(String args[]) throws Exception { BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in)); DatagramSocket clientSocket = new DatagramSocket(); InetAddress IPAddress = InetAddress.getByName("hostname"); byte[] sendData = new byte[1024]; byte[] receiveData = new byte[1024]; String sentence = inFromUser.readLine();

sendData = sentence.getBytes();

Createinput stream

Create client socket

Translate hostname to IP

address using DNS


Example: Java client (UDP), cont’d

DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, 9876); clientSocket.send(sendPacket); DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); clientSocket.receive(receivePacket); String modifiedSentence = new String(receivePacket.getData()); System.out.println("FROM SERVER:" + modifiedSentence); clientSocket.close(); }

}

Create datagram with data-to-send,

length, IP addr, port

Send datagramto server

Read datagramfrom server


Example: Java server (UDP)

import java.io.*; import java.net.*; class UDPServer { public static void main(String args[]) throws Exception { DatagramSocket serverSocket = new DatagramSocket(9876); byte[] receiveData = new byte[1024]; byte[] sendData = new byte[1024]; while(true) { DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);

serverSocket.receive(receivePacket);

Createdatagram socket

at port 9876

Create space forreceived datagram

Receivedatagra

m


Example: Java server (UDP), cont

String sentence = new String(receivePacket.getData()); InetAddress IPAddress = receivePacket.getAddress(); int port = receivePacket.getPort(); String capitalizedSentence = sentence.toUpperCase();

sendData = capitalizedSentence.getBytes(); DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, port); serverSocket.send(sendPacket); } }

}

Get IP addrport #, of

sender

Write out datagramto socket

End of while loop,loop back and wait foranother datagram

Create datagramto send to client


Chapter 2: Summary

Application architectures client-server P2P hybrid

application service requirements: reliability, bandwidth, delay

Internet transport service model connection-oriented, reliable:

TCP unreliable, datagrams: UDP

Our study of network apps now complete!

specific protocols: HTTP FTP SMTP, POP, IMAP DNS

socket programming


Chapter 2: Summary

typical request/reply message exchange: client requests info or

service server responds with

data, status code

message formats: headers: fields giving

info about data data: info being

communicated

Most importantly: learned about protocols

control vs. data msgs in-band, out-of-band

centralized vs. decentralized

stateless vs. stateful reliable vs. unreliable msg

transfer “complexity at network

edge”

Documents

2: Application Layer1 School of Computing Science Simon Fraser University CMPT 371: Data Communications and Networking Chapter 2: Application Layer