Upload
vancong
View
216
Download
0
Embed Size (px)
Citation preview
1
Chapter 2: Application Layerp pp yOur goals: Conceptual aspects
learn about protocols by examining some
of network application protocols transport-layer
application-level protocols HTTP transport layer
service models client-server
HTTP SMTP DNS
P2Pparadigm peer-to-peer
paradigm
P2P Content distribution
networks (CDNs)paradigm
programming network applications sockets API
Application Layer (SSL) 1
sockets API10/3/2017
2
Chapter 2: outline
2 1 principles of 2.5 P2P applications2.1 principles of network applications
2.5 P2P applications2.6 video streaming
and content pp2.2 Web and HTTP2.3 electronic mail
distribution networks (CDNs)
• SMTP2.4 DNS
2.7 socket programming with UDP d TCPUDP and TCP
Application Layer 2-2
3
Creating a network appapplication
Write programs that run on (different) end
systems
applicationtransportnetworkdata linkphysical
systems communicate over network e.g., web server software
i t ith bcommunicates with browser software
No need to write software applicationt tfor network-core devices
This approach allows for rapid app development
applicationtransportnetworkdata link
transportnetworkdata linkphysical
p pp pmphysical
Application Layer (SSL) 310/3/2017
4
Application architectures Client-server
to scale –> racks of servers in a datacenter
to improve throughput and response time -> geographically distributed clusters of servers in CDNs
Peer-to-peer (P2P) difficult to manage difficult to manage
-> super nodes (with some server characteristics) needed in addition to regular peers
Application Layer (SSL) 410/3/2017
5
Client-server architectureserver:
always-on hostP dd permanent IP address
data centers for scalingscaling
clients: communicate with server
( k f )(speak first) may be intermittently
connectedclient/server
may have dynamic IP addresses
do not communicate
Application Layer (SSL) 5
directly with each other10/3/2017
6
Pure P2P architecturel no always-on server
arbitrary end systems directly communicatedirectly communicate
peers are intermittently connected and may h IP ddchange IP addresses
difficult to manage find content or another
peer-peer find content or another
peer by broadcast (flooding)
NAT traversal problem NAT traversal problem
Example: BitTorrent has trackers (super nodes that manage the peers and chunks of a torrent)
Application Layer (SSL) 610/3/2017
peers and chunks of a torrent)
7
Processes communicatingProcess: program running
within a host.Client process: process
that initiates i ti within same host, two
processes communicate using inter-process
communicationServer process: process
that waits to beusing inter process communication (defined by OS).
that waits to be contacted
N t li t & processes in different
hosts communicate by
Note: client & server processes run in P2P as well as client-server hosts communicate by
exchanging messages apps
N A i h lid b h d i d
Application Layer (SSL) 710/3/2017
Note: A process in these slides may be a thread instead
8
Addressing processesd f l d h to receive messages,
process must have an identifier
identifier includes both IP address and port number associated with identifier
host device has unique 32-bit IP address
um wprocess on host.
Example port numbers:H P 0 Q: Is IP address of
host on which process runs enough for
HTTP server: 80 Mail server: 25
to send HTTP messageruns enough for identifying the process?
to send HTTP message to gaia.cs.umass.edu web server: IP dd ss: 128 119 245 12 A: No, many
processes can be running on same host
IP address: 128.119.245.12 Port number: 80
more shortly…
Application Layer (SSL) 8
g m y
10/3/2017
9
App-layer protocol specs Types of messages
exchanged, Public-domain protocols:g e.g., request, response
Message syntax:h t fi ld i &
p defined in RFCs which
enable interoperability what fields in messages &
how fields are delineated Message semantics
e.g., HTTP, SMTP
Proprietary protocols: meaning of information in
fields Rules for when and how
Proprietary protocols: e.g., Skype
Rules for when and how processes send & respond to messages
Application Layer (SSL) 910/3/2017
10
What transport service does an app need?
Data loss some apps (e.g., audio) can
tolerate some loss
Throughput some apps (e.g., audio,
video) require minimumtolerate some loss other apps (e.g., file
transfer, telnet, email) i 100% li bl d t
video) require minimum amount of throughput to be “effective”
require 100% reliable data transfer
Timing
other apps (“elastic apps”) make use of whatever throughput they getTiming
some apps (e.g., Internet telephony, i t ti )
throughput they get Security Encryption, data interactive games)
require low delay to be “effective”
yp ,integrity, … (in app layer, e.g. SSL/TLS)
Application Layer (SSL) 1010/3/2017
11
Transport service requirements of common appsTransport service requirements of common apps
Application Data loss Throughput Time SensitiveApplication
file transfere-mail
Data loss
no lossno loss
Throughput
elasticelastic
Time Sensitive
nono
Web documentsinteractive audio/video
no lossno lossloss-tolerant
elasticelastic (few kbps)audio: 5kbps-1Mbpsvideo:10kbps-5Mbps
noyes, 100’s of ms
streaming stored audio/video
interactive gamesi t t i
loss-tolerant
loss-tolerant
p psame as above
few kbps up
yes, few secs
yes, 100’s of msdinstant messaging no loss elastic yes and no
Application Layer (SSL) 1110/3/2017
12
Internet transport protocols servicesTCP service: UDP service:TCP service: connection-oriented: setup
required between client and server processes; state info
UDP service: unreliable data
transfer between server processes; state info
reliable transport: between sending and receiving process
sending and receiving processes
does not provide: flow control: sender won’t
overwhelm receiver congestion control:
pconnection setup, reliability, flow control, congestion control, timing,
congestion control:throttle sender when network is overloaded
does not provide: timin
throughput guarantee, or security
does not provide: timing, minimum throughput guarantee, security
Q: why bother? Why is there a UDP?
Application Layer (SSL) 1210/3/2017
13
Internet apps: application & transport protocolspp pp p p
ApplicationApplicationlayer protocol
Underlyingtransport protocolApplication
e-mailremote terminal access
layer protocol
SMTP [RFC 5321]
Telnet [RFC 854]
transport protocol
TCPTCPremote terminal access
Web file transfer
streaming multimedia
Telnet [RFC 854]
HTTP [RFC 2616]
FTP [RFC 959]
HTTP (e. g., Youtube)
TCPTCPTCPTCP g
Internet telephony
( g , )
SIP[RFC 3261],RTP[RFC 3550], proprietary(e.g., Skype) UDP or TCP
Application Layer (SSL) 1310/3/2017
14
Chapter 2: outline
2 1 principles of 2.5 P2P applications2.1 principles of network applications
2.5 P2P applications2.6 video streaming
and content pp2.2 Web and HTTP2.3 electronic mail
distribution networks (CDNs)
• SMTP2.4 DNS
2.7 socket programming with UDP d TCPUDP and TCP
Application Layer 2-14
15
Web and HTTPFirst some jargon Web page consists of base HTML-file which
includes several referenced objects
Obj t b HTML fil JPEG i J Object can be HTML file, JPEG image, Java applet, audio file,…
Each object is addressable by a URL, for example
www.someschool.edu/someDept/pic.gifwww.someschool.edu/someDept/pic.gif
host name path name
Application Layer (SSL) 1510/3/2017
16
HTTP overviewHTTP: hypertext
transfer protocol Web’s application layer
protocol client/server model
PC runningExplorer
client: browser that requests, receives, displays Web objects
Explorer
p y j server: Web server
sends objects in response to requests
Server running
Apache Webserverp q
Mac runningChrome
Application Layer (SSL) 1610/3/2017
17
HTTP overview (continued)HTTP overv ew (cont nued)
Uses TCP: HTTP is “stateless” client initiates TCP
connection (creates socket) to server, port 80
server maintains no information about past client requests, p
server accepts TCP connection from client
HTTP messages exchanged
p q
Protocols that maintain “state” are complex
aside HTTP messages exchanged
between browser (HTTP client) and Web server (HTTP server)
state are complex past history (state) must
be maintained if / li t h( )
TCP connection closed if server/client crashes,
their views of “state” may be inconsistent, must be reconciled
Application Layer (SSL) 17
reconciled
10/3/2017
18
Nonpersistent HTTPSuppose user enters URL
www.someSchool.edu/someDepartment/home.index
(contains text, references to 10
jpeg images)
1a. HTTP client initiates TCP connection to HTTP server (process) at 1b. HTTP server at host
S h l d iti(process) atwww.someSchool.edu on port 80 www.someSchool.edu waiting
for TCP connection at port 80, “accepts” connection, notifying client
2. HTTP client sends HTTP request message (containing URL) into TCP connection socket Message indicates
cl ent
3. HTTP server receives request message forms responsesocket. Message indicates
that client wants object someDepartment/home.index
message, forms response message containing requested object, and sends message into its socket
Application Layer (SSL) 18
time10/3/2017
19
Nonpersistent HTTP (cont.)
4. HTTP server closes TCP connection
5. HTTP client receives response message containing html file, displays html. Parsing html
connection.
p y gfile, finds 10 referenced jpeg objects
6. Steps 1-5 repeated for each time p pof 10 jpeg objects
Application Layer (SSL) 1910/3/2017
20
Non-Persistent HTTP: Response timeDefinition of RTT: time for
a small packet to travel from client to serverfrom client to server and back.
Response time:initiate TCPconnectionp
one RTT to initiate TCP connection
RTT f HTTP time to
RTTrequestfile
one RTT for HTTP request and first few bytes of HTTP response
time to transmit file
RTT
filereceivedy p
to return file response time total
received
time time
2RTT+ file transmissi n time
Application Layer (SSL) 2010/3/2017
= 2RTT+ file transmission time
21
Nonpersistent vs. Persistent HTTPNonpersistent HTTP delay requires 2 additional RTTs
f n bj t
Persistent HTTP delay server leaves connection
open after sending responsefor an object
most browsers open parallel P f h
open after sending response subsequent HTTP messages
between client/server are sent over same connectionTCP connections to fetch
referenced objects very little additional time
f ll f bj ts
sent over same connection client sends a new request
as soon as it encounters a referenced objectfor all reference objects
but OS overhead to allocate host resources for all TCP connections
referenced object (pipelining)
as little as one additional RTT for all referencedall TCP connections RTT for all referenced objects (plus their transmission time)
Application Layer (SSL) 2110/3/2017
22
HTTP historyNonpersistent HTTP At most one object is
Persistent HTTP Version 1.1 (since 1999)
lti l bj t bsent over a TCP connection Version 0 9 (in 1991)
multiple objects can be sent over single TCP connection between Version 0.9 (in 1991)
• only one method (GET) • only one object type
(HTML)
client and server. pipelining of requests
(HTML)
Version 1.0 (in 1996)
Version 2 (since 2015)
allows server pushlti l i t Version 1.0 (in 1996)
multiplexing requests over a single TCP connection – without
Application Layer (SSL) 22
HOL blocking10/3/2017
23
HTTP request message
two types of HTTP messages: request, response two types of HTTP messages request, response HTTP request message:
ASCII (human-readable format)
GET /somedir/page.html HTTP/1.1\r\nHost: www someschool edu\r\n
request line(GET, POST,
HEAD commands) Host: www.someschool.edu\r\n User-agent: Mozilla/5.0\r\nConnection: close\r\n Accept-language:fr\r\n
HEAD commands)
headerlines
Accept language:fr\r\n\r\n Carriage return,
line feed indicateend of header lines
Application Layer (SSL) 2310/3/2017
25
Uploading form inputUpload ng form nput
Post method: Web page often
includes form inputl
GET method: Uses GET method
Input is uploaded to server in entity body
Entity body is empty Input is uploaded in
URL field of requestURL field of request line:
www.somesite.com/animalsearch?monkeys&banana
Application Layer (SSL) 2510/3/2017
26
Method typesMethod types
HTTP/1.0 HTTP/1.1 GET POST
GET, POST, HEAD PUT
HEAD asks server to leave
requested object out of
uploads file in entity body to path specified in URL fieldrequested object out of
response DELETE deletes file specified in
the URL fieldthe URL field
… (others)
Application Layer (SSL) 2610/3/2017
27
HTTP response messagep m gstatus line(protocol
HTTP/1.1 200 OK\r\n Connection close\r\nDate: Tue, 09 Aug 2011 12:00:15 GMT\r\n
(protocolstatus code
status phrase)
Server: Apache/2.2.3 (CentOS)\r\n Last-Modified: Tue, 09 Aug 2011...\r\n Content-Length: 6821\r\n
/h l\ \
headerlines
Content-Type: text/html\r\n\r\n data data data data data ... data, e.g.,
requestedrequestedHTML file
Aside: several other methods to indicate
Application Layer (SSL) 2710/3/2017
Aside: several other methods to indicate length of a response
28
Cookies: keeping state info p g
client serverusual http request msgusual http response +Set-cookie: 1678
Amazon servercreates ID
1678 for user
Cookie file
ebay: 8734 Set coo e: 6 8
usual http request msgcookie: 1678 cookie-
1678 for user
Cookie file
amazon: 1678
y
cookie: 1678
usual http response msgspecificaction
o : 678ebay: 8734
one week later:
usual http request msgcookie: 1678
usual http response msg
cookie-spectific
action
Cookie fileamazon: 1678ebay: 8734
Application Layer (SSL) 28
usual http response msg actiony
10/3/2017
29
Cookies Cook es Application end points keep state at client
and server across multiple transactionsp http messages carry state info but http
protocol is stateless
What cookies can bring: authorization
Cookies and privacy: cookies permit sites
shopping carts recommendations i t t
(including third parties) to learn a lot about you
user session state (Web e-mail)
about you
Application Layer (SSL) 2910/3/2017
30
Web cache (proxy server)
user sets browser: web accesses via
Goal: satisfy client request without involving origin servers
web accesses via cache
browser sends all Proxy
origin server
HTTP requests to cache if object is not in
client
Proxyserver
if object is not in cache, request object from origin server, then return object tothen return object to client
else cache returns object
clientorigin server
Application Layer (SSL) 30
object server
10/3/2017
31
More about Web cachingMore about Web cach ng
cache acts as both Why Web caching?client and server
typically cache is i st ll d b ISP
y g reduce response time
for client requestinstalled by ISP (university, company, residential ISP)
reduce traffic on an institution’s access link.n .
Application Layer (SSL) 3110/3/2017
32
Caching example origin
Assumptions average object size = 1 Mbits avg request rate from institution’s
originservers
publicI t t avg. request rate from institution s
browsers to origin servers = 15/sec Internet delay (ISP router to any
origin server and back) = 2 sec
Internet
origin server and back) = 2 secConsequences utilization on LAN = 15% institutional
15 Mbps access link
utilization on access link = 100% total delay = Internet delay + access
link delay + LAN delay
network 100 Mbps LAN
= 2 seconds + minutes + 0.01 second
many lost packets in reponses
institutionalcache
Application Layer (SSL) 3210/3/2017
Assumption: request is very short and takes no time to transmit
33
Caching example (2)g mp ( )possible solution in s b nd idth f ss
originservers
increase bandwidth of access link to, say, 100 Mbps public
Internet
consequence utilization on LAN = 15% utilization on access link = 15%
100 Mbps access link
Total delay = Internet delay + access link delay + LAN delay
= 2 + 0.01 + 0.01 seconds
institutionalnetwork 100 Mbps LAN
access l nk
often a costly upgradeinstitutional
cache
Application Layer (SSL) 33
cache
10/3/2017
34
Caching example (3)possible solution: install cache suppose hit rate is 0.4
originservers
consequence 40% requests will be satisfied
almost immediately
publicInternet
almost immediately 60% requests satisfied by
origin servers utilization of access link
d d 60% l i i
15 Mbps access linkreduced to 60%, resulting in
negligible delay (say 70 ms) ave. total delay = Internet
delay + access link delay + LAN
institutionalnetwork 100 Mbps LAN
access l nk
y ydelay= 0.6*(2 + 0.07 + 0.01) +
0.4* (0 + 0 + 0.01) seconds1 25
institutionalcache
Application Layer (SSL) 34
= 1.25 sec cache
10/3/2017
35
Conditional GET Goal: don’t send object if
cache has up-to-date cached version
cache serverversion
cache: indicates date of cached copy in HTTP request to server
HTTP request msgIf-modified-since:
<date>object
notto serverIf-modified-since:
<date>
server: response contains no
HTTP responseHTTP/1.1
304 Not Modified
not modified
server: response contains no object if cached copy is up-to-date: HTTP/1.1 304 Not
HTTP request msgIf-modified-since: /
Modified <date>
HTTP responseHTTP/1 1 200 OK
object modified
Application Layer (SSL) 35
HTTP/1.1 200 OK<data>
10/3/2017
36
Chapter 2: outline
2 1 principles of 2.5 P2P applications2.1 principles of network applications
2.5 P2P applications2.6 video streaming
and content pp2.2 Web and HTTP2.3 electronic mail
distribution networks (CDNs)
• SMTP2.4 DNS
2.7 socket programming with UDP d TCPUDP and TCP
Application Layer 2-36
37
Electronic MailMajor components:
outgoing message queueMajor components:
user agents mail servers
user mailbox
message queue
useragent
simple mail transfer protocol: SMTP
Us A nt
useragent
mailserver
agent
SMTPUser Agent a.k.a. “mail reader” composing, editing, reading mail
mailserver user
agent
SMTP
SMTPmessagese.g., Outlook, alpine, Mozilla Thunderbird, iPhone mail client
useragent
mailserver
SMTP
useragent
user
Application Layer (SSL) 37
agent
10/3/2017
38
Electronic Mail: mail serversMail Servers mailbox contains incoming
messages for useruseragentmessages for user
message queue of outgoing (to be sent) mail messages
useragent
mailserver
agent
SMTPSMTP protocol
between mail servers to
mailserver user
agent
SMTP
SMTPbetween mail servers to send email messages “client”: sender “server”: receiver
useragent
mailserver
SMTP
server : receiver server
useragent
user
Looks like it is the very first (hybrid) P2P
!Application Layer (SSL) 38
useragent
10/3/2017
system!
39
Mail access protocolsMa l access protocols
useruserSMTP SMTP Mail access
t l agent
sender’s mail
useragent protocol
receiver’s mail
SMTP: for delivery to receiver’s server Mail access protocol: for retrieval from server
server server
p POP: Post Office Protocol [RFC 1939]
• authorization (agent <-->server) and download IMAP: Internet Mail Access Protocol [RFC 3501] IMAP: Internet Mail Access Protocol [RFC 3501]
• more features (keeps user state across sessions)• manipulation of stored msgs in folders on server
W b l HTTP f b h d d
Application Layer (SSL) 39
Web email: use HTTP for both send and receive10/3/2017
40
Electronic Mail: SMTP uses TCP to reliably transfer email message from
client to server, port 25 direct transfer: sending server to receiving server three phases of transfer
h ndsh kin ( tin ) handshaking (greeting) transfer of messages (persistent connection) closure closure
command/response interaction commands: ASCII text response: status code and phrase
messages must be in 7-bit ASCII
Application Layer (SSL) 4010/3/2017
41
Sample smtp interaction(after TCP connection established)S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <[email protected]> S: 250 [email protected]... Sender ok C: RCPT TO: <[email protected]> S: 250 [email protected] ... Recipient ok C: DATA S 354 E t il d ith " " li b it lfS: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C:C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection
Application Layer (SSL) 41
g g
10/3/2017
42
Try SMTP interaction for yourself:Try SMTP interaction for yourself
telnet servername 25
see 220 reply from server enter HELO, MAIL FROM, RCPT TO, DATA, QUIT
commandsabove lets you send email without using email client
(reader)(reader)
Log on a UTCS linux machine. Then doLog on a UTCS linux machine. Then dotelnet mailbox2.cs.utexas.edu 25 (opens TCP conn.)
Application Layer (SSL) 4210/3/2017
43
Mail message formatSMTP (rfc 5321): protocol for
exchanging email msgsf 5322 t d d f t trfc 5322: standard for text
message format:header
blankline
header lines, e.g., To: From:
body
Subject:different from SMTP
commands and responsescommands and responses body
the “message”, ASCII characters only
Application Layer (SSL) 43
characters only
10/3/2017
44
Message format: multimedia extensions
Multipurpose Internet Mail Extensions (MIME) additional lines in msg header declare media additional lines in msg header declare media
content types (also multipart content types)
From: [email protected] To: [email protected] Subject: Picture of yummy crepe. MIME-Version: 1.0method used
MIME version
MIME Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg
b 64 d d d t
multimedia datatype subtype
to encode data
base64 encoded data ..... ......................... ......base64 encoded data
type, subtype, parameter declaration
encoded data
Application Layer (SSL) 4410/3/2017
45
Chapter 2: outline
2 1 principles of 2.5 P2P applications2.1 principles of network applications
2.5 P2P applications2.6 video streaming
and content pp2.2 Web and HTTP2.3 electronic mail
distribution networks (CDNs)
• SMTP2.4 DNS
2.7 socket programming with UDP d TCPUDP and TCP
Application Layer 2-45
46
DNS: Domain Name SystemD D m m y mPeople: many identifiers:
SSN, name, passport #Domain Name System: distributed database, , p p
Internet hosts, routers: IP address (32 bit)
d f dd i
distributed databaseimplemented as a hierarchy of many name servers
- used for addressing datagrams
“name”, e.g.,
application-layer protocolused to resolve names (address/name translation)
www.yahoo.com - used by humans
Required: map between
(address/name translation) note: core Internet function,
implemented as application-layer protocolequ red map between
name and IP addresseslayer protocol
complexity at network’s “edge”
Application Layer (SSL) 4610/3/2017
47
DNS Why not centralized DNS? single point of failure
DNS services Hostname to IP single point of failure
traffic volume distant centralized
address translation Host alias
C i l d li database maintenance/update
Canonical and alias names
Mail server alias
doesn’t scale!
Mail server alias Load distribution
Replicated Web pserver: a set of IP addresses for one canonical host name
Application Layer (SSL) 47
canonical host name10/3/2017
48
Distributed, Hierarchical DatabaseRoot DNS Servers
Top-level domains
.com DNS servers .org DNS servers .edu DNS servers
poly edu umass edub poly.eduDNS servers
umass.eduDNS serversyahoo.com
DNS serversamazon.comDNS servers
pbs.orgDNS servers
Client wants IP address for www amazon com: 1st approx:Client wants IP address for www.amazon.com: 1 approx: Client queries a root server to find .com DNS server Client queries .com DNS server to get amazon.com q g
DNS server Client queries amazon.com DNS server to get IP
dd f
Application Layer (SSL) 48
address for www.amazon.com10/3/2017
49
DNS: root name servers 13 root name13 root name “servers”worldwide
c. Cogent, Herndon, VA (5 other sites)d. U Maryland College
e. NASA Mt View, CAf Internet Software C
i. Netnod, Stockholm (37 other sites)
k. RIPE London (17 other sites)
m WIDE Tokyo
y gPark, MDh. ARL Aberdeen, MDj. Verisign, Dulles VA (69 other sites )
a. Verisign, Los Angeles (5 other sites)
f. Internet Software C.Palo Alto, CA (and 48 other sites)
m. WIDE Tokyo(5 other sites)
(5 other sites)b. USC-ISI Marina del Rey, CAl. ICANN Los Angeles, CA
(41 other sites)
g. US DoD Columbus, OH (5 other sites)
Application Layer (SSL) 4910/3/2017
50
TLD and Authoritative Servers Top-level domain (TLD) servers: responsible
for .com, .org, .net, .edu, …, and all top-level t d i k f jcountry domains .uk, .fr, .ca, .jp
Verisign maintains servers for .com TLD Educause for edu TLD Educause for .edu TLD
Authoritative DNS servers: organization’s DNS servers, providing authoritative p ghostname-to-IP mappings for organization’s named hosts (e.g., Web and mail). Can be locally maintained by organization or a Can be locally maintained by organization or a
service provider
Application Layer (SSL) 5010/3/2017
51
Local Name ServerLocal Name Server
does not strictly belong to hierarchy does not strictly belong to hierarchy each ISP (residential ISP, company,
university) has one.y) also called “default name server”
when host makes DNS query, query is sent q y, q yto its local DNS server which acts as proxy, forwards query into
hi hhierarchy
Application Layer (SSL) 5110/3/2017
52
root DNS serverDNS name s l ti l
23
TLD DNS
resolution example Host at cis.poly.edu
t IP dd f4
5
TLD DNS serverwants IP address for gaia.cs.umass.edu
It sends a recursivelocal DNS server
dns.poly.edu
1678
It sends a recursive query to local DNS server
1
authoritative DNS serverdns.cs.umass.edu
8iterated query: contacted server
replies with name of requesting host
cis.poly.edu
gaia.cs.umass.edu
dns.cs.umass.edupserver to contact“I don’t know this name, but ask this
Application Layer (SSL) 52
gaia.cs.umass.edu,other server”
10/3/2017
53
DNS: caching when (any) name server learns mapping, it caches
mapping cached entries timeout (disappear) after
some time (e.g., 2 days) TLD servers typically cached in local name TLD servers typically cached in local name
servers -Thus root name servers not often visited
cached entries may be out-of-date (best effort service)
if h st h d IP dd ss this t b if host changed IP address, this may not be known Internet-wide until all TTLs expire
Application Layer (SSL) 5310/3/2017
54
DNS recordsDNS: distributed db storing resource records (RR)
RR format: (name value type ttl)RR format: (name, value, type, ttl)
Type=A name is hostname
Type=CNAMEl f
T NS
name is hostname value is IP address
name is alias name for some “canonical” (the real) namee.g., www.ibm.com is really
Type=NS name is domain (e.g.
foo.com)
servereast.backup2.ibm.com
value is canonical name
value is hostname of authoritative name server for this domain
Type=MX value is host name of mail
server associated with name
Application Layer (SSL) 5410/3/2017
55
DNS protocol messagesDNS protocol : query and reply messages, both with
same message format
msg header identification: 16 bit #
f lfor query, reply to query uses same #
flags: query or reply recursion desired recursion availabler cur n a a a reply is authoritative
Application Layer (SSL) 5510/3/2017
56
DNS protocol messages (cont.)D p m g ( .)
N t fi ldName, type fieldsfor a query
RR iRRs in responseto query
records forrecords forauthoritative servers
additional “helpful”additional helpfulinfo that may be used
Application Layer (SSL) 5610/3/2017
57
Inserting records into DNS Example: just created startup “Network Utopia” Register name networkutopia.com at a registrar (e.g.,
Network Solutions)Network Solutions) Need to provide registrar with names and IP addresses of
your authoritative name servers (primary and secondary)
Registrar inserts NS and A resource records for each authoritative server into .com TLD servers:(networkutopia.com, dns1.networkutopia.com, NS)(dns1 networkutopia com 212 212 212 1 A)(dns1.networkutopia.com, 212.212.212.1, A)
Put into authoritative servers, CNAME record for www networkutopia com MX record forwww.networkutopia.com, MX record for networkutopia.com, and …
How do people get the IP address of your Web site?
Application Layer (SSL) 57
How do people get the IP address of your Web site?10/3/2017
58
Chapter 2: outline
2 1 principles of 2.5 P2P applications2.1 principles of network applications
2.5 P2P applications2.6 video streaming
and content pp2.2 Web and HTTP2.3 electronic mail
distribution networks (CDNs)
• SMTP2.4 DNS
2.7 socket programming with UDP d TCPUDP and TCP
Application Layer 2-58
59
P2P architecture no always-on server arbitrary end systems arbitrary end systems
directly communicate peers are intermittently
d d h IPconnected and change IP addresses peer-peer
Hybrid deployment examples: File distribution, e.g.,
BitTorrent (has trackers)BitTorrent (has trackers) Internet telephony, e.g.,
original Skype (had super peers)
Application Layer (SSL) 59
peers),
10/3/2017
60
File Distribution: Client-Server vs P2PQuestion : How much time to distribute a file
from one server to N peers?p
S
us server upload capacity (bps)
uu2d1 d2
u1
Server ui peer i upload capacity (bps)di peer i downloadus
d2
dN Network (with
File, size Fdi peer i download capacity (bps)
uN
Network (with abundant bandwidth)
Application Layer (SSL) 6010/3/2017
61
Time to distribute file to N users : Client-Server
u
Server
F
server sequentially sends N copies:
us
u2d1 d2u1
dNNetwork (with
F NF/us time
client i takes F/di ti t d l d
uN
dN abundant bandwidth)time to download
Tcs ≥ max { NF/us , F/min(di) }Time to distribute Fto N clients is
Wh N is l mi (d ) > /N
cs { s , ( i) }i(lower bound)
Application Layer (SSL) 6161
When N is large, min(di) > us/N10/3/2017
i
62
Time to distribute file to N users: P2P
u
Server
F
server must send one copy: F/us time
us
u2d1 d2u1
dNNetwork (with
Fpy s
client i takes F/di time to download
l d duN
dN abundant bandwidth) NF bits must be uploaded (aggregate)
fastest possible upload rate: u + Σ ufastest possible upload rate: us + Σ1≤i≤N-1 ui
TP2P ≥ max { F/us , F/min(di) , NF/(us + Σui) } Largest TP2P ≥ max { F/us , F/min(di) , NF/(us Σui) }i
us > (us + Σui)/N and, for large N, min(di) > (us + Σui)/N
gvalue as N ↑
Application Layer (SSL) 626210/3/2017
s ( s i) , g , ( i) ( s i)i
63
Client-server vs. P2P: exampleCli t l d t f ll i F/ 1 i 10 d
3.5P2P
Client upload rate = u for all i, F/u = 1 min., us = 10u, dmin ≥ u
2.5
3
tion
Tim
e
Client-Server
/ / (10 )sNF u NF u=
1.5
2
m D
istri
but ( )/ 10 min.
s
N=1
as
/ ( ) 1 min.N
s i
N
NF u u = F / u−
→ ∞
+ =
2 servers
0.5
1
Min
imum 1
/ ( ) .s ii
N u u / u=
00 5 10 15 20 25 30 35
N
Application Layer (SSL) 6310/3/2017
What if more than 1 server? I.e., a data center
64
Chapter 2: outline
2 1 principles of 2.5 P2P applications2.1 principles of network applications
2.5 P2P applications2.6 video streaming
and content pp2.2 Web and HTTP2.3 electronic mail
distribution networks (CDNs)
• SMTP2.4 DNS
2.7 socket programming with UDP d TCPUDP and TCP
Application Layer 2-64
65
Sockets process sends/receives
messages to/from its socket
host orserver
host orserversocket
socket analogous to “door” sending process shoves
server se ve
controlled byapp developer
message out of door it relies on transport
infrastructure on other
process
TCP with
socket
process
TCP with
socket
infrastructure on other side of door which delivers message to
k t t i i
C w tbuffers,variables
buffers,variables
Internet
socket at receiving process
controlledby OS
Application Layer (SSL) 6510/3/2017
66
Socket programmingGoal: learn how to write client and server programs
which communicate by sending data into sockets, di d t t f k t
Socket API introduced in BSD4 1 UNIX 1981
reading data out of sockets
introduced in BSD4.1 UNIX, 1981 sockets are explicitly created, used, then
released by applications y pp client/server paradigm API: choice of a transport protocol and ability
t if f tto specify a few parameters reliable byte stream unreliable datagram
Application Layer (SSL) 66
unreliable datagram 10/3/2017
67
Socket programming with TCPBefore client contact: server process must first
be running
When contacted by client, server TCP creates a new connection socket for server be runn ng
server must have created socket (door) that welcomes client’s contact
process to communicate with client allows server to talk with
l l lw m
Client contacts server by: creating client-local TCP
s k t
multiple clients source IP address and port
number used to distinguish l ( h )socket
specifying IP address, port number of server process
h l k
clients (more in Chap 3)
CP d l bl dapplication viewpoint
When client makes connect call: client TCP establishes connection to server TCP
TCP provides reliable, in-ordertransfer of byte stream
between client and server
Application Layer (SSL) 6710/3/2017
68
Socket API for TCP (BSD Unix) S id Client side
socket(), returns client socket id
Server side socket(), returns server
socket idsocket id connect(), need to
specify server IP address and port sends
bind(), binds server socket to server IP address and portaddress and port, sends
conn req send(), sends to client
socket
listen(), willing to accept conn req on server socket
accept(), accepts new conn socket recv(), receives from
client socket close() closes
p preq and returns its connection socket id
recv(), receives from close(), closes
connection
note: OS supplies local IP address
connection socket send(), sends to connection
socket
Application Layer (SSL) 68
note: OS supplies local IP address and port for client close(), closes connection
10/3/2017
69
Client/server socket interaction: TCPserver (running on hostid) client
create socket: port=xserverSocket = socket()
S k t bi d()
server (running on hostid) client
wait for incoming connection request
serverSocket.bind()serverSocket.listen()
create socket:clientSocket = socket()connection request
connectionSocket =serverSocket.accept()
connect to hostid, port=xclientSocket socket()
TCP connection setup
send request usingclientSocketrecv request from
connectionSocket
send reply toconnectionSocket
close
recv reply fromclientSocket
close
Application Layer
connectionSocket closeclientSocket
10/3/2017
69Note: syntax is neither complete nor precise
70
Socket programming with UDPUDP: no “connection”
between client and serverh d h k
application viewpoint
UDP provides unreliable transfer no handshaking sender explicitly attaches IP
address and port of destination k
pof groups of bytes (“datagrams”)
between client and server
to every packet server must extract IP
address, port of sender from i d kevery received packet
UDP: transmitted data may be received out of orderbe received out of order, or lost
Application Layer (SSL) 7010/3/2017
71
Socket API for UDP (BSD Unix) Client side
socket(), returns client socket id
Server side socket(), returns server
socket idsocket id sendto(), sends to client
socket, need to specify destination’s IP address and
bind(), binds server socket to server IP address and portdestination s IP address and
port recvfrom(), receives from
client socket: data and
p recvfrom(), receives
from server socket: data and sender’s IP client socket data and
sender’s IP address and port
bind() optional
address and port sendto(), sends to
server socket, need to bind(), optional note: needs timeout
management;OS supplies local IP address and port for
,specify destination’s IP address and port
Application Layer (SSL) 71
OS supplies local IP address and port for client if bind() not used10/3/2017
72
Client/server socket interaction: UDP
create socket:cli ntS ck t
create socket: port= xserverSocket =
server (running on serverIP) client
clientSocket =socket(AF_INET, SOCK_DGRAM)
create datagram with server IP
socket(AF_INET, SOCK_DGRAM)serverSocket.bind()
create datagram with server IP address and port=x; send datagramto clientSocket
recv datagram with client’s address fromserverSocket
recv datagram with server’s address froml k
serverSocket
send reply withclient’s address to
clientSocket
closeclientSocket
serverSocket
clientSocket
10/3/201772Application Layer
Note: syntax is neither complete nor precise
73
Chapter 2: outline
2 1 principles of 2.5 P2P applications2.1 principles of network applications
2.5 P2P applications2.6 video streaming
and content pp2.2 Web and HTTP2.3 electronic mail
distribution networks (CDNs)
• SMTP, POP3, IMAP2.4 DNS
2.7 socket programming with UDP d TCPUDP and TCP
Application Layer 2-73
74
Video Streaming and CDNs: context
• Netflix, YouTube: 37%, 16% of downstream
video traffic: major consumer of Internet bandwidth
Netfl x, You ube 37%, 6% of downstreamresidential ISP traffic in 2015
• ~1B YouTube users, ~75M Netflix users challenge: scale ~1B users?
• single mega-video server won’t workch ll n : h t n it challenge: heterogeneity different users have different capabilities
and bandwidths (e.g., wired versus mobile)( g , m ) solution: distributed, application-level
infrastructure
Application Layer
2-74
75
Multimedia: videospatial coding example: instead of sending N values of same color (all purple), send only
CBR: (constant bit rate): video encoding rate fixed ……………………..
co or (a purp ), s n on ytwo values: color value (purple) and number of repeated values (N)
……………….…….
VBR: (variable bit rate): video encoding rate changes as g gamount of spatial, temporal coding changes examples: frame imp• MPEG-1 (CD-ROM) 1.5
Mbpstemporal
• MPEG-2 (DVD) 3-6 Mbps• MPEG-4 (often used in
Internet < 1 Mbps)frame i+1
mpcoding example: instead of sending complete frame at i+1, send only Internet < 1 Mbps) differences from frame i
Application Layer
2-75
76
Streaming stored video: g
simple scenario:p
video server client
Internet
(stored video)client
Video can be compressed to essentially any bit rate by trading image qualityrate by trading image qualityE.g., Internet video from 100Kbps for low quality
to over 3 Mbps for high-definition movies
Application Layer
2-76
77
Streaming multimedia: DASH Dynamic, Adaptive Streaming over HTTP server:
di id id fil i t lti l h k divides video file into multiple chunks chunks stored, encoded at different rates manifest file: provides URLs for different chunks manifest file: provides URLs for different chunks
client: periodically measures server-to-client bandwidthp y consulting manifest, requests one chunk at a time
• chooses maximum coding rate sustainable at current bandwidth
• can choose different coding rates at different points in timetime
Application Layer
2-77
78
Streaming multimedia: DASH (cont.) “intelligence” at client:
client determinesclient determines when to request chunk - so that buffer
starvation, or overflow does not occur what encoding rate to request - higher quality
when more bandwidth available
where to request chunk – may possibly request from another server that has higher available f m gbandwidth (less congested path) [actually client will need help from CDN]
Application Layer
2-78
79
Content distribution networkschallenge: how to stream content (selected
from millions of videos) to hundreds of thousands of simultaneous users?
option 1: single, large “mega-server”single point of failurepoint of network congestionlong path to distant clients
lti l i f id t t imultiple copies of same video sent over outgoing links
this solution doesn’t scale….this solution doesn t scale
Application Layer
2-79
80
Content distribution networks option 2: third-party CDN multiple copies
of videos at geographically distributed , t / itstore/serve sites: enter deep: place CDN servers deep into many
access networksaccess networks • close to users• e.g., used by Akamai, 1700 locations
bring home: smaller number (10’s) of larger bring home: smaller number (10 s) of larger clusters in IXPs/POPs near (but not within) access networks
• e.g., used by Limelight
option 3: private CDN, e.g., Google and NetFlix– use both enter-deep and bring-home servers
Application Layer 2-80
81
CDN content access: DNS redirectBob (client) requests video http://video.netcinema.com/6Y7B23V video stored in KingCDN.com
11. Bob clicks on URL for video http://video netcinema com/6Y7B23Vhttp://video.netcinema.com/6Y7B23Von netcinema.com web page
22. Client resolves video.netcinema.comvia local DNS
4&5. Local DNS resolves 56. request video from Bob’s
netcinema.com 3. netcinema’s DNS returns hostname 1105 KingCDN com 4
1105.KingCDN.comfrom KingCDN’s authoritative DNS, which returns the IP address of a KingCDN server
6. request video fromKINGCDN server,streamed via HTTP
local DNSserver
netcinema.com
3
hostname 1105.KingCDN.com 4 g
Bob’s client then establishes a direct TCP connection with the KingCDN server and issues an HTTP GET request for video
KingCDN.comnetcinema’s
authoratative DNSHTTP GET request for video
KingCDNauthoritative DNS Application
Layer2-81
82
Cluster selection strategy (proprietary)f d D
subscriber requests content from CDN
copies of content stored at CDN servers
subscriber requests content from CDN• directed to nearby copy, retrieves content• may choose different copy if network path is m y ff py f p
congested
if t filwhere’s Madmen?
manifest file
Application Layer
2-82
83
Case study: Netflix
Amazon upload copies of lti l i f
Netflix
Netflix registration,
Amazon cloud CDN
server
multiple versions of video to CDN servers
gaccounting servers
22. Bob browsesNetflix video
3. Manifest filereturned for requested video
CDNser er
1
1. Bob manages
2Netflix video 3 requested video server
1. Bob manages Netflix account
4. DASH
CDNserver
4. DASH streaming
Application Layer
2-83