97
Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ETH Zürich

Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

OperatingSystemsandNetworks

NetworkLecture12:ApplicationLayer

AdrianPerrigNetworkSecurityGroupETHZürich

Page 2: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

2

WhereweareintheCourse• StartingtheApplicationLayer!

– Buildsdistributed“networkservices” (DNS,Web)onTransportservices

PhysicalLink

Application

NetworkTransport

Page 3: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

3

Recall• Applicationlayerprotocolsareoftenpartofan“app”

– Butdon’tneedaGUI,e.g.,DNS

TCPIP

802.11

HTTPapp

OS

User-level

(NIC)

Page 4: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

4

Recall(2)• Applicationlayermessagesareoftensplitovermultiplepackets– Ormaybeaggregatedinapacket…

802.11 IP TCP HTTP

802.11 IP TCP HTTP

802.11 IP TCP HTTP

HTTP

Page 5: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

ApplicationCommunicationNeeds• Varywidelywithapp;mustbuildonTransportservices

5

UDP

DNS

TCP

Seriesofvariablelength,reliablerequest/replyexchanges

Web

UDP

Real-time(unreliable)

streamdelivery

Skype

SeeBook

Short,reliablerequest/replyexchanges

Messagereliability!

Page 6: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

OSISession/PresentationLayers• Rememberthis?Tworelevantconcepts…

6

– Providesfunctionsneededbyusers– Convertsdifferentdatarepresentations– Multiplesessionsbetweensamesrc-dst– Providesend-to-enddelivery– Sendspacketsovermultiplelinks– Sendsframesofinformation– Sendsbitsassignals

Butconsiderpartoftheapplication,notstrictlylayered!

Page 7: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

7

SessionConcept• Asessionisaseriesofrelatednetworkinteractionsinsupportofanapplication task– Ofteninformal,notexplicit

• Examples:– Webpagefetchesmultipleimages– Skypecallinvolvesaudio,video,chat

Page 8: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

8

PresentationConcept• Appsneedtoidentifythetypeofcontent,andencodeitfortransfer– ThesearePresentationfunctions

• Examples:– Media(MIME)types,e.g.,image/jpeg, identifythetypeofcontent

– Transfer encodings,e.g.,gzip,identifycontentencoding– Applicationheadersareoftensimpleandreadableversuspacked forefficiency

Page 9: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

9

Topics• EvolvingInternetapplications• DNS(DomainNameSystem)• HTTP(HyperText TransferProtocol)• Webproxiesandcaching• ContentDistributionNetworks• Peer-to-peer (BitTorrent)

• Real-timeapplications(VoIP)

Thislecture

SeeBook

Page 10: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

EvolutionofInternetApplications• Alwayschanging,andgrowing…

10

20101970 19901980 2000

Traffic

FileTransfer(FTP)Email(SMTP)

News(NTTP)

SecureShell(ssh)Telnet

Email

Web(HTTP)Web(CDNs)

P2P (BitTorrent)Web(Video)

???

Page 11: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

EvolutionofInternetApplications(2)• ForapeekatthestateoftheInternet:

– Akamai’sStateoftheInternetReport(quarterly)– Cisco’sVisualNetworkingIndex– MaryMeeker’s InternetReport

• RobustInternetgrowth,esp.video,wirelessandmobile– Mosttrafficisvideo,willbe90%ofInternetinafewyears– Wirelesstrafficwillsoonovertakewiredtraffic– Mobiletrafficisstillasmallportion(15%)ofoverall– GrowingattacktrafficfromChina,alsoU.S.andRussia

11

Page 12: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

EvolutionoftheWeb

12

Source:http://www.evolutionoftheweb.com, Vizzuality,Google,andHyperakt

Page 13: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

EvolutionoftheWeb(2)

13

Source:http://www.evolutionoftheweb.com, Vizzuality,Google,andHyperakt

Page 14: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

14

DomainNameSystem(DNS)(§7.1.1-7.1.3)

• TheDNS(DomainNameSystem)– Human-readable hostnames,andmore– Part1:thedistributed namespace

www.uw.edu?

Network

128.94.155.135

Page 15: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

NamesandAddresses• Names:higher-level (user-understandable) resourceidentifiers• Addresses: lower-level resource locators

– Multiplelevels,e.g.,fullnameà emailà IPaddressà Ethernetaddress• Resolution (orlookup):mappinganametoanaddress

15

Directory

Name,e.g.“AndyTanenbaum,”or“flits.cs.vu.nl”

Address,e.g.“Vrije Universiteit,Amsterdam”

orIPv4“130.30.27.38”Lookup

Page 16: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

16

BeforetheDNS– HOSTS.TXT• DirectorywasafileHOSTS.TXTregularlyretrievedforallhostsfromacentralmachineattheNIC(NetworkInformationCenter)

• Nameswereinitiallyflat,becamehierarchical(e.g.,lcs.mit.edu)~1985

• NeithermanageablenorefficientastheARPANETgrew…

Page 17: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

17

DNS• AnamingservicetomapbetweenhostnamesandtheirIP

addresses(andmore)– www.uwa.edu.auà 130.95.128.140

• Goals– Easytomanage(especiallywithmultipleparties)– Efficient(goodperformance,fewresources)

• Approach– Distributeddirectorybasedonahierarchicalnamespace– Automatedprotocoltotiepiecestogether

Page 18: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

DNSNamespace• Hierarchical,startingfrom“.”(dot,typicallyomitted)

18

robot.cs.washington.edu

Page 19: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

TLDs(Top-LevelDomains)• RunbyICANN(InternetCorp.forAssignedNamesandNumbers)

– Startingin‘98;namingisfinancial,political,andinternational

• 22+genericTLDs– Initially.com,.edu ,.gov.,.mil,.org,.net– Added.aero,.info,.museum,etc.from’01through.xxxin’11– DifferentTLDshavedifferentusagepolicies

• ~250countrycodeTLDs– Twoletters,e.g.,“.au”,plusinternationalcharacterssince2010– Widelycommercialized,e.g.,.tv (Tuvalu)– Manydomainhacks,e.g., instagr.am(Armenia),goo.gl(Greenland)

19

Page 20: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

DNSZones• Azone isacontiguousportionofthenamespace

20

AzoneDelegation

Page 21: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

21

DNSZones(2)• Zonesarethebasisfordistribution

– EDURegistraradministers.edu– UWadministerswashington.edu– CS&Eadministerscs.washington.edu

• Eachzonehasanameserver tocontactforinformationaboutit– Zonemustincludecontactsfordelegations,e.g.,.eduknowsnameserver forwashington.edu

Page 22: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

DNSResourceRecords• AzoneiscomprisedofDNSresourcerecordsthatprovideinformationaboutitsdomainnames

22

Type MeaningSOA Start of authority, has main zone parametersA IPv4 address of a hostAAAA (“quad A”) IPv6 address of a hostCNAME Canonical name for an aliasMX Mail exchanger for the domainNS Nameserver of domain or delegated subdomain

Page 23: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

23

DNSResourceRecords(2)

IPaddressesofcomputers

Nameserver

Mailgateways

Page 24: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

24

DNSResolution• DNSprotocolletsahostresolveanyhostname(domain)toIPaddress

• Ifunknown,canstartwiththerootnameserver andworkdownzones

• Let’sseeanexamplefirst…

Page 25: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

DNSResolution(2)• flits.cs.vu.nlresolvesrobot.cs.washington.edu

25

Page 26: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

26

Iterativevs.RecursiveQueries• Recursivequery

– Nameserver completesresolutionandreturnsthefinalanswer– E.g.,flitsà localnameserver

• Iterativequery– Nameserver returnstheanswerorwhotocontactnextfortheanswer

– E.g.,localnameserver à allothers

Page 27: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

27

Iterativevs.RecursiveQueries(2)• Recursivequery

– Letsserveroffloadclientburden(simpleresolver) formanageability

– Letsservercacheoverapoolofclientsforbetterperformance

• Iterativequery– Letsserver“fileandforget”– Easytobuildhighloadservers

Page 28: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

28

Caching• Resolutionlatencyshouldbelow

– Addsdelaytowebbrowsing• Cachequery/responsestoanswerfuturequeriesimmediately– Includingpartial(iterative) answers– ResponsescarryaTTLforcaching

Nameserver

query out

responseCache

Page 29: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

Caching(2)• flits.cs.vu.nlnowresolveseng.washington.edu

– Andpreviousresolutionscutoutmostoftheprocess

29

1:query 2:query

UWnameserver(forwashington.edu)

3:eng.washington.edu4:eng.washington.edu

Localnameserver(forcs.vu.nl)

Iknowtheserverforwashington.edu!

Cache

Page 30: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

30

LocalNameservers• Localnameservers typicallyrunbyIT(enterprise,ISP)

– ButmaybeyourhostorAP– Oralternativese.g.,GooglepublicDNS

• Clientsneedtobeabletocontacttheirlocalnameservers– TypicallyconfiguredviaDHCP

Page 31: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

31

RootNameservers• Root(dot)isservedby13servernames

– a.root-servers.nettom.root-servers.net– Allnameservers needrootIPaddresses– Handledviaconfigurationfile(named.ca)

• Thereare>250distributedserverinstances– Highlyreachable,reliableservice– MostserversarereachedbyIPanycast (Multiplelocationsadvertise

sameIP!Routestakeclienttotheclosestone.See§5.2.9)– ServersareIPv4andIPv6reachable

Page 32: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

RootServerDeployment

32

Source:http://www.root-servers.org.Snapshoton27.02.12.Doesnotrepresentcurrentdeployment.

Page 33: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

33

DNSProtocol• Queryandresponsemessages

– BuiltonUDPmessages,port53– ARQforreliability;serverisstateless!– Messageslinkedbya16-bitIDfield

Query

Response

Time

Client ServerID=0x1234

ID=0x1234

Page 34: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

34

DNSProtocol(2)• Servicereliabilityviareplicas

– Runmultiplenameservers fordomain– Returnthelist;clientsuseoneanswer– Helpsdistributeloadtoo

NSforuw.edu?

A

B

C

UseA,BorC

Page 35: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

35

DNSProtocol(3)• Securityisamajorissue

– Compromiseredirectstowrongsite!– Notpartofinitialprotocols..

• DNSSEC(DNSSecurityExtensions)– Longunderdevelopment,nowpartiallydeployed

Um,security??

Page 36: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

36

HTTP,theHyperText TransferProtocol(§7.3.1-7.3.4)

• HTTP,(HyperText TransferProtocol)– Basisfor fetchingWebpages

requestNetwork

Page 37: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

37

SirTimBerners-Lee(1955–)• InventoroftheWeb

– DominantInternetappsincemid90s– HenowdirectstheW3C

• DevelopedWebatCERNin‘89– Browser,serverandfirstHTTP– Popularized viaMosaic(‘93),Netscape– FirstWWWconference in’94…

Source:ByPaulClarke,CC-BY-2.0,viaWikimediaCommons

Page 38: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

WebContext

38

HTTPrequest

HTTPresponse

PageasasetofrelatedHTTPtransactions

Page 39: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

39

WebProtocolContext• HTTPisarequest/responseprotocolforfetchingWebresources– RunsonTCP,typicallyport80– Partofbrowser/server app

TCPIP

802.11

browserHTTP

TCPIP

802.11

serverHTTP

request

response

Page 40: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

40

FetchingaWebpagewithHTTP• StartwiththepageURL:

http://de.wikipedia.org/wiki/Chuchichäschtli

• Steps:– ResolvetheservertoIPaddress(DNS)– SetupTCPconnectiontotheserver– SendHTTPrequestforthepage– (AwaitHTTPresponseforthepage)– Execute/fetchembeddedresources/render– CleanupanyidleTCPconnections

Protocol PageonserverServer

**

Page 41: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

Staticvs DynamicWebpages• Staticwebpageiscontentofafile,e.g.,image• Dynamicwebpageistheresultofprogramexecution

– Javascript onclient,PHPonserver,orboth

41

Page 42: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

EvolutionofHTTP• Considersecurity(SSL/TLSforHTTPS)later

42

20101990 20001995 2005

1.0developed

1.1developed(persistentconnections)

0.9RFC1945

RFC2068,2109RFC2616

CookiesSSL2.0

SPDY(HTTP2.0)

Proliferationofcontenttypes

andbrowser/server

scriptingtechnologies

RFC2965

Page 43: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

43

HTTPProtocol• Originallyasimpleprotocol,withmanyoptionsaddedover

time– Text-basedcommands,headers

• Tryityourself:– Asa“browser”fetchingaURL– Run“telnetwww.scion-architecture.net 80”– Type“GET/HTTP/1.1”followedby“Host:www.scion-

architecture.net”followedbyablankline– ServerwillreturnHTTPresponsewiththepagecontents(orother

info)

Page 44: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

44

HTTPGetResult$telnet www.scion-architecture.net 80Trying129.132.85.42...Connected toscion-architecture.net.Escapecharacter is'^]'.GET/HTTP/1.1Host:www.scion-architecture.net

HTTP/1.1 200OKDate:Wed,01 Jun201621:04:24 GMTServer: Apache/2.2.15 (RedHat)X-Powered-By: PHP/5.3.3Transfer-Encoding: chunkedContent-Type: text/html; charset=UTF-8

3565

<!DOCTYPE HTML>

<html>...

Page 45: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

45

HTTPProtocol(2)• Commandsusedintherequest

Method DescriptionGET ReadaWebpageHEAD ReadaWebpage'sheaderPOST AppendtoaWebpagePUT StoreaWebpageDELETE RemovetheWebpageTRACE EchotheincomingrequestCONNECT ConnectthroughaproxyOPTIONS Queryoptionsforapage

FetchpageUploaddata

Page 46: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

HTTPProtocol(3)• Codesreturnedwiththeresponse

46

Code Meaning Examples1xx Information 100=serveragreestohandleclient'srequest2xx Success 200=requestsucceeded;204=nocontentpresent3xx Redirection 301=pagemoved;304=cachedpagestillvalid4xx Clienterror 403=forbiddenpage;404=pagenotfound5xx Servererror 500=internalservererror;503=tryagainlater

Yes!

Page 47: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

HTTPProtocol(4)• Manyheaderfieldsspecifycapabilitiesandcontent

– E.g.,Content-Type: text/html,Cookie: lect=12-1-http

47

Function ExampleHeadersBrowsercapabilities(client à server)

User-Agent, Accept,Accept-Charset,Accept-Encoding,Accept-Language

Cachingrelated(mixeddirections)

If-Modified-Since, If-None-Match,Date,Last-Modified,Expires,Cache-Control,ETag

Browsercontext(clientà server) Cookie,Referer,Authorization,Host

Contentdelivery(serverà client)

Content-Encoding, Content-Length,Content-Type,Content-Language,Content-Range,Set-Cookie

Page 48: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

48

HTTPPerformance (§7.3.4)• PerformanceofHTTP

– Parallelandpersistentconnections

requestNetwork

Page 49: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

49

PLT(PageLoadTime)• PLTisthekeymeasureofwebperformance

– Fromclickuntiluserseespage– SmallincreasesinPLTdecreasesales

• PLTdependsonmanyfactors– Structureofpage/content– HTTP(andTCP!)protocol– NetworkRTTandbandwidth

Page 50: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

50

EarlyPerformance(1)• HTTP/1.0usesoneTCPconnectiontofetcheachwebresource– MadeHTTPveryeasytobuild– ButgavefairlypoorPLT…

Page 51: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

51

EarlyPerformance(2)• ManyreasonswhyPLTislargerthan

necessary– Sequentialrequest/responses,evenwhen

todifferentservers– MultipleTCPconnectionsetupstothesame

server– MultipleTCPslow-startphases

• Network isnotusedeffectively– Worsewithmanysmallresources/page

Page 52: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

52

WaystoDecreasePLT1. Reducecontentsizefortransfer

– Smallerimages,gzip2. ChangeHTTPtomakebetteruseofavailable

bandwidth(e.g.,avoidTCPslowstart)3. ChangeHTTPtoavoidrepeatedtransfersofthe

samecontent– Caching,andproxies

4. Movecontentclosertoclient– CDNs[later]

Page 53: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

53

ParallelConnections• OnesimplewaytoreducePLT

– Browserrunsmultiple(8,say)HTTPinstancesinparallel– Serverisunchanged;alreadyhandlesconcurrentrequestsformanyclients

• Howdoesthishelp?– SingleHTTPwasn’tusingnetworkmuch…– Soparallelconnectionsaren’tslowedmuch– Pullsincompletiontimeoflastfetch

Page 54: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

54

PersistentConnections• Parallelconnectionscompetewitheachotherfornetworkresources– 1parallelclient≈8sequentialclients?– Exacerbatesnetworkbursts,andloss

• Persistentconnectionalternative– Make1TCPconnectionto1server– UseitformultipleHTTPrequests

Page 55: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

PersistentConnections(2)

55

Onerequestperconnection

Sequentialrequestsperconnection

Pipelinedrequestsperconnection

Page 56: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

56

PersistentConnections(3)• WidelyusedaspartofHTTP/1.1

– Supportsoptionalpipelining– PLTbenefitsdependingonpagestructure,buteasyonnetwork

• Issueswithpersistentconnections– HowlongtokeepTCPconnection?– Canitbeslower?(Yes.Butwhy?)

Page 57: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

57

HTTPCachingandProxies (§7.3.4,§7.5.2)

• HTTPcachingandproxies– Enablingcontentreuse

ServerClients

ProxyCache

Page 58: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

58

WebCaching• Usersoftenrevisitwebpages

– Bigwinfromreusinglocalcopy!– Thisiscaching

• Keyquestion:– WhenisitOKtoreuselocalcopy?

NetworkCache

Localcopies

Server

Page 59: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

59

WebCaching(2)• Locallydetermineifcopyisstillvalid

– Basedonexpiryinformation suchas “Expires”header fromserver

– Oruseaheuristictoguess(cacheable, freshlyvalid,notmodified recently)

– Contentisthenavailablerightaway

NetworkCacheServer

Page 60: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

60

WebCaching(3)• Revalidatecopywithremoteserver

– Basedontimestampofcopysuchas“Last-Modified” headerfromserver

– Orbasedoncontentsuchas“ETag”header fromserver:EntityTag,computedbyserverasauniqueobjectidentifier

– Contentisavailableafter1RTT

NetworkCacheServer

Page 61: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

WebCaching(4)• Puttingthepiecestogether:

61

Page 62: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

62

WebProxies• Placeintermediarybetweenpoolofclientsandexternalwebservers

• Proxycaching– Clientsbenefitfromlarger,sharedcache(otherclientsmayhavealreadyaccessedcontent)

– Benefitslimitedbysecure/dynamicper-clientcontent,aswellas“longtail”dataaccesspattern

– Enablesapplicationoforganizationalaccesspolicies

Page 63: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

WebProxies(2)• Clientscontactproxy;proxycontactsserver

63

Cache

NearclientFarfromclient

Page 64: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

64

CDNs (ContentDeliveryNetworks)(§7.5.3)

• CDNs(ContentDeliveryNetworks)– Efficientdistributionofpopularcontent;fasterdelivery forclients

ContentReplica

Consumers

Page 65: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

65

Context• Asthewebtookoffinthe90s,trafficvolumesgrewandgrew.This:1. Concentrated loadonpopularservers2. Ledtocongestednetworksandneedtoprovisionmore

bandwidth3. Gaveapooruserexperience

• Idea:– Placepopularcontentnearclients– Helpswithallthreeissuesabove

Page 66: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

66

BeforeCDNs• Sendingcontentfromthesourceto4userstakes4x3=12“networkhops”intheexample

Source

User

User

...

Page 67: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

67

AfterCDNs• Sendingcontentviareplicastakesonly4+2=6“networkhops”

Source

User

User

...Replica

Page 68: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

68

AfterCDNs(2)• Benefitsassumingpopularcontent:

– Reducesserver,network load– Improvesuserexperience (PLT)

Source

User

User

...Replica

Page 69: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

69

PopularityofContent• Zipf’s Law:fewpopularitems,manyunpopularones(“heavytail”ofprobabilitydistribution);bothmatter

Zipf popularity(kth itemis1/k)

Rank

Source:Wikipedia

GeorgeZipf (1902-1950)

Page 70: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

70

Howtoplacecontentnearclients?• Usebrowserandproxycaches

– Helps,butlimitedtooneclientorclientsinoneorganization

• WanttoplacereplicasacrosstheInternetforusebyallnearbyclients– DonebycleveruseofDNS

Page 71: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

ContentDeliveryNetwork

71

Page 72: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

ContentDeliveryNetwork(2)• DNSresolutionofsitegivesanswerdependingonclient

– Directeachclient tothenearestreplica(usingIPgeolocation)

72

Page 73: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

Consumer

site

73

BusinessModel• ClevermodelpioneeredbyAkamai

– PlacingsitereplicaatanISPiswin-win– Improvessiteexperience andreducesbandwidthusageofISP

ISPUser

User

...Replica

Page 74: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

74

TheFutureofHTTP• TheFutureofHTTP

– Howwillwemakethewebfaster?– Abrieflookatsomeapproaches

requestNetwork

Page 75: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

ModernWebPages• Waterfalldiagramshowsprogressionofpageload

75

webpagetest toolforhttp://coursera.org (Firefox,5/1Mbps, fromVA,3/1/13)

Page 76: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

76

ModernWebPages(2)

Yikes!-23requests-1Mbdata-2.6secs

webpagetest toolforhttp://coursera.org (Firefox,5/1Mbps, fromVA,3/1/13)

Page 77: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

• WaterfallandPLTdependsonmanyfactors– Verydifferent fordifferent browsers– Verydifferent forrepeatpageviews– Dependsonlocalcomputationaswellasnetwork

ModernWebPages(3)

77

Yay!(Networkusedwell)

Page 78: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

78

RecentworktoreducePLTPagesgrowevermorecomplex!

– Larger,moredynamic,andsecure– HowwillwereducePLT?

1. Betteruseofthenetwork– HTTP/2effortbasedonSPDY

2. Bettercontentstructures– mod_pagespeed serverextension

Page 79: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

79

SPDY(“speedy”)• AsetofHTTPimprovements

– Multiplexed (parallel)HTTPrequestsononeTCPconnection– Clientpriorities forparallel requests– CompressedHTTPheaders– Serverpushofresources

• Nowbeingtestedandimproved– DefaultinChrome,Firefox– BasisforHTTP/2

Page 80: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

80

mod_pagespeed• Observation:

– Thewaypagesarewrittenaffectshowquicklytheyload– Manybooksonbestpractices forpageauthorsanddevelopers

• Keyidea:– Haveserverre-write (compile)pagestohelpthemloadquickly!

– Apachemod_pagespeed isanexample

Page 81: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

81

mod_pagespeed (2)• Apacheserverextension

– Software installedwithwebserver– Rewritespages“onthefly”withrulesbasedonbestpractices

• Examplerewriterules:– MinifyJavascript– Flattenmulti-level CSSfiles– Resizeimagesforclient– …andmuchmore(100sofspecific rules)

Page 82: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

82

Peer-to-PeerContentDelivery(BitTorrent)(§7.5.4)

• Peer-to-peercontentdelivery– Runswithoutdedicatedinfrastructure– BitTorrent asanexample

Peer

Peer

Peer

PeerPeer

Page 83: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

83

Context• Deliverywithclient/serverCDNs:

– Efficient, scalesupforpopularcontent– Reliable,managedforgoodservice

• …butsomedisadvantagestoo:– Needfordedicated infrastructure– Centralized control/oversight

Page 84: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

84

P2P(Peer-to-Peer)• Goalisdeliverywithout dedicatedinfrastructureorcentralizedcontrol– Stillefficientatscale,andreliable

• Keyideaistohaveparticipants(orpeers)helpeachother– InitiallyNapster‘99formusic(gone)– NowBitTorrent ‘01onwards(popular!)

Page 85: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

85

P2PChallenges• Noserversonwhichtorelyon

– Communicationmustbepeer-to-peer andself-organizing,notclient-server

– Leadstoseveralissuesatscale…

Peer

Peer

Peer

PeerPeer

Page 86: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

86

P2PChallenges(2)1. Limitedcapabilities

– Howcanonepeerdelivercontenttoallotherpeers?

2. Participationincentives– Whywouldpeershelpeachother?

3. Decentralization– Howwillpeersfindcontent?

Page 87: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

87

OvercomingLimitedCapabilities• Peercansendcontenttoallotherpeersusingadistributiontree– Typicallydonewithreplicasovertime– Self-scalingcapacity (morenodesà morecapacity)

Source

Page 88: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

88

ProvidingParticipationIncentives• Peersplaytworoles:

– Download()tohelpthemselves,andupload()tohelpothers

Source

Page 89: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

89

ProvidingParticipationIncentives(2)• Couplethetworoles:

– I’lluploadforyouifyouuploadforme– Encouragescooperation

Source

Page 90: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

90

EnablingDecentralization• Peermustlearnwheretogetcontent

– UseDHTs (DistributedHashTables)

• DHTsarefully-decentralized,efficientalgorithmsforadistributedindex– Indexisspreadacrossallpeers– Indexlistspeerstocontactforcontent– Anypeercanlookuptheindex– Startedasacademicworkin2001

Page 91: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

91

BitTorrent• MainP2Psysteminusetoday

– DevelopedbyCohenin‘01– Veryrapidgrowth,largetransfers– BigfractionofInternettraffic– Usedforlegalandcopyrightedcontent

• Deliversdatausing“torrents”:– Transfersfilesinpiecesforparallelism– Notablefortreatmentofincentives– Trackerordecentralized index(DHT)

ByJacobAppelbaum, CC-BY-SA-2.0, fromWikimediaCommons

BramCohen(1975—)

Page 92: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

92

BitTorrent Protocol• Stepstodownloadatorrent:

1. Startwithtorrentdescription2. Contacttrackertojoinandgetlistofpeers(withatleast

seedpeer)2. Or,useDHTindexforpeers3. Tradepieceswithdifferentpeers4. Favorpeersthatuploadtoyourapidly;“choke”peers

thatdon’tbyslowingyouruploadtothem

Page 93: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

BitTorrent Protocol(2)• Allpeers(exceptseed)retrievetorrentatthesametime

93

Page 94: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

BitTorrent Protocol(3)• Dividingfileintopiecesgivesparallelismforspeed

94

Page 95: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

BitTorrent Protocol(4)• Chokingunhelpfulpeersencouragesparticipation

95

STOPSTOP

STOP

XXX

Page 96: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

BitTorrent Protocol(5)• DHTindex(spreadoverpeers)isfullydecentralized

96

DHT

DHT

DHTDHT

DHT

DHT

DHT

DHT

Page 97: Operating Systems and Networks Network Lecture 12 ... · Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ... Application

97

P2POutlook• AlternativetoCDN-styleclient-servercontentdistribution– Withpotentialadvantages

• P2PandDHTtechnologiesfindingmorewidespreaduseovertime– E.g.,partofskype,Amazoncloudcomputing– Expecthybridsystemsinthefuture