Client/Server Networking

Protocol Stack Summary

“turtles all the way down”

each layer uses the services of the layer below and provides a service to the layer above

Python lets you work at the layer of your choice

programs are “cleaner” the higher the layer you use

layers work by “hiding” the layers below behind function calls

What lies under Socket?

TCP/UDP

IP

“link layer”

Internet Protocol Stack

Networking

About “sharing” resources.

Compare to sharing of disk, IO devices, etc done by programs running on a computer

Computer Example: OS is the master controller

Network Example: Each participant “plays by the rules” but no “controller”

Networking

All network cards share the same ethernet cable

all wireless transmitters share the same frequency channels

fundamental unit of sharing is “packet”

individual packets carry addressing info sufficient to arrive at final destination.

Addressing

two layers: one hop at a time and end-to-end

single hop addressing performed by link layer

end-to-end addressing is IP

process addressing is TCP or UDP

IP Addresses

a.b.c.d: 0 <= a,b,c,d <= 255

New Paltz: 137.140.*.*

localhost: 127.0.0.1

private networks: 10.*.*.*, 172.16-31.*.*, 192.168.*.*

Domain Name Service

DNS converts host names into host IP addresses.

corresponds to directory assistance

address = socket.gethostbyname(name);

How DNS Works

gethostbyname() first looks in /etc/hosts

if this fails then it looks in /etc/resolv.conf for the address of “directory assistance”, also called a DNS Server.

sends the request to this address

Observation: If your DNS server is down, you won't get anywhere on the Internet.

Routing

Each time a packet arrives at a new node a decision must be made at that node as to where to send the packet next.

Guiding principle of routing on the Internet is that each time a packet “hops” from one node to another it is always one hop closer to its final destination.

Exercise: Difference between host and node.

Lots of Reading

The classic text is TCP/IP Illustrated: Vol I by Richard Stevens.

PDF file available on the web at books.google.com among other places

We will concentrate on Chapters 1-4, 9, 11, 14, 17-19.

EXAMPLE:

Given:

Find longitude and latitude

207 N. Defiance St, Archbold, OH

Protocol Stack:

GoogleMaps

?????

TCP

IP

Ethernet

GoogleMaps

googlemaps library (3rd party) uses

urllib, uses httplib, uses Socket, uses TCP, IP,

Ethernet

GoogleMaps

URL

HTTP

Socket

protocol stackinside the actualprogram itself

TCP, IP andEthernet make upthe OS part of theprotocol stack

APIs vs Sockets:

well-tested

written by experts

common practice to use them

we still need to understand Sockets toappreciate things that depend upon them

Wireshark:

lets you look at packets crossing the wire

needs root permissions

easy to filter out unneeded traffic

I saved some traffic and you can view it with Wireshark (see course web page).

Highest Level API Example:

Fetch a JSON document without realizing it:

#!/usr/bin/env python# Foundations of Python Network Programming - Chapter 1 - search1.py# Not even clear you are using a web service

from googlemaps import GoogleMapsaddress = '207 N. Defiance St, Archbold, OH'print GoogleMaps().address_to_latlng(address)

GET Syntax:

#!/usr/bin/env python# Foundations of Python Network Programming - Chapter 1 - search2.py# HTML-level abstraction

import urllib, urllib2try: import jsonexcept ImportError: # for Python 2.5 import simplejson as json

params = {'q': '207 N. Defiance St, Archbold, OH', 'output': 'json', 'oe': 'utf8'}url = 'http://maps.google.com/maps/geo?' + urllib.urlencode(params)

rawreply = urllib2.urlopen(url).read()

reply = json.loads(rawreply)print reply['Placemark'][0]['Point']['coordinates'][:-1]

GET Syntax:

#!/usr/bin/env python# Foundations of Python Network Programming - Chapter 1 - search3.py# HTTP level abstraction

import httplibtry: import json # json built in with Python 2.6except ImportError: # for Python 2.5 import simplejson as json

path = ('/maps/geo?q=207+N.+Defiance+St%2C+Archbold%2C+OH' '&output=json&oe=utf8')

connection = httplib.HTTPConnection('maps.google.com')connection.request('GET', path)rawreply = connection.getresponse().read()

reply = json.loads(rawreply)print reply['Placemark'][0]['Point']['coordinates'][:-1]

GET Syntax:

#!/usr/bin/env python# Foundations of Python Network Programming - Chapter 1 - search4.py

import socketsock = socket.socket() # OS functionalitysock.connect(('maps.google.com', 80))sock.sendall( 'GET /maps/geo?q=207+N.+Defiance+St%2C+Archbold%2C+OH' '&output=json&oe=utf8&sensor=false HTTP/1.1\r\n' 'Host: maps.google.com:80\r\n' 'User-Agent: search4.py\r\n' 'Connection: close\r\n' '\r\n')rawreply = sock.recv(4096)print rawreply

GET Syntax:

# search4.py output

HTTP/1.1 200 OKContent-Type: text/javascript; charset=UTF-8Vary: Accept-LanguageDate: Wed, 21 Jul 2010 16:10:38 GMTServer: mafeCache-Control: private, x-gzip-ok=""X-XSS-Protection: 1; mode=blockConnection: close

{ "name": "207 N. Defiance St, Archbold, OH", "Status": { "code": 200, "request": "geocode" }, "Placemark": [ { ... "Point": { "coordinates": [ -84.3063479, 41.5228242, 0 ] } } ]}

data transmitted byweb server

data read intoprogram variable

Things We've Seen:

protocols stacked on top of one another

higher level protocols using services of lower levels

programs get more specific and harder to maintain the lower down you go

the idea behind high-level protocols is precisely to hide lower levels

there's a whole lot going on below Socket.

The Stack:

Fundamental unit of shared information is the packet.

Typical packet structure:

Transmitted as a single unit (but serially)

Routing is generally at the packet level

Things packets contain: data, addresses, layering, sequencing, protocol bytes, checksums

ethernet packets are called frames.

programdata

TCP/UDPheader

IP header

ethernetheader

Ethernet:

14-byte header

addresses: two 6-byte addresses – source and destination

type: 2 bytes – 0800 == IP datagram

the two network cards involved can process the header without using the CPU, RAM, etc.

cable length (100m) and MTU

CSMA/CD

Some of the details:

http://serverfault.com/questions/422158/what-is-the-in-the-wire-size-of-a-ethernet-frame-1518-or-1542

IP Addresses:

32 bits: a.b.c.d

network address – n bits; host id – (32-n) bits

some times the network part has a subnet component; some times the subnet component is carved out of the hostID bits

a.b == 137.140 == New Paltz network address

a.b.c == 137.140.8 == CS subnet at New Paltz

the part of the network address that is not subnet identifies an organization like New Paltz.

IP Address Classes:

IP Special Addresses:

127.*.*.*: local to the current machine

10.*.*.*, 172.16-31.*.*, 192.168.*.*: private subnets.

none of these address found on the larger Internet.

IP Routing:

Guiding principle: after each hop you are one step closer to your destination

typical local routing table contains a default entry pointing to the Internet together with one entry for each local subnet the host belongs to.

[pletcha@archimedes PPT]$ netstat -nrKernel IP routing tableDestination Gateway Genmask Flags Iface0.0.0.0 192.168.1.1 0.0.0.0 UG wlan0192.168.1.0 0.0.0.0 255.255.255.0 U wlan0192.168.122.0 0.0.0.0 255.255.255.0 U virbr0

IP Routing Next Hop Algorithm:

Search Destination column of table entries with H-flag set which is an exact match to Destination IP in packet

If found and Flag is G or H then Gateway is next hop; otherwise Destination IP is next hop.

If not found then calculate Dest IP && Genmask for each entry that is not the default. If Dest IP && Genmask == Destination column entry then if Flag is G or H then Gateway is next hop; otherwise Destination IP is next hop.

Otherwise use the default entry. Flag is almost always G so Gateway is next hop IP.

IP Routing Next Hop Algorithm:

Once you have the next hop IP you need to determine the next hop ethernet.

The Address Resolution Protocol (ARP) converts the next hop IP into a next hop ethernet. More recently replaced by the ip neigh command

Exercise: Read up on ARP in TCP/IP Illustrated.

[pletcha@archimedes PPT]$ ip neigh137.140.39.139 dev enp0s25 lladdr 00:c0:17:c2:14:f3 STALE137.140.193.250 dev wlp3s0 lladdr 00:1f:29:07:e4:6a STALE137.140.39.250 dev enp0s25 lladdr 00:21:a0:39:65:00 DELAY

ARP Example

● From my laptop (137.140.8.104) I try to locate joyous (137.140.8.101)

● Because of my routing table I know it is locally connected so 137.140.8.101 is “next hop”.

[pletcha@archimedes PPT]$ ping 137.140.8.101PING 137.140.8.101 (137.140.8.101) 56(84) bytes of data.64 bytes from 137.140.8.101: icmp_seq=1 ttl=64 time=0.266 ms^C

[pletcha@archimedes PPT]$ netstat -nrKernel IP routing tableDestination Gateway Genmask Flags MSS Window irtt Iface0.0.0.0 137.140.8.250 0.0.0.0 UG 0 0 0 enp0s25137.140.8.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s25137.140.192.0 0.0.0.0 255.255.254.0 U 0 0 0 wlp3s0

ARP Request

ARP Reply

Packet Fragmentation:

The Internet Protocol Suite supports 64k packets but specific IP networks support much smaller packets.

Ethernet networks support 1500 byte packets.

IP headers contain a Don't Fragment (DF) flag, set by sender.

– DF not set, then a router can fragment a packet too large to be forwarded on a particular interface.

– DF set, router sends an ICMP message to original sender so sender can fragment the original message and try again.

UDP: DF unset by OS

TCP: DF set by OS

Packet Fragmentation (continued):

Each subnet has an MTU – Maximum Transmission Unit.

Path MTU = min hop MTU over all hops in a path

DSL providers make MTU = 1492.

– Initially many service providers used MTU = 1500 and disabled ICMP so never knew their “large” traffic was being dropped.

TCP/IP Illustrated discusses how fragmentation actually happens (Read Section 11.5).

TCP/IP Illustrated:

Pages to Look at

25, 38, 43, 44, 48, 58, 61, 63