Writing Network Programs with Python

Nile University, CIT-614 Network Programming and Distributed Objects, By Dr Sameh Al Ansary, modified by Ahmed Kishk

Agenda

Talk - Are We really Engineers?Python (Programming language).

Engineering?

Software: is it really engineering? If it is not engineering, then what is it?

Art? Craft? Science?

Engineering?

Art: the production or expression of what is beautiful, appealing, or of more than ordinary significance.

Craft: A trade or occupation requiring special skill, especially manual skill.

Science: The systematic study of man and his environment based on the deductions and inferences that can be made, and the general laws that can be formulated, from reproducible observations and measurements of events and parameters within the universe.

Engineering?

Engineering: The art of making practical application of the knowledge of pure science.

Simply it is a profession.

Engineering?

The profession in which a knowledge of the mathematical and natural sciences gained by study, experience, and practice is applied with judgment to develop ways to utilize, economically, the materials and forces of nature for the benefit of mankind.

Accreditation board for Engineering & Technology, 1996

Talk

SEI (Software Engineering Institute)Software Product Line (Development, Testing and

Production Environment)Skill y0ur self with Software Engineering

professionalism (Testing -Architecture Design and Documentation)

Doing Unit Testing, Regression Testing, Integration Testing, System Testing and Smoke Testing.

Web Application EngineeringYou have to have critical thinking.Read Code Complete

To Find More…

Let’s Start our Lecture

Python

Python is a programming language that lets you work more quickly and integrate your systems more effectively.

Download the compiler from http://www.python.org/

PyDev is a plugin for python in Eclipse IDE.You can find installation steps at below link

http://pydev.org/manual_101_install.html

Python

no explicit variable declarations

Variables spring into existence by being assigned a value

automatically destroyed when they go out of scope.

Python

Run: Python >>>print “Hello MUFIX”

Weak typed language: >>> x = 5 >>>x 5 >>>type(x) <type 'int'> >>>x = “MUFIX” >>>x MUFIX >>>type(x) <type 'str'>

Python

.py file is called a module.To declare a function use “def:”Args separated by “,”No return typesuse the “return” keyword to return values

Python Indenting Code

no explicit begin or end, and no curly braces.

Python Lists

Like java ArrayList but more powerful.>>> list = [“a”, “b”, “c”, “d”]>>>list[“a”, “b”, “c”, “d”]>>> List[0]“a”>>> List[3]“d”

Python Lists

Negative List indices>>>list[“a”, “b”, “c”, “d”]>>>list[-1]“d”>>>list[-3]“b”

Python Lists

Slicing a list>>>list[“a”, “b”, “c”, “d”]>>>list[1:3][“b”, “c”]>>>list[1, -1][“b”, “c”]>>>list[:3][“a”, “b”, “c”]>>>list[:][“a”, “b”, “c”, “d”]

Python Lists

Operations: like java ArrayList methods

List.append()List.insert()List.extend()List.index()List.pop()

Python Dictionaries

Like Java HashMap but a simple one.A key / value pairs

>>> Info = {“name”:”Ahmed”, “job”:”Software Engineer”}

>>>Info[“name”]“Ahmed”>>>Info[“College”] = “MUFIC”>>>Info{“name”:”Ahmed”, “job”:”Software Engineer”,

“College”:”MUFIC”}

Python Dictionaries

• len(d)• d[key]• d[key] = value• del d[key]• key in d• key not in d• clear()• copy()• items()• keys()• values()

Tuples

>>> T = (“a”, “b”, “c”, “d”)>>> T(“a”, “b”, “c”, “d”)

Read-Only ListsHave no methodsLike java Enums

Python Strings

>>> names = “C++, Java, Python, Ruby, C#”>>>list = names.split(“,”)['C++', ' Java', ' Python', ' Ruby', ' C#']>>>names = ” ”.join(list)>>>names”C++ Java Python Ruby C#”

Python and Files

To Find More…

Network Programming

Browsers, chat clients, Downloads…..etc

Internet Socket and TCP/IP

an Internet socket or network socket is an endpoint of a bidirectional inter-process communication flow across an Internet Protocol-based computer network, such as the Internet.

Internet sockets is also used as a name for an (API) for the TCP/IP protocol stack

provided by the operating system. Internet sockets delivers incoming data packets to

the appropriate application process or thread, based on a combination of local and remote IP addresses and port numbers.

Socket types

Datagram sockets, also known as connectionless sockets, which use User Datagram Protocol (UDP)

Stream sockets, also known as connection-oriented sockets, which use Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP).

Raw sockets (or Raw IP sockets), typically available in routers and other network equipment. Here the transport layer is bypassed, and the packet headers are not stripped off, but are accessible to the application. Application examples are Internet Control Message Protocol (ICMP, best known for the Ping sub operation), Internet Group Management Protocol (IGMP), and Open Shortest Path First (OSPF).

Connection-Oriented Services

socket()

bind()

listen()

accept()

read()

write()

[blocked]

socket()

connect()

write()

read()

[blocked]

[blocked]

Server Client

When interaction is over, server loops to accept a new connection

First Network Program – Echo Server

Client

File Like Objects

HTTP RFC 2616

HTTP Request and Response

HTTP Request

• The format of the initial line for requests andresponses is different• The Request line contains:– Method:• GET: The most common• Others: POST, HEAD, PUT,…– Request-URI• Exampe: bla/directory17/hello.html– HTTP-Version• Example The string: “HTTP/1.1”

HTTP Request

Request Methods

Response Initial line

• The Response initial (Called Status line): – HTTP-Version: – Status-Code: three digits, e.g 200, 404 – Reason-Phrase: OK, Not found

• Examples: – 200 OK – 404 Not Found • (The requested resource doesn't exist.)

Response status codes

Minimal HTTP Client

HTTP Connection Management

Serial connection:

Parallel connection

Each transaction opens/closes a new connection, costing time and bandwidth.

Each new connection has reduced performance because of TCP slow start.

There is a practical limit on the number of open parallel connections.

Persistent Connections

TCP connections that are kept open after transactions complete are called persistent connections

Pipelined Connections

Multiple requests can be enqueued before the responses arrive

To Find More…

http://www.w3.org/Protocols/rfc2616/rfc2616.html

YouTube Downloader

http://www.youtube.com/watch?v=QvsQ9hYKq7c

Proxy Demo

http://www.youtube.com/watch?v=QvsQ9hYKq7c

Non Blocking Sockets

socket()

bind()

listen()

accept()

read()

write()

[blocked]

socket()

connect()

write()

read()

[blocked]

[blocked]

Server Client

When interaction is over, server loops to accept a new connection

Non Blocking Sockets

socket.setblocking(0) Simply In Python, you use to make it non-blocking.

send, recv, connect and accept can return without having done anything.

You can check return code and error codes and generally drive yourself crazy.

Your app will grow large, buggy.

let’s skip the brain-dead solutions and do it right.

Use select.

Multiplexing Servers with select

Let’s pick one of the previous solustion (Threads)

threads and processes don't really run in parallel.

your operating system divides the computer's processing power among all active tasks.

This process of switching between tasks is sometimes called multiplexing.

And this is the idea behind SELECT

Multiplexing Servers with select (CONTIUNED)

Servers can apply this technique without Threading nor forking.

By multiplexing client connections and the main dispatcher with the select system call, a single event loop can process clients and accept new ones in parallel .

the magic behind this server structure is the operating system select call

select is asked to monitor a list of input sources, output sources, and exceptional condition sources and tells us which sources are ready for processing

select-based echo server

select-based echo server (continued)

Questions