Python and Web Development - IEEEoccs.ieee.org/cosst/presentations/COSST2003-Esterbrook... Who is...

Python and Web Development Can’t possibly do justice in 60 minutes So we’ll be flying at 30,000 feet. Quick URLs: http://python.org/ http://webware.sourceforge.net/ http://www.python.org/cgi-bin/moinmoin/WebProgramming

Who is your presenter? Chuck Esterbrook Independent Contractor/Consultant since Spring

2000 Most work since then has been in Python But also Java, C# and even (gasp!) VB Going backwards: Project Manager, Senior Software

Engineer, B.S. in Comp Sci Have used a variety of tools for a variety of

companies in a variety of roles My programming language of choice is Python Author of O.S. product: Webware for Python Co-founder of SANDPYT

Why Python? At a really high level: Productivity In IT, Productivity = Happiness Python was designed to be

“-able” as in readable, writeable and maintainable Easy to use Powerful Balances all of these

Contrasted with lack of balance in other langs: Perl: writeability to the detriment of others C++: performance to the detriment of others VB: legacy BASIC to the detriment of everything

Python productivity: Ubiquity Can’t stress this enough: Python works well in almost all areas:

web, gui, sys admin, simulation, finances, etc. So you can take your acquired skills and libraries

everywhere Hit the ground running Easily reuse your own home-brewed libs, 3rd party

libs, etc. Contrast:

C++: Not ideal for sys admin or rapid development Perl: Not ideal for large scale or team-based VB: Not ideal for anything

Python: Selective popularity Python has no marketing budget, but Still gets vote of confidence by successful, well-

known companies: Google Yahoo! Industrial Light & Magic

These companies can afford and use any tools they want. Their choices include Python.

Dr. Dobb’s Journal lists Python across the top of their mag. cover right beside Linux, XML, Win32, etc.

Python programs are problem-oriented Common sensation among new Python

programmers, including myself: “In Python, I’m dealing with my problem instead of

my language.” Again, contrast:

C++: obscure compilation problems Perl: reading obscure code VB(6): no inheritance, no exception handling, etc.

What does Python look like? Let’s get this out of the way:

# HelloWorld.py

print “Hello, world”

#!/usr/bin/env python

# What does Python really look like?

# like "wc -l <filenames> | sort -n"

import sys

def main(args=None):

if args is None:

args = sys.argv

linesPerFile = []

for filename in args[1:]: # arg[0] is prog

n = len(open(filename).readlines())

t = (n, filename)

linesPerFile.append(t)

linesPerFile.sort(myCompare)

# print it

for lines, filename in linesPerFile:

print "%4i %s" % (lines, filename)

def myCompare(t1, t2):

return cmp(t1[0], t2[0])

if __name__=='__main__':

main()

What my Python really looks like. Almost all my Python is Object-Oriented. We’re talking classes, objects and methods. Python is easiest OO language I’ve used to date. Like rest of language:

Simple; easy to learn Powerful Gets to the point

More OO specifics: Full dynamic binding Easy introspection (aka “reflection”) Easy hooks for operator overloading (see next slide) Multiple inheritance (see next slide)

What once was evil is now good! I left C++ thinking “multiple inheritance is bad” and

“operator overloading is bad” After experiencing them in Python, I realize both are

good if done right Multiple inheritance

Full dynamic binding No strange compilation errors, or mysterious crashes Used mostly as a “mix-in” style

http://www.linuxjournal.com/article.php?sid=4540 Imagine if Java interfaces provided “default” or “canonical”

implementations of some methods. Bottom line: m.i. increases productivity

Operator overloading Just normal operators

# What does Python OOP look like?

class Node:

def __init__(self, name, superNode=None):

self.name = name

self.superNode = superNode

if superNode:

superNode.subNodes.append(self)

self.subNodes = []

def dump(self, out=None):

if out is None:

out = sys.stdout

self._dump(out, 0)

def _dump(self, out, indent):

out.write(' '*4*indent + str(self) + '\n')

indent += 1

for node in self.subNodes:

node._dump(out, indent)

def __repr__(self):

return '<%s %r %i-subnodes 0x%x>' % (

self.__class__.__name__, self.name,

len(self.subNodes), id(self))

Outputanimal = Node('animal')

mammal = Node('mammal', animal)

cat = Node('cat', mammal)

dog = Node('dog', mammal)

insect = Node('insect', animal)

animal.dump()

<Node 'animal' 2-subnodes 0x8fa260>

<Node 'mammal' 2-subnodes 0x8fa8c8>

<Node 'cat' 0-subnodes 0x8fa378>

<Node 'dog' 0-subnodes 0x8fa418>

<Node 'insect' 0-subnodes 0x8fa440>

Other Python Highlights Built in lists and dictionaries Exception handling

Try…except…else…finally raise SomeError(args)

Full garbage collection Imperative with common constructs

for, while, break, continue, func() Var args by position or keyword Doc strings Modules and packages Platform independent by default and

specific by choice

Learning Python Lots of options! I like:

http://python.org/doc/current/tut/tut.html All the links you need:

http://python.org/topics/learn/ Books abound (check BookPool.com, Amazon) SANDPYT – San Diego Python User’s Group

http://sandpyt.org/

Web dev One hour presentation as intro to

both Python and Python web dev! (heh) Web dev options:

CGI: Bleck. Might be fast enough for some sites, but certainly feels

wasteful. Encourages nekkid scripts. i.e., non-OO

FastCGI: A band-aid useful for existing CGI apps. App Servers

Webware Zope Others

What is Webware for Python? Server-side web development tools that leverage

Python. Most similar to Java web tools and Apple WebObjects Some overlap with CGI, CF, Zope, PHP, ASP… Covers common needs of web developers Open source development and community Python license Cross-platform; works equally well on:

Posix in its many flavors (Linux, BSD, Solaris, UNIX…) Windows NT/2000/XP

Modular architecture: components can easily be used together or independently

Object-oriented

Comparisons to other Tools No religious fervor allowed during this slide. I designed Webware after using various web tools.

So I addressed shortcomings from the start. Unlike PHP, Webware leverages a general purpose

language (Python) and everything that comes with it Same with ColdFusion; also not closed-source Unlike CGI, Webware is fast and provides

a good OO structure Similar to Java web tools, but better language and

less bureaucratic APIs Zope: WW is less monolithic, direct access to Python,

programmer-oriented, etc.

What is in Webware? The heart of Webware is WebKit, the application

server And:

Python Server Pages (PSP) TaskKit MiddleKit UserKit

All of these are Python packages WebKit includes the App Server which you will run

continuously as with other servers (web, db, etc.)

WebKit A fast, easy-to-use Python application server Multi-threading, not forking

Makes persistent data easier Works well on Windows

Supports multiple styles of development: Servlets Python Server Pages (PSP) Custom file extension handling

Extensible Servlet factories Plug-ins Import any Python module. ;-)

Is it real? Yes! Been around since spring 2000 including contractual

work Stable and mature Used in several real-world, commercial projects: http://webware.sf.net/wiki//WhoIsUsingWebware http://StockAlerts.com/ http://StevesStockPicks.com/ http://www.Vorbis.com/ - open free audio http://www.ElectronicAppraiser.com/ - real estate http://PatientWire.com - e-commerce for optometry And others including many private Intranets

Server

Architecture

Browser

Apache

WebKit

Servlets PSPs

Filesystem

mod_webkit

XML-RPC client

80 80

8086

WebKit.cgi

8086

Database

Starting the app server Installation instructions are

included with Webware Ways to connect web server & app server:

WebKit.cgi – least common denominator mod_webkit – fast

In your working directory, run: Unix:

cd /usr/local/webapps/webinator ./AppServer

Windows: cd C:\MyWebApps\Webinator AppServer

Using the Example servlets and PSP’s To use the CGI adapter, surf to:

http://localhost/cgi-bin/WebKit.cgi To use the mod_webkit adapter, surf to:

http://localhost/webkit Experiment and enjoy!

Servlets A Python class located in a module of the same name Must inherit from WebKit.Servlet or one of its

subclasses: WebKit.HTTPServlet WebKit.Page

A common technique is to make your own subclass of WebKit.Page called SitePage which will contain: Utility methods Overrides of default behavior in WebKit.Page

Simplest servlet:from WebKit.Page import Page

class HelloWorld(Page):def writeContent(self):

self.writeln(‘Hello, World!’)

The Request-Response Cycle User initiates a request:

http://localhost/webkit/MyContext/MyServlet This activates the MyContext context, and the MyServlet servlet,

based on settings in Application.config Note: no extension was specified, even though the file is called

MyServlet.py There are settings in Application.config that control the way

extensions are processed An instance of the MyServlet class is pulled out of a pool of

MyServlet instances, OR if the pool is empty then a new MyServlet instance is created.

A Transaction object is created. These methods are called on the MyServlet instance:

Servlet.awake(transaction) Servlet.respond(transaction) Servlet.sleep(transaction)

The MyServlet instance is returned to its pool of instances.

HTTPRequest Derived from generic Request base class Contains data sent by the browser:

GET and POST variables: .field(name, [default]) .hasField(name) .fields()

Cookies: .cookie(name, [default]) .hasCookie(name) .cookies()

If you don’t care whether it’s a field or cookie: .value(name, [default]) .hasValue(name) .values()

CGI environment variables Various forms of the URL Server-side paths etc.

HTTPResponse Derived from generic Response base class Contains data returned to the browser

.write(text) – send text response to the browser Normally all text is accumulated in a buffer, then sent all

at once at the end of servlet processing .setHeader(name, value) – set an HTTP header .flush() – flush all headers and accumulated text; used for:

Streaming large files Displaying partial results for slow servlets

.sendRedirect(url) – sets HTTP headers for a redirect

Page: Convenience Methods Access to the transaction and its objects:

.transaction(), .response(), .request(), .session(), .application()

Writing response data: .write() – equivalent to .response().write() .writeln() – adds a newline at the end

Utility methods: .htmlEncode() .urlEncode()

Passing control to another servlet: .forward() .includeURL() .callMethodOfServlet()

Whatever else YOU decide to add to your SitePage

Page: Methods Called During A Request .respond() usually calls .writeHTML() Override .writeHTML() in your servlet if you want

your servlet to provide the full output But, by default .writeHTML() invokes a convenient

sequence of method calls: .writeDocType() – override this if you don’t want to use

HTML 4.01 Transitional .writeln(‘<html>’) .writeHead() .writeBody() .writeln(‘</html>’)

Forwarding & Including self.forward(‘AnotherServlet’)

Analogous to a redirect that happens entirely within WebKit Bundles up the current Request into a new Transaction Passes that transaction through the normal Request-

Response cycle with the indicated servlet When that servlet is done, control returns to the calling

servlet, but all response text and headers from the calling servlet are discarded

Useful for implementing a “controller” servlet that examines the request and passes it on to another servlet for processing

self.includeURL(‘AnotherServlet’) Similar to .forward(), except that the output from the

called servlet is included in the response, instead of replacing the response.

Sessions Store user-specific data that must persist from one

request to the next Sessions expire after some number of minutes of

inactivity Controlled using SessionTimeout config variable

The usual interface: .value(name, [default]) .hasValue(name) .values() .setValue(name, value)

And dictionary-like access for values: sess = self.session() sess[‘userId’] = userId

PSP: Python Server Pages Mingle Python and HTML in the style of JSP or ASP Include code using <% … %> Include evaluated expressions using <%= … %> Begin a block by ending code with a colon:

<%for I in range(10):%>

End a block using the special tag:<%end%>

When the user requests a PSP: It is automatically compiled into a servlet class derived from

WebKit.Page The body of your PSP is translated into a writeHTML()

method

PSP Example<%def isprime(number): if number == 2: return 1 if number <= 1: return 0 for i in range(2, number/2): for j in range(2, i+1): if i*j == number: return 0 return 1%> <p>Here are some numbers, and whether or not they are prime:<p> <%for i in range(1, 101):%> <%if isprime(i):%>        <font color=red><%=i%> is prime!</font>    <%end%><%else:%>        <%=i%> is not prime.    <%end%>    <br><%end%>

Web Services: XML-RPC Turn your Webware site into a “web service” Write a servlet derived from XMLRPCServlet

Define exposedMethods() method that lists the methods you want to expose through XML-RPC

Write your methods Sorry, no time for an example. Bottom line:

Creating XML-RPC services in Webware is easy Using XML-RPC services in Python is easy

Error Reports (i.e., Tracebacks) If an unhandled exception occurs in a servlet:

Application.config settings: If ShowDebugInfoOnErrors = 1, an HTML version of

the traceback will be shown to the user; otherwise, a short generic error message is shown.

You can configure WebKit so that it sends the traceback by email: EmailErrors, ErrorEmailServer, ErrorEmailHeaders

Include “fancy” traceback using IncludeFancyTraceback and FancyTracebackContext

Your users will NOT report tracebacks, so set up emailing of fancy tracebacks!

MiddleKit Object-Relational mapper Supports MySQL and MS SQL Server.

PostgreSQL support soon? @@ check this Can be used anywhere, not just WebKit applications. Write an object model in a Comma-Separated Values (CSV) file

using a spreadsheet Inheritance is supported Numbers, strings, enums, dates/times, object references, lists of

objects (actually sets of objects) Compile the object model

This generates Python classes for each of your objects that contain accessor methods for all fields

Also, an empty derived class is provided where you can add your own methods

And, a SQL script is generated that you can run to create the tables

Cheetah http://www.cheetahtemplate.org/ A Python-powered template engine and code

generator Uses the “dollar sign-pound sign” $# syntax found in

Velocity, WebMacro, et al. @@ Integrates tightly with Webware Can also be used as a standalone utility or combined

with other tools Compared with PSP:

Much more designer-friendly Perhaps less programmer-friendly?

Zope I used Zope before Webware even existed. Then I wrote Webware. Zope has strong “through the web” CMS and some

nice built-in features if they matched your application.

I found it monolithic and “interfering”. It squirreled Python away and wrapped it with DTML

and UI. But Python was designed to be user-friendly from the

start, so I wanted to use it in a natural environment.

Webware-Zope Quotes “In Zope, I find myself writing a lot of External

Methods to do the 'heavy lifting', and call them from within the DTML. In WebKit, the 'heavy lifting' is just part of the Python, not a requisite separate entity. Gary Perez - Jun 11, 2003

“I guess, it's the thinness of Webware. WW is ‘pro-Python’, which means, that everything you do is more than less directly done in Python. No DHTML, ... This -at least IMO- is one of the strong points in Webware: it doesn't put anything between the application developer and Python.” Frank Barknecht - Jun 11, 2003

Choosing Webware or Zope If you’re more of a non-technical user,

Zope’s point-and-click WUI interfaceand templating, might appeal to you more.

If you like programming in Python,Webware is more likely to appeal to you.

Other App Servers Zope was only mature app server when I built

Webware. I haven’t tracked the others at all. Most popular ones seem to be SkunkWeb, Quixote

and possibly Twisted Matrix. Zope 3 is in development as successor to Zope 2. Webware is still going strong with new developers,

new users and thousands of downloads.

That’s All! Any questions?

URLs: http://python.org/ http://webware.sourceforge.net/ http://www.python.org/cgi-bin/moinmoin/

WebProgramming