Hypertext Transfer Protocol - Harvard Universitycscie12.dce.harvard.edu/lecture_notes/2007-08/20080423.pdf · script > 12. 13. < div id = "myContent" > 14. < p > Alternative

Hypertext Transfer Protocol http://localhost:8080/cocoon/projects/cscie12/slides/20080423/handout.html

1 of 87 4/23/2008 10:53 AM

Table of Contents | All Slides | Link List | CSCI E-12

Hypertext Transfer ProtocolApril 23, 2008

Harvard University Division of Continuing Education

Extension School

Course Web Site: http://cscie12.dce.harvard.edu/

Copyright 1998-2008 David P. Heitmeyer

Instructor email: [email protected] Course staff email: [email protected]


2 of 87 4/23/2008 10:53 AM

Adobe SWF and Flash and Flex

Flash Player is the browser pluginSWF is file format ("swif")Authoring tools: Flash and FlexActionScript is the proprietary scripting language (based on JavaScript)

Video

Charts

Webkinz

MLB Gameday

ESPN Gamecast

Flash Earth


3 of 87 4/23/2008 10:53 AM


4 of 87 4/23/2008 10:53 AM

Example Flash with XML/SWF Charts

XML/SWF ChartsCharts SWF is providedYou provide the data in an XML format

Files involved:

chart-object.htmlcharts.swfdata.xml

A closer look

Markup (note use of "object" with nested "embed"):

Screenshot of the chart itself:

XML Data (data.xml):

view plain print ?

<body> 1. <p>A Chart</p> 2. <object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" 3. codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,4. width="400" height="250" id="charts" align=""> 5. <param name="movie" value= 6. "charts.swf?library_path=charts_library&xml_source=data.xml" /> 7. 8. <param name="quality" value="high" /> 9. <param name="bgcolor" value="#666666" /> 10. <embed src= 11. "charts.swf?library_path=charts_library&xml_source=data.xml" 12. quality="high" bgcolor="#666666" width="400" height="250" name= 13. "charts" align="" swliveconnect="true" 14. type="application/x-shockwave-flash" pluginspage= 15. "http://www.macromedia.com/go/getflashplayer" /> 16. </object> 17.</body> 18.


5 of 87 4/23/2008 10:53 AM

view plain print ?

<chart> 1. <chart_data> 2. <row> 3. <null/> 4. <string>2004</string> 5. <string>2005</string> 6. <string>2006</string> 7. <string>2007</string> 8. </row> 9. <row> 10. <string>Region A</string> 11. <number>5</number> 12. <number>10</number> 13. <number>30</number> 14. <number>63</number> 15. </row> 16. <row> 17. <string>Region B</string> 18. <number>100</number> 19. <number>20</number> 20. <number>65</number> 21. <number>55</number> 22. </row> 23. <row> 24. <string>Region C</string> 25. <number>56</number> 26. <number>21</number> 27. <number>5</number> 28. <number>90</number> 29. </row> 30. </chart_data> 31.</chart> 32.


6 of 87 4/23/2008 10:53 AM

Use "swfobject.js" to Embed SWF

swfobjectSWFObject is an easy-to-use and standards-friendly method to embed Flash content, which utilizes one small JavaScript fileExample of Chart using SWFObject

view plain print ?

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xh1.<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> 2. <head> 3. <title>SWFObject v2.0</title> 4. <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> 5. <script type="text/javascript" src="swfobject.js"></script> 6. </head> 7. <body> 8. <script type="text/javascript"> 9. flashvars = { library_path: 'charts_library', xml_source: 'data.xml' } 10. swfobject.embedSWF("charts.swf", "myContent", "600", "400", "9.0.0",'expressInstall.sw11. </script> 12. 13. <div id="myContent"> 14. <p>Alternative content</p> 15. </div> 16. </body> 17.</html> 18.


7 of 87 4/23/2008 10:53 AM

Java Applets

UC Irvine Chemistry Educational Applets Chemical Kinetics Simulation

Math, Physics, and Engineering Applets Ripple Tank


8 of 87 4/23/2008 10:53 AM

My own applet made with Scratch

Learn more about this project


9 of 87 4/23/2008 10:53 AM

HyperText Transfer Protocol

GET /

view plain print ?

minerva% telnet www.npr.org 80 1.Trying 216.35.221.77... 2.Connected to www.npr.org. 3.Escape character is '^]'. 4.GET / HTTP/1.1 5.Host: www.npr.org 6. 7.HTTP/1.1 200 OK 8.Date: Tue, 10 Apr 2008 20:07:33 GMT 9.Server: Apache 10.Set-Cookie: Apache=140.247.197.240.289451144786054516; path=/ 11.Cache-Control: max-age=0 12.Expires: Tue, 10 Apr 2008 20:07:33 GMT 13.Transfer-Encoding: chunked 14.Content-Type: text/html 15. 16.76c 17.<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 18. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 19.<html xmlns="http://www.w3.org/1999/xhtml"> 20.<head> 21.<title>NPR - National Public Radio - News, Arts, World, US.</title> 22. 23.</html> 24. 25. 26.


10 of 87 4/23/2008 10:53 AM

The Internet

(adapted from The Internet Book, 2nd edition by Douglas E. Comer)


11 of 87 4/23/2008 10:53 AM

TCP/IP: Transmission Control Protocol/Internet Protocol

The TCP/IP Internet Protocol Suite

IP (Internet Protocol): provides basic communication rules

TCP (Transmission Control Protocol): provides additional facilities

IP Addresses

The name www.fas.harvard.edu resolves to the IP address of 140.247.34.66. You can use the tool host or dig to lookup the IP to name or name to IP number.

view plain print ?

[dheitmey@minerva dheitmey]$ host www.fas.harvard.edu 1.www.fas.harvard.edu has address 140.247.34.66 2.[dheitmey@minerva dheitmey]$ host minerva.dce.harvard.edu 3.minerva.dce.harvard.edu has address 140.247.197.240 4.[dheitmey@minerva dheitmey]$ host www.npr.org 5.www.npr.org has address 216.35.221.77 6. 7.


12 of 87 4/23/2008 10:53 AM

Hostnames

cscie12.dce.harvard.edu

.eduharvard.edudce.harvard.educscie12.dce.harvard.edu

Numbers

As of January 2008, Internet Domain Survey reports 541,677,360 hosts in the Domain Name Service(Source: Internet Software Consortium ( http://www.isc.org/).


13 of 87 4/23/2008 10:53 AM

Clients and Servers

client-server computing The interaction between two programs when they communicate across a network. A program at onesite sends a request to a program at another site and awaits a response. The requesting program iscalled a client; the program satisfying the request is called the server. (definition from The Internet Book,2nd edition by Douglas E. Comer)

Application Layer of Network

HTTP (default port 80)FTP (port 21)SMTP (port 25)telnet (port 23)ssh (port 22)


14 of 87 4/23/2008 10:53 AM


Specifies the grammar of a conversation between an HTTP-client (Web Browser) and an HTTP-server(Web Server) is to take place.

GET /

view plain print ?

minerva% telnet www.npr.org 80 1.Trying 216.35.221.77... 2.Connected to www.npr.org. 3.Escape character is '^]'. 4.GET / HTTP/1.1 5.Host: www.npr.org 6. 7.HTTP/1.1 200 OK 8.Date: Tue, 10 Apr 2008 20:07:33 GMT 9.Server: Apache 10.Set-Cookie: Apache=140.247.197.240.289451144786054516; path=/ 11.Cache-Control: max-age=0 12.Expires: Tue, 10 Apr 2008 20:07:33 GMT 13.Transfer-Encoding: chunked 14.Content-Type: text/html 15. 16.76c 17.<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 18. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 19.<html xmlns="http://www.w3.org/1999/xhtml"> 20.<head> 21.<title>NPR - National Public Radio - News, Arts, World, US.</title> 22. 23.</html> 24.


15 of 87 4/23/2008 10:53 AM

HTTP

Example Request and Loading of a Web Page

As an example, an XHTML page with 7 images and an external CSS file and an external Javascriptfile, the client would make 10 separate requests (1 request for the XHTML resource, 1 request foreach of the seven images, 1 request for the CSS, and 1 request for the JS).

Client Server

HTTP Request for "example.html"

HTTP Response with "example.html" content

Parses through XHTML and determines what other requests it needs to make.

HTTP Request for CSS document

HTTP Response for CSS document

Parses through CSS and determines what other requests it needs to make.

HTTP Request for Javascript document

HTTP Response for Javascript document

HTTP Request for image 1

HTTP Response for image 1













Render page


16 of 87 4/23/2008 10:53 AM

HTTP

HTTP is Stateless

Each requested resource is a separate, independent, request to the server -- it is a stateless protocol.

HTTP Versions

W3C and Internet Engineering Task Force (IETF) oversees the Hypertext Transfer Protocol.

HTTP 1.0 (1996)HTTP 1.1 (1999)Extensions to HTTP

WebDAV

An HTTP Conversation

Client RequestMETHOD Resource HTTP VersionClient Generated HeadersRequest Body

Server ResponseStatus LineServer Generated HeadersData

HTTP 1.1 Methods

GET

POST

HEADPUTDELETETRACEOPTIONS


17 of 87 4/23/2008 10:53 AM

HTTP Response Codes

Code Range Meaning

100's Informational

200's Success

300's Redirected

400's Request Incomplete

500's Server Error

HTTP 1.1 status codes

Common ones:

200 OK301 Moved permanently302 Moved temporarily304 Not modified403 Forbidden404 Not found500 Internal server error

The complete list:

100 Continue101 Switching protocols200 OK201 Created202 Accepted203 Non-authoritative information204 No content205 Reset content206 Partial content300 Multiple choices301 Moved permanently302 Moved temporarily303 See other304 Not modified305 Use proxy400 Bad request401 Unauthorized402 Payment required403 Forbidden404 Not found405 Method not allowed406 Not acceptable407 Proxy authentication required408 Request timeout409 Conflict410 Gone411 Length required412 Precondition failed413 Request entity too large414 Request-URI too long415 Unsupported media type500 Internal server error501 Not implemented502 Bad gateway503 Service unavailable504 Gateway timeout505 HTTP version not supported


18 of 87 4/23/2008 10:53 AM

Common Request Headers

AcceptAccept-LanguageAuthorizationCookieHostIf-Modified-SinceRefererUser-Agent


19 of 87 4/23/2008 10:53 AM

Sample Request Header Values

View some of the Headers your browser is sending to the server

Mozilla Firefox

HTTP_ACCEPT text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=

HTTP_ACCEPT_CHARSET ISO-8859-1,utf-8;q=0.7,*;q=0.7

HTTP_ACCEPT_ENCODING gzip,deflate

HTTP_ACCEPT_LANGUAGE en-us,en;q=0.5

HTTP_CONNECTION keep-alive

HTTP_COOKIE __utma=76898816.1019426635.1173814303.1173814303.1173814303.1__utmz=76898816.1173814303.1.1.utmccn=(referral)|utmcsr=localhost:8nde-textsize=16px

HTTP_HOST cscie12.dce.harvard.edu

HTTP_KEEP_ALIVE 300

HTTP_REFERER http://localhost:8080/cocoon/projects/cscie12/slides/20070410/slide16.htm

HTTP_USER_AGENT Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/2007

Opera 9

HTTP_ACCEPT text/html, application/xml;q=0.9, application/xhtml+xml, image/png,image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1

HTTP_ACCEPT_CHARSET iso-8859-1, utf-8, utf-16, *;q=0.1

HTTP_ACCEPT_ENCODING deflate, gzip, x-gzip, identity, *;q=0

HTTP_ACCEPT_LANGUAGE en-US,en;q=0.9

HTTP_CONNECTION Keep-Alive


HTTP_USER_AGENT Opera/9.01 (Windows NT 5.1; U; en)

Internet Explorer 7

HTTP_ACCEPT image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,application/x-shockwave-flash, application/vnd.ms-excel,application/vnd.ms-powerpoint, application/msword, application/xaml+xml, application/vnd.ms-xpsdocument,application/x-ms-xbap, application/x-ms-application, */*

HTTP_ACCEPT_ENCODING gzip, deflate

HTTP_ACCEPT_LANGUAGE en-us



HTTP_UA_CPU x86

HTTP_USER_AGENT Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; InfoPath.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)

Amaya


20 of 87 4/23/2008 10:53 AM

HTTP_ACCEPT */*;q=0.1,image/svg+xml,application/mathml+xml,application/xhtml+xml

HTTP_ACCEPT_ENCODING *,gzip

HTTP_CONNECTION TE,Keep-Alive


HTTP_TE trailers,deflate

HTTP_USER_AGENT amaya/9.51 libwww/5.4.0

Lynx

HTTP_ACCEPT text/html, text/plain, audio/mod, image/*, application/msword,application/pdf, application/postscript, application/x-java-jnlp-file,text/sgml, video/mpeg, */*;q=0.01

HTTP_ACCEPT_LANGUAGE en


HTTP_USER_AGENT Lynx/2.8.5dev.16 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.7a

wget

HTTP_ACCEPT */*



HTTP_USER_AGENT Wget/1.9+cvs-stable (Red Hat modified)


21 of 87 4/23/2008 10:53 AM

Experimenting with HTTP

Viewing HTTP Request and Response Headers

Firefox Extension - Live HTTP Headers

Command Line

telnet

lwp-request Documentation:

minerva% man lwp-request

minerva% lwp-request -h

An example:

minerva% lwp-request -USed http://www.harvard.edu/


22 of 87 4/23/2008 10:53 AM

HTTP Method: HEAD

http://cscie12.dce.harvard.edu/http/raspberry.gifcscie12.dce.harvard.edu is 140.247.197.240we must explicitly use port 80Note the two "returns" after we are done with the HTTP request

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.HEAD /http/raspberry.gif HTTP/1.1 5.Host: cscie12.dce.harvard.edu <return> 6.<return> 7.HTTP/1.1 200 OK 8.Date: Tue, 10 Apr 2008 20:23:14 GMT 9.Server: Apache/2.2 (Fedora) 10.Last-Modified: Wed, 06 Apr 2005 19:30:42 GMT 11.ETag: "461fb8-348c-a0f67c80" 12.Accept-Ranges: bytes 13.Content-Length: 13452 14.Connection: close 15.Content-Type: image/gif 16. 17.Connection closed by foreign host. 18. 19.


23 of 87 4/23/2008 10:53 AM

Response 404

Sometimes a file is not there...

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.HEAD /http/blueberry.gif HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6. 7.HTTP/1.1 404 Not Found 8.Date: Tue, 10 Apr 2008 20:24:27 GMT 9.Server: Apache/2.2 (Fedora) 10.Connection: close 11.Content-Type: text/html; charset=iso-8859-1 12. 13.Connection closed by foreign host. 14. 15.


24 of 87 4/23/2008 10:53 AM

HTTP Header: Host

Problem: "Infinite" domain names; finite IP addresses.

Solution: "Virtual Hosts"

Example: cscie12.dce.harvard.edu, cscie153.dce.harvard.edu, cscisl.dce.harvard.edu andminerva.dce.harvard.edu resolve to the same IP address (140.247.197.240).

Host Header

This is required for HTTP 1.1 requests.


25 of 87 4/23/2008 10:53 AM

HTTP Redirect

http://www.fas.harvard.edu/http://www.fas.harvard.edu/home/

view plain print ?

minerva% telnet www.fas.harvard.edu 80 1.Trying 140.247.34.66... 2.Connected to www.fas.harvard.edu. 3.Escape character is '^]'. 4.HEAD / HTTP/1.1 5.Host: www.fas.harvard.edu 6. 7.HTTP/1.1 301 Moved Permanently 8.Date: Wed, 06 Apr 2008 20:11:43 GMT 9.Server: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6g mod_perl/1.24 10.Location: http://www.fas.harvard.edu/home/ 11.Content-Type: text/html; charset=iso-8859-1 12. 13. 14.


26 of 87 4/23/2008 10:53 AM

Media Types (MIME Types)

How a Browser Knows What Kind of File it is Getting

Multipurpose Internet Mail Extensions (media types). Server will return a media type to client.Client will handle the media appropriately. Some common media types are:

text/htmltext/cssimage/jpegimage/pngimage/gifapplication/pdfapplication/mswordapplication/vnd.ms-excelAll media types listed in /etc/mime.types on minerva

More information about MIME Types is available.

Questions:

How does the server know the media type?How does the client know the media type?How does the client know "what to do with" the file?


27 of 87 4/23/2008 10:53 AM

Content Negotiation

Resource can be in multiple formats and languages. Client preferences can determine which resourceis returned.

Content Negiation Resources

ApacheWeek: Content Negotiation Explained http://www.apacheweek.com/features/negotiationApache: Content Negotiation http://httpd.apache.org/docs/content-negotiation.html


28 of 87 4/23/2008 10:53 AM

Content Negotiation: MIME Types

A file listing :

raspberry.gifraspberry.jpgraspberry.pngraspberry

The HTTP Transaction:

view plain print ?

minerva% ls -l raspberry* 1.16 -rw-r--r-- 1 e12 e12 13452 Apr 6 15:30 raspberry.gif 2.16 -rw-r--r-- 1 e12 e12 16255 Apr 6 15:30 raspberry.jpg 3.12 -rw-r--r-- 1 e12 e12 8899 Apr 6 15:30 raspberry.png 4.

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.HEAD /http/raspberry HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6. 7.HTTP/1.1 200 OK 8.Date: Tue, 10 Apr 2008 20:39:20 GMT 9.Server: Apache/2.2 (Fedora) 10.Content-Location: raspberry.png 11.Vary: negotiate,accept 12.TCN: choice 13.Last-Modified: Wed, 06 Apr 2005 19:30:42 GMT 14.ETag: "461fba-22c3-a0f67c80;4edcb400" 15.Accept-Ranges: bytes 16.Content-Length: 8899 17.Connection: close 18.Content-Type: image/png 19. 20.Connection closed by foreign host. 21.


29 of 87 4/23/2008 10:53 AM

Content Negotiation: MIME Types

Client specifies MIME Types it accepts through HTTP "Accept" header.

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.HEAD /http/raspberry HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6.Connection: close 7.Accept: image/jpeg 8. 9.HTTP/1.1 200 OK 10.Date: Tue, 10 Apr 2008 20:40:17 GMT 11.Server: Apache/2.2 (Fedora) 12.Content-Location: raspberry.jpg 13.Vary: negotiate,accept 14.TCN: choice 15.Last-Modified: Wed, 06 Apr 2005 19:30:42 GMT 16.ETag: "461fb9-3f7f-a0f67c80;4edcb400" 17.Accept-Ranges: bytes 18.Content-Length: 16255 19.Connection: close 20.Content-Type: image/jpeg 21.


30 of 87 4/23/2008 10:53 AM

Content Negotiation: Language

lang.htmlEnglish VersionFrench VersionGerman Version

A file listing:

HTTP Transaction:

view plain print ?

minerva% ls -l lang* 1.-rw-r--r-- 1 e12 e12 191 Apr 6 15:30 lang.html.de 2.-rw-r--r-- 1 e12 e12 193 Apr 6 15:30 lang.html.en 3.-rw-r--r-- 1 e12 e12 191 Apr 6 15:30 lang.html.fr 4.

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.HEAD /http/lang HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6.Connection: close 7. 8.HTTP/1.1 200 OK 9.Date: Tue, 10 Apr 2008 20:41:02 GMT 10.Server: Apache/2.2 (Fedora) 11.Content-Location: lang.html.en 12.Vary: negotiate,accept-language 13.TCN: choice 14.Accept-Ranges: bytes 15.Content-Length: 193 16.Connection: close 17.Content-Type: text/html; charset=UTF-8 18.Content-Language: en 19. 20.Connection closed by foreign host. 21.


31 of 87 4/23/2008 10:53 AM

Content Negotiation: Language

Content Negotiation: Gotcha

Permissions must be set to rwxr-xr-x on directories.

Why?

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.HEAD /http/lang HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6.Connection: close 7.Accept-Language: de 8. 9.HTTP/1.1 200 OK 10.Date: Tue, 10 Apr 2008 20:44:16 GMT 11.Server: Apache/2.2 (Fedora) 12.Content-Location: lang.html.de 13.Vary: negotiate,accept-language 14.TCN: choice 15.Accept-Ranges: bytes 16.Content-Length: 191 17.Connection: close 18.Content-Type: text/html; charset=UTF-8 19.Content-Language: de 20. 21.Connection closed by foreign 22.


32 of 87 4/23/2008 10:53 AM

HTTP: Moved

Attempt to access: http://www.fas.harvard.edu/~cscie12/

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.>HEAD /~cscie12/ HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6. 7.HTTP/1.1 301 Moved Permanently 8.Date: Tue, 10 Apr 2008 20:47:19 GMT 9.Server: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6g mod_perl/1.24 10.Location: http://www.courses.fas.harvard.edu/~cscie12/ 11.Content-Type: text/html; charset=iso-8859-1 12.


33 of 87 4/23/2008 10:53 AM

HTTP: trailing 'slash' for directories

view plain print ?

minerva% telnet 140.247.197.240 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.HEAD /images HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6. 7.HTTP/1.1 301 Moved Permanently 8.Date: Tue, 10 Apr 2008 20:48:42 GMT 9.Server: Apache/2.2 (Fedora) 10.Location: http://cscie12.dce.harvard.edu/images/ 11.Content-Type: text/html 12. 13.HEAD /images/ HTTP/1.1 14.Host: cscie12.dce.harvard.edu 15. 16.HTTP/1.1 200 OK 17.Date: Tue, 10 Apr 2008 20:48:42 GMT 18.Server: Apache/2.2 (Fedora) 19.Connection: close 20.Content-Type: text/html; charset=UTF-8 21.


34 of 87 4/23/2008 10:53 AM

Connection: keep-alive

Allows multiple HTTP requests to be made over the same TCP/IP connection.

view plain print ?

[dheitmey@minerva dheitmey]$ telnet www.fas.harvard.edu 80 1.Trying 140.247.197.240... 2.Connected to www.fas.harvard.edu. 3.Escape character is '^]'. 4.HEAD /home/ HTTP/1.1 5.Host: www.fas.harvard.edu 6.Connection: keep-alive 7. 8.HTTP/1.1 200 OK 9.Date: Tue, 13 Apr 2008 19:24:16 GMT 10.Server: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6g mod_perl/1.24 11.Keep-Alive: timeout=15, max=100 12.Connection: Keep-Alive 13.Content-Type: text/html 14. 15.HEAD /home/images/ HTTP/1.1 16.Host: www.fas.harvard.edu 17.Connection: close 18. 19.HTTP/1.1 200 OK 20.Date: Tue, 13 Apr 2008 19:24:34 GMT 21.Server: Apache 22.Connection: close 23.Content-Type: text/html 24. 25.Connection closed by foreign host. 26.


35 of 87 4/23/2008 10:53 AM

Caching Related Headers

Local cache and Proxy-server cache

If-Modified-SinceAgeExpiresLast-ModifiedCache-ControlETag


36 of 87 4/23/2008 10:53 AM

HTTP Example

view plain print ?

minerva% telnet www.apache.org 80 1.Trying 192.87.106.226... 2.Connected to www.APACHE.org. 3.Escape character is '^]'. 4.HEAD /images/asf_logo.gif HTTP/1.1 5.Host: www.apache.org 6.Connection: close 7. 8.HTTP/1.1 200 OK 9.Date: Tue, 10 Apr 2008 20:52:43 GMT 10.Server: Apache/2.2.0 (Unix) 11.Last-Modified: Sun, 20 May 2001 02:19:30 GMT 12.ETag: "c558a6-1c6f-bf85080" 13.Accept-Ranges: bytes 14.Content-Length: 7279 15.Cache-Control: max-age=86400 16.Expires: Wed, 12 Apr 2006 20:52:43 GMT 17.Connection: close 18.Content-Type: image/gif 19.


37 of 87 4/23/2008 10:53 AM

If-Modified Since

view plain print ?

minerva% telnet www.harvard.edu 80 1.Trying 128.103.60.55... 2.Connected to zooey.harvard.edu. 3.Escape character is '^]'. 4.GET /images/global/banner.gif HTTP/1.1 5.Host: www.harvard.edu 6.Keep-Alive: 300 7.Connection: keep-alive 8.If-Modified-Since: Thu, 12 Sep 2002 21:51:13 GMT 9. 10.HTTP/1.x 304 Not Modified 11.Date: Tue, 10 Apr 2008 20:57:00 GMT 12.Server: Apache/1.3.28 (Unix) PHP/4.3.2 mod_perl/1.28 13.Connection: Keep-Alive 14.Keep-Alive: timeout=15, max=100 15.Etag: "ff-2f98-3d810c51" 16.


38 of 87 4/23/2008 10:53 AM

Proxy Servers


39 of 87 4/23/2008 10:53 AM

HTTP Cookies

HTTP is a stateless protocol. Cookies provide a mechanism to "maintain state".

Cookie Central: The Unofficial Cookie FAQ http://www.cookiecentral.com/faq/ http://www.cookiecentral.com/

Maintaining State with Cookies

HTTP State Management Mechanism http://www.ics.uci.edu/pub/ietf/http/rfc2109.txt


Persistent Client State HTTP Cookies http://www.netscape.com/newsref/std/cookie_spec.html


40 of 87 4/23/2008 10:53 AM

Cookie Example

Server returns cookie to HTTP client ("Set-Cookie" response header)HTTP client returns cookie to server ("Cookie" request header)

ESPN Cookies

Set-Cookie:SWID=C8F9AF31-F170-42BF-9471-50A95DA24C17;path=/;expires=Tue, 10-Apr-2027 03:20:59 GMT;domain=.go.com;

Set-Cookie:DE2=dXNhO21hO2NhbWJyaWRnZTt0MTs1OzQ7NDs1MDY7MDQyLjM4MDstMDcxLjEzNTspath=/; expires=Tue, 17 Apr 2008 03:00:00 GMT; domain=.go.com

view plain print ?

minerva% lwp-request -USed http://www.espn.com/ 1.GET http://espn.go.com/ 2.User-Agent: lwp-request/2.07 3. 4.GET http://www.espn.com/ --> 301 Moved Permanently 5.GET http://espn.go.com/ --> 200 OK 6.Cache-Control: no-cache 7.Date: Tue, 10 Apr 2008 03:20:58 GMT 8.Pragma: no-cache 9.From: SPORTBARWEB08 10.Accept-Ranges: bytes 11.ETag: "802e571f7bc71:1762" 12.Server: Microsoft-IIS/5.0 13.Vary: Accept-Encoding 14.Content-Length: 122217 15.Content-Type: text/html; charset=iso-8859-1 16.Content-Type: text/html; charset=windows-1252 17.Last-Modified: Tue, 10 Apr 2008 03:19:21 GMT 18.Cache-Expires: Tue, 10 Apr 2008 03:24:22 GMT 19.Client-Date: Tue, 10 Apr 2008 03:21:02 GMT 20.Client-Peer: 198.105.193.43:80 21.Client-Response-Num : 1 22.P3P: CP="CAO DSP COR CURa ADMa DEVa TAIa PSAa PSDa IVAi IVDi CONi OUR SAMo OTRo BUS PHY ONL 23.Refresh: 3600 24.Set-Cookie: SWID=C8F9AF31-F170-42BF-9471-50A95DA24C17; path=/; expires=Tue, 10-Apr-2027 03:225.Set-Cookie: DE2=dXNhO21hO2NhbWJyaWRnZTt0MTs1OzQ7NDs1MDY7MDQyLjM4MDstMDcxLjEzNTs4NDA7MjI7ODg526. 27.


41 of 87 4/23/2008 10:53 AM

Cookie Properties/Attributes

nameexpiresdomainpathsecure

HTTP State Management Mechanism, RFC 2965

RFC 2109, February 1997RFC 2965, October 2000

namecommentcomment URLdiscarddomainmax-agepathportsecureversion

Additional Cookie Notes

Client: 300 total cookies4 kb per cookie20 cookies per server or domain


42 of 87 4/23/2008 10:53 AM

Cookie Example: Server Sets a Cookie

Form that will set a Cookie: http://cscie12.dce.harvard.edu/http/cookie.cgi

Set-Cookie HTTP Response Header:

Set-Cookie: YourName=David%20P.%20Heitmeyer; domain=cscie12.dce.harvard.edu; path=/http/;expires=Fri, 13-May-2008 18:05:04 GMT

view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.GET /http/cookie.cgi?name=David%20P.%20Heitmeyer HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6.Connection: close 7. 8.HTTP/1.1 200 OK 9.Connection: close 10.Date: Wed, 13 Apr 2008 18:05:04 GMT 11.Server: Apache/2.0.49 (Fedora) 12.Content-Type: text/html; charset=ISO-8859-1 13.Client-Date: Wed, 13 Apr 2008 18:05:04 GMT 14.Client-Peer: 140.247.197.240:80 15.Client-Response-Num : 1 16.Client-Transfer-Encoding : chunked 17.Set-Cookie: YourName=David%20P.%20Heitmeyer; \ 18. domain=cscie12.dce.harvard.edu; \ 19. path=/http/; \ 20. expires=Fri, 13-May-2008 18:05:04 GMT 21. 22.<?xml version="1.0" encoding="iso-8859-1"?> 23.<!DOCTYPE html 24. PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 25. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 26.<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US"><head><title>Form</27.</head><body> 28.<h1>Hello, David P. Heitmeyer</h1> 29.</body></html> 30.Connection closed by foreign host. 31. 32.


43 of 87 4/23/2008 10:53 AM

Cookie Example: Returning a Cookie


view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.GET /http/cookie.cgi HTTP/1.1 5.Cookie: YourName=David%20P.%20Heitmeyer 6.Host: cscie12.dce.harvard.edu 7.Connection: close 8. 9.HTTP/1.1 200 OK 10.Connection: close 11.Date: Wed, 13 Apr 2008 18:11:40 GMT 12.Server: Apache/2.0.49 (Fedora) 13.Content-Type: text/html; charset=ISO-8859-1 14.Client-Date: Wed, 13 Apr 2008 18:11:40 GMT 15.Client-Peer: 140.247.197.240:80 16.Client-Response-Num : 1 17.Client-Transfer-Encoding : chunked 18. 19.<?xml version="1.0" encoding="iso-8859-1"?> 20.<!DOCTYPE html 21. PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 22. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 23.<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US"> 24.<head><title>Form</title></head><body> 25.<h1>Hello, David P. Heitmeyer</h1> 26. 27.


44 of 87 4/23/2008 10:53 AM

Your Cookies

Firefox Webdeveloper Toolbar has a "Cookies" section.

Mozilla Cookie Manager


45 of 87 4/23/2008 10:53 AM

Cookies and Session IDs

A UserID or SessionID (a long character/number string that is uniquely assigned) is often stored incookie. The SessionID is used as the key or identifier when storing information about the user orsession.

For example, a user logs in to a site. If the username and password match, the server sets a cookie("Set-Cookie") in the browser that contains a session id; the server also makes an entry in websitedatabase that maps the session id to the username. When the cookie is returned, the session id isread and the username is looked up in the database.


46 of 87 4/23/2008 10:53 AM

Google Cookie Example

Using Google's "Preference" page and setting:

Search Language preference to: English, French, GermanSafeSearch Filtering: Strict FilteringNumber of Results: 50

The Cookie name is: PREF The Value is:ID=bb504f37cd318aa9:FF=1:LR=lang_en|lang_fr|lang_de:LD=en:NR=50:TM=1113416195:LM=11134

This cookie contains a session id as well as the values of certain preferences in a colon-separateddata structure.


47 of 87 4/23/2008 10:53 AM

Cookies and Ad Tracking


48 of 87 4/23/2008 10:53 AM


GET /

HTTP is a stateless protocol. Cookies provide a mechanism to "maintain state".


Maintaining State with Cookies

HTTP State Management Mechanism http://www.ics.uci.edu/pub/ietf/http/rfc2109.txt


Persistent Client State HTTP Cookies http://www.netscape.com/newsref/std/cookie_spec.html

view plain print ?

minerva% telnet www.npr.org 80 1.Trying 216.35.221.77... 2.Connected to www.npr.org. 3.Escape character is '^]'. 4.GET / HTTP/1.1 5.Host: www.npr.org 6. 7.HTTP/1.1 200 OK 8.Date: Tue, 10 Apr 2008 20:07:33 GMT 9.Server: Apache 10.Set-Cookie: Apache=140.247.197.240.289451144786054516; path=/ 11.Cache-Control: max-age=0 12.Expires: Tue, 10 Apr 2008 20:07:33 GMT 13.Transfer-Encoding: chunked 14.Content-Type: text/html 15. 16.76c 17.<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 18. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 19.<html xmlns="http://www.w3.org/1999/xhtml"> 20.<head> 21.<title>NPR - National Public Radio - News, Arts, World, US.</title> 22. 23.</html> 24.


49 of 87 4/23/2008 10:53 AM

Cookie Example

Server returns cookie to HTTP client ("Set-Cookie" response header)HTTP client returns cookie to server ("Cookie" request header)

ESPN Cookies

Set-Cookie:SWID=C8F9AF31-F170-42BF-9471-50A95DA24C17;path=/;expires=Tue, 10-Apr-2027 03:20:59 GMT;domain=.go.com;

Set-Cookie:DE2=dXNhO21hO2NhbWJyaWRnZTt0MTs1OzQ7NDs1MDY7MDQyLjM4MDstMDcxLjEzNTspath=/; expires=Tue, 17 Apr 2008 03:00:00 GMT; domain=.go.com

view plain print ?

minerva% lwp-request -USed http://www.espn.com/ 1.GET http://espn.go.com/ 2.User-Agent: lwp-request/2.07 3. 4.GET http://www.espn.com/ --> 301 Moved Permanently 5.GET http://espn.go.com/ --> 200 OK 6.Cache-Control: no-cache 7.Date: Tue, 10 Apr 2008 03:20:58 GMT 8.Pragma: no-cache 9.From: SPORTBARWEB08 10.Accept-Ranges: bytes 11.ETag: "802e571f7bc71:1762" 12.Server: Microsoft-IIS/5.0 13.Vary: Accept-Encoding 14.Content-Length: 122217 15.Content-Type: text/html; charset=iso-8859-1 16.Content-Type: text/html; charset=windows-1252 17.Last-Modified: Tue, 10 Apr 2008 03:19:21 GMT 18.Cache-Expires: Tue, 10 Apr 2008 03:24:22 GMT 19.Client-Date: Tue, 10 Apr 2008 03:21:02 GMT 20.Client-Peer: 198.105.193.43:80 21.Client-Response-Num : 1 22.P3P: CP="CAO DSP COR CURa ADMa DEVa TAIa PSAa PSDa IVAi IVDi CONi OUR SAMo OTRo BUS PHY ONL 23.Refresh: 3600 24.Set-Cookie: SWID=C8F9AF31-F170-42BF-9471-50A95DA24C17; path=/; expires=Tue, 10-Apr-2027 03:225.Set-Cookie: DE2=dXNhO21hO2NhbWJyaWRnZTt0MTs1OzQ7NDs1MDY7MDQyLjM4MDstMDcxLjEzNTs4NDA7MjI7ODg526.


50 of 87 4/23/2008 10:53 AM

Cookie Properties/Attributes

nameexpiresdomainpathsecure

HTTP State Management Mechanism, RFC 2965

RFC 2109, February 1997RFC 2965, October 2000

namecommentcomment URLdiscarddomainmax-agepathportsecureversion

Additional Cookie Notes

Client: 300 total cookies4 kb per cookie20 cookies per server or domain


51 of 87 4/23/2008 10:53 AM

Cookie Example: Server Sets a Cookie


Set-Cookie HTTP Response Header:

Set-Cookie: YourName=David%20P.%20Heitmeyer; domain=cscie12.dce.harvard.edu; path=/http/;expires=Fri, 13-May-2008 18:05:04 GMT

view plain print ?

minerva% telnet 140.247.197.240 80 1. Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.GET /http/cookie.cgi?name=David%20P.%20Heitmeyer HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6.Connection: close 7. 8.HTTP/1.1 200 OK 9.Connection: close 10.Date: Wed, 13 Apr 2008 18:05:04 GMT 11.Server: Apache/2.0.49 (Fedora) 12.Content-Type: text/html; charset=ISO-8859-1 13.Client-Date: Wed, 13 Apr 2008 18:05:04 GMT 14.Client-Peer: 140.247.197.240:80 15.Client-Response-Num : 1 16.Client-Transfer-Encoding : chunked 17.Set-Cookie: YourName=David%20P.%20Heitmeyer; \ 18. domain=cscie12.dce.harvard.edu; \ 19. path=/http/; \ 20. expires=Fri, 13-May-2008 18:05:04 GMT 21. 22.<?xml version="1.0" encoding="iso-8859-1"?> 23.<!DOCTYPE html 24. PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 25. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 26.<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US"><head><title>Form</27.</head><body> 28.<h1>Hello, David P. Heitmeyer</h1> 29.</body></html> 30.Connection closed by foreign host. 31.


52 of 87 4/23/2008 10:53 AM

Cookie Example: Returning a Cookie


view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.GET /http/cookie.cgi HTTP/1.1 5.Cookie: YourName=David%20P.%20Heitmeyer 6.Host: cscie12.dce.harvard.edu 7.Connection: close 8. 9.HTTP/1.1 200 OK 10.Connection: close 11.Date: Wed, 13 Apr 2008 18:11:40 GMT 12.Server: Apache/2.0.49 (Fedora) 13.Content-Type: text/html; charset=ISO-8859-1 14.Client-Date: Wed, 13 Apr 2008 18:11:40 GMT 15.Client-Peer: 140.247.197.240:80 16.Client-Response-Num : 1 17.Client-Transfer-Encoding : chunked 18. 19.<?xml version="1.0" encoding="iso-8859-1"?> 20.<!DOCTYPE html 21. PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 22. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 23.<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US"> 24.<head><title>Form</title></head><body> 25.<h1>Hello, David P. Heitmeyer</h1> 26.


53 of 87 4/23/2008 10:53 AM

Your Cookies

Firefox Webdeveloper Toolbar has a "Cookies" section.

Mozilla Cookie Manager


54 of 87 4/23/2008 10:53 AM

Cookies and Session IDs

A UserID or SessionID (a long character/number string that is uniquely assigned) is often stored incookie. The SessionID is used as the key or identifier when storing information about the user orsession.

For example, a user logs in to a site. If the username and password match, the server sets a cookie("Set-Cookie") in the browser that contains a session id; the server also makes an entry in websitedatabase that maps the session id to the username. When the cookie is returned, the session id isread and the username is looked up in the database.


55 of 87 4/23/2008 10:53 AM

Google Cookie Example

Using Google's "Preference" page and setting:

Search Language preference to: English, French, GermanSafeSearch Filtering: Strict FilteringNumber of Results: 50

The Cookie name is: PREF The Value is:ID=bb504f37cd318aa9:FF=1:LR=lang_en|lang_fr|lang_de:LD=en:NR=50:TM=1113416195:LM=11134

This cookie contains a session id as well as the values of certain preferences in a colon-separateddata structure.


56 of 87 4/23/2008 10:53 AM

Cookies and Ad Tracking


57 of 87 4/23/2008 10:53 AM

Method: POST


view plain print ?

minerva% telnet 140.247.197.240 80 1.Trying 140.247.197.240... 2.Connected to 140.247.197.240. 3.Escape character is '^]'. 4.POST /http/cookie.cgi HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6.Content-Length: 10 7.Content-Type: application/x-www-form-urlencoded 8. 9.name=David 10.HTTP/1.1 200 OK 11.Date: Wed, 13 Apr 2008 19:31:11 GMT 12.Server: Apache/2.0.49 (Fedora) 13.Set-Cookie: YourName=David; domain=cscie12.dce.harvard.edu; path=/http/; expires=Fri, 13-May14.Content-Length: 319 15.Connection: close 16.Content-Type: text/html; charset=ISO-8859-1 17. 18.<?xml version="1.0" encoding="iso-8859-1"?> 19.<!DOCTYPE html 20. PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 21. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 22.<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US"> 23.<head><title>Form</title> 24.</head><body> 25.<h1>Hello, David</h1> 26.


58 of 87 4/23/2008 10:53 AM

WebDAV: an extension of HTTP

Web-based Distributed Authoring and Versioning

WebDAV Resources http://www.webdav.org/From the WebDAV Resources :

WebDAV stands for "Web-based Distributed Authoring and Versioning". It is a set ofextensions to the HTTP protocol which allows users to collaboratively edit andmanage files on remote web servers.


59 of 87 4/23/2008 10:53 AM

HTTP Resources

W3C HTTP http://www.w3.org/Protocols/HTTP Pocket Reference http://www.oreilly.com/catalog/httppr/ by Clinton Wong (O'Reilly).Illustrated Guide to HTTP http://www.manning.com/hethmon/ by Paul Hethmon (Manning Publications; ISBN 0138582262) see sample chapters and resources online.

Other Readings:

W3C Recommendations Reduce 'World Wide Wait' http://www.w3.org/Protocols/NL-PerfNote.htmlApache Week: HTTP version 1.1 http://www.apacheweek.com/features/http11WebTechniques: HTTP 1.1: What's in it for Me? http://www.webtechniques.com/archives/1997/08/webm/



60 of 87 4/23/2008 10:53 AM

Apache HTTP Server

Apache Software FoundationApache HTTP Server Project

Apache 1.3Apache 2.x

Apache ModulesPHPPerlPythonmany, many others

Netcraft Web Server Survey


61 of 87 4/23/2008 10:53 AM

In this Unit: Configuring Apache with .htaccess files

Custom Error DocumentsRedirectRewriteDirectory IndexSetting HTTP Headers

ExpiresHeaders

Requiring a Secure Connection (SSL)


62 of 87 4/23/2008 10:53 AM

Apache Configuration Overview

Server Configuration (httpd.conf) Unless you are the server administrator, you generally will not have access to this account. Onthe DCE systems, you do not have read or write access to this file. Server configuration is readat server start or restart.

Per Directory (.htaccess) Certain configuration directives for Apache can be placed within per-directory .htaccess files..htaccess file is read on a per request basis.


63 of 87 4/23/2008 10:53 AM

.htaccess File Example

document root: /home/e12/htdocs filename: .htaccess location: /home/e12/htdocs/apache/.htaccess contents:

filename: status404.html location: /home/e12/htdocs/apache/status404.html

http://cscie12.dce.harvard.edu/apache/ZZZ.html

Apache Default 404 document:

Custom 404 document:

ErrorDocument 404 status404.html 1.


64 of 87 4/23/2008 10:53 AM

Scope of .htaccess files

Directives within .htaccess files apply to the directory that contains the .htaccess file and all its descendants.

Directives within the file, /home/e12/htdocs/.htaccess would apply to all files within and "under" the public_html directory for the user cscie12.

Directives within the file, /home/e12/htdocs/assignments/.htaccess would apply to all files within and "under" the public_html/assignments directory for the user cscie12.


65 of 87 4/23/2008 10:53 AM

Problems You Will Have with .htaccess files

Internal Server ErrorCan't "see" the fileIncorrect Permissions


66 of 87 4/23/2008 10:53 AM

Problems You will encounter when using .htaccess files (Internal Server Error 500)

500 Internal Server Error If you see begin seeing 500 Internal Server Error responses from the server after you have createdor edited an .htaccessfile, the most likely cause of the problem is incorrect permissions and/or an error in the directivesyntax.

Permissions on the .htaccessfile are not set correctly. Just like HTML and image files, the server must be able to read the.htaccess file. The simplest way to allow that is to make your .htaccess file readable by "other".

Syntax Error. An error in the syntax of a directive the .htaccess file will result in a 500 Internal Server Error. In addition, correct usage of a directive that is not allowed in the .htaccess file will result in a 500status code. Whether or not a directive is allowed depends upon the server configuration file(httpd.conf; AllowOverride) and the directive itself.

view plain print ?

minerva% pwd 1./home/courses/j/h/jharvard/public_html 2.minerva% ls -l .htaccess 3.-rw------- 1 jharvard founder 349 Nov 27 00:03 .htaccess 4.minerva% chmod o+r .htaccess 5.minerva% ls -l ~/public_html/.htaccess 6.-rw----r-- 1 jharvard founder 349 Nov 27 00:03 .htaccess 7.


67 of 87 4/23/2008 10:53 AM

Problems You will encounter when using .htaccess files (Can't see the .htaccess file)

You can't "see" your .htaccess file.

HTTP The web server is typically configured to deny requests for .htaccess files. For example, the file corresponding to the URL, http://cscie12.dce.harvard.edu/.htaccess exists and is readable by the Web server, but if we try to follow the link, we get a 403 Forbidden response.

UNIX The ls command will not list files or directories that begin with a '.' (dot). In order to see the .htaccess file when you do a directory listing, use the -a (all) option:

SFTP Sometimes your SFTP program will hide the "dot" files unless explicitly told to show them.


68 of 87 4/23/2008 10:53 AM

Apache Configuration Sections

Configuration directives can be limited by using "sections", such as

DirectoryLocationFilesVirtualHostDirectoryMatchLocationMatchFilesMatch

Within .htaccess

Note that only Files and FilesMatch can be used within .htaccess files.

Examples:

Examples:

<Files .htaccess> 1. Order allow,deny 2. Deny from all 3.</Files> 4.

# deny access to any tilde backup files 1.<Files *~> 2. Order allow,deny 3. Deny from all 4.</Files> 5.


69 of 87 4/23/2008 10:53 AM

Custom Error Documents

.htaccess

ErrorDocument directive http://www.apache.org/docs/2.0/mod/core.html#errordocumentCustom Error Responses http://www.apache.org/docs/2.0/custom-error.html

ErrorDocument 401 /apache/status401.html 1.ErrorDocument 403 /apache/status403.html 2.ErrorDocument 404 /apache/status404.html 3.


70 of 87 4/23/2008 10:53 AM

HTTP Redirect

Fight Linkrot!

Do your part to Fight Linkrot! (Jakob Nielson's Alertbox, http://www.useit.com/alertbox/980614.html )

RedirectRewriteMeta http-equiv refresh

Redirecting Requests

HTTP Status Codes: 301 Moved permanently 302 Moved temporarily

Redirecting client requests can be very useful:

URL moves to a new location Do your part to Fight Linkrot! (Jakob Nielson's Alertbox,http://www.useit.com/alertbox/980614.html )

resource removedsite structure is reorganized

Provide "friendly" or additional URLs to access a resource


71 of 87 4/23/2008 10:53 AM

Redirect

Redirect directive http://www.apache.org/docs/2.0/mod/mod_alias.html#redirect

.htaccess

Try it:

http://cscie12.dce.harvard.edu/apache/dce.htmlhttp://cscie12.dce.harvard.edu/apache/church_st

Redirect 302 /apache/dce.html http://www.dce.harvard.edu/ 1.Redirect 301 /apache/church_st http://map.harvard.edu/level3.cfm?mapname=camb_allston&ti2.

view plain print ?

minerva% telnet cscie12.dce.harvard.edu 80 1.Trying 140.247.197.240... 2.Connected to cscie12.dce.harvard.edu. 3.Escape character is '^]'. 4.GET /apache/dce.html HTTP/1.1 5.Host: cscie12.dce.harvard.edu 6.Connection: close 7. 8.HTTP/1.1 302 Found 9.Date: Wed, 13 Apr 2008 20:03:10 GMT 10.Server: Apache/2.0.49 (Fedora) 11.Location: http://www.dce.harvard.edu/ 12.Content-Length: 302 13.Connection: close 14.Content-Type: text/html; charset=iso-8859-1 15.

view plain print ?

minerva% lwp-request -USed http://cscie12.dce.harvard.edu/apache/dce.html 1.GET http://www.dce.harvard.edu/ 2.User-Agent: lwp-request/2.06 3. 4.GET http://cscie12.dce.harvard.edu/apache/dce.html --> 302 Found 5.GET http://www.dce.harvard.edu/ --> 200 OK 6.Connection: Close 7.Date: Wed, 13 Apr 2008 20:01:26 GMT 8.Accept-Ranges: bytes 9.Server: Orion/2.0.6 10.Content-Length: 3619 11.Content-Type: text/html 12.Content-Type: text/html; charset=iso-8859-1 13.Last-Modified: Wed, 27 Oct 2007 18:45:00 GMT 14.Client-Date: Wed, 13 Apr 2008 20:01:49 GMT 15.Client-Peer: 140.247.198.100:80 16.Client-Response-Num : 1 17.


72 of 87 4/23/2008 10:53 AM

Rewrite

mod_rewrite http://www.apache.org/docs/2.0/mod/mod_rewrite.htmlA Users Guide to URL Rewriting with the Apache Webserver http://www.engelschall.com/pw/apache/rewriteguide/

Rewrite uses regular expressions to match on a pattern and rewrite to a new location. For example,the Derek Bok Center site used to be a "user" account and had the "~bok_cen/" base. When moved toits own virtual host, all of the "~bok_cen" requests could be rewritten to the new site with a singlerewrite rule.

Old URL: http://www.fas.harvard.edu/~bok_cen/tf/resources.html(.*) matches on: /tf/resources.html

New URL: http://bokcenter.fas.harvard.edu/tf/resources.html

# rewrite for Bok Center 1.RewriteRule ^/~?bok_cen(.*) http://bokcenter.fas.harvard.edu$1 [R=301] 2.


73 of 87 4/23/2008 10:53 AM

Examples of Rewrite Uses

Provide a standard mechanism to access course Web sites within HarvardCollege.

http://www.courses.harvard.edu/<4 digit catalog number>

For example, Chemistry 7 has a catalog number of 5118, so the URL for the course Web site can be reached through:

http://www.courses.harvard.edu/5118

The "real" location of the site is:

http://my.harvard.edu/icb/icb.do?course=fas-chem7

HASCS Site Restructure

Dozens of rewrite directives were put in place when the HASCS site was restructured so that links todocuments within the previous site would get redirected to the appropriate page in the new site.


74 of 87 4/23/2008 10:53 AM

Rewrite: Can be conditional

Rewrite rules can conditional (match against almost any environment variable).

Here we match on host and user-agent to deliver an error page explaining that their browser is not supported.

RewriteEngine On 1.RewriteCond %{HTTP_USER_AGENT} ^Lynx 2.RewriteRule ^(index.html)?$ text/ [R=302] 3.

# rewrite rule to catch IE Mac browsers since 1.# the PIN Service does not support them as of 10/16/2006 2.RewriteCond %{HTTP_HOST} "^login.icommons.harvard.edu$" 3.RewriteCond %{HTTP_USER_AGENT} "MSIE 5.*\; Mac_PowerPC" 4.RewriteRule ^/pinproxy.* /pin_error_ie_mac.html [R,L] 5.


75 of 87 4/23/2008 10:53 AM

Meta Refresh

Note: redirection may also be achieved on some browsers by using the http-equiv attribute of the <meta> element. The recommended method is to do it at the server level.

view plain print ?

 1. 2.<meta http-equiv="Refresh" content="10; URL=http://www.harvard.edu/"/> 3.


76 of 87 4/23/2008 10:53 AM

Directory Index and Listings

Note: Remember the difference between a directory having rwx-----x and rwx---r-x permissions?

DirectoryIndex http://www.apache.org/docs/2.0/mod/mod_dir.html Would you prefer main.html or overview.html to be the default files returned when a directory isrequested?mod_autoindex http://www.apache.org/docs/2.0/mod/mod_autoindex.html Provides for automatic indexing of a directory.

DirectoryIndex

DirectoryIndex index.html main.html overview.html slide1.html 1.


77 of 87 4/23/2008 10:53 AM

More Control over Directory Listings

mod_autoindex

Basic

Custom

The details:


78 of 87 4/23/2008 10:53 AM

Example with Course Lecture Notes

view plain print ?

minerva% pwd /home10/c/s/cscie12/public_html/autoindex/ex2 1.minerva% ls -la 2.total 28 3.drwxr-xr-x 2 cscie12 courses 8192 Nov 27 13:28 . 4.drwxr-xr-x 6 cscie12 courses 8192 Nov 27 13:11 .. 5.-rw-r--r-- 1 cscie12 courses 207 Nov 27 13:12 .htaccess 6.-rw-r--r-- 1 cscie12 courses 147 Nov 27 13:09 HEADER.html 7.-rw-r--r-- 1 cscie12 courses 66 Nov 27 13:09 README.html 8.-rw-r--r-- 1 cscie12 courses 4168 Nov 27 12:58 client-server.gif 9.-rw-r--r-- 1 cscie12 courses 906 Nov 27 12:58 slide1.html 10.-rw-r--r-- 1 cscie12 courses 743 Nov 27 12:58 slide2.html 11.-rw-r--r-- 1 cscie12 courses 1208 Nov 27 12:58 slide3.html 12.minerva% cat .htaccess 13.IndexOptions FancyIndexing 14.IndexOptions IconsAreLinks IconHeight=22 IconWidth=20 \ 15. NameWidth=* ScanHTMLTitles SuppressLastModified \ 16. SuppressSize SuppressColumnSorting \ 17. SuppressHTMLPreamble 18.IndexIgnore *.gif .. 19.minerva% 20.


79 of 87 4/23/2008 10:53 AM

Expires HTTP Header

Module mod_expires http://www.apache.org/docs/2.0/mod/mod_expires.html

.htaccess

Or, expire based upon modification time of document:

ExpiresActive On 1. 2.ExpiresByType text/html A3600 3.# HTML expires in 1 hour 4. 5.ExpiresByType image/gif A2592000 6.# GIF expires in 30 days 7. 8.ExpiresByType image/jpeg A2592000 9.# JPEG expires in 30 days 10. 11.ExpiresByType image/png A2592000 12.# PNG expires in 30 days 13. 14.# types not specified 15.ExpiresDefault "now plus 1 day" 16.# expires in 1 day 17.

ExpiresActive On 1.ExpiresByType text/html M86400 2.# HTML expires 1 day after it was last modified 3.ExpiresDefault M86400 4.


80 of 87 4/23/2008 10:53 AM

Do not cache

If you do not want your page cached, set these HTTP response headers:

In .htaccess in Apache, this would translate to:

view plain print ?

Cache-control: no-cache 1.Pragma: no-cache 2.Expires: <set to now> 3.

ExpiresDefault "now" 1.Header set Pragma "no-cache" 2.


81 of 87 4/23/2008 10:53 AM

Headers

mod_headers http://www.apache.org/docs/2.0/mod/mod_headers.html

The optional headers module allows for the customization of HTTP response headers. Headers canbe merged, replaced or removed. The server will always add a "Server" and "Date" header to the HTTP response.

view plain print ?

Header set Author "David P. Heitmeyer" 1.


82 of 87 4/23/2008 10:53 AM

Usertrack with Cookies

mod_usertrack

.htaccess file:

view plain print ?

minerva% lwp-request -USed http://cscie12.dce.harvard.edu/apache/usertrack/ 1.GET http://cscie12.dce.harvard.edu/apache/usertrack/ 2.User-Agent: lwp-request/1.38 3. 4.GET http://cscie12.dce.harvard.edu/apache/usertrack/ --> 200 OK 5.Connection: close 6.Date: Wed, 13 Apr 2008 20:32:54 GMT 7.Accept-Ranges: bytes 8.Server: Apache/2.2.6 (Fedora) 9.Content-Length: 59 10.Content-Type: text/html; charset=UTF-8 11.Client-Date: Wed, 13 Apr 2008 20:32:54 GMT 12.Client-Peer: 140.247.197.240:80 13.Client-Response-Num : 1 14.Set-Cookie2: MyCookie=140.247.197.240.1113424374035983; \ 15. path=/; max-age=2858400; \ 16. domain=.dce.harvard.edu; version=1 17.

CookieTracking on 1.CookieStyle RFC2965 2.CookieName MyCookie 3.CookieExpires "1 month 3 days 2 hours" 4.CookieDomain .dce.harvard.edu 5.


83 of 87 4/23/2008 10:53 AM

Details about enabling .htaccess and allowed directives

Context: can these directives be in .htaccess files?AllowOverride: is the server configured to allow this group of directives to be overriden in thislocation?Is the required module loaded?


84 of 87 4/23/2008 10:53 AM

Legal Directives I: Context

Certain Apache directives are legal within .htaccess files. Some are not. See the Apache Documentation for details. Specifically, look at the Context line that is given for thedirective in question.

Apache Core Features http://www.apache.org/docs/2.0/mod/core.htmlApache Module List http://www.apache.org/docs/2.0/mod/standard Apache Directives http://www.apache.org/docs/2.0/mod/directives.html

The following is an excerpt from the Apache HTTP Server Version 2.2 documentation

ErrorDocument directive

Syntax: ErrorDocument error-code document Context: server config, virtual host, directory, .htaccess Status: core Override: FileInfo Compatibility: The directory and .htaccess contexts are only available in Apache 1.1 and later.

Also, the "a" indicator on the Apache Quick Reference Card indicates that the directive is valid within an .htaccess file.


85 of 87 4/23/2008 10:53 AM

Legal Directives II: AllowOverride

Users are allowed to override certain aspects of the main server configuration. The main server configuration file (httpd.conf) contains an AllowOverride directive that determines which directives within .htaccess files Apache will process. The Override line that is given for eachdirective in the Apache documentationindicates which configuration directive must be active in order to use that directive with an .htaccessfile.

For the FAS system, the main server configuration file has the following directive in place for users'public_html directories:

The following is an excerpt from the Apache HTTP Server Version 1.3 documentation

ErrorDocument directive

Syntax: ErrorDocument error-code document Context: server config, virtual host, directory, .htaccess Status: core Override: FileInfo Compatibility: The directory and .htaccess contexts are only available in Apache 1.1 and later.

AllowOverride FileInfo AuthConfig Limit Indexes Options 1. 2.


86 of 87 4/23/2008 10:53 AM

Legal Directives III: Apache Modules

Apache is distributed with several modules. These modules may or may not be active within the Apache server with which you are working. The Core features will always be available.

For example, if the Rewrite Module (mod_rewrite) has not been activated, none of the Rewritedirectives will be available to use.

Refer to the Status and Modulelines in the documentation for each directive and to the documentation for the specific Apacheinstallation you are using.


87 of 87 4/23/2008 10:53 AM

Apache Modules

On the Apache (Apache/2.2.6) minerva.dce.harvard.edu web server, the following Apache modulesare active:

LoadModule auth_basic_module modules/mod_auth_basic.soLoadModule auth_digest_module modules/mod_auth_digest.soLoadModule authn_file_module modules/mod_authn_file.soLoadModule authn_alias_module modules/mod_authn_alias.soLoadModule authn_anon_module modules/mod_authn_anon.soLoadModule authn_dbm_module modules/mod_authn_dbm.soLoadModule authn_default_module modules/mod_authn_default.soLoadModule authz_host_module modules/mod_authz_host.soLoadModule authz_user_module modules/mod_authz_user.soLoadModule authz_owner_module modules/mod_authz_owner.soLoadModule authz_groupfile_module modules/mod_authz_groupfile.soLoadModule authz_dbm_module modules/mod_authz_dbm.soLoadModule authz_default_module modules/mod_authz_default.soLoadModule ldap_module modules/mod_ldap.soLoadModule authnz_ldap_module modules/mod_authnz_ldap.soLoadModule include_module modules/mod_include.soLoadModule log_config_module modules/mod_log_config.soLoadModule logio_module modules/mod_logio.soLoadModule env_module modules/mod_env.soLoadModule ext_filter_module modules/mod_ext_filter.soLoadModule mime_magic_module modules/mod_mime_magic.soLoadModule expires_module modules/mod_expires.soLoadModule deflate_module modules/mod_deflate.soLoadModule headers_module modules/mod_headers.soLoadModule usertrack_module modules/mod_usertrack.soLoadModule setenvif_module modules/mod_setenvif.soLoadModule mime_module modules/mod_mime.soLoadModule dav_module modules/mod_dav.soLoadModule status_module modules/mod_status.soLoadModule autoindex_module modules/mod_autoindex.soLoadModule info_module modules/mod_info.soLoadModule dav_fs_module modules/mod_dav_fs.soLoadModule vhost_alias_module modules/mod_vhost_alias.soLoadModule negotiation_module modules/mod_negotiation.soLoadModule dir_module modules/mod_dir.soLoadModule actions_module modules/mod_actions.soLoadModule speling_module modules/mod_speling.soLoadModule userdir_module modules/mod_userdir.soLoadModule alias_module modules/mod_alias.soLoadModule rewrite_module modules/mod_rewrite.soLoadModule proxy_module modules/mod_proxy.soLoadModule proxy_balancer_module modules/mod_proxy_balancer.soLoadModule proxy_ftp_module modules/mod_proxy_ftp.soLoadModule proxy_http_module modules/mod_proxy_http.soLoadModule proxy_connect_module modules/mod_proxy_connect.soLoadModule cache_module modules/mod_cache.soLoadModule suexec_module modules/mod_suexec.soLoadModule disk_cache_module modules/mod_disk_cache.soLoadModule file_cache_module modules/mod_file_cache.soLoadModule mem_cache_module modules/mod_mem_cache.soLoadModule cgi_module modules/mod_cgi.so

Table of Contents | All Slides | Link List | CSCI E-12