Marlon Pierce, Geoffrey FoxCommunity Grids Lab

Indiana University

Tutorial OutlineTechnology Survey

AJAX, JSON Social bookmarking and tagging REST Services

Amazon S3 example Atom and RSS Newsfeeds Start Pages, Gadgets, and Widgets Microformats

Basic lessons: Not much is actually new Reuse and exploit existing technologies in interesting ways. Brush up your JavaScript

Too much to cover, so we will skim some sections.Discussions and questions are welcome at any time.Tutorial materials available from

http://www.servogrid.org/slide/iSERVO/Web20/

AcknowledgementsThe following people helped create the

slides and material for this presentation:Siddharth MainiJoshua RosenHuapeng YuanYu Deng

Please see also talk by Fatih Mustacoglu during Wednesday afternoon’s session.

A survey of techniques and toolkits for improving browser based client interfaces. We will focus on open source technologies

and alternatives to Flash/Flex.

Sovereign or Transitory?Nice discussion in “Ajax in Action” from Manning Press.Sovereign applications:

IDEs, Word Processors and other office software, desktop tools

Things you use for hours per day.Transitory applications:

Calendars, email clientsAlmost all Web applications

HTTP’s request/response cycle limits interactivity Problem for Science Portals…

AJAX and related techniques have the promise of enabling browser-based applications to be sovereign.

AJAX Stands for Asynchronous JavaScript + XML Ajax is not a technology but a combination of:

Standards-based presentation using XHTML and CSS Dynamic update and display content using DOM (Document

Object Model) Data communication using JavaScript’s XMLHttpRequest Event-driven without direct action required by user Use other XHTML/Dynamic HTML features

Asynchronous calls Make requests to the server without a page refresh. Parse and work with XML documents

Extensively used by Google E.g. Google Suggest

Web page can communicate with web server online as user enters data into an HTML formSource: http://www.google.com/webhp?complete=1&hl=en

<script type="text/javascript" language="javascript"> var http_request ; function makeRequest(url) {if (window.XMLHttpRequest) { // For Mozilla, Safari http_request = new XMLHttpRequest();

} else if (window.ActiveXObject) { // For IEhttp_request = new ActiveXObject("Msxml2.XMLHTTP");}

http_request.onreadystatechange = alertContents; http_request.open('GET', url, true); http_request.send(null);

function alertContents() { if (http_request.readyState == 4) {

if (http_request.status == 200) { alert(http_request.responseText); } else { alert('There was a problem with the request.'); }

}} </script><span style="cursor: pointer; text-decoration: underline;"

onclick="makeRequest('test.html')">Make a request</span>

Make an HTTP request using

XMLHttpRequest class

Provide HTTP request object the name of

JavaScript object which will process

onreadystatechange property

Once the state of request AND status

code of server response is

checked, we can process the data

User makes the request in browser

OutputUser clicks the link “Make a Request in the Browser”This calls the makerequest(“test.html”) with test.html in the

same directoryRequest is made and then (onreadystatechange) execution is

passed to alertContents()alertContents() verifies the response received and then

alert()s the content of test.html file (as part of processing the data)

Limits of AJAXXMLHttpRequest object lets JavaScript make

GET HTTP requestsBut as a security feature you cannot call third

party domains through latest web browsers Exception IE 5 and 6 only under low security

settingsYou can only make requests back to the original

server/hostnameSolution: Some hacks/methods

Server-side application proxies Apache Proxy JSON

Called as a “fat-free-alternative” to XML and a serialized JavaScript Object

It is an ultra-simple lightweight data interchange format

Based on subset of JavaScript syntax, array and object literals

Supported as an alternative output format by many Web 2.0 ServicesYahoo APIs, Google Maps, Del.icio.us, etc.

Built-in support in Browsers

JSON: JavaScript Object Notation

JSON, ContinuedCan make cross-domain requests unlike AJAX which

uses XMLHttpRequestTo make cross-domain requests, just dynamically create your

<script> tags using the DOMAdd a “src” attribute to point to the desired JSON URL.Your browser will download the additional code.For more info:

http://www.xml.com/pub/a/2005/12/21/json-dynamic-script-tag.html

CSS can also be expressed in JSON Example to followhttp://www.featureblend.com/css-json.html

JavaScript's eval() method to convert the string into a real JavaScript objectVar data = eval('(' + req.responseText + ')');

JSON Example

Defining Arrays in JavaScript

JSON Syntax

CSS JSON CSS Structure

XML -> JSONXML can be converted to reversible JSON if:

all subelements names occur exactly once, or … Subelements with identical names are in sequence.

XML can be converted to irreversible JSON if: Sub elements are not unique Element order doesn’t matter

Source :Goessner, Stephen. "XML.com: Converting Between XML and JSON." XML.com. 31 May 2006. O'REILLY. 16 May 2007 <http://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html?page=2>.

JSON Compared to XMLXML String JSON String

<classinfo> <students> <student> <name>Michael Smith</name> <average>99.5</average> <age>17</age> <graduating>true</graduating> </student> <student> <name>Steve Johnson</name> <average>34.87</average> <age>17</age> <graduating>false</graduating> </students></classinfo>

{ "classinfo" : { "students" : [ { "name" : "Michael Smith", "average" : 99.5, "age" : 17, "graduating" : true }, { "name" : "Steve Johnson", "average" : 34.87, "age" : 17, "graduating" : false } ] }}

Information repeated repeatedly Information isn’t repeated

More bytes needed to store the same information Less Bytes needed to store the same information

Have to convert string into JavaScript using:var books = req.responseXML.getElementsByTagName(‘<element_name>');

JavaScript's eval() method to convert the string into a real JavaScript object E.g. var data = eval('(' + req.responseText + ')');

Source: Zakas, Nicholas C., Jeremy McPeak, and Joe Fawcett. Professional AJAX. 1st ed. WROX Press, 2006.

JavaScript/AJAX/JSON ToolkitsTo make writing Web Sites or Web Applications

easier to developGWT (Google Web Toolkit)

http://code.google.com/webtoolkit/ YUI (Yahoo! User Interface Library)

http://developer.yahoo.com/yui/ DWR

http://getahead.org/dwr script.aculo.us

http://script.aculo.us/ Prototype

http://www.prototypejs.org/

GWT (Google Web Toolkit)Open source Java software development framework

Develop Web applications as if they were Swing-like GUIsEasier AJAX application development

1. You write your front end in Java using any Java IDE available (e.g. Eclipse, JProfiler, JUnit, IntelliJ…)

2. GWT complier will automatically convert it into browser-complaint JavaScript and HTML

3. Confirm that the Web App. Runs successfully in each browserGWT Architecture

GWT Java-to-JavaScript Compiler: Java-to-JavaScript compiler GWT Hosted Web Browser: run and execute your GWT applications JRE emulation library: contains JavaScript implementations of the

most widely used classes in the Java standard class library. GWT Web UI class library: set of custom interfaces and classes that

let your create web browser "widgets” with use of CSS

Benefits of GWTStatic type checking in the Java language

boosts productivity while reducing errors.Common JavaScript errors are easily caught at

compile time rather than at runtime.Code prompting/completion is widely

available.Java-based OO designs are easier to

communicate and understandMakes your AJAX code base more

comprehensible with less documentation.Use GWT’s set of User Interface (UI) to build

UI elements making an AJAX application

GWT Example CODEpackage com.google.gwt.sample.kitchensink.client;import com.google.gwt.core.client.EntryPoint;import com.google.gwt.sample.kitchensink.client.Sink.SinkInfo;import com.google.gwt.user.client.History;import com.google.gwt.user.client.HistoryListener;import com.google.gwt.user.client.ui.DockPanel;import com.google.gwt.user.client.ui.HTML;

/** Application that demonstrates all of the built-in widgets. */public class KitchenSink implements EntryPoint, HistoryListener { protected SinkList list = new SinkList(); private SinkInfo curInfo; private Sink curSink; private HTML description = new HTML(); private DockPanel panel = new DockPanel(); private DockPanel sinkContainer;

GWT Example

Source: http://code.google.com/webtoolkit/documentation/examples/kitchensink/

YUI (Yahoo! User Interface Library)Collection of JavaScript and CSS resourcesMakes RIA (Rich Internet Applications) easier to buildOpen sourceYahoo! User Interface Library components fall into

three groups: UtilitiesUI Controls, andCSS resources

“Better documented than GWT” (Sidd’s personal opinion)

Can be used with Yahoo! Design Patterns Library to easily implement design patterns

Utilities Animation: to Create "cinematic effects” Browser History Manager: just like GWT’s Back Button functionality Connection Manager: to manage XMLHttpRequest transactions in a cross-browser fashion Data Source Utility: to provide an interface for retrieving data Dom Collection: Drag & Drop: Create draggable objects that can be picked up and dropped elsewhere on

the page Event: gives you easy and safe access to browser events

Controls AutoComplete: Button Control Calendar DataTable Control Logger: Provides easy way to write logs to an on-screen console Menu Slider Tab View Tree View

CSS Resources CSS Grids CSS Fonts CSS Reset

Yahoo Components

DWR (Direct Web Remoting) Java open source libraryAllows writing AJAX web sitesTwo Main parts:

Java Servlet running on the server to process requests and sends responses back to the browser.

JavaScript running in the browser to send requests and can dynamically update the webpage.

It dynamically generates JavaScript1. Feels like the execution is happening in the browser2. But server is executing the code , and 3. DWR converts data back and forth using a callback function

In the example above, DWR dynamically generates the AjaxService class

It then converts all parameters and return values between Java and JavaScript

DWR featuresIntegrates with

Spring, Struts, WebWork, JSF, Hibernate, Beehive

Supported Environments and Application ServersTomcat, Weblogic, Websphere, Jboss, Jetty,

Resin, Sun ONE, GlashfishAbility to asynchronously send data

from a web-server to a browserSource: http://getahead.org/dwr/integration

DWR Example (Wal-Mart)1. Wal-Mart uses the

DWR technology in the “Quick View”

2. Clicking “Quick View” will pop-up a dialog with more details, fetched asynchronously using DWR.

Features - Script.aculo.us Visual Effects Library: To add animationDrag and Drop JavaScript Library: enables drag-

and-drop of elements in your Web App.Dom utilities: Create DOM elements dynamicallyAjax Controls

Drag and Drop, Draggables, Autocompletion, In Place Editing

Integration with development frameworks Ruby On Rails, Perl, Nitro, PHP, Java Plone, DotNet, Symfony, Seaside AIDA/Web, Open ACS, Django

Java Script Unit Testing Includes unit and functional tests for itself Catch scripting and syntax errors and presenting them in a useful

way without breaking the tests Source: http://wiki.script.aculo.us/scriptaculous/show/UnitTesting

Example Test Results

EXAMPLEUsed animation and effects library to replace the

Flash animation on the home page. Allows easier maintenance of the drop-down menu

SummaryGWT YUI DWR Script.aculo.us

Code in Java and generate

JavaScript/HTML automatically

YES NO YES NO

Internationalization(easily create

international Libraries)YES NO NO NO

Integration with frameworks

Eclipse, IntelliJ,

JProfiler, JUnit

NOSpring, Structs, WebWork, JSF,

Hibernate, beehive

Ruby on Rails, Perl, Nitro,

Eclipse, SeaSide, Django, Plone,

Java, PHP

Barebones social web sites

Social Bookmarking / TaggingUsers save a list of internet resourcesThey assign keywords or tags to each

of these resourcesThis method of categorization is also

called as folksonomyMost bookmarking websites allow

users to search for bookmarks on tags or keywords.

Advantages / Disadvantages Advantages

You can get to your bookmarks from anywhere You can share your bookmarks with your friends,

coworkers, supervisor etc. You can categorize or group/bundle your bookmarks

according to your wish You can search of other users registered on that website

and vice versa You also have the option to make your tags private

Disadvantages No controlled vocabulary No standard for structure of tags e.g. capitalization,

singular, plural etc. Spelling mistakes in tags Tags having multiple meanings No hierarchical relationship Spamming – bookmarking same page multiple times

Del.icio.us Store links to your favorite Internet resources Share links/favorites with friends, family,

coworkers, and the del.icio.us community. Discover new things. On del.icio.us, you use tags to organize and

remember your bookmarks, which is a much more flexible system than folders.

You can bundle the tags into groups Example Uses

Research: keeping track of your research materials Wish list: maintain a wish list Collaboration: friends, coworkers and other groups

can use a shared account

Many Interfaces of del.icio.usWeb interfaceRSS Feeds of tags

More laterApplication Programming Interfaces (API)

Typically REST, more laterEmbedded JavaScript code inside your

web pageThis is a general theme of Web 2.0

interfaces.

Personal Bookmarks* Option to make bookmarks hidden

Bookmarks of other users who used common tags* Option to add any user to your network

Add users to your network and browse their tags* Option to disable sharing of your network

Subscribe to interesting tags to be aggregated

Group / Network Feed

Individual Feed

Google Reader

Del.icio.us API All del.icio.us APIs are done over https and require HTTP-Auth Example

https://api.del.icio.us/v1/tags/get Returns a list of tags and number of times used by a

user. Output: XML Code <?

 require_once 'HTTP/Request.php'; require_once '/home/smaini/vals.php';

 $req =& new HTTP_Request("https://api.del.icio.us/v1/tags/get"); $req->setBasicAuth(delTagUser,delTagPass); $response = $req->sendRequest();

 echo $req->getResponseBody();>

Output ->

API (continued)Update

https://api.del.icio.us/v1/posts/update Returns the last update time for the user

https://api.del.icio.us/v1/posts/all? Returns to check if the data has changed since last

fetch

Tagshttps://api.del.icio.us/v1/tags/rename?

old=horse&new=horsesRename an existing tag with a new tag name

Arguments required &old (required) - tag to rename. &new (required) - new name.

DELICIOUS APIPosts

https://api.del.icio.us/v1/posts/get? https://api.del.icio.us/v1/posts/recent? https://api.del.icio.us/v1/posts/all? https://api.del.icio.us/v1/posts/dates? https://api.del.icio.us/v1/posts/add? https://api.del.icio.us/v1/posts/delete?

Bundleshttps://api.del.icio.us/v1/tags/bundles/all https://api.del.icio.us/v1/tags/bundles/set? https://api.del.icio.us/v1/tags/bundles/delete?

JavaScript Widget for Del.icio.us JavaScript can be embedded into your HTML code as a “Bookmarklet”

This code can load a JavaScript Object that contains your latest set of tags

<a href="javascript:location.href='http://del.icio.us/post?v=2&url='+encodeURIComponent(document.location.href)+'&title='+encodeURIComponent(document.title)+' '">Post to del.icio.us</a>

Source: http://ekstreme.com/seo/socialbookmarkingcode.php

Connotea Free online references management for scientists and

clinicians One can save and organize references

Step 1: Add the reference to Connotea’s library as a Bookmark (using browser button, add form or DOIs)

Step 2: Tag the reference and type in the title Find users who used your tags Find Bookmarks matching your tags Entire library can be exported to the following formats:

RIS: Suitable for importing into Reference Manager and similar software

EndNote: Exporting to MS EndNote BibTex: For LATEX MODS: U.S. Library of Congress Metadata Object

Description Schema (MODS) format RSS Feeds, RDF (Resource Description Framework)

Can Import links / references using local file RIS, BibTeX, EndNote, MODS, ISI Web of knowledge, Firefox

bookmarks

Automatic Collection of Bibliographic Information Connotea will add automatic bibliographic

information for pages saved from the following sources

Nature.com, PubMed, PubMed Central, Science PloS, BioMed Central, Supported Eprints

repositories Supported Highwire Press publications Blackwell Synergy, Wiley Interscience, Scitation arXiv.org, Smithsonian/NASA Astrophysics Data

system Amazon, HubMed, D-Lib Magazine

RSS Feeds

Individual RSS Feed

Group RSS Feed

Multiple Users and Tags Clicking around on user and tag names allows you

to view the articles for one user or one tag But you can construct a URL for combining the

tags using AND and OR operators To see a list of articles for the users fdr and jfk,

construct the URL as: http://www.connotea.org/user/fdr/jfk

This, in fact, filters for fdr OR jfkhttp://www.connotea.org/user/fdr+jfk

A plus signs means AND, whereas a forward slash means OR.

More Tag Searching Rules This works for tag names too, and you can

combine user names, tag names, +'s and /'s in any combination.

For example: http://www.connotea.org/user/fdr+hst/jfk+lbj/tag/topsecret/classified

gives you a list of articles tagged as 'topsecret' or 'classified' by both fdr and hst, or by both jfk and lbj.

After getting the results you have the option to export the list as any of the formats mentioned including RSS Feeds

Programming Interface API Version 1 released wrapper libraries in Perl and Ruby URL Structure:

http://www.connotea.org/data /bookmarks or /tags or '' (empty string, which means 'posts')/user/ [username] /tag/ [tagname] /date/ [date of form YYYY-MM-DD ] /uri/ [uri or hash] ? q= [free text search string] & num= [number of results per] & start= [result number to start at]

All these are standard HTTP requests Use GET to retrieve the URL Output Format: RDF Format (Resource Descriptor Framework

Format) Example: http://www.connotea.org/data/tags/user/sidds1601

Embedding Connotea JavaScript Widgets

<a style="cursor:pointer;" onclick="javascript:w=open('http://www.connotea.org/addpopup?');<img src="images/connotea.png" border="0“ alt="add bookmark to connotea" align="absmiddle">Add to Connotea</a>

Will display “Add to Connotea” icon in your webpage.

CiteULike Purpose: To share, store, and organize

the papers Provides “Bookmarklets” to extract

information from current page Can manually post an article Can add URLs/DOIs and bibliographic

metadata using its supported sites You can add tags to your own or other

entries

CiteULike It also provides extra bibliographic information from various

databases It is NOT open source It can only import references from BibTex It can export reference list to:

EndNote or BibTex format

Supports Watch Lists: Page with papers that may be relevant to your field of study in the future.

Note: you can watch a page only on one tag and not more than one.

You only have the option to watch a page when you click on the active tags on the site.

Web Interface

Click to save it as a reference

RSS FeedsEntire libraries can be exported as an RSS feed.

Inserting JavaScript Code<script type="text/javascript"

src="http://static.citeulike.org/button.js"> </script>

CiteUlike icon in your Web Page

Similarities / ComparisonConnotea CiteUlike

Bookmarklets Yes Yes

Export Formats RISEndNoteBibTexMODSRSS FeedsRDF

EndNoteBibTex

Tag Sorting Order of Entry Alphabetically

OpenSource Yes No

Social Networking SitesThese are sites that build online social networks

of people with shared interests.For the big list, see

http://en.wikipedia.org/wiki/List_of_social_networking_websites

Note the scale of usersFlickr has 4,000,000 registered users.It is a medium-sized site.

We’ll look at Flickr as an example.I chose it because it has the second most used API

(after Google Maps) at programmableweb.com.

Pre-History of FlickrFlickr began life as a side project of a massive

multiplayer game“Game Neverending” ended in 2004

It continued to evolve... Early capabilities that were dropped:

Chats, real-time photo-sharing Capabilities added later

Tags (and tag clouds), favorites, group photo poolsInterestingness: a proprietary algorithm for

determining most interesting photos for a particular time period. Who tags, how many click, number of comments, etc.

Synchronous to asynchronous collaborationSource: http://en.wikipedia.org/wiki/Flickr

Flickr Interface OverviewAs with the del.icio.us, connotea, etc, there

are several ways to interact with FlickrGo to the websiteSubscribe to feedsUse the very impressive APIEmbed JavaScript badges

Flickr API OverviewIt is quite extensive.Supported request formats:

RESTXML-RPCSOAP

Response FormatsThe above, plus JSON and serialized PHP

API Kits:PHP, Python, Ruby, Perl, .NET, Java, Cold

Fusion, …http://www.flickr.com/services/api/

What Can You Do with the API?Activity: see recent photos and comments by a

particular user.Favorites: add, remove, and get a list of favorites.Interestingness: get a list of photos with high

interestingness scores.People: get information (email, group

membership, etc).Photos: add a tag, add a comment, find the

location, upload, etc. Reflection: get information about other API

methods.And more.

FeedsFlickr supports Atom 1.0

and various versions of RSS.

You can subscribe to the following feedsPublic photos, friends’

photos, group posts, discussions, news, etc.

Feed URLs support optional query parameters.Useful for REST style

programming.Specify a specific user ID,

a specific set of tags, etc.

Badges: Flickr WidgetsSmall, embedded

HTML or Flash “widget” that you can place in other websitesLike your blog.

Yet another example of client side integration.

A short guide to REST-style Web Services.

Representational State TransferREST is HTTP as a CS Ph. D. thesis.

Roy Fielding, co-author of HTTP specificationStandard Web Services have

WSDL is an API language. SOAP is the network message envelop

REST proposes to use Internet as programming platform with only HTTP. Use HTTP to ship machine process-able content, not just HTML. Simple (simplistic) but scales.

Clients to REST services have one API for all occasions. HTTP GET, PUT, POST, DELETE Operations are performed on URLs. Very suitable for AJAX and JSON programming

REST services are stateless (or idempotent). Identical requests give identical responses.

REST Services, ContinuedContent of URLs should be XML or similar encoded data (not

just HTML).No universal message format like SOAP, but RSS and Atom

are commonly used.You could use SOAP also.Or RDF, custom XML, and so on.

REST does not tell you what kind of message you will get.This is usually application specific.Assume a human developer is involved rather than a lot of

generated code.Simple but a workable tactical solution.Sometimes humans are better than machines.

Amazon’s Mechanical Turk

An April Fools Critique of RESTREST tends to attract passionate religious debatesSanjiva Weerawarana sees the light:

http://www.bloglines.com/blog/sanjiva?id=196Weerarana is co-author of WSDL specification, project

leader of IBM Web Service efforts, Apache Axis 2, CEO of WSO2

See also his more serious comments athttp://www.infoq.com/articles/sanjiva-rest-myths

echo “Groundskeeper Willie: It won't last. Brothers and sisters are natural enemies. Like Englishmen and Scots! Or Welshmen and Scots! Or Japanese and Scots! Or Scots and other Scots! Damn Scots! They ruined Scotland! Principal Skinner: You Scots sure are a contentious people. Groundskeeper Willie: You just made an enemy for life!” | sed “s/Scots/RESTifarians/”

We examine Amazon’s Simple Storage System as a REST case study.

• S3 issues two codes to each account. • An account access key: your identity• Shared secret key: used to digitally sign communications

• In each HTTP request, you have to add an Authorization field. • It will use the account access key and a “Signature” which is a HMAC-SHA1 hash of the request (sans “Authorization” line) using the secret access key as the key.

Authorization: AWS 1ATXQ3HHA59CYF1CVS02:SZf1cHmQ/nrZbsrC13hCZS061yw=

“Authorization: AWS “+ AWSAccessKeyID + “:” + Signature

The authorization line is formed like so:

An example :

Buckets: Grouping Your FilesObjects (files) are stored in buckets. An account can have multiple buckets. These bucket names are not user specific.

In other words, if an S3 user is already using a desired bucket name, that name is unavailable for everyone else.

This bucket name will be used in the url for your resources. An example would be : http://s3.amazonaws.com/bucket_name/sample.jpeg

Buckets (cont’d) Buckets are created with an http PUT formed like this :

PUT /[bucket-name] HTTP/1.0 Date: Wed, 08 Mar 2006 04:06:15 GMT Authorization: AWS [aws-access-key-id]:[header-signature] Host: s3.amazonaws.com

If properly formed it would return a response of

HTTP/1.1 200 OK x-amz-id-2: VjzdTviQorQtSjcgLshzCZSzN+7CnewvHA+6sNxR3VRcUPyO5fmSmo8bWnIS52qa

x-amz-request-id: 91A8CC60F9FC49E7 Date: Wed, 08 Mar 2006 04:06:15 GMT Location: /[bucket-name] Content-Length: 0 Connection: keep-alive Server: AmazonS3

Objects: Stored FilesThere is more to these objects than the

content of the file. Metadata can be included with each object. These are name/value pair collections

The objects must have unique names (keys) in regards to their own bucket. In other words, “s3.txt” can exist in multiple buckets,

but only once in a single bucket. There are no “sub-buckets” so many programmers

decide to group files by prefixing. For instance, all pictures would start with “Pictures.” This works well with the “list” operation.

Replacing/Overwriting ObjectsIf a file is uploaded that has a key that

already exists, that file is then replaced. This queue is based on when the file completes

upload. So if one user starts to upload a large file to

foo.bar and another one starts a much smaller file to that same key, even thought the smaller one started last, it is quite possible the larger one will overwrite the smaller one when it finishes. The first S stands for “simple”

PUT /[bucket-name]/[key-name] HTTP/1.0 Date: Wed, 08 Mar 2006 04:06:16 GMT Authorization: AWS [aws-access-key-id]:[header-signature] Host: s3.amazonaws.com Content-Length: 14 x-amz-meta-title: my title Content-Type: text/plain

this is a test

The HTTP request to upload a file will look like this:

This will give the following response:

HTTP/1.1 200 OK x-amz-id-2: wc15E1LUrjDZhNtT4QZtsbtadnOMKGjw5QTxkRDVO1owwbA6YoiqJJEuKShopufw x-amz-request-id: 7487CD42C5CA7524 Date: Wed, 08 Mar 2006 04:06:16 GMT ETag: "54b0c58c7ce9f2a8b551351102ee0938" Content-Length: 0 Connection: keep-alive Server: AmazonS3

File PermissionsThere are extensive rules to read and

write access to objects and buckets. These rights are stored in an ACL (access

control list), which is an XML document. Rights can be granted to users on a one to

one basis, or to pre-defined groups.

• READ: For a bucket, allows listing of the objects in that bucket. For an object, allows reading of data and/or metadata

• WRITE: For a bucket, allows the creation and deletion of all new and existing objects in this bucket. There is no support or WRITE on an object.

• READ_ACP: Allows the reading of a bucket or object’s ACL.

• WRITE_ACP: Allows the changing of a bucket or object’s ACL.

• FULL_CONTROL: Grants all of the above permissions. No other rights are added with this type.

Permissions (Grantee Types)User : Has to be a user of S3.

Can be identified either by e-mail address or by an id supplied by Amazon (canonical).

Owner : Always has full rights and is always the creator of the object.

Group : Currently there are only two groups: all users and authenticated usersRights given by these groups do not overwrite other

access control considerations.http://acs.amazonaws.com/grouops/global/AllUsers:

Represents everyone, anonymous or authenticated.http://acs.amazonaws.com/groups/global/

AuthenticatedUsers: Represents non-anonymous users.

<AccessControlPolicy> <Owner> <ID>a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9</ID> <DisplayName>chriscustomer</DisplayName> </Owner> <AccessControlList> <Grant> <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"> <ID>a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9</ID> <DisplayName>chriscustomer</DisplayName> </Grantee> <Permission>FULL_CONTROL</Permission> </Grant> <Grant> <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group"> <URI>http://acs.amazonaws.com/groups/global/AllUsers<URI> </Grantee> <Permission>READ</Permission> </Grant> </AccessControlList> </AccessControlPolicy>

Single User Entry

Group Entry

Owner

How to Use S3With PHP programming examples

define (“amazonKey”,”15B4D3491F177624206A”);define (“amazonSecret”,”(secret code given by S3)”);define (“amazonURL”,”http://s3.amazonaws.com/” );define (“amazonBucket”,”BucketName”);

Define Constants• When a new account is created, it will have a key

and “secret code” attached to it.• These should be placed in a separate file and be

defined as constants. • For these purposes, they will be known as

‘amazonKey’ and ‘amazonSecret’, respectively.

$ pear install HTTP_Request$ pear install Crypt_HMAC

require_once ‘Crypt/HMAC.php’;require_once ‘HTTP/Request.php’;

S3 requires the use of HTTP Requests and RFC 2104 compliant hashes. Luckily, the Pear framework (which comes with PHP) has made packages for these purposes.

Before using these packages, they must be added to PHP. Simply run these commands (pear is found in /bin under the /php directory).

Any php script that will use these packages must include these two lines.

function hex2b64($str) { $raw = ''; for ($i=0; $i < strlen($str); $i+=2) { $raw .= chr(hexdec(substr($str, $i, 2))); } return base64_encode($raw);}

function constructSig($str) { $hasher =& new Crypt_HMAC(amazonSecret, "sha1"); $signature = hex2b64($hasher->hash($str)); return($signature);}

Two functions will need to be created to facilitate the use of S3.

Converts a string from hex to base 64

Constructs the “Signature” using the secret key to hash the given string and encode it

function createBucket($bucket, $acl = 'private') { $httpDate = gmdate("D, d M Y G:i:s T"); echo $httpDate; $stringToSign = "PUT\n\n\n$httpDate\nx-amz-acl:$acl\n/$bucket"; $signature = constructSig($stringToSign); $req =& new HTTP_Request(amazonUrl . $bucket); $req->setMethod("PUT"); $req->addHeader("Date", $httpDate); $req->addHeader("Authorization", "AWS " . amazonKey. ":" . $signature); $req->addHeader("x-amz-acl", $acl); $response = $req->sendRequest(); $responseCode=$req->getResponseCode(); if ($responseCode == 200) { return true; } else { return false; }}

This section constructs the signature

This section constructs the headers, and creates the signature

Send the request and return whether or not it was successful

createBucket(amazonBucket);

• Once that function is created, it’s pretty simple to create a bucket. • It is usually more desirable to keep the ACL private, so we’ll keep that blank. • The objects under it can still be made publicly readable, which allows browser access

• This prevents others from searching the directory, and adding objects of their own.

PUT / BucketName HTTP/1.0 Content-Length: 0 Authorization: AWS 15B4D3461F177624206A:YFhSWKDg3qDnGbV7JCnkfdz/IHY= Date: Thu, 17 Nov 2005 02:40:52 GMT

So in this instance, running this line (see previous slide):

should create an HTTP request looking like :

function putObject($bucket, $key, $filePath, $contentType, $contentLength, $acl, $metadataArray=array(), $md5=""){ sort($metadataArray); $resource = "$bucket/$key"; $resource = urlencode($resource); $req = & new HTTP_Request($this->serviceUrl.$resource); $req->setMethod("PUT"); $httpDate = gmdate("D, d M Y G:i:s T"); $req->addHeader("Date", $httpDate); $req->addHeader("Content-Type", $contentType); $req->addHeader("Content-Length", $contentLength); $req->addHeader("x-amz-acl", $acl); if($md5){ $MD5 = $this->hex2b64(md5_file($filePath)); $req->addHeader("Content-MD5", $MD5); } $req->setBody(file_get_contents($filePath)); $stringToSign="PUT\n$MD5\n$contentType\n$httpDate\nx-amz-acl:$acl\n"; foreach($metadataArray as $current){ if($current!=""){ $stringToSign.="x-amz-meta-$current\n"; $header = substr($current,0,strpos($current,':')); $meta = substr($current,strpos($current,':')+1,strlen($current)); $req->addHeader("x-amz-meta-$header", $meta); } } $stringToSign.="/$resource"; $signature = $this->constructSig($stringToSign); $req->addHeader("Authorization", "AWS " . $this->accessKeyId . ":" . $signature); $response = $req->sendRequest(); $responseCode = $req->getResponseCode(); if (responseCode == 200) { return true; } else { echo $req->getResponseBody(); return false; }}

Prepare the request

Add the necessary Headers

Includes an md5 if specified

Creates the signature, with metadata

Add the contents of the file to the body of the request

Send the request and return the response

putObject (amazonBucket, $_FILES[‘upFile’][‘name’], $_FILES[‘upFile’][‘tmp_name’], $_FILES[‘upFile’][‘type’], filesize($_FILES[‘upFile’][‘tmp_name’]), ‘public_read’);

In this instance, a file is being uploaded that is simply called “Message”. This is a simple text file with the contents “Paper or Plastic”.

PUT /BucketName/Neo HTTP/1.0 Content-Length: 16 Authorization: AWS 15B4D3461F177624206A:xQE0diMbLRepdf3YB+FIc8F2Cy8= Date: Thu, 17 Nov 2005 07:48:33 GMT Content-Type: text/plain

Paper or Plastic

This will produce the HTTP request seen here :

Which will upload this file to the S3 server which can be accessed either by the REST service or directly by accessing the link: http://s3.amazonaws.com/BucketName/Neo

A brief overview of news feeds, how to create, and how to consume.

News Feed ReadersWe have already

seen several of these.SageGoogle Reader

Have become ubiquitous as part of browser Start PagesiGoogleNetvibesYahoo

Atom Feed for Bay1Temp<?xml version="1.0" encoding="utf-8" ?> <feed xmlns="http://www.w3.org/2005/Atom">

  <id>urn:uuid:29fefa08:112967cfbb9:-8000</id> <updated>2007-05-16T16:07:16.376-05:00</updated> <title type="text">Bay1Temp Atom Feed</title> <author>

<name>Yu(Carol) Deng</name> </author><link rel="self" type="application/atom+xml" href="http://hagar.cs.indiana.edu:8181/foo.xml" />

<entry><id>urn:uuid:29fefa08:112967cfbb9:-7fff</id> <updated>2007-05-16T16:03:32.423-05:00</updated> <title type="text">SensorName, TimeStamp, DoubleData</title> <author>

<name>Yu(Carol) Deng</name> </author><content type="html">Bay1Temp 2007-05-16 20:06:35Z

25.2<br>Bay1Temp 2007-05-16 20:06:55Z 25.1<br>Bay1Temp 2007-05-16 20:07:15Z 25.1</content> </entry>

</feed>

AtomsphereAn Atom feed library written in Javahttp://www.colorfulsoftware.com/projects/

atomsphereDownload packages

atomsphereatomsphere-taglibatomsphere-webappatomsphere-weblib

Add jar files included in the packages above to the project

Create Atom Feed for Bay1TempCreate an atom feed document

add attribute “xmlns” to be “http://www.w3.org/2005/Atom” to the feed

add elements “Author”, “Id”, “Title”, “Link” , “Updated”

Add an entry document to the feed add elements “Author”, “Id”, “Title”,

“Updated” add element “Content” which contains

parameters’ value of Bay1Temp

Java Code for creating an atom feed doc// New a feedFeed theFeed = new Feed();

// Add "xmlns" attribute to the feedtheFeed.addAttribute(new Attribute("xmlns","http://www.w3.org/2005/Atom"));

// Add AuthortheFeed.addAuthor(new Author("Yu(Carol) Deng"));

// Set a universally unique Id for the feedtheFeed.setId(new Id("urn:uuid:" + new UID().toString()));

// Add TitleTitle feedTitle = new Title("text"); // Set title type for the feedfeedTitle.setText("Bay1Temp Atom Feed"); // Set title contenttheFeed.setTitle(feedTitle); // Set the title

// Add LinkLink feedLink = new Link(); // New a Link in the feedfeedLink.setRel(new Attribute("rel", "self")); //Set "rel"

attribute of the link feedLink.setType(new Attribute("type", "application/atom+xml")); //Set "type"

attribute of the linkfeedLink.setHref(new Attribute("href", FeedHref)); //Set "href" attribute of the

linktheFeed.addLink(feedLink);

// Set Updated to the entry theFeed.setUpdated(new Updated(new Date()));

// New an EntryparcelEntry = new Entry();

// Add AuthorparcelEntry.addAuthor(new Author("Yu(Carol) Deng"));

// Add TitleTitle parcelTitle = new Title("text"); // Set title type for the feedparcelTitle.setText("SensorName, TimeStamp, DoubleData"); // Set title

contentparcelEntry.setTitle(parcelTitle); // Set the title

// Set a universally unique Id for the entryparcelEntry.setId(new Id("urn:uuid:" + new UID().toString()));

// Set Updated to the entry Calendar cal = new GregorianCalendar();parcelEntry.setUpdated(new Updated(cal.getTime()));

// Set the current data to the ContentparcelEntry.setContent(nodeSensorName + nodeTimeStamp +

nodeDoubleData);

// Add the Entry to the feedcurrentFeed.addEntry(parcelEntry);

Code for adding an entry doc to feed

Atom Feed for Bay1Temp<?xml version="1.0" encoding="utf-8" ?> <feed xmlns="http://www.w3.org/2005/Atom">

  <id>urn:uuid:29fefa08:112967cfbb9:-8000</id> <updated>2007-05-16T16:07:16.376-05:00</updated> <title type="text">Bay1Temp Atom Feed</title> <author>

<name>Yu(Carol) Deng</name> </author><link rel="self" type="application/atom+xml" href="http://hagar.cs.indiana.edu:8181/foo.xml" />

<entry><id>urn:uuid:29fefa08:112967cfbb9:-7fff</id> <updated>2007-05-16T16:03:32.423-05:00</updated> <title type="text">SensorName, TimeStamp, DoubleData</title> <author>

<name>Yu(Carol) Deng</name> </author><content type="html">Bay1Temp 2007-05-16 20:06:35Z

25.2<br>Bay1Temp 2007-05-16 20:06:55Z 25.1<br>Bay1Temp 2007-05-16 20:07:15Z 25.1</content> </entry>

</feed>

Bay1Temp Atom Feed in iGoogle

Making Your Own Feed ConsumerThere are plenty of libraries for consuming

feeds that you can embed in your own Web pages, blogs, wikis, etc.

I looked at two for PHP:Magpie RSS: wasted an afternoon trying to get

this to work.SimplePie: worked in 5 minutes.

We’ll look at SimplePie for MediaWiki

Adding SimplePie to MediaWiki, Part IThese steps require access to MediaWiki’s

directories under ApacheDownload SimplePie and put the simplepie.inc

file in your MediaWiki’s “extensions” folder.Download the MediaWiki plugin and put it

(simple_pie.php) in your extensions folder also.Edit LocalSettings.php to add the line

include("./extensions/simplepie_mediawiki.php");

After installation, next steps can be done by anyone.

Adding a Feed, Part II

To show just dates and titles, do this:<feed showdesc="false" showdate="true">

... </feed>

In the text area, add a line like<feed>http://myblog.host.org/rss/</feed>

Let’s say you want to add your blog’s RSS feed to your team’s wiki.

Create a new Wiki page and edit it. Part of the art of a wiki...

Use blog to create posts.

Display blog RSS feed in MediaWiki.

Aggregating Rich Internet Experiences and other buzz phrases

What Is a Portal?Traditionally, a portal is personalized Web page

that recognizes you.You have to log in and set cookies.

You get customized content and layouts.Typically newsfeeds but this is also a good model

for science gateways.Portals have a component model. Collections of

components (portlets) are managed by a portal container.

Enterprise portals are based on standards like JSR 168 (Java) for portlets and containersWeb Services for Remote Portlets (WSRP)

Web 2.0 equivalent is called a Start Page.

Science Portals Science Portals are

often built using portlet components.Ex: TeraGrid Science

Gateways Portlets are a server

side technology.Can be built with

enterprise technologies like JSF.

Users can select from available portlets and customize their layouts.

Web 2.0 Challenges for Science PortalsClient-side integration of components from multiple

service providers. Start Pages do this all the time with widget “ecosystems”

Multiple views of the same Web application. Consider del.icio.us: Browser, JSON, REST/XML, JavaScript

Widget interfaces all to the same application. You don’t have to go to http://del.icio.us to use it.

Simple programming interfaces that encourage “do it yourself” science mash-ups. JSON, AJAX compatible REST APIs.

Widgets/gadgets that allow portal capabilities to be exported to other Web sites and start pages. Easily add stuff like Digg, YouTube, MySpaces, etc.

Google Gadgets

Netvibes Widgets

Integrate content from anywhere.

Content is under complete control of user.

Universal widget modules with supporting JavaScript libraries.

Any standalone HTML application can be converted into a widget/gadget.

Javascript expertise needed to make sophisticated widgets.

Gadgets are easily created, published and shared.

Anyone can write a gadget. Gadgets can be anywhere on

the Web Netvibes ecosystem

But you don’t have access to Netvibes or Google container source code.

Downloadable Start Page containers?

Containers only show content deployed on the portal server.

Users can only choose from portlets deployed in the container.

Portlets have Java programming API.

Requires expertise in Java web programming (JSP, JSF, Struts, etc).

Portlets must be deployed on the server that runs the portal container.

Only the portal administrator can add a new portlet

No way to share running JSR 168 portlets between portal containers.

WSRP is supposed to solve this.

Iframes are more practical But the portal administrators

have complete control over the container and content.

You download and run everything on your server.

Start Pages, Gadgets Portals, Portlets

Writing simple gadgets and widgets

Google and Netvibes examples

Writing a Google GadgetWe first create a simple XML description for

the gadget and place in a URL. For example, content of the gadget

descriptor located will be located at http://hostname:8080/gridsphere/ogcegadget.html

Content of this URL is on the next slide.

Gadget XML DescriptionContent of ogcegadget.html is

<?xml version="1.0" encoding="UTF-8" ?> <Module> <ModulePrefs title="OGCE

Portal" /> <Content type="url" href="http://hostname:8080/gridsphere/ogce1.html" />

</Module>

Add it to your iGoogle page in the usual fashion (click “Add Stuff”).

Gadget shows up in your layout.

Writing a Netvibes WidgetBasic steps are as follows:

Write an HTML page.Decorate with Netvibes metatags (required).Use Netvibes CSS style sheets (optional)Use Netvibes JavaScript libraries (optional)Add to your Netvibes content.

<?xml version="1.0" encoding="utf-8"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"> <head>

<meta name="author" content="Huapeng Yuan" /> <meta name="description" content="A Netvibes Widget for OGCE" /> <meta name="apiVersion" content="1.0" /> <meta name="inline" content="true" /> <meta name="debugMode" content="false" /> <link rel="stylesheet" type="text/css" href="http://www.netvibes.com/themes/uwa/style.css" />  <script type="text/javascript" src="http://www.netvibes.com/js/UWA/load.js.php?env=Standalone"> </script> 

<title>OGCE Portal</title> <link rel="icon" type="image/png" href="http://www.netvibes.com/favicon.ico" />

</head> <!—Begin HTML web form --> <body></body>

Add required Netvibes meta tags.

Use Netvibes style sheets.

Import Netvibes JavaScript

OGCE login widget placed in Netvibesstart page.

The widget is also accessible in standalone mode via is URL.

After login, the user is redirected to the OGCE portal site.

Building Better Netvibes WidgetsThe previous example was for a minimal widget.

Only HTMLNetvibes JavaScript libraries let you do the

following:“widget” object: replaces JS document and window.

Sandboxes the widget from other widgets in the same page.

“el” object: allows you to manipulate HTML in the widget.

AJAX utility methods: UWA.Data.{getFeed(), getXML(), getJson(), getText(), request()}

You need these to process results inline (i.e. not redirect away to a new page as a result of a form action).

Google Gadgets & WidgetsVisually appealing mini-applications that work with Google

Homepage, Google Desktop, or any page on the webE.g. Calendar, Mail Checker, NASA TV, Stock Prices,

Weather Globes, Free SMS etc.Enable a wide variety of visual effects and animationSupports rich markup languages such as Flash, XMLTwo Types of Google Gadgets

Universal Gadgets Desktop Gadgets

Date & Time (Universal Gadget) Google Gadget in Google Desktop (undocked and

docked view

Universal Gadgets Desktop Gadgets

Work in Google Desktop, Google Page Creator, any Web Page, Google Personalized Page, BloggerEasy to Create - No downloads necessary, No libraries to learn, No Web server required

Works only with Google Desktop

Features Supported

Anything that can be done on a WebPageHTML FormsEasy integration with Google Maps/ Web SearchActiveX

Client Win32 LibrariesMulti-user support through Google Talk e.g. Interactive GamesFree form shapes and transparenciesSearch files, emails etc across computersEasy to use standard UI elementsReact to user actions outside a gadgets

Create without Download

Yes No

Offline Availability

No Yes

Languages Supported

HTML, JavaScript, generated HTML (eg PHP, Java, Perl, ASP)

JavaScript, C, C++, C#, and/or VB.Net

"Google Code - Google Gadgets." Google Code. Google Inc.. 17 May 2007 <http://code.google.com/apis/gadgets/>.

Creating Google GadgetsGoogle Desktop SDK is used for creating

Desktop Gadgets using Google Desktop Gadget API

Google Gadgets API is used to develop Universal Google Gadgets

How to Make a Desktop GadgetDownload the Google Desktop SDK and create the following filesgadget.gmanifest files specifies (required)

Metadata XML file describing Gadget Components needed to be installed before gadget creation

<gadget> element with minimumGoogleDesktopVersion parameter<about> element (required)<install> element (optional)

main.xml file (required) Defines the <view> element to specify the overall appearance of the pane

main.js file to write the code for gadget functionality (required) To handle the event

options.xml file (optional) to add options view (optional) Strings. xml file

Contains all language-specific strings that will be visible in your gadget's UI

Contains variable assignments for the strings in a particular language.

gadget.gmanifest fileThe <about> element can include these sub-

elements: <id> : The gadget's CLSID. <name> : Your gadget's name. (Required) <description> : A short description of what the gadget does. (Required) <version> : The gadget's version number. <author> : Who wrote the gadget. <authorEmail> : An email address for contacting the developer. <authorWebsite> : A URL for your website. <copyright> : A copyright notice. <aboutText> : Text displayed in the gadget's About dialog. <smallIcon> : A 24x24 pixels icon shown in your gadget's title bar. <icon> : A 32x32 pixels icon shown in the gadget's About dialog and in the Alerts UI <eventAPI> : the gadget can use the Google Desktop Event API. <displayAPI usesNotifier="true" /> : When set to true, enables Sidebar UI

notifications. <queryAPI allowModifyIndex="true" /> : When set to true, allows the gadget to use

the Query API and index notifications.

main.xml file (example)Specified that the gadget has main.js as the

scripting fileDefines 250 X 150 pixel view for the labelSet the horizontal and vertical positionSet the alignment, size of text, width, horizontal

and vertical pin of the label “HELLO_WORLD” (present in strings.xml file)

Retrieves the name “iulabs” for the labelEnable the element to fire mouse/keyboard eventsCalls “onTextClick() function defined in main.js file

Main.js file (example)Specify the code to spin the text clockwiseTaking 1500 milliseconds between 0 – 360

degreesIt also displays the caption

“GADGET_COPYRIGHT” in the expanded view

Google Gadget in Google Desktop (undocked and docked view

Strings.xml file (example)Specify the metadata about the stringHere “HELLO_WORLD” element contains the

string “IU Research” which is displayed when the gadget is run in Google Desktop

I want to say just one word to you. Just one word. Are you listening? Microformats.

MicroformatsMicroformats, like Ajax, follow a technical

methodology.Not a new technology.

Main Idea: Use existing XHTML extension mechanisms to create tag sets the represent data objects.<div> and <span> tags

Use community conventions to drive adoption of specific formats.

Stay away from hard problems that have bogged down the Semantic Web.Logics, inference, and artificial intelligence.

An Example: An Earthquake FaultNote this is just

HTML.Note we have used

custom tags to encode data meaning and not data formatting.

CSS style sheets would be used to format the display.

<div class=”earthquake.fault”>

<div class=”faultName”>

Northridge

</div>

<div class=”latStart>…</div>

<div class=”lonStart>…</div>

<div class=”latEnd”>…</div>

<div class=”lonEnd”>…</div>

<div class=”strike”></div>

<div class=”dip”>…</div>

</div>

Other ExamplesPeople:

hCard: the Microformat version of the IETF standard vCard.

Dates: hCalendar: the Microformat version of IETF

standard iCalendar.Both of these are characterized by sets of

name/value pair conventionsDublin Core publication metadata is

another potential application.Unlike Semantic Web, no way to express

inter-format relationships.

vCard-->hCardBEGIN:VCARDVERSION:3.0N:Çelik;TantekFN:Tantek ÇelikURL:http://

tantek.com/ORG:TechnoratiEND:VCARD

<div class="vcard"> <a class="url fn"

href="http://tantek.com/">Tantek Çelik

</a> <div class="org">

Technorati </div></div>

http://microformats.org/wiki/hcard

Should You Care?Microformats’ value really only appears when

they are adopted as regular conventions.JavaScript libraries and related browser tool support.

Both Firefox 3 and IE 8 may support some standard microformats.Ex: automatically detect hCard and hCalendar.Allow these to be exported to your favorite address book

and calendar application automatically.Our fault microformat may be associated with a Google

Map or an HTML form interface generator, for example.Nifty hcard/hcalendar demo:

http://spaces.live.com/editorial/rayozzie/demo/liveclip/liveclipsample/clipboardexample.html

View source when you are done playing.

More InformationSlides and a related document will be

available fromhttp://grids.ucs.indiana.edu/ptliupages/

presentationshttp://grids.ucs.indiana.edu/ptliupages/

publicationsContact: mpierce@cs.indiana.eduBlog: http://communitygrids.blogspot.com