WS-Interoperability and RESTful Web Services

Mag. Philipp Leitner Distributed Systems Group

Institute of Information Systems http://www.infosys.tuwien.ac.at/Staff/leitner

www.VitaLab.tuwien.ac.at

Overview

!  WS-Interoperability

!  HTTP and RESTful Web Services

!  Mashups

2

WS-Interoperability

Web Specifications

!  Web (service) specifications

!  Are a (the?) central element of Web services !  You already know a few of them …

!  SOAP !  WSDL !  WS-CDL !  WS-BPEL !  WS-AtomicTransactions !  WS-RM

!  And you will get to know more …

!  UDDI / WS-MEX 4

WS-Reliability

WS-Transaction

Web Specifications

!  However, there’s (still) more …

5

WS-Federation WS-SecureConversation WS-SecurityPolicy

WS-Policy

WS-Notification ebXML

WSDM

WS-BaseNotification

WS-Eventing

WSLA

WS-Agreement

WSRF

WS-Discovery

WS-Coordination

WS-ReliableMessaging

Web Standards

!  Many of these standards are: !  Redundant (but incompatible) !  Sub- or supersets of each other !  Poorly developed !  Seldom or never used !  Not implemented !  Company-specific

!  But it gets worse: !  Even the better specifications (SOAP, WSDL) have

many facets / variations / uncertainties

!  " Design by Commitee

6

Case in Point (1)

7

!  WSLA (Web Service Level Agreements)

!  Currently exactly one implementation !  http://www.research.ibm.com/wsla/implementation.html

!  Java prototype of IBM Research (as part of the Emerging Technologies Toolkit)

!  But also more or less discontinued

!  Strong overlap with WS-Agreement

!  WS-Agreement seen as the successor by some

!  No known practical applications

Case in Point (2)

8

!  RPC/encoded

!  RPC/literal!

!  document/encoded

!  document/literal!

<myMethod>! <x xsi:type="xsd:int">5</x> ! <y xsi:type="xsd:float">5.0</y> ! </myMethod>!<myMethod> ! <x>5</x> ! <y>5.0</y> !</myMethod>!

<xElement>5</xElement> <yElement>5.0</yElement>!

WS-I

!  Web Services Interoperability !  ‘Meta’ Web standards !  Built on top of real-world experiences !  As of July 2010 " Part of OASIS

!  “Organization for the Advancement of Structured Information Standards”

!  Main goals:

!  Clarifying ambiguities in existing standards !  Define best practices !  ‘Ban’ certain aspects which are known to cause troubles

9

WS-I Basic Profile

!  WS-I Basic Profile (BP):

!  Main outcome of WS-I !  Small subset of WS features, which are expected to

be interoperable among any (compliant) platform and WS middleware

!  Current version: 1.1 !  SOAP 1.1, WSDL 1.1, UDDIv2

!  Current draft (but quite stable): 1.2 !  MTOM, WS-Addressing

!  Work-in-progress: 2.0 !  SOAP 1.2, WSDL 2, UDDIv3

10

WS-I Basic Profile

!  WS-I BP 1.1

!  Provides >200 concrete restrictions / extensions in usage of Web standards

!  Basically a checklist for middleware providers

!  Most important: !  ‘Ban’ of */encoded !  Definition of array types !  Usage of SOAP intermediaries is specified in more detail !  Details of HTTP binding are specified:

-  POST method has to be used -  Receivers cannot rely on a SOAPAction HTTP header -  Status codes are defined

!  Details of WSDL/UDDI integration are specified

14

WS-I BP Example

14

WS-I Basic Security Profile

!  WS-I BSP 1.0

!  Similar to WS-BP !  But focuses on interoperability of Security solutions !  Defines mostly a subset of WS-Security and XML-

Signature !  > 100 rules !  Some of which can be checked statically, others need

runtime validation, e.g.

!  Ciphers which can/cannot be used … !  Suggested minimal key lengths !  Defines handling of XML signatures

14

HTTP and RESTFul Web Services

15

Overview

REST … is an acronym for Representational State Transfer

… is an architectural style for distributed systems

No Communication protocol No Web services standard No deployable technology

… is the basic architecture of the WWW

That means: REST is the foundation of the largest deployed Information System today !

16

Goals

!  Many design goals

!  But among the most important are !  Scalability !  Robustness !  Simplicity of interfaces !  Independence of components

!  Other goals are subordinated !  e.g., ‚Enterprise‘ features (transactions, …)

17

Main Characteristics

!  Main characteristics:

!  Resource-Orientation !  Key elements of any RESTful service are resources !  Activities are not explicitly modelled

!  Statelessness !  Every request is self-contained

!  Uniform Interface !  Every resource is accessed through the same interface

!  Naming !  Every resource is associated with an unique and descriptive name

!  Layering !  Intermediaries can be inserted transparently !  Proxies, caches, …

18

HATEOAS

!  HATEOAS

!  ‚Hypermedia as the Engine of Application State‘ !  Often seen as the defining principle of RESTFul

systems

“ (…) the principle is that the hypermedia in each server response will contain links that correspond to all the actions that the client can currently perform. (…) every server response describes the new actions that are available. (…) A client of a RESTful application need only know a single fixed URL to access it. All future actions should be discoverable dynamically from hypermedia links included in the representations of the resources that are returned from that URL. (…) ” [Wikipedia]

19

Implementation

!  Main Protocols:

!  RFC 2068 / 2616 !  HTTP !  Communication protocol !  Today mostly RFC 2616 (HTTP 1.1)

!  RFC 2396 !  URI !  Resource identifiers

!  RFC 2045, 2046, 2047, 2049, 4288, 4289 !  MIME !  Definition of Representation

20

Recap: HTTP Basics

!  Stateless, text-based protocol !  Used to transmit resources (representations)

over a network

!  Request/Response Structure: GET /index.html HTTP/1.1!Host: localhost!

HTTP/1.1 200 OK!Date: Tue, 02 Nov 2010 14:38:00 CET!Content-Type: text/html!

<html>!...............!

Recap: HTTP Interface

21

Verb Safe Idemp. Body Description

GET Yes Yes No GET a resource identified by an URL

PUT No Yes Yes UPDATE a resource identified by a client-specified URL

POST No No Yes CREATE a new resource

DELETE No Yes No DELETE a resource identified by a URL

See: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

22

Recap: HTTP Interface

!  Safe: !  Intended purely for retrieving data (no changes on

server side) !  HTTP GET

!  Idempotent: !  Many identical requests should have the same effect

as a single request !  Safe implies Idempotent !  HTTP DELETE, HTTP PUT, (HTTP GET)

!  Body: !  In principle, every operation is allowed to have a body !  But: in practice, bodies are only used in PUT and

POST

23

Recap: HTTP Status Codes Code Name Description

1xx Informational Used to indicate that something has happened triggering a response, but the request is not yet completed, e.g., protocol switch

2xx Success Used to indicate that all is good and the respective request was handled

3xx Redirect Further action is needed to finish the request, e.g., because of a server move the request needs to be sent to a different URL

4xx Client Error Used to indicate that there is a problem with the request, e.g., URL not found, or forbidden

5xx Server Error Used to indicate that there is a problem on server side, e.g., a bug

24

... And now back to REST

!  Characteristics revisited:

!  Resource-Orientation !  Resources are represented in a MIME type !  e.g., text/html, text/plain, image/jpeg, …

!  Statelessness !  HTTP is a stateless protocol

!  Uniform Interface !  HTTP interface !  GET / POST / PUT / HEAD / DELETE

!  Naming !  Every resource is identified by a URI !  URIs should be descriptive !  e.g., http://en.wikipedia.com/wiki/REST

!  Layering !  HTTP works over caches, proxies, gateways, routers…

25

From REST to RESTful Web Services

“The problem is, most of today‘s web services have nothing to do with the Web. (…) Today‘s web service architectures

reinvent or ignore every feature that makes the Web successful.“

[RESTFul Web Services]

‚Simple‘ solution: !  Create Web services that are in line with

the Web !  Create Web services according to the

REST principles

26

RESTful Web Service Properties

!  REST is just a way of building services !  No concrete protocol !  No toolbox !  RESTful services employ usually ‚ad hoc‘ protocols

!  Resources are important !  Main focus is on CRUD

!  Addressing is important !  Every logical or physical resource needs an URI !  HATEOAS

!  Statelessness is important !  No transactions along >1 service invocations !  Ordering of methods is transparent

27

RESTful Web Service Technology

!  HTTP + URI + MIME !  Every RESTful service uses HTTP !  Plain TCP / SMTP / … are not supported

!  Unlike SOAP-based Web services

!  XML !  Prevalently used for resource representation !  Other possibilities:

!  JSON (in conjunction with AJAX) !  HTML !  Plain text !  …

28

Alternative Encodings - JSON

!  JSON – JavaScript Object Notation

!  Lightweight data interchange format !  Easy to read and write for humans !  Easy to parse and generate for machines !  Basically, serialized JavaScript objects

!  Built on two basic structures

!  Name/Value pairs !  Ordered lists of values

29

Alternative Encodings - JSON

{! “firstname“:“John“,! “lastname“:“Smith“,! “address“: {! “street“:“21th Street“,! “city“:“Vienna“! },! “phoneNumbers“: [! “+43664/337652“,! “+43664/332876“! ]!} !

30

Alternative Encodings – JSON vs. XML

!  Comparing JSON vs. XML:

!  JSON may have smaller footprint (no closing tags) !  JSON is easier to read to some !  JSON is better suited to handle binary data

!  XML has more features (for instance, attributes) !  XML has explicit support for reference types (XLink) !  XML Schema, XSLT, XQuery, RelaxNG, …

31

Alternative Encodings – Badgerfish

!  In practice, it is often necessary to translate between JSON and XML !  E.g.: Apache CXF uses a conversion chain

JAXB " XML " JSON to serialize objects into JSON

!  Some conversion rules are necessary !  Most popular: Badgerfish

!  Note that the transformation is not necessarily equivalent (some information may be lost) !  Attributes ? !  Namespaces ?

32

Alternative Encodings – Badgerfish

XML Example JSON Example

Text bob Special $ property

“$” : “bob”

Elements <name>bob</name> <add>vienna</add>

Properties “name” : { “$” : “bob”}, “add” : { “$” : “vienna”}

Attributes <name n=“bob” /> Nested properties starting with @

“name” : { “@n” : “bob”}

Namespaces <alice xmlns= “http://test.com” />

Special @xmlns property

“alice” : { “@xmlns” : “http://test.com” }

33

REST Interface Description

!  How can one describe RESTful services?

!  Easy answer: !  It‘s not necessary – REST services use the HTTP interface

and subsequent URIs are discoverable (HATEOAS) !  Often argued by REST purists

!  More realistic answer: !  No established machine interpretable format available

!  Possibilities: -  WSDL 2.0 -  WADL

34

WSDL for REST Services

!  Possible starting with WSDL 2.0 !  Implemented using the HTTP Binding (instead of

SOAP Binding) of WSDL 2.0 !  Advantages:

!  Allows for unified description of SOAP-based and RESTful Web services (e.g., done by Apache Axis2)

!  Some tool support (although not great) !  Disadvantages:

!  Limited expressiveness !  Not very RESTful (WSDL 2.0 still uses operations and

messages) !  No support for non-XML hypermedia

35

WSDL for REST Services - Example

<interface name="UserListInterface"> <operation name="getUsers" pattern="http://www.w3.org/ns/wsdl/in-out" style="http://www.w3.org/ns/wsdl/style/iri"

wsdlx:safe="true">

<input element=”GetBookList”/><!-– empty element --> <output element=”BookList” /> </operation>

</interface>

36

WADL

!  Alternative: use a description language specifically for REST

!  WADL – Web Application Description Language !  XML-based, but very different to WSDL

!  (Somewhat) simpler !  Describes resources, not operations

!  Basic constructs: !  Grammars (defines data, e.g., using XSD) !  Resources (defines resources of data) !  Methods (defines what you can do with resources) !  Representation (what format is used?)

37

WADL Example <grammars> <include href=”Users.xsd"/> </grammars>

<resources base="http://my.server.com/userservice/">

<resource path="users/{id}"> <method name="GET" id="getusers"> <request> <param name="userid" type="xsd:string” style="template" required="true" /> </request> <response status="200"> <representation mediaType="application/xml" element="Users" /> </response> </method> </resource>

</resources> 38

WADL Advantages and Disadvantages

!  Advantages: !  Simple language !  Expressive !  Conceptually close to REST specs such as JAX-RS

!  Which is to be expected since JAX-RS and WADL have the same main author

!  Disadvantages: !  Very little tool support (more or less only Sun Jersey) !  Not widely used !  Which grammar would you use for media types

without grammar language (i.e., most everything except XML?)

39

RESTful Web Service Technology – JAX-RS

!  JAX-RS (JSR-311) !  Java API for building RESTful Web services !  Pure server-side API !  POJO-based development !  Extensive use of annotations (cp. JAX-WS for SOAP)

!  Implementations: !  Jersey (Reference implementation) !  Apache CXF !  JBoss Resteasy

@Path(”/management") @ProduceMime("application/json”) @ConsumeMime("application/json”) public class MyServer {

@GET @Path(“/item/{id}”) public int getInt(@PathParam(“id) String id) {

40

RESTful Web Service Technology – JAX-RS

!  Some important annotations:

!  @GET, @POST, @DELETE, @PUT!!  The verb to use

!  @Path(“/item/{id}”)!!  The resource is identified by the URI http://<hostname>/

<resource>/item/{id} !  {id} is a placeholder for a path parameter !  E.g., http://<hostname>/<resource>/item/5 (the path parameter

is 5) !  Obviously, only simple types can be used as path parameters

!  @ConsumeMime(“application/json”)!!  The service expects the body of the request to be JSON !  Analogous: @ProduceMime!!  Obviously, @ConsumeMine can only be used with verbs that

actually have a body

41

RESTful Web Service Technology – Parameters

!  Default: parameter in body !  Expects parameter in the request body !  Any type allowed that the framework knows how to

(de-)serialize !  Only one body parameter per operation

!  Path parameters: !  /items/5!!  Only simple types, no arrays

!  Query Parameters: !  /items?id=5!!  Only simple types, arrays allowed (?id=5&id=5)

42

RESTful Web Service Technology – Parameters

!  Matrix parameters: !  /items;id=3;name=REST!!  Special type of query parameters, same rules apply

!  Form parameters: !  Maps form-encoded data to parameters

!  HTTP Header parameters: !  Maps a HTTP header to an operation parameter !  Mostly useful to get easy access to a specific header

field (e.g., content type)

!  Cookie parameters: !  Similar to header parameters for cookies

43

RESTful Web Service Technology – Serialization

!  Serialization: !  We have seen that only `simple’ types can be used

for most parameter types !  These are types that are easily marshallable to and

and from a java.lang.String !  For parameter in body, types need to be one of the

following types:

!  byte[], String, InputStream, File, Reader!!  Some other more esoteric types !  Or a JAXB-annotated type

!  These types can be serialized to XML and JSON !  For other types or other MIME formats, a custom

MessageBodyProvider needs to be implemented 44

RESTful Web Service Technology – .NET WCF

!  WCF – Windows Communication Foundation !  Contains some support for REST

!  Principles comparable to JAX-RS

!  Arguably, JAX-RS is somewhat ‘cleaner’ and more mature

[OperationContract]![WebGet( ! ResponseFormat = WebMessageFormat.Json,! UriTemplate = ”items/{id}")]!ItemEntity GetInt(int id);!

45

How ‚RESTful‘ are REST Services Today?

!  Typical answer: not very much !  Often seen misconceptions:

!  One endpoint catches all !  http://my.example.com/rest/endpoint

!  RPC semantics encoded in HTTP !  http://my.example.com/invoke/methodA?param1=1&param2=2

!  HTTP GET is used even for ‚non-safe‘ operations !  http://my.example.com/photo12345/delete

!  ‚Methods‘ are encoded as GET or POST parameters !  http://my.example.com/photo12345/?method=showMetaData

!  Authentication via API keys !  http://my.example.com/photo/12345/?key=philippleitner !  Security? Non-Reputiation?

!  URIs are often not human-readable !  http://my.example.com/photo/aaaabbb1234ksdhf76

46

Hybrid Services “(…) web services that fit somewhere in between the RESTful web

services and the purely RPC-style services. These services are often created by programmers who know a lot about real-world

web applications, but not about the theory of REST.“ [RESTFul Web Services]

!  Most real-life RESTful services !  Have some properties of REST !  But also some RPC-Like attributes !  Trade-Off: REST theory and real-world requirements !  Rarely satisfy strict HATEOAS criteria

!  Common terms: !  POX over HTTP, HTTP+POX

!  (Plain Old XML over HTTP) !  Hybrid services

47

Example 1: Amazon

http://ecs.amazonaws.com/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=[KEY]&Operation=ItemSearch&SearchIndex=PCHardware&Title=ATI&Version=2007-07-16

48

Example 2: Flickr

http://api.flickr.com/services/rest/?method=flickr.interestingness.getList&api_key=[key]&per_page=5

49

SOAP vs. REST

!  ‚Format Wars‘ are no stranger in computer science

!  ‚It‘s just Emacs vs. VI all over again…‘ !  Often purely ideological debates !  From a technical point the similarities are predominant

50

Activities vs. Resources

SOAP REST

SOAP Services Model ACTIVITIES

e.g., a bank transaction

REST Services Model DATA (HATEOAS)

e.g., a lyrics database

See: http://www-128.ibm.com/developerworks/webservices/library/ws-restvsoap/

51

Enterprise vs. Web 2.0

!  SOAP

!  Main focus: !  SOA !  EAI

!  Supports different protocol bindings

!  WS-* stack supports enterprise features

!  REST

!  Main focus: ! Web 2.0 ! Mashups

!  Simple usage, ad-hoc

!  Light-weight

!  ‚Per API‘ tool support (libraries specific for certain APIs, e.g., Amazon)

52

Generality vs. Simplicity

!  SOAP

!  Idea:

!  Have a spec for every possible use case

!  Cover everything (and introduce extensibility to cover everything else)

!  Web Services as the silver bullet of distributed computing

!  REST

!  Idea:

!  Keep the ‚standards‘ as simple and high-level as possible

!  Let engineers figure out the details on demand

!  Web services as simple Web integration technology

Mashups

54

Mashup Basics

!  Mashups originate from the music domain

!  Songs ‚blended‘ by mixing two or more songs, e.g., by overlaying the instrumental parts of one song with lyrics from the other

!  Similarly, service mashups integrate two or more services in a way that the end result is (hopefully) more than the sum of the parts

!  Example:

55

Core Properties

!  „(...) uses and combines data, presentation or functionality from two or more sources to create new services.”

!  Properties: !  Easy to build !  Easy to use !  Uses open APIs and services !  (Very often) data is used for different purposes as originally

indented

[Wikipedia]

56

Relationship to Service Composition

!  From a high level, mashups seem similar to service composition, but there are some differences:

Composition Mashup Purpose Create value-added

composite application Create value-added composite application

Paradigm Implement a business process (activity driven)

Implement data aggregation (data driven)

Atomic services SOAP Web services Data sources (often RESTFul services)

Nr. Of Users Intended for many users Often done for a limited number of users (private)

Development Process

Structured, planned activity Often ad hoc (simple)

57

Aggregation and Integration

!  Mashups often use a set of typical data sources … !  Feeds (ATOM, RSS) !  Maps (Google Maps) !  Videos (YouTube) !  Images (Flickr) !  Reviews (e.g., TripAdvisor) !  Search Results (Google)

!  ... aggregate the data … !  … and integrate the data with personalized info

!  Location (Where am I?) !  Context (What am I doing?) !  Preferences (What do I like best?)

58

Mashup Tools

!  There are some industrial tools: !  Yahoo Pipes (Web mashups) !  Intel Mashmaker (Web mashups) !  WSO2 Mashup Server (business mashups) !  IBM Mashup Center (business mashups) !  ExpressFlow (collaboration-centric, business

mashups)

!  However, the big hype is over: !  Microsoft Popfly (" discontinued) !  Google Mashup Editor (" discontinued)

59

Yahoo Pipes Example

60

Platforms

!  Mashup Platforms:

!  Allow users to share, discover and rate mashups !  Mashup registry !  Arguably even more important than actual mashup tools

!  ProgrammableWeb: http://www.programmableweb.com/ !  Currently hosts over 5300 mashups !  Quality is varying … very often proof-of-concept or toy mashups !  … but at least some interesting mashups exist (often using Twitter)

!  Example: twittering “#Ask4Stuff Mashups” triggers the Ask4Stuff Mashup, which twitters back to you some information about mashups

61

Summary

!  WS-I: improves interoperability of Web Services

!  WS-I Basic Profile !  Set of rules (‚meta-specification‘) !  Mainly clears up ambiguities in core WS specs

!  WS-I Basic Security Profile !  Does the same for interoperability of security

solutions

62

Summary

!  RESTful Web services !  Web services built according to the same principles

as the WWW (HATEOAS principles) !  Resource-centric instead of activity-centric !  Uses HTTP and various different media types

!  E.g., XML and JSON !  Interface description is not straight-forward

!  E.g., WADL !  Most RESTful Web services today are actually hybrid

Web services

63

Summary

!  Mashups

!  Are a light-weight and ad hoc service composition approach

!  Composition for RESTful Web services

!  Example tool: !  Yahoo Pipes

!  Also very important: !  Platforms, e.g., ProgrammableWeb