04 October 2007Kaiser: COMS W4156 Fall 20071 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser [email protected]

04 October 2007 Kaiser: COMS W4156 Fall 2007 1

COMS W4156: Advanced Software Engineering

Prof. Gail Kaiser

[email protected]

http://york.cs.columbia.edu/classes/cs4156/

mailto:[email protected]



Reprise: Web Services

• Web Services = distributed applications, services and components, described using XML-encoded WSDL interfaces, that process XML-encoded SOAP messages

• XML, SOAP and WSDL constitute baseline specifications that provide a foundation for application integration


But…

• Additional standards beyond this baseline become necessary as WS applications become more complex, integrating multiple components across multiple organizations

• Otherwise, WS developers are compelled to implement higher-level functionality in proprietary and non-interoperable ways

• Solution?: WS-* Set of Specifications


Example: Department Store Chain Enterprise

Application Integration• Background: The chain discovered that different

credit approval applications had been developed in various parts of the company.

• Solution: The chain exposed one credit approval application as a Web Service. They linked this to their point-of-sale, warehouse and financial applications.


Example: Department Store Chain Enterprise

Application Integration• Business Benefits: The chain was able to use

the same credit approval application with the three distinct applications. As a result:

• Credit approvals became more consistent • Maintenance costs decreased


Tier 1 -- Enterprise Application Integration

• Companies initially use Web Services to integrate internal applications

• Web Services allow them to expose legacy applications in heterogeneous environments without having to rewrite significant amounts of code


Example: Car Rental Company Interoperability

with Key Partner• Background: A major airline approached the

car rental company about putting a link to the car reservation system on the airline’s website. Linking the two proprietary reservation systems presented an extreme challenge.

• Solution: The car rental company created a translation engine for sending data between the two systems.


Example: Car Rental Company Interoperability

with Key Partner• Business Benefits: • Car rental company developed another large

sales channel

• Solution got to market quickly


Tier 2 -- Interoperability with Key Partners

• The next step is to integrate one or two key partners outside the company

• Web Services allow for interoperability between applications across the public Internet

• But companies must agree upon the technologies they will use to develop these interoperating Web Services


Example: Insurance Company Interoperability

across Several Companies• Background: A large insurer needed to generate

quotes for dental coverage and make them available on the intranet of one of their large corporate customers. But it had outsourced the maintenance of the dental providers directory and the credit rating service.

• Solution: The insurance company, credit rating service, and dental provider orchestrated these applications to generate a quote that was requested by the customer on a corporate intranet.


Example: Insurance Company Interoperability

across Several Companies• Business Benefits: The insurance company

considered this a transformational competitive advantage for the following reasons:

• It generated quotes in half the time of its competitors and provided them via a corporate intranet to one of its major customers.

• It automated existing business relationships at the level of multiple, interoperating applications. As a result, outsourcing became much more valuable, cutting the cost of quote generation by one third.


Tier 3 -- Interoperability across Multiple Companies

Companies want to extend their computing out to more partners and customers to build business ecosystems


Business Ecosystem Requirements: Security

• The most common concern for companies implementing Web Services solutions

• Developers need an end-to-end security architecture that is straightforward to implement across companies and trust boundaries


Business Ecosystem Requirements: Addressing

• Companies building Web Services solutions are concerned about the scalability and fault-tolerance of the business ecosystems they are building

• Developers need a way of specifying messaging paths and the ability to configure those message paths dynamically


Business Ecosystem Requirements:

Reliable Messaging• A key requirement for mission-critical

applications

• Developers need an end-to-end guarantee of message delivery across a range of semantics such as: – at-least-once– at-most-once– exactly once


Business Ecosystem Requirements: Coordination

• Some applications require database-like transactions across companies

• ACID properties – atomic, consistent, isolated, durable

• But because of the nature of decentralized computing, developers need flexible compensation-based transaction schemes


Building Business Ecosystems

• The basic Web Services specifications enable interoperability between software components developed by different companies and residing on different infrastructures

• But most component model frameworks for use within an enterprise support security, reliability, transactions, etc.

• How can we add those capabilities to WS?


Composable Services

• By adding more specialized Web Service specifications that are independent but can be combined

• For example, it is possible to independently add transaction identifiers and reliable messaging sequence numbers

• The two extensions do not conflict with each other and are compatible with pre-existing message structures

• Developers and providers can integrate selected specifications that fulfill the requirements of their communicating processes



SOAP Inherently Supports Composition

• SOAP uses a regular, multi-part message structure: New message elements supporting new services may be added to message headers in a manner that does not alter the processing of existing functionality

• SOAP body is for the ultimate recipient, SOAP header blocks may be targeted at any entity along the message path





Transports

• HTTP, HTTPS, SMTP (Simple Mail Transport Protocol)

• Core communication mechanisms • Move blocks of "bytes" between Web Services• This is only useful if participants can convert the

bytes into data structures that their code processes


Messaging

• XML, SOAP• XML and XML Schema Definition (XSD) provide

the mechanisms for abstractly agreeing on message [data] structures

• SOAP defines the standard encoding for representing XML messages in the byte information that Web Services exchange over transports


Addressing

• Messages and responses both go somewhere and come from somewhere (and errors also need to be reported somewhere)

• By default, SOAP encodes the destination for a message with a URL placed in the HTTP transport

• The destination for the response is determined by the HTTP return address

• Builds on the basic browser-server model


Addressing

• The source and destination information are not part of the message itself

• But information can be lost if a transport connection terminates (e.g., if the response takes a long time and the connection times out)

• Or if the message is forwarded by an intermediary, perhaps routed over multiple transports

• Also does not allow for directing a response to a third party (e.g., request sent over HTTP but returned via SMTP)


WS-Addressing• Provides a mechanism to place the target, source

and other addressing information directly within the message

• Decouples address information from any specific transport model

• Supports both asynchronous and extended duration communication patterns

• But not very exciting for request/response over a single HTTP connection (see blog entry)

http://www.pacificspirit.com/blog/2004/12/20/ruminations_on_wsaddressing_and_transfer_protocols


Message Addressing Properties• To -- message destination, required (URI)• Action -- an action value indicating the semantics of the

message, corresponds to WSDL porttype, required (URI)

• From -- the endpoint of the service that dispatched this message (EPR)

• ReplyTo -- the endpoint to which reply messages should be dispatched (EPR)

• FaultTo -- the endpoint to which fault messages should be dispatched (EPR)

• Unique MessageId, required if there will be any response (URI)

• RelatesTo previous messages (a pair of URIs, indicating previous From and MessageId)


Example with Simple Endpoint References

<S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://www.w3.org/2004/12/addressing"> <S:Header> <wsa:MessageID> http://example.com/SomeUniqueMessageIdString </wsa:MessageID> <wsa:ReplyTo>

<wsa:Address>http://myClient.example.com/someClientUser</wsa:Address> </wsa:ReplyTo> <wsa:FaultTo>

<wsa:Address>http://myserver.example.com/DemoErrorHandler</wsa:Address> </wsa:FaultTo> <wsa:To>mailto:[email protected]</wsa:To> <wsa:Action>http://myserver.example.com/DoSomething</wsa:Action> </S:Header> <S:Body> … </S:Body></S:Envelope>


WS-Addressing Endpoint Reference

• A standard XML element providing a structured approach to encoding fine-grained addressing

• Only the address is required• May include reference properties that distinguish

between multiple services (or multiple versions of the same service) at the same address

• May include reference parameters that identify resources managed by a particular service


Example Extended Endpoint Reference

<wsa:EndpointReference xmlns:wsa="..." xmlns:example="...">

<wsa:Address>http://example.com/weather</wsa:Address>

<wsa:ReferenceProperties>

<example:ServiceLevel>Basic</example:ServiceLevel> </wsa:ReferenceProperties> <wsa:ReferenceParameters> <example:CityCode>NYC</example:CityCode> </wsa:ReferenceParameters></wsa:EndpointReference>



Description

• The transport and message specs allow Web services to communicate using messages

• But how do participants know what the messages are?

• How does a Web Service describe the messages it sends and receives?

• Using a Web Service requires an understanding of the messages the Web Service will consume and produce


XSD and WSDL

• XML Schema allows developers and service providers to define XML types for data structures, e.g., a purchase order, and messages, e.g., the CreatePO message

• WSDL allows a Web Service to document the messages it receives and sends - what "actions" or "functions" the service performs in terms of the messages it receives and sends


Requirements and Expectations between

Requesters and Receivers• WSDL and XSD define the service's interface

syntax, but do not express information about what the service requires/expects of the caller and vice versa

• For example, does the service require security or implement transactions?

• WS-Policy enables a service to specify the functional assurances that they expect from and provide to callers (and intermediaries)


WS-Policy• Extensible language for expressing the

requirements, capabilities and preferences of a service

• Assertions represent an individual preference, requirement or capability

• Usage: Required, Optional, Rejected, Observed, Ignored

• Operators: All, ExactlyOne, OneOrMore• Applied to domain-specific policy subjects (e.g.,

from WS-SecurityPolicy)


Example<wsp:Policy xmlns:wsse="..." xmlns:wsp="...">

<wsp:ExactlyOne> <wsse:SecurityToken wsp:Usage="wsp:Required"

wsp:Preference="100"> <wsse:TokenType>wsse:Kerberosv5TGT</wsse:TokenType> </wsse:SecurityToken> <wsse:SecurityToken wsp:Usage="wsp:Required"

wsp:Preference="1"> <wsse:TokenType>wsse:X509v3</wsse:TokenType>

</wsse:SecurityToken></wsp:ExactlyOne>

</wsp:Policy>


Obtaining Descriptions

• XML, XSD, WSDL and WS-Policy support describing the interface and service assurances for a Web Service

• But how do potential users of the service find this information?

• Currently the most common approach is through email exchanges, manual (human) online lookup, or word of mouth

• A more general purpose, scalable model is necessary – although has not (yet) proven practical outside an intranet setting


UDDI

• UDDI = Universal Description and Discovery Interface

• Query UDDI at design time, to find services compatible with requirements

• Query UDDI at runtime, when the caller "knows" the interface it requires and searches for a service that meets its functional requirements or is provided by a well-known partner


WS-MetadataExchange

• The caller service may go directly to the callee service to obtain information via SOAP messages following WS-MetadataExchange specification

• Used when developers already have a reference to a service and need to obtain details

• Obtains relevant WSDL, XSD, WS-Policy, etc.



Requirements and Expectations Again (Assurances)

• It is not enough to simply exchange messages• Applications and services reside in middleware

and systems that provide valuable component services such as security, reliable messaging and transacted operations

• Web Services need a mechanism for interoperability between these facilities


Security

• Support authentication, message integrity, confidentiality, trust, privacy

• Federation of security between different organizations


Security Requirements

• A sends a message to service B• B partially processes the message and forwards

it to service C• HTTPS allows authentication, integrity, and

confidentiality between A-B and B-C• However, C and A cannot authenticate each

other, or hide information from B • For A, B and C to use BASIC-Auth for

authentication, they must share the same replicated user and password information


WS-Security

• Defines mechanisms for associating security related claims with a message

• Signed, encrypted security tokens– Username/password– x509 certificates– Kerberos tickets– XML-based tokens: XrML eXtensible rights Markup

Language) and SAML (Security Assertion Markup Language)


WS-Security

• A can generate a token that C can verify as having come from A, B cannot forge the token

• A can sign selected elements or the entire message, this allows B and C to confirm that the message has not changed since A sent it

• A can seal the message or selected elements, this ensures that only the intended service for those elements can use the information and his prevents B from seeing information intended for C and vice versa


Trust

• Security relies on pre-defined trust relationships• Kerberos works because participants "trust" the

Kerberos Key Distribution Center• PKI (public key infrastructure) works because

participants trust the root certificate authorities


WS-Trust

• Defines an extensible model for setting up and verifying trust relationships

• Allows Web Services to set up and agree on which security servers they "trust" and to rely on these servers

• The key concept is a Security Token Service (STS) - a distinguished Web Service that issues, validates and exchanges security tokens

• An STS might issue a Kerberos token asserting that the key holder is Susan, based on an X.509 certificate issued by a trusted Certificate Authority

• Enables organizations using different security technologies to federate


STS


Secure Conversations

• Some Web Service scenarios only involve the short sporadic exchange of a few messages – which is what WS-Security was intended for

• Other scenarios involve long duration, multi-message conversations between the Web Services

• WS-Security may not be good enough for:– Repeated use of computationally expensive

cryptographic operations such as public key validation– Sending and receiving many messages using the

same cryptographic keys, providing more information that allows brute force attacks to "break the code"


Secure Conversations

• Protocols like HTTPS use public keys to perform a simple negotiation that defines conversation specific keys

• This key exchange allows more efficient security implementations and also decreases the amount of information encrypted with a specific set of keys


WS-SecureConversation

• WS-SecureConversation provides similar support to WS-Security

• Participants may use WS-Security with public keys to start a "conversation" or "session“, and use WS-SecureConversation to agree on session specific keys for signing and encrypting information


Federation

• Sometimes a set of organizations need to establish a single virtual security domain

• For example, a travel agent, an airline and a hotel chain may set up such a federation

• An end-user that "logs into" any member of the federation has effectively logged into all of the members


WS-Federation

• WS-Federation defines several models for providing federated security through WS-Trust and WS-SecureConversation

• Customers often have "properties" when they deal with an enterprise

• An example is a preference for window or aisle seats, or a midsize car

• WS-Federation allows the members to set up a federated property space


WS-Federation

• Properties about individuals may be closely held for privacy protection or because the information provides a competitive advantage to a specific federation member

• WS-Federation supports a pseudonym model• For example, users that have authenticated to

the travel agency have agency-generated "aliases" in their interactions with the airline or hotel



Reliable Messaging

• In an Internet world, almost all communication channels are unreliable - messages disappear or are duplicated, connections break

• Without a reliable messaging standard, Web Service application developers must build these functions into their applications

• The basic approaches and techniques are well understood, e.g., many middleware systems ensure messages have unique identifiers, provide sequence numbers, and use retransmission when messages are lost

• If Web Service developers implement these models in their applications, they may make different assumptions or design choices, resulting in little if any reliable messaging


WS-ReliableMessaging

• Defines mechanisms that enable Web Services to ensure delivery of messages over unreliable communication networks

• Supports bridging two different infrastructures into a single, logically complete, end-to-end model



• The RM Source MUST assign each reliable message a sequence number beginning at 1 and increasing by exactly 1 for each subsequent reliable message

• Every acknowledgement issued by the RM Destination MUST include within an acknowledgement range or ranges the sequence number of every message successfully received by the RM Destination and MUST exclude sequence numbers of any messages not yet received



• Delivery Assurances – AtMostOnce, AtLeastOnce, ExactlyOnce, InOrder

• Protocol Elements – Sequence, Sequence Acknowledgement, Request Acknowledgement, Sequence Creation, Sequence Termination

• Policy Assertions – SequenceCreation, SequenceExpiration, InactivityTimeout, RetransmissionInterval, AcknowledgementInterval




Transactions• A complex business scenario may require

multiple parties to exchange multiple sets of messages

• The multiple messages exchanged between participants constitute a logical "task" or "objective"

• The parties must be able to: – Start new coordinated tasks. – Associate operations with their logical task - the

parties may be performing multiple such tasks at the same time

– Agree on the outcome of the computation


WS-Coordination

• General mechanism for starting and agreeing on the outcome of multiparty, multi-message Web service tasks

• Coordination context is a message element that flows on all messages that Web Services exchange during the computation

• The coordination context contains the WS-Addressing endpoint reference to the coordination service and the endpoint contains information to identify the specific task being coordinated


Coordination Service

• Starts a coordinated task, terminates a coordinated task, allows a participant to register in a task, and produces a coordination context that is part of all messages within a group

• Includes an interface that participating services use in order to be informed of the outcome of the coordinated task



WS-AtomicTransaction

• Defines a specific set of protocols that plug into WS-Coordination to implement traditional two-phase atomic transactions

• For activities that require the traditional atomic, consistent, isolated, and durable (ACID) properties

• Usually short-lived


WS-AtomicTransaction

• Atomic, two-phase model is only with respect to the services involved

• Sites or infrastructure offering services may advertise two-phase commit, but use some other intra-enterprise model like compensation or versioning

• This makes a simple two-phase commit model more useful for long-running Internet computations


Business Activities

• May consume many resources over a long duration

• May involve a significant number of atomic transactions

• Individual tasks within a business activity can be “seen” prior to the completion of the business activity - their results may have an impact outside of the computer system

• Responding to a request may take a very long time - human approval, assembly, manufacturing or delivery may have to take place before a response can be sent


Business Activities

• In the case where a business exception requires an activity to be logically undone, abort is typically impractical/impossible

• Exception handling mechanisms may require business logic, e.g., in the form of a compensation task, to reverse the effects of a completed business task


WS-BusinessActivity• Another set of protocols that plug into WS-

Coordination, to coordinate activities that apply business logic to handle business exceptions

• Actions are applied immediately and are permanent

• Compensating actions may be invoked in the event of an error

• Enables existing business process and work flow systems to wrap their proprietary mechanisms and interoperate across trust boundaries and different vendor implementations



BPEL4WS = Business Process Execution Language for Web Services

• Supports service composition• Enables developers to define the structure and

behavior of a set of Web Services that together implement a shared business solution

• The composed solution is itself a Web Service, which supports HTTP/SOAP messages and defines its interface using WSDL (and WS-Policy)


BPEL4WS Processes

• Executable processes model actual behavior of each participant in a business interaction

• Abstract processes specify the mutually visible message exchange behavior of each of the parties involved in the protocol, without revealing their internal behavior

• And much much more…• Basically an entire programming system where

conventional Web Services are the atomic functions



Revised Project ConceptDue Next Week!

• Tuesday 9 October, 10am

• Assignments posted on course website

• Submit via CourseWorks

• Revised Project Concept

http://york.cs.columbia.edu/classes/cs4156/assignments.htm

https://courseworks.columbia.edu/

http://york.cs.columbia.edu/classes/cs4156/homeworks/revisedconcept.htm


Upcoming Deadlines

• First iteration plan due October 16th • First iteration progress report due October 23rd • First iteration demo week October 30th –

November 8th • First iteration final report due November 9th

http://york.cs.columbia.edu/classes/cs4156/homeworks/firstplan.htm


COMS W4156: Advanced Software Engineering

Prof. Gail Kaiser

[email protected]


mailto:[email protected]


Documents

04 October 2007Kaiser: COMS W4156 Fall 20071 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser [email protected]