Delivering Process-Driven, Dynamic Applications

Welcome David RR Webber

SOA ArchitectIntegrity One Partners

Model-Driven, Open Architecture BPM:

Delivering Process-Driven,

Dynamic Applications

2 April 21-23, 2008

Renaissance Washington, DC

Overview• Change is constant - How do we make self-

adaptive, smarter implementations that are better able to cope?

• How do we leverage and align standards based mechanisms and tools within our business processes to make them compatible?

• How does this fit into modern SOA approach?

• Ability to align business conceptual details with actual implementation through use of XML templates as intermediary common shared representation.

3 April 21-23, 2008


“The devil is in the details”

• XML is multi-faceted – it can represent:– data (information exchanges)– rules– workflows and processes– forms and reports– configuration and control– semantics (metadata)– documents and contracts– multimedia and presentations

4 April 21-23, 2008


Template Driven Approach

StaticConventional

Models, Artefacts, CodeWSDL,XSD,UML,XML

Analysis

Design

Coding

TestMaintenance

SW SW DEVELOPERSDEVELOPERS

Specification

Agile DynamicComponentsTEMPLATES

Adoption, integration

Installation

Implementation/Use

Needs

Requirements

Procurement

BUSINESS USERS

BPMN modelsExcel spreadsheetXML visualizationXML artifacts

DynamicDynamic

5 April 21-23, 2008


Template Wizard runtime configuration

LibraryObject templates/Components

design time

FacetsQuestions / Data

Technology Targeting Syntax specific production rules

Partner / Component

Interface Agreement

builder + review / support /

completion workflow

11

33

44

XML22 Conceptual

Factors Stored

XMLXMLXMLXML

Solution Specific Syntax

55

6 April 21-23, 2008


• Context is the pervasive driver to effective engineering

• Drives dynamic process configuring and control

• Ensures accurate information capture, packaging and delivery

• Key to correct relationships between partners in a collaboration

Lack of context control mechanisms is the most prominent reason why legacy business systems are difficult and complex to extend and support

Date: circa 15681 : the parts of a discourse that surround a word or passage and can throw light on its meaning2 : the interrelated conditions in which something exists or occurs

Date: circa 15681 : the parts of a discourse that surround a word or passage and can throw light on its meaning2 : the interrelated conditions in which something exists or occurs

Understanding about Context

7 April 21-23, 2008


Context, role and use

• Context, context and context– Interoperability absolutely requires the software to

understand and apply context from the ground up in a consistent way that links to the external world functional needs – and that is not hard-coded into rigid software methods – but exposed across the information space

• Role and context– Role is a convenient way to group together context and

manage that for classes of functional users• Use based solutions

– This determines how and what results are expected given role and context that allows users to perform their business use cases effectively without information issues.

8 April 21-23, 2008


Today’s SOA / Mashup’s have extended service model needs

9 April 21-23, 2008


Common Data Issues

Information Security ManagementElectronic Document ManagementPoor and Inconsistent DocumentationData / Customer / Process IntegrationCollaboration and MessagingAuthentication TechnologiesStorage TechnologyRemote communication challengesSystem Development Life CycleChange ManagementSegregation of Duties

Exchange & Information Technology

10 April 21-23, 2008


Solution / Problem Metrics

• Can I create a standard simple open format to describe my message structures and data content rules?

• Can my partners validate their transactions in test BEFORE they send them?

• How do people know what I will send them?• I want something that’s simple and standards

based – leverages existing XML components• Can I generate HTML documentation that is

readable by business analysts?

11 April 21-23, 2008


Defining Information Services

Create Samples

RulesEditor

XML

AnalystTemplates

Samples

Develop

Verify

StructureRulesContextVocabulary

3

2

Rules

html

XML

Resultshtml

Publish

1

Detail Use Rules

Test Rules on Samples

4

Share Results

EnablingAgile Information Exchanges

Structure + Vocabulary

Rules + Context Templates

Verify Template Outcomes

12 April 21-23, 2008


ContentAssemblyTemplate

Configure the Business Exchange

XMLinstance

structure(s)

Business context rules

+Content Reference rules

+Data Validation rules

+External Mapping rules

Logical business transaction can have more than one structure layout

can relate rules directly with structure

makeOptional()excludeTree()

item type="noun" name="Agency/@AgencyID" UIDReference="ukt000910"

<conditional expression="'//SupplierID' and lookup(value,'SGIRWSDL:supplierID_check')"

<MapRule output="type" input="Sales/Company/Year/Qtr/Product@type"/>

Simple Declarative Statements

BPMStep

transactiontemplate

OASIS Content Assembly Mechanism(CAM)

13 April 21-23, 2008


Business Functionality• Simple and obvious domain solution templates

• Local deployment and simple change support

• Handling local context drivers and flexibility

• Opening up business integration rules so that partners can confirm their usage + validating

• HTML reporting of rules and layouts

• Processing content with open public specification and sharable rules base

OASIS Content Assembly Mechanism Specification:

http://wiki.oasis-open.org/camhttp://www.jcam.org.uk

Organization forAdvancement ofStructuredInformationStandards

14 April 21-23, 2008


SOA, BPM and Data Services

TransportTransport

SecuritySecurity

Internet Infrastructure

DeliveryDelivery

Proc

ess

Proc

ess 33

22

55

Query / Response

WSDL

11

SOAP / http / https

Data Services

Data Services

44

Vocabulary / Semantics

Packaging / Encoding

XML / edi

Validation / Assembly

Mapping / Transform

XSD, CAM, Schematron

Business Model

ServiceAgreement

Context/ Roles

BusinessProcesses

Description

Msg ExchangeProfile

State & Context

Addressing / Envelope

Transactions

Coordination

Push / Pull

C A M

15 April 21-23, 2008


Exchange Facilitation Needs

For business communities:– Ability to create sharable templates for

communities of practice that need consistent XML transaction handling definitions that are open and public.

For business data analysts:– Printable rule documentation support and features. – Example domain templates within communities of

practice – Supporting codelists implementation (Genericode).

For programmers:– XML content manipulation support and rules ( xslt,

XPath …)– Web services and SOA support

16 April 21-23, 2008


Metrics of Information Sharing

To effectively exchange information, there must be a common semantic understanding of data among participants, and the data must be formatted in a consistent manner that matches the business process context

CommonLexicon

17 April 21-23, 2008


Interoperability – what is it really?What it really is:

– Solve the business functional needs as the priority - ability for systems to accommodate each others functional requirements in a holistic and agile way so that users can maximize the business value obtained today and adapt that to tomorrows changing needs rapidly and easily

– Predictability – when the mechanisms being used are using known, consistent and extensible methodology

– Need to support semantically rich information (human), contextual mechanisms, and not just rigid lists of enforced content (machine)

– Reduce collaboration and adoption costs by making exchanges quick and easy to build and self-adaptive – cost is barrier to interoperability

– Ability to support routing and intermediary services with views of exchange content based on role and use context

How do you get there?

18 April 21-23, 2008


Example: NIEM XSD Reference Architecture

Exchange xmlExchange xml

*Exchange xsd*Exchange xsd

*Extension xsd*Extension xsd

*Code listxsd

Namespace

* Optional

XML Schema

NIEM xsd subsetNIEM xsd subset *Constraint xsd*Constraint xsd

IEPDs have their own target

namespaces

IEPDs have their own target

namespaces

ConstraintValidationConstraintValidation

ConformanceValidation

ConformanceValidation

NIEM = National Information Exchange Model (www.niem.gov) for DOJ/DHS

19 April 21-23, 2008


Domain-specific XML Schemas

NIEM – a comprehensive model of all entities of interest in government domain– Defines terms shared between various agencies:

persons, places, things

LEXS – information-sharing protocols and messages, with NIEM as foundation– Establishes patterns of requests and responses– Provides structure around NIEM data elements

20 April 21-23, 2008


21 April 21-23, 2008


Versioning Challenges

• If the schema version changes – how to ensure it does not break our in place validations?

• How to rapidly adapt to rule changes in a production environment?

• How to develop user context driven deep version control and re-use of sub-components?

• Enhance and automate Test release cycle by improving transparency for bug fix process and expose change deltas to speed testing process? Support for regression testing?

22 April 21-23, 2008


Problem: XSD is non-deterministic!

• XSD does not have direct context mechanisms• The schema contains the superset of every

exchange component variation• People make everything in schema optional• Dependencies are not clear• It is difficult to understand the constructs and to

document the rules clearly for business users to verify

• It is hard to create test cases and instances (the “want list” tough to visualize)

• Disconnect between XML data types and legacy data – e.g. dates, telephone formats, post codes

23 April 21-23, 2008


Consequences for Interoperability

• Multiple WSDL’s introduces risk that services won’t talk to each other

• Bypassing validation permits exchange of incorrect documents

• Propagation of incorrect documents in the system pushes problems to the perimeter

• User interfaces in each participating system must adapt to proliferating special cases

24 April 21-23, 2008


What makes something Agile?

Re-usable methods that can be applied to many areas– Based on open standards and approach; not proprietary

Context and role driven and aware– This allows tailoring to specific profiles and use pattern templates dynamically

Self-adaptive– When requirements change can be adjusted on-the-fly in real time

Fault tolerant and not brittle– Ability to ignore non-critical interchange items and especially not to fail for trivial

reasons or slight version nuances

– Leveraging XML capabilities to make self-describing transactions possible rather than static fixed legacy exchanges

Able to support new uses without extensive reprogramming– Usage patterns set via external configuration allowing broad but controlled uses

25 April 21-23, 2008


Extract XSD to Assembly Template

Exchange xmlExchange xml

*Exchange xsd*Exchange xsd

*Extension xsd*Extension xsd

*Code listxsd

Namespace

* Optional

XML Schema

NIEM xsd subsetNIEM xsd subset *Constraint xsd*Constraint xsd

EXTRACTEXTRACT

Structure

Rules

Documentation

XSD 2 CAMXSLT Tools

XSD 2 CAMXSLT Tools

26 April 21-23, 2008


Example Generation – OASIS EDXL

Structure

Declarative Rules

XSLT

XPath structure flexibility

Assertion logic

27 April 21-23, 2008


Example Generated Documentation

EnhancedData typeLogic

Better code listhandling

Clear use pattern

XPath functions

28 April 21-23, 2008


CAM - Just re-shuffled syntax?

• If we built the CAM template directly from the XSD how can we have added rules value?

• CAM validation significantly more capable and agile than rigid XSD parsing:– XPath based assertions– 30 added functions add control – Mask functions better content matching– Context mechanism allows adaptive rules/structure– Cross-dependencies– Enhanced code list handling– Flexible error handling and reporting

• CAM syntax exposes logic for human verification

29 April 21-23, 2008


Value Proposition - Deterministic

• Making XML transaction handling simpler and predictable

• Quick and easy rule/example building from sample XSD

• Extends and clarifies your existing XSD schema structures

• Enabling more robust fault tolerant processing + versioning

• Providing open sharable templates and documentation

• Re-use easier through support for includable components

• Ability to integrate to business processes and context

• Open source, open public standard toolkit – editor + engine

30 April 21-23, 2008


Eclipse CAM Editor

1

2

3

4

5

Available structures

Structure Rule Viewer

Rule Details

Results Viewer

ValidationProcess

31 April 21-23, 2008


How does CAM work?

• CAM uses WYSIWYG approach to XML

• Starting with your XML structure layout makes template

from that + default data content model

• Next – add your structure use rules – optional / repeatable,

date fields, allowed values, lookups

• Then make context business rules – cross field use rules,

exclude, include, variables

• Save template – run against samples

• Eclipse editor tool makes this all easy to do!

• Deploy to production using jCAM processor

32 April 21-23, 2008


Modelling & Simulation Testing ScenarioEmergency Response Services Workflow using OASIS EDXL exchanges

33 April 21-23, 2008


Conceptual View of EDXL transaction

Transaction

Templates

StructureRulesContext3

2

Use Rules

html

XML

Reportshtml

Publish

1

Common Details

Rules

4

Share Details

Address

Facility

Contact

Vehicles

Emergency

Resources

Activity

Content Rules

Lookup Values

Context

VersioningResources

34 April 21-23, 2008


Illustrative EDXL requirements

When Admissions Total > 50– AND Deaths > 0

When AdultICU Bed type > 0– AND Triage Quantity > 5

Require Facility State = CA, NV, NM

35 April 21-23, 2008


CAM rules syntax

<as:BusinessUseContext> <as:Rules> <as:default> <as:context>

<as:constraint condition="//Activity24Hr /Admissions > 50 and( // Activity24Hr /Deaths > 0)" action="restrictValues(//

Activity24Hr /Admissions, ‘Alert – possible outbreak’) "/>

<as:constraint action="restrictValues(//ns5:AdministrativeArea /ns5:NameElement , 'CA|NV|NM')"/>

</as:context> </as:default> </as:Rules> </as:BusinessUseContext>

36 April 21-23, 2008


CAM Summary• XSD to CAM template extraction via XSLT makes quick

adoption and low learning curve• CAM templates provide agile mechanisms to enhance

control over information exchanges• Context mechanisms support BPM techniques and

tailoring handling to match processing needs• Better documentation support for human verification• Link templates to partner agreements and use patterns• More flexible versioning and non-brittle handling• Test suites and conformance checking with regression

testing and deterministic rule based examples • Open public standard and open source implementation• Sharable public templates for partner collaboration

37 April 21-23, 2008


Process Driven Dynamic Applications

• Templates provide linkage between business needs and software mechanics

• Semantics can be morphed into different renderings depending on interfacing needs– Validation scripts– Forms logic– Data storage and sharing– Workflow steps– HTML documentation

• Exploiting agile adaptable mechanisms using XML

38 April 21-23, 2008


Questions?

39 April 21-23, 2008


www.jcam.org.uk

wiki.oasis-open.org

www.oasis-open.org/committees/cam

docs.oasis-open.org/cam

www.oasis-open.org/committees/emergency www.niem.gov

Resources:

40 April 21-23, 2008


Thank You!David RR WebberSOA ArchitectIntegrity One Partners

Contact Information:[email protected]