1 Quick Guide to CAM Blueprints Using blueprints to develop XML exchange templates and schema (CAM – Content Assembly Mechanism Specification) Author:

1

Quick Guide to CAM Blueprints

Using blueprints to develop XML exchange templates

and schema

(CAM – Content Assembly Mechanism Specification)

Author:

David RR Webber

Chair OASIS CAM TC

January, 2010

http://www.oasis-open.org/committees/cam

2

Introduction

• Configuring formal information exchanges using XML transactions today requires a set of exchange schema that describe those interactions

• An exchange schema provides the structural layout for the XML transaction along with rules detailing the content expected

• Exchange schema development requires knowledge of XML best practices, industry specifications, data modelling techniques and guidelines for exchanging XML transactions

• How does the CAM blueprint approach enable a shift away from a purely manual process today to one that can be highly automated, consistent and repeatable?

3

Solution Approach• Top Down development

• Allows developers to quickly outline the information exchange• Provides XML dictionaries of canonical reusable components• CAM tool then expands the details and fill-in the blanks• Review in visual editor to confirm or extend results• Iterative approach until complete

• Automated exchange package generation• Schemas, XML, documentation, mapping crosswalk• Test generation of example XML with rules validation

• Principles and Rules evaluator• Provides best practices for interoperability • Checks for specific schema techniques and details

• Dictionary driven reuse• Provides industry components / Enterprise Data Model• Ensures consistency of definition and use• Roll your own dictionaries

4 January, 2010 – CAM Draft Specification Development Related Materials

Agenda• Blueprints and Dictionaries

• Exchange Assembly Process Steps Overview

• Leveraging re-use – canonical XML dictionaries

• Introducing Exchange Assembly Blueprints

• Modelling information structures in XML

• Using namespaces and prefixes

• Constructing your Blueprint templates

• Running the Expander

• Finishing your Exchange Template

• Generating Exchange Artifacts• NDR evaluation, Exchange schema, mapping crosswalk, XML

instances, realistic data use, business rules documentation

• Summary


Web based

Desktop

Excel

Search tools

Exchange Assembly Process Overview

Project InceptionProject

Inception

dictionariesdictionaries

Exchange Content

Modelling

Exchange Content

Modelling

Blueprints Dictionary

Dictionary

Expand and Evaluate Results

Expand and Evaluate Results

Review / Tailor Exchange Structures

Review / Tailor Exchange Structures

Generate Package

Artifacts and

Test them

Generate Package

Artifacts and

Test them

Exchange Package

Exchange Package

Completed

Exchange Structures

NDR Evalu

- ation

Dictionary Generation /Selection

Completed Exchange artifacts

Test Cases

Validation Engine

XMLexamples

XSD schemas

Subset SchemaCrosswalk

MappingSpreadsheet

RulesReport

RenderingStylesheet

Import XSD

Naming / NDR /

spell check Generat

e Dictionar

y

Import EDM

Test Examples and Rules

Model Exchange

Expand, Review, Refine

CAM Enabled

11

22

33

44

55

66

BusinessModelling

Requirements

Charter

Work Group

Cross Referenc

e Mapping

XSD Schema Writer

Schema Generation


Leveraging Re-use :Canonical XML Dictionaries

• What is a canonical dictionary

• Styles of dictionary

• How do I obtain a dictionary?

• How do I use it with CAM?


Canonical XML dictionary

• A collection of distinct components that represent discreet business information for an application domain

• Includes singleton components and combinations of related components together as sub-assemblies

• Information is represented in a simple neutral conceptual data format that captures the critical concepts about the data e.g. name, description, content type, contextual usage pattern, hierarchy

• Wikipedia definition:http://en.wikipedia.org/wiki/Canonical#Computer_science


XSD Schema v CAM Dictionary• There are generally two “styles” of XSD schema

• Distinct information exchange schema structures• Collections of schema model components, possibly abstract,

and possibly recursive

• CAM dictionaries built from distinct exchange structures generally mimic those components directly

• Parent / Child sets are visible in dictionary as ABIE/BBIE pairings

• “children” column in dictionary contains direct descendents• “context” column reflects component relationships

• CAM dictionaries built from models with abstract and recursive parts may have fragments of components only

• Assembly of exchange structures using references to piece parts, rather than complete structure assemblies

• Reuse is then around these extension part collections• Context may need to be overridden to meet desired use pattern


Available XML Dictionaries• LEXS 3.1.4 dictionary• OASIS EDXL dictionary• OASIS EML dictionary

• NIEM 2.1 dictionaries• CBRN dictionary• Emergency dictionary• Family dictionary• Immigration dictionary• Infrastructure dictionary• Intelligence dictionary• Justice dictionary• Maritime dictionary• Screening dictionary• Trade dictionary• NIEM core dictionary• Immigration blueprint

Available from download site

direct link:

http://sourceforge.net/projects/camprocessor/files

XMLXMLXMLXML XMLXML XMLXML XMLXML XMLXML

+ includes spreadsheets and sample blueprint

Packaged with CAM editor

see dictionary folder of install

+ spreadsheet

+ blueprint samples

XMLXMLXMLXML

XMLXML

Note: Those marked in bold are model style dictionaries with recursive components.


Drag and Drop dictionary into Spreadsheet

Drag and Drop XML dictionary

format into Microsoft Excel

Drag and Drop XML dictionary

format into Microsoft Excel

Item namesItem names

Child item indicatorChild item indicator

List of Child itemsList of Child items

Parent item indicatorParent item indicator

Unique IDUnique ID

Item PropertiesItem Properties


Creating your own Dictionaries

• The CAM toolkit includes dictionary generation tools• can harvest components from existing exchange schema

• will import model components from ERwin modelling tool

• can ingest and use subset schema or component schema

• A separate companion Quick Guide describes the dictionary creation processes and tools

• Look for existing dictionary packs from Sourceforge files location of CAM downloadhttp://www.sourceforge.net/projects/camprocessor/files


Using Dictionaries

• Determine your business information exchange components at conceptual level

• Search and locate candidate components from appropriate domain dictionary collections

• Catalogue the parts to be used• Dictionary components can be referenced individually or

as collections by an assembly blueprint that puts them all together to create a complete information exchange

• Components can be selected from multiple dictionaries

• Note any new extension pieces as needed• A control file configures which physical dictionary files to

reference and associates one or more particular namespace prefixes with each dictionary


Dictionary Control File

Control files should be manually edited in a XML editor tool to make configuration changes as needed (e.g. arrowed text areas).

Associates physical dictionary files to namespace prefix details


Introducing Exchange Assembly Blueprints

• What is a blueprint template• Example blueprints• Blueprint tools overview• How do I create a blueprint?• How do I use it with CAM?


Introducing Blueprints

• A blueprint is the outline of your information exchange and the top level sets of components needed

• Allows rapid iterative exchange structure design• Start with a basic idea of what is needed, extend and refine as you clarify

groups of components and their relationships

• Can import components from one or more domain dictionary collections

• Allows you to focus on the conceptual and hierarchical information model components and their purpose• The low level specific information details will be filled in for you by the

expander tool from the matching dictionary definitions

• Review results and add local extensions / exclusions as needed

• Blueprints themselves also have high re-use value• Can be sub-assemblies and patterns not just exchange models


Example BlueprintsLEXS messaging blueprint

Reusable messaging envelope constructs

OASIS EDXL HAVE message

Business functional components

Message handling,

delivery and controlPayload goes here

Top level sets of

business

information

components

Individual

component

these examples available from CAM editor install package~ CAMeditor\eclipse\workspace\CAMEditor\dictionary\blueprints\


Blueprint Tools Overview

Expander Tool

Expander Tool

Industry dictionaryIndustry

dictionary

Domain dictionaryDomain dictionary

Component Definitions

44

Component Definitions

Web toolWeb tool

ExcelExcel

Search

Tools

Search

Tools

22

BlueprintDesignerBlueprintDesigner

11

InsertDictionary

ParentComponents

InsertDictionary

ParentComponents

33

Completed ExchangeTemplate

Completed ExchangeTemplate

55


How do I create a blueprint?

• Use CAM Eclipse visual editor• create basic blueprint from scratch or from XML instance

• Clone and edit an existing blueprint• use a XML editor to manipulate raw XML of blueprint (e.g.

search and replace namespace details)

• modify blueprint using CAM Eclipse visual editor

• Beware of namespaces!• You need to pre-declare namespaces in the blueprint –

that may appear later in the finished expanded exchange

• Use CAM Eclipse visual editor root node menu to add namespace definitions

• Tip: use your XML editor to cut and paste namespaces into raw blueprint XML file directly


Blueprint Structure Overview

Namespace

declarations

Short Name for Exchange

Root node name and optional namespace

XML container elements to facilitate use patterns

Dictionary component references

Dictionary Type annotation reference

(note red paper clip icon)


Adding Components from a Dictionary

• Understanding the 4 basic types of component referencing available

• Direct component (BBIE)

• Direct collection (ABIE)

• Indirect type (extension)

• Indirect sub-component (contextual re-use)

• Control of the XML structure hierarchy• Grouping collections of components

• Extending an existing component

• Building new extension from existing components

• Namespaces – when to use them


Example XML Techniques

Direct component referencing (BBIE)

Direct collections referencing (ABIE)

Indirect type extension via annotation

Type

(red paper clip icon denotes annotation)

Building new component from existing ones

Grouping collection of components

Namespaces – root namespace optional (aaa)

Dictionary component namespaces are required

(Immigration Alien Alert example blueprint)


Making New Blueprint from scratch

Open new template

11

Assign root element22

33

Complete structure by adding elements and dictionary components


Running Blueprint Expander

• Blueprint check list • Using Expander tool dialogue• Reviewing results• Hints and Tips


Using Your Blueprints

• What you need -• CAM Eclipse visual editor• Blueprint template• Dictionary or dictionaries of components• Dictionary control file

• Contains linkage of namespace prefix to each dictionary• Contains physical filename path of each dictionary

• Getting Started• Install CAM editor tool from Sourceforge.net• Open example blueprint from EDXL folder

~ CAMeditor\eclipse\workspace\CAMEditor\dictionary\blueprints\EDXL

• Try blueprint expander with control file EDXL-dictionary-control-file.xml


Running Blueprint Expander

Select Dictionary Control File

Pick output result

filename (default is

-expanded.cam)

Determine rule generation

selections from dictionary

definitions

Note: entries are “sticky” so

are remembered for next time

This example is available in the NIEM dictionaries download pack from Sourceforge


Blueprint followsmodel Console Log detail

Blueprint Expander Example

Exchangecomponents outline

EXCHANGE BLUEPRINTEXCHANGE BLUEPRINT

COMPLETED EXCHANGE TEMPLATECOMPLETED EXCHANGE TEMPLATE

11

33

StructureDetails Expanded

Expander Tool

Expander Tool

22

Dictionary Lookups

Dictionary Lookups


Reviewing Results

Exchange Structure items

Rules for each item

DomainComponents Namespaces

1122

33


Hints and TipsBlueprint Remedy

Node not expanded, or removed! Override default dictionary structure context definition using an annotation Type reference to the node instead – see Alien alert example blueprint – aaa:OfficerDetails

Added parent and child combination. All children added instead of just one.

Expander inserts all children of parent. Either delete unwanted children from expanded result (see next section), or create custom parent node, and pick children components directly – see Alien alert blueprint example.

Node not expanded Make sure namespace prefix and spelling exactly matches dictionary and dictionary control file prefixes.

Undefined namespace prefix error when opening expanded template in CAM editor

Expander added dictionary components from another namespace. Declare that namespace prefix back in blueprint and re-run expander again

Expanded result missing all desired sub-components

Create your own blueprint extension parent and combine parts to create desired result – see Alien alert example – aaa:FurtherDetails

Expanded result has too many sub-components

In expanded result mark unneeded parts with “excludeTree” rules, then run File / Export / Compress to strip out; or use extension technique above to pick only few child elements instead of whole parent

29

Completing the Exchange Template

• Adding nodes• Deleting nodes• Editing Rules• Adding annotations


Exchange Template Editor

Rules for each item

Can add new custom domain Items

1122

33

44May need to add a namespace for new items


Deleting Exchange Components

Right click for popup menu22

From Rules exclude item or tree

Select item to be removed11

33

44 After marking all items to be excluded, save template, then run Files / Export / Compress template to physically remove them.


Editing Rules

Right click for popup menu

Select item to add new rule11

22

33Use rule wizard to complete


Adding annotations

Right click for popup menu

Select item to add new annotation11

22

33Use annotation wizard to complete

34

Generating Exchange Artifacts

• Dictionary crosswalk mapping • Exchange Schema • XML instances • Business rules documentation


Exchange Generation Steps

Toolkit for exchange artifacts

generation

Toolkit for exchange artifacts

generation

Suite of menu options and tools in desktop CAM toolkit editor

(each run custom xslt scripts on the exchange CAM template xml)

CAM Template

1. Run template Evaluation Report

2. Compare to industry dictionary

- create mapping spreadsheet

- create crosswalk xml

Generate business rules report

3. Generate exchange XSD schema

4. Generate XML test instance(s)

22

11

33

44

55

EXCHANGE TEMPLATEEXCHANGE TEMPLATE

Structure

Rules

Documentation


Using NDR Evaluation Report

• Provides scoring, alerts, warnings and potential

issues including:

• Naming and content model conventions

• Naming and restriction consistency checks

• Interoperability enablers/inhibitors checks

• Rules integrity and duplicates

• Statistics on exchange size

• Spell checking on component names

Access from Tools menu, Evaluate Template option


Evaluation Report (NDR) example

Part of an example report for LEXS getDataItemRequest template displayed in toolkit HTML viewer


Compare to industry dictionary

• References industry dictionary of names and properties• Matches on physical names• Reports mapping details• Compatible with Microsoft Excel• Report can be used to do spell checking• Generates crosswalk xml file• When NIEM-dictionary selected:

• generates NIEM SSGT wantlist.xml file

• provides % re-use statistics for NIEM components( note NIEM-dictionary.xml is located in install directory

~ CAMeditor\eclipse\workspace\CAMEditor\dictionary folder )


Example cross-reference spreadsheet

Formatted view in Microsoft Excel of import of cross-reference report details (from generated XML file)


Generate Documentation and Schema

• Documentation:• Create HTML report of exchange schema details and

associated content and business rules

• Report layout and content designed to be reviewed by business analysts

• Schema:• Generate XSD schema for exchange

• Customizable exchange folder layout management by namespace for extension, subset and exchange schema components

• Writes XSD schema in syntax that is clear, simple and compatible with deployment tooling environments


Business Rules Documentation

Part of the example rules for LEXS getDataItemRequest template displayed in toolkit HTML viewer


Export Exchange to XSD Schema

Completed Exchange Structure

Set Exchange Output Option

Complete set of exchange schemas generated

Export Template to Exchange XSD:


Exchange Schema Generated

Each namespace file is import for those specific type definitions

Reviewing XSD results in a schema editor tool

Set of XSD files with filename and namespace suffix


XML Testing Examples Generation

• Support for software development testing process• Designed to allow creation of concrete realistic

examples not just random value based• Hinting system allows insertion of actual test system

values into XML examples• Can create both valid and invalid examples to support

unit testing of application software• Exclude capability allows generator to create examples

that contain only a portion of the entire exchange• Control over random seed value used allows re-

generation of identical test cases


XML example generation wizard

View of CAM toolkit with LEXS getDataItemRequest and dialogue for XML test example generator tool


Running validation rules tests

• Built-in CAM validation engine allows testing of XML instances against actual exchange rules (CAMV).

• Critical to ensure that the exchange validates actual live production example scenarios correctly

• Allows deployed solution to match exchange schema details

• Errors can be reviewed interactively in exchange visual interface

• Post-processing of validation results allows unit regression tests to be created with reporting of errors, warnings and information level notes


Run Exchange Template

Pick XML test case to validate Run validationReview results in visual editor


Example Exchange Packaging Details

Package Artefact Description

CAM Generates Files

NDR Evaluation Report Evaluation of exchange template compared to set of Naming and Design Rules (NDR) and interoperability factors / principles

Crosswalk XML Itemized list of each dictionary component element and attribute included in the exchange.

Exchange Schema Base document schema that defines the full XML structure for the exchange and is generally named after the exchange itself.

Extension Schema Specification for extended components—separate local name-spaces of components not contained in dictionary

Sample XML Instance Example instance(s)

Documentation

Master DocumentationThe Master Document is the main document for which all of the context and details around the exchange are explained. This

document includes, the overview, as well as details surrounding the exchange, business drivers and requirements

Business rules Business rules in plain or structured English, generated by CAM development tool.

Mapping to Dictionary Mapping of domain components, tagged with constraints (i.e., cardinality, etc.) to dictionary components as a spreadsheet.

Extended componentsComponents created because they were not in dictionary—may be part of mapping spreadsheet and include structure and definitions

of new components.


Summary

Dictionary driven exchangesBlueprint enabled reuseAutomated exchange package generationAlignment to NDR Principles and RulesTesting and validation support


Review• Top Down development

• Reference dictionary components

• Create exchange blueprint

• Run Expander tool

• Refine desired structure in visual editor

• NDR Principles and Rules• Best practices for interoperability and schema techniques

• Dictionary driven reuse• Select Enterprise Data Model and industry components

• Ensures consistency of definition and use

• Automated exchange package generation• Schemas, XML, documentation, mapping crosswalk

• Test generated example XML with rules validation


Links and Resources

• CAM Toolkit download• http://sourceforge.net/projects/camprocessor

• NIEM dictionaries and sample blueprint• http://sourceforge.net/projects/camprocessor/files

• NIEM tools and IEPD tools tips and hints• http://wiki.oasis-open.org/cam/NIEM_Tools_and_CAM

Documents

1 Quick Guide to CAM Blueprints Using blueprints to develop XML exchange templates and schema (CAM – Content Assembly Mechanism Specification) Author: