Collaborative Expedition Workshop #47, Tuesday, January 24, 2006,

1

Collaborative Expedition Workshop #47, Tuesday, January 24, 2006,

at the National Science Foundation

Advancing Credible Agreements Across Networked Improvement Communities: Bootstrapping Service-Oriented

Architecture and Semantic Interoperability Toward Transformative Practice

Brand Niemann (US EPA), Chair,Semantic Interoperability Community of Practice (SICoP)

January 23, 2006

2

Overview• 1. Some History:

– Architecture and Infrastructure Committee (AIC) and Federal Enterprise Architecture (FEA) Perspective

• 2. Service-Oriented Architecture:– Ontology and Flow

• 3. Some Resources:– Previous Work

• 4. Chief Architects Forum Comments:– See http://colab.cim3.net/cgi-bin/wiki.pl?

AnnouncementofSOACoP

3

1. Some History• In the reorganization of the AIC there was the:

– Governance Subcommittee:• Government Enterprise Architecture Framework.

– Components Subcommittee:• Definition (1) and White Paper I (2).• Core.Gov Phase I - CollabNet (see next slide).

– Emerging Technology Subcommittee:• XML Working Group – XML Vocabularies, XML Schema

Registry, and Vendor Demonstrations.• Web Services Working Group – Pilots, Components, and

Registries and Repositories:– E-Forms and Business Gateway for OMB and SBA, and

Componenttechnology.org (six quarterly conferences for the AIC).

(1) A self contained business process or service with predetermined functionality that may be exposed through a business or technology interface”(2) The three Subcommittees were on different pages and the White Paper I went nowhere like DRM 1.0!

4

1. Some History• The Web Services Working Group was an outgrowth

of an award winning SOA with Web Service (see slides 28-36) and was tasked to provided successful pilots and components to the Business Gateway, namely:– Collaborative XML Schema Development – The XML

Collaborator from Blue Oxide:• Pursued by the Intelligence Community Metadata Working

Group.– E-Forms – The Bureau of Census – Fenestra Open Source

Tool.• Fenestra is now a subcontractor on the six year contract to

Lockheed Martin to build the Electronic Records Archives (ERA) system for NARA.

– Collaborative Development Environment Registry/Repository – CollabNet.

• Adopted for Core.Gov I– Native XML Data Storage – Tamino XML Server from

SoftwareAG.• The leading native XML storage data base in the world.

5

1. Some History

• In the current phase of the AIC:– Governance Subcommittee:

• Value of Enterprise Architecture and Reference Model Maintenance Process.

• SOA Governance (January 19, 2006).– Components Subcommittee:

• Service Component Based Architecture – “FEA Version of SOA” and White Paper II.

• Core.Gov Phase II – Tomoye (see next slide).– Emerging Technology Subcommittee:

• XML CoP – ET.Gov component registration, XML Vocabularies (e.g. StratML), and Naming and Design Rules and Guidelines.

• SICoP – Pilots, DRM 2.0 Implementation Components, Collaboration Workshops and Cross-CoP Meetings, and Semantic SOAs.

6

1. Some History

• Core.Gov Phase II:– Register E-Gov Initiatives Components.

• Done as SOA’s?

– Offer Component Development Environment.

• Using Tomoye?.

– Discuss how to include components from:• Open Source, Componenttechnology.Org, XML

CoP, SICoP, etc.

7

1. Some History

Source: Gartner, June 30, 2005, Magic Quadrant for Metadata Repositories, 2H05 to 1H06, Michael J. Blechar, ID Number: G00129274.

*DHS MCOEVendor Proofof Conceptfor MetadataRepositorySelection.

Note: For 2005 Computer Associates is removed from the Leader Quadrant.

**

*

* See next slide.

8

1. Some History

• CIO Council’s Web Services WG and SICoP Pilots:– MetaMatrix – Visionaries Quadrant.– Flashline – “Flashline has been at the leadership front in

terms of promoting market interest in areas such as software asset portfolio management, governance, compliancy and Web services/SOA.”

• See http://www.componenttechnology.org/Awards/– Unicorn - Visionaries Quadrant.

• Also Improving Rapid First Response Event Ontology Pilot and Water Data Harmonization Pilot.

– LogicLibrary – “LogicLibrary has helped drive market interest in service and component reuse, SOA governance and integration with leading service-oriented development products.”

• See http://www.componenttechnology.org/Awards/• Demo: http://www.logidexassetcenter.com/assetcenter.jsp

9

1. Some History

• The FEA Has Used the Term Component Based Architecture for Linking the Five Reference Models Together.

• But This Has been Difficult to Understand and Implement Because the First Four FEA Reference Models Are Taxonomies.

• This Has Been Dealt With in DRM 2.0 Where Implementation Testing Has Paralleled Development and Used SOA.

• Registries and Repositories are Being Built Into Operating Systems (e.g. Sun OS – ebXML and mapping to UDDU) and as SOA Platforms (e.g. Digital Harbor - PIES)!

10

1. Some History

• Gartner characterizes the Federal Enterprise Architecture as “a set of Reference Models backed by law and administrative rule” and “not a roadmap, but a guide to getting there.”

• A 2004 GAO report said the FEA is more akin to a classification scheme (taxonomy) for government operations than a true enterprise architecture, questioned (since the terms are not well-defined) if the expected relationship between the FEA and agencies’ architectures were clear enough for agencies to “map” and “align” their architectures with the FEA, and if the agencies’ enterprise architectures would provide sufficient content for driving implementation of systems.

11

1. Some History

• The Industry Advisory Council EA SIG recently said “The FEA Reference Models do not constitute a comprehensive EA methodology or approach by themselves. These FEA Reference Models, as populated, specifically serve mainly as mechanisms for identifying and coding initiatives via a common taxonomy and as checklists for coverage in an EA.”

• OMB Chief Architect, Dick Burk, said recently the FEA is not a “real architecture” – only four Reference Models that are taxonomies, but we are evolving it to a “real and target architecture” (Chief Architect Forum, October 6, 2005).

12

1. Some History

Evolving toServiceOrientedArchitecture

SemanticInteroperabilityArchitecture

Evolving to aData-DrivenApproach forthe DRM

“The problem is not that there are no semantics, the problem is thatthe semantics is hidden in software components”, Stefan Decker,quoted by Christopher Welty in “Towards a Semantics for the Web”.

Convergence in the FEA Paradigm Shifts

13

1. Some History

• FEA Reference Model Taxonomies

• FEA “Common Language”

• DRM 1.0 by committee– Implementation after

development.

• FEA Reference Model Ontology

• FEA Semantic Model

• DRM 2.0 by open, collaborative process– Implementation though

iteration and testing during development.

Paradigm Shifts

14

1. Some History

Source: Expanding E-Government, Improved Service Delivery for the American People Using Information Technology, December 2005, pages 2-3.http://www.whitehouse.gov/omb/budintegration/expanding_egov_2005.pdf

15

1. Some History

• Essentially, SICoP and others have evolved the FEA Taxonomy/Component Based Architecture over the past year to a Ontology-Based/Semantic Interoperability Architecture (SIA) or Semantic SOA (SSOA) following the same process as DRM 2.0, namely testing while developing to demonstrate it works in pilots and real-world implementations, instead of the Components Subcommittee approach of developing White Papers and providing Core.gov without that open collaborative process and iteration and testing during development, and having essentially no uptake.

16

2. Service-Oriented Architecture

• Ontology and Flow:– What is Abstraction and Indirection?– What is the SOA Abstraction and Indirection?– Why is “Publish” the First and Most Important

Step?– What is a SOA With Web Services?– Why is Semantic Interoperability Important for

SOA?– What Are Some Best Practices of SOA,

Composite Applications, SSOA, and Their Platforms?

– What are Some SOA Governance Approaches?

17


Topic Maps Ontology

Topics Concepts

Associations(between Topics & Occurrences)

Relationships(between Concepts and Instances)

Occurrences Instances

Information ModelTwo Connected Layers: Knowledge Map and the Information Resources

18


• Information Model:– Introduce a concept in the form of a question.– Answer that question with a definition and an

instance that illustrates the relationship we mean between the concept and the instance.

– Provide a flow of concepts and instances that supports logic and reasoning.

– This illustrates the Knowledge Reference Model we are working towards!

19

2. Service-Oriented Architecture• What is Abstraction and Indirection?

– Definition: Architects of both software and physical structures routinely use the principle of abstraction to isolate complex components and reduce the scope of a problem to be solved (“see the forest for the trees”). By definition, ontology is abstraction and is the ultimate abstraction tool for information.

• Example: Imagine a scenario of using a pivot data model without abstraction – it would require the aggregation of all of the data elements in a particular community – the result could be the a community of 500+ applications, each application with approximately 100 data elements, requiring a pivot model with about 50,000 data elements – an abstracted model could conceivably be capable of representing this information in far fewer than about 100 date elements!

– Pilot: Demonstrations of SICoP Pilot Projects for EPA Managers, August 16, 2004, Semantic Information Management (Unicorn): Integrating Health and Environmental Information to Protect American Children”, at http://web-services.gov

20


• What is Abstraction and Indirection?– Definition: Indirection is a concept that is use to

plan for future uncertainty. Simply put, indirection is when two things need to be coupled, but instead of coupling them directly, a third thing is used to mediate direct, brittle connections between them. By leveraging indirection in the fundamental aspects of the technology, semantic interoperability is built for change, and this built-in flexibility differentiates semantic technologies from other information-driven approaches.

• Example: Model-View-Controller (see next two slides)– Pilot: E-Forms for E-Gov, see

http://www.fenestra.com/eforms/

21


Source: http://www.enode.com/x/markup/tutorial/mvc.html

22


• Model-View-Controller (MVC) is a classic design pattern often used by applications that need the ability to maintain multiple views of the same data. The MVC pattern hinges on a clean separation of objects into one of three categories — models for maintaining data, views for displaying all or a portion of the data, and controllers for handling events that affect the model or view(s).

• Because of this separation, multiple views and controllers can interface with the same model. Even new types of views and controllers that never existed before can interface with a model without forcing a change in the model design.

Source: http://www.enode.com/x/markup/tutorial/mvc.html

23


• What is the SOA Abstraction and Indirection?– IBM created a model to depict Web services

interactions which is referred to as a “service-oriented architecture” comprising relationships among three entities (see next slide):

• A Web service provider;• A Web service requestor; and a• A Web service broker.

– Note: IBM’s service-oriented architecture is a generic model describing service collaboration, not specific to Web services.

• See http://www-106.ibm.com/developerworks/webservices/

24


Serviceprovider

Servicebroker

ServicerequestorFind

BindPublish

Service-oriented architecture representation (Courtesy of IBM Corporation)

25


Web Services Network:Security

ReliabilityQoS

Billing

Service providersService requestors

Web services networks act as intermediariesin Web services interactions.

26


• ZapThink’s* ZapForum (2005):– May 4th - The Great Debate – Enterprise Service Bus (ESBs),

SOA Fabrics, or Something Else?– June 8th – End-to-End Metadata Management – Registries,

Repositories and Governance.– July 13th – Building Composite Applications with Legacy

Systems.– September 7th – Transformation and Semantics.– October 5th – Improving Performance of SOA Systems.– November 2nd – Implementing Reliable Service-Oriented

Architectures.– December 1st – Policies and Governance.

* The “Garner for SOA” at http://www.zapthink.com/ See SOA Roadmap.

27


• Why is “Publish” the First and Most Important Step?– These need to be done in order or there will be needless

infrastructure and cost.– “Publish” is to expose the data in an interoperable format

(XML) otherwise there is nothing to find and bind to:• Rule 1: Every new application or system modernization should

be a state-of-the-art web service.– “Find” requires a registry and/or repository which has been

very difficult and expensive historically (e.g. XML.Gov, Core.gov, DoD, etc.) with very limited ROI:

• Rule 2. Don’t put the registry/repository cart before the horse or that will become the focus.

– “Bind” requires tools and expertise to write and test WSDL files like XML Spy and others.

28


• What is a SOA With Web Services?– Definition: XML for the data and XML for

the messages linking two or more services together.

• Example: The Award Winning Emergency Response Web-to-VoiceXML Application and Distributed Content Network (see next slide).

– Features: Data made visible and exchangeable and reusable and many network nodes (county, state, agency, subject matter, etc.)

29


http://www.epa.gov/ceppo/lepclist.htm

30


http://130.11.57.49/lepc/FMPro?-db=LEPC.FP5&-format=-fmp_xml&zip_lepc::zip_code=22181&-find=

XML Data XML Message for Voice

31


Source: Expanding E-Government, Improved Service Delivery for the American People Using Information Technology, December 2005, pages 2-3.http://www.whitehouse.gov/omb/budintegration/expanding_egov_2005.pdf

32

2. Service-Oriented Architecture• Metamodel: Precise

definitions of constructs and rules needed for abstraction, generalization, and semantic models.

• Model: Relationships between the data and its metadata.

• Metadata: Data about the data.

• Data: Facts or figures from which conclusions can be inferred.

Relationships and associations

The purpose of this schematic is to show that we need to describe information model relationships and associations in a way that can be accessed and searched.

Source: Professor Andreas Tolk, August 16, 2005

33


See http://web-services.gov and Dynamic Knowledge Repositories

This Data Architecture Provides the Three S’s: Structure, Searchability, and Semantics.

34


Note: Can Highlight Table and Copy and Paste to Spreadsheet Because of XML Markup.

Metamodel

Model

Metadata

Data

Data Story

35


Data &Metadata(see nextslide)

http://web-services.gov/statabs2003no1.htm

Separation of the Data Presentation from the Data & Metadata.

DataPresentation/Visualization

36


Data &Metadatain XML

http://web-services.gov/statabs2003no1.htm

The Data & Metadata Travel Together in XML Format!

37


• Why is Semantic Interoperability Important for SOA?– Dimensions of Interoperability– Evolution of the SOA Platform– Line of Sight– Example 1 - Web Services for E-

Government– Example 2 - Development of the FEA Data

and Information Reference Model

38

2. Service-Oriented Architecture• Dimensions of Interoperability:

– Organizational Interoperability is about streamlining administrative processes and information architecture top the institutional goals we want to achieve – and to facilitate the interplay of technical and organizational concerns. It requires the identification of “business interfaces”, and coordination throughout MS and EU.

– Technical Interoperability is about knitting together IT-systems and software, defining and using open inter-faces, standards, and protocols. It relies on cooperation as well as on technical infrastructures.

– Semantic Interoperability is about ensuring that the meaning of the information we exchange is contained and understood by the involved people, applications, and institutions. It needs the know-how of sector institutions and publication of specifications.

Source: Barbara Held, The European Interoperability Framework for pan-European eGovernment Services, IDABC, Enterprise & Industry Directorate-General, European Commission, February 17-18, 2005:

39


• Evolution of the SOA Platform:– Simple Web Services – exposing data and

actions– Composite Applications – business

processes consumed by portals– Service Infrastructure

Sources: (1) David Chappell, Business Process Management in a Service-Oriented World, Federal Architect Forum, May 11, 2005, (2) Bruce Graham, Taking SOA from Pilot to Production with Service Infrastructure, May 12, 2005; and (3) David Martin, Semantic Web Services: Promise, Progress, and Challenges, SWANS Conference Tutorial, April 8, 2005.

40


Simple Composite Infrastructure

Organizational

Technical

Semantic

Dimensions of Interoperability

Evolution of the SOA Platform

Line of Sight

1

2

3

41


• Example 1 - Web Services for E-Government:– 1. Organizational-Simple:

• Led CIO Council award winning VoiceXML Web Service for EPA Emergency Response pilot that has subsequently been commercialized and implemented as Infrastructure.

– 2. Technical-Composite:• Lead the CIO Council’s E-Forms for E-Gov Pilot that saw 13 E-

forms vendors each build an XML Web Service using a common XML Schema for E-Grants to increase their collective technical interoperability with one another.

– 3. Semantic-Infrastructure:• Our recent Semantic Web for Military Applications Conference

featured 40 vendors implementing RDF/OWL including the “Putting Context to Work: Semantic Keys to Improve Rapid First Response” that used an event ontology to achieve semantic interoperability across five vendors.

42


Source: Putting Context to Work: Semantic Keys to Improve Rapid First Response,Semantic Web Applications for National Security Conference, April 8, 2005, TradeShow, Broadstrokes, ImageMatters, MyStateUSA, Starbourne, and TargusInfo.

Event Type Ontology in Context:Application in Unicorn Workbench: http//www.unicorn.com

43


• Example 2 - Development of the FEA Data and Information Reference Model:– 1. Organizational-Simple:

• A Wiki is being used to support a Community of Practice in “the publish, find, and bind” of SOA in their development of the basic documents and items 2 and 3 below.

– 2. Technical-Composite:• A Wiki is also being used as a registry of the taxonomy of XML

Schemas to organize the government’s data and information for sharing within the context of the taxonomy.

– 3. Semantic-Infrastructure:• A Wiki is also being used for coordination of taxonomy and

ontology development, sharing, and reuse across the government and non-government organizations.

44

2. Service-Oriented Architecture• What Are Some Best Practices of SOA, Composite

Applications, SSOA, and Their Platforms?– N2 problem, need if N is large, otherwise do direct

connections (Tolk)– Semantic Interoperability at Work: Improving Rapid First

Response:• See Collaborative Expedition Workshops and SICoP Public

Meetings.– Digital Harbor, Composite Applications Platform, SICoP

Pilots – Four Use Cases:• SICoP Fact Sheet at

http://web-services.gov/SICoPFactSheet.doc• See Leaderboard, page 70, in InfoWorld, May 2, 2005, Issue 18

– NSA (Chance and Kendall), Semantic Service Oriented Architecture and Standards for Model-Driven Semantics:

• 4th Semantic Interoperability for E-Government Conference, February 9-10, 2006, MITRE, McLean, Virginia.

45


• What are some SOA Governance Approaches?– Opportunistic – Make every new application and system

modernization a state-of-the-art Web Service.• EPA’s Award Winning Web Services.

– Kim Nelson: “I wish we had created more Web Services than data warehouses.”

– Mandated from on high – The Joint Chiefs of Staff said we will have a medical readiness information system to go to war in Iraq.

• Medical Operational Data System (MODIS).– Incremental bootstrapping approach. July 22, 2003, see

http://www.mods.army.mil/

46


• What are some SOA Governance Approaches (continued)?– Specified in Collaborative Software Component

Development and Reuse:• Dr. Jeffrey Poulin, “Measuring Software Reuse: Principles,

Practices, and Economic Models”, Addison-Wesley, May 14, 2004, Workshop (next slide).

– Fostered in Communities of Interest:• DoD CoI Forum: Propose pilot to expose and share data with

Web Services, get sign-off by 1-2 star general, complete the pilot, and expose to acquisition (January 19, 2006, meeting).

– Visionary Agencies and Organizations:• NSA SSOA - Participation in standards organizations and

piloting of state-of-the art emerging technologies.

47

3. Some Resources• Emerging Technology Innovations in Software

Components Development, Reuse, and Management – Applications to Government Enterprise Architecture, May 14, 2004:– See http://colab.cim3.net/cgi-bin/wiki.pl?

ExpeditionWorkshop/SoftwareComponentsWorkshop_2004_05_11

• Suggested Roadmap from the FEA to SOA/SIA, Management of Change Conference, May 25, 2005:– See http://web-services.gov/scopemoc05252005.ppt

• High Performance Government: SOA Led Organization Transformation – Racing Towards Business Improvement, SOA Executive Event, May 12, 2005:– See http://web-services.gov/scopesoa05122005.ppt

48

3. Some Resources• Tutorial 3 : Understanding Service-Oriented

Architectures and Government Applications– What Attendees Will Learn:

• Real-world examples of government agencies that have implemented SOAs

• How SOAs can streamline government IT and facilitate broad-scale interoperability

• How to build a SOA using and Enterprise Service Bus (ESB)• Why ESB technology is emerging as the foundation for

successful SOA deployments• How the ESB supports the specific requirements of a

distributed, federal government application environment• Correlations between benefits of web services, SOA, and ESBs

in the public and private sectors• How ESB’s differ from traditional integration approaches• How to select the right ESB for your agency’s integration needs

http://www.e-gov.com/events/2005/ea/conference/tutorials.asp

49

3. Some Resources

• David Chappell, Vice President and Chief Technology Evangelist Sonic Software, and Author:– Enterprise Service Bus (ESB), O’Reilly Media,

June 2004.

• Eric Newcomer, Chief Technology Officer, IONA Technologies and Author:– Understanding SOA with Web Services, with Greg

Lomow, Addison Wesley, December, 2004.

50

3. Some Resources

• E-Government needs a comprehensive guide to SOA with Web Services standards and best practices for implementation to get from the current "as is" to the future "to be" architecture. This book meets that need superbly.– Brand Niemann, Ph.D.,

Co-Chair, SICoP, U.S. Federal CIO Council

For example: My review:

51

3. Some Resources

• Caution: Be Prepared to Slow Down – Road Work Ahead:– David Chappell, Federal Architect Forum, April 8, 2004: The "Big

Bet" - Has anyone ever tried to create a complete, multi-vendor security framework before? Will this work? Keep an eye on the progress of WS-Security implementations - The success of SOA may depend on this technology.

– David Martin, SRI International, April 8, 2005: Sociological (crossing the chasm) – getting to where the payoff exceeds the overhead (for significant numbers).

– Russ Reopell, MITRE, Intelligence Community Metadata Working Group Meeting, May 4-5, 2005: The SOA Threat

– Greg Lomow, Bearing Point: Work on a Multi-vendor Security SOA Framework for DHS (Source: JP Morgenthal, May 26, 2005).

– SOA Leaders, Building the Business Case for SOA, June 9, 2005.– SecurE-Biz CXO Summit Conference, SOA/ Web Services Track,

June 29-30, 2005.

52

53

3. Some Resources

• W3C Workshop on Rule Languages for Interoperability, 27-28 April 2005, Washington, D.C., USA.– Ontology and Rules on the

same level, connected, and both treated as data.

• W3C Workshop on Frameworks for Semantics in Web Services, June 9-10, 2005, Digital Enterprise Research Institute (DERI), Innsbruck, Austria.

54

3. Some Resources• Enterprise Architecture and Service-Oriented Architecture Fad

or Foundation?:– The goal of this series was to provide a clear path (J2EE/.NET for

building Web services/SOA-based applications) in complex environments using open-standards to improve interoperability across all Governmental Agencies. It is far from perfect, but we gave it an honest effort:

• Part One: http://www.dmreview.com/article_sub.cfm?articleId=1025078• Part Two: http://www.dmreview.com/article_sub.cfm?articleId=1025567• Part Three: http://www.dmreview.com/article_sub.cfm?articleId

=1025870• Part Four: http://www.dmreview.com/article_sub.cfm?articleID

=1026284• Semantic Technologies and Ontology Engineering for

Enterprise Architecture, Ralph Hodgson, TopQuadrant:– EA Summit 2005, May 22-24, Miami, Florida.

• Composite Applications with Multiple Ontologies:– For example, Digital Harbor (http://www.digitalharbor.com/) at the

SWANS Conference Trade Show.• See Leaderboard, page 70, in InfoWorld, May 2, 2005, Issue 18.

http://www.dmreview.com/article_sub.cfm?articleId=1025078









http://www.dmreview.com/article_sub.cfm?articleID=1026284



http://www.digitalharbor.com/

55

4. Chief Architects Forum Comments

• Service Component Based Architecture (SCBA) and Core.Gov for Enterprise Architecture (January 26, 2006):– ‘Did not want to throw “component” away in SCBA

V3.4,’ but you should drop components and component registration since it causes confusion and implementation problems because the FEA does not couple EA to SD (software development) and manage SD across the enterprise:

• Services are things that you “publish, find, and bind to” like in a SOA, while

• “Components” implies physical things you move around – put in a Core.gov - and try to get reused in some way.

56

4. Chief Architects Forum Comments

• Service Component Based Architecture (SCBA) and Core.Gov for Enterprise Architecture (January 26, 2006) (continued):– Three lessons learned:

• We are not using industry best practices of governance for ROI in software component development and reuse. (See May 14, 2004, Expedition Workshop).

• The E-Gov Initiatives (e.g. E-Grants) were generally not architected with open source, reusable components (See January 26, 2006, Expedition Workshop).

• The use of services in SOA is a more general and useful architectural pattern than is components in the FEA (See January 26, 2006, Expedition Workshop).

Documents

Collaborative Expedition Workshop #47, Tuesday, January 24, 2006,