© RJD 2005 / 04 / 26

Working Notes on

Protocol Centric Compositional Architecture:

Section 1 – Basics of Compositional Architecture

by Robert J. DuWors

04 / 26 / 2005

2© RJD 2005 / 04 / 26

Innovating the next Dominant Design for Large Scale Distributed Systems

Section 1: Compositional Architecture

Need: Architectural understanding of what we really do in practice everyday on the Web and to jump start better architectures for more effective solutions.

Solution: Compositional Architecture is the first architectural framework as flexible as the Web itself and offers a powerful framework going forward.

Section 2: Protocol Translating User Agents and Models

Need: Provision of well structured customized functionality and unique User Experience for specialized User Community markets at competitive cost.

Solution: Application Level Protocols (ASLP) link Service Oriented Models (SO-M) that extend core product functionality flexibly and inexpensively, to Service Oriented User Agents (PTUA) that tailor User Experience on the fly.

Section 3: Distributed Capability Architecture

Need: Trustworthy security from top to bottom of all architectural levels without too much or too little securing of highly scalable, dynamic, distributed systems.

Solution: Permission Rule Based Capability model for coarse through fine grained control, and for scalable, adaptable “just right” security controls from top-to-bottom and end-to-end: a big step beyond the currently dominant but troubled Access Control List (ACL) model.

3© RJD 2005 / 04 / 26

Compositional Architecture:The Great Leap Forward

Opportunity: Gain major competitive advantage from an innovative architectural framework that describes what we do in practice every day on the web and propels product development going forward.

Needed: A new dominant design to kick start the next generation of distributed system architecture

Solution: Compositional Architecture

Challenge

• Distributed databases replication as the central mechanism of distributed system design failed in the 1980s

• Boeing, GE, etc

• The Distributed Object Orientation Approach fell short in the 1990s

• CORBA is moribund and OSF RPC/DCOM/COM+ is of limited utility in a highly proprietary environment

• ISO Open Distributed Processing (ODP) went nowhere

• Distributed Objects work best, if at all, for relatively tightly coupled applications in LAN environments

• Distributed Objects form really hideous de facto protocols at application level (multiple method

invocations running through a “Sea of Objects” that are almost never documented as such)

• The original HTML hypertext model is worn out in the 2000s (1st decade – what do we call this period? The 00s?)

• the Web involves a great deal of computation and persistence as well as display dynamically delivered

• but the URL/URI did break wide open the notion of linkage in distributed systems (Internet style)

• The line between “data” and “executable” has been irretrievably breached

• network resources are freely distributed, combined into new entities that are executed and used, and

recombined as needed

• there are precious few programming languages or data models with this ability and none that can deal

with the rich interaction of multiple technologies on the Web today, e.g. HTML, CCS, Javascript, XML,

XLST in one dynamically constructed entity (“transclusion”) such as a Protocol Translating User Agents

4© RJD 2005 / 04 / 26

A Quick Introduction to Compositional Architecture

5© RJD 2005 / 04 / 26

Compositional Architecture in a Nutshell: A Sneak Preview

• Compositional Architecture describes distributed systems in which:• the basic entities are composed and recomposed out of network resources as needed on the fly (dynamically)• all entities communicate with each other exclusively by means of appropriately defined “by value” protocols

“over the wire” for basic retrieval, transfer, loading and composition of entities, and expressing the application

level semantics of interaction between entities• the entities in Compositional Architecture are made out 4 elemental types (fundamental units):

“Earth” - Units of Persistence (uP)

“Air” - Units of Exchange (uE)

“Fire” - Units of Computation (uC)

“Water” - Units of Transduction (uT)• the basic rules of Compositional Architecture govern how these 4 elemental types can be combined and

transmuted

• URIs lubricate the ability to reach out to network resources by means of the appropriate references and protocols:• to retrieve the network resources as needed • or to make use of them in place as appropriate• URIs are to Compositional Architecture as symbol tables are to compilers and loaders (linkage mechanism)• blame the explosion of Compositional Architecture on the wide spread use of URIs! “Transclusion”!

• Computational Architecture replaces the long outdated Hypertext model as the dominate Web model

• Because so much dynamic composition is going on:• Compositional Architectures often employ conventional multi-level “uses” relationships between layers in

“stacks” • but Compositional Architectures distinctively also make extensive use of “meta” relationships (“describes” or

“defines”) between layers in “meta-meta stacks” ala the 4 level UML meta-meta layers• Compositional Architectures make extensive use of three types of linkage binding:

• Early binding - pre-defined linkages established well before use• Late binging - pre-defined linkages established at the time of use• Meta-binding - rules for creating the entity itself & defining linkages at the time of use

6© RJD 2005 / 04 / 26

Compositional Architecture in Context: A Further Sneak Preview

• Compositional Architecture (CA) is a dynamic End-to-End network architecture

• Subsumes other common architectures: Peer-to-Peer, Client-Server, Star, n-Tier

• Subsumes other common paradigms: Object Orientation, File Processing, and Relational Data Model

• CA is a real architecture, not an overgrown Software Engineering technique

• CA is not in direct conflict with OO, but extends beyond the constricting limits of OO

• Object Orientation corsets data and computation to the point of strangulation: information hiding is evil!

• CA is a genuine “post-OO” paradigm

• CA emphasizes the dynamic as much as the static

• in entity relationships

• in meta-level relationships

• CA overthrows 60 years of “data versus executable”

• in most software languages and methodologies, data and computation are segregated or rigidly combined

• CA introduces meta-binding in addition to early and late binding

• fits naturally with Document Oriented Design containing both data and business rules in one document

• promotes higher level knowledge representations

• CA is uniquely suited for Distributed Computing

• highly dynamic: topology independent and self constructing

• universal: technology independent and can describe anything that can be built

• equally friendly with any imperative, OO, functional, logic, or temporal frameworks

• the only theoretical architecture that encompasses what we do in practice every day on the Web!

• Protocol Translating User Agents (PTUA) / Service Oriented Models (SO-M), and the Distributed Security Rules

Capability models follow naturally

7© RJD 2005 / 04 / 26

• A network is composed of nodes, and edges called links that represent communications channel (“connectivity”) linkage between the nodes

• A directed graph defines which nodes are connected to which nodes by links, and is known as the network’s topology:

• Beware: perceived network topology typically changes by level of abstraction, e.g,

• A network diagram (graphic) provides a (partial) abstract representation of network resources:

• traditionally nodes represent entities of some kind, e.g. database servers, routers, “objects”, etc.

• links represent communications channels in the Information Theory sense of the word.

• network topology can be highly dynamic, but usually isn’t, except possibly in networks with a large number of mobile devices, or some kinds of application level network services, e.g. IM.

• A Distributed Computing System is a network that supports four fundamental units (types):

• All four types can reside at a node:

• Units of Computation (roughly “programs”, “processes”, “scripts” and “rules”),

• Units of Persistence (roughly “data”, “memory” and “archives”)

• Units of Exchange (roughly “messages”)

• Units of Transduction (roughly “inputs-outputs” including the “User Interface”).

• Only the Unit of Exchange can move across links between nodes while carrying potentially all four fundamental units within it.

A Universal Model of Distributed Computing and the 4 Fundamental Types

Physical Network Topology IP Transport Level Topology Application Client Server Topology

8© RJD 2005 / 04 / 26

Can contain all fundamental units:uC, uP, uE, uTCan be created by entities external to the network (import) Can be created uC (export)Can be received (import) or sent (export) from edge node by uCCan be read and updated by uCCan be persisted by uPCan be carried by uECan be consumed by entities external to the network (export)Can be destroyed uC (import)

Passive, except to import or export externally to the network

Define the “inside” from the “outside of the network: represents information in the form of all fundamental types destined to or arriving from outside of the network at edge nodes.Can only sent or received by uC at edge node.Can only be created by uC and consumed externally (export) or created externally and received by uC (import).Carried inward and outward by some from of channel that is NOT internal to the network, i.e. not by a network link.

uTUnits of Transduction

“I/O including UI”

Can carry all fundamental units:uC, uP, uE, uTCRUD by uCFilled with fundamental units and emptied by uC at each node

Passive, except to cross edges

Carrier mechanism between nodes: information in the form of the fundamental units that can be sent or received from node to node.Can only be carried across links between nodes.Can only be filled with fundamental units and transmitted, or received and emptied by uC at each node.

uEUnits of Exchange

“messages”

Can make persistent all units:uC, uP, uE, uTCRUD by uCCan be carried between nodes by uE

PassivePreserves unaltered state of all fundamental units until externally transformed to new state or destroyed by uC.

uPUnits of Persistence

“data and memory”

Can Create, Read, Update, and Destroy (CRUD) all fundamental units: uC, uP, uE, uTCan be persisted by uPCan be carried by uECan put in and take out all fundamental units from uECan import and export uT

ActiveCan change their own state (compute) and the state of all fundamental units.Can only be active when resident at a node.Can send uE outward from node, and receive uE inward to node, across communications channels (links).Can send uT outward from edge nodes (export) or receive uT inward (import) at edge nodes.

uCUnits of Computation

“programs, scripts and rules”

InteractionAction TypeDescriptionAbbreviationFundamentalUnits

The Four Fundamental Units of Distributed Computing

9© RJD 2005 / 04 / 26

The Four Fundamental Elements of Distributed Systems Architecture Applied to Protocol Translating User Agents

• Units of Computation (uC)

• Units of Persistence (uP)

• Units of Exchange (uE)

• Units Transduction (uT)

uE carrying

uP and uC

Logical Protocol Translating User Agent

Application Protocol Handler

Translation of

User Inteface

uP and uC

Transduction

Physical Display / Output

Physical Input

External World Physical Device

uT

Signal

Signal

Network Boundary

Outside World

10© RJD 2005 / 04 / 26

What’s the Big Deal?

“IT architecture‘s attention shifts from the fixed structure of functionality towards active composition of functionality”

Two basic concepts: Dynamic Composition and Protocols.

• Unlike Actors and Distributed OO, nodes are NOT classic self contained and bashfully self hiding Objects.

• Nodes represent network resources that can hold (be bound to) the fundament units whose content are assumed to be fully visible subject only to the constraints of security policies which enforce whole or partial opaqueness.

• Thus nodes are containers whose total content is composed dynamically (composition as a function of time) out of ALL fundamental units: Units of Computation and Units of Persistence with Units of Exchange and Units of Transduction:

node total stateT: = (uC1T & uC2

T … & uCnT) & (uP1

T … & uPmT) & (uE1

T … & uEpT) & (uT1

T … & uToT)

where & is the composition operator (“transclusion” in Web Architecture)

• Messages carry much more than just “data” but also “program and rules” and “I/O” and even other “messages”.

• Messages are not rendezvous tuples like LOTOS or memory tuples like Linda which are major alternates to the Distributed OO model (although both of tuple models do allow their elements to be uC or uP or uE or uT!)

• Distributed System messages (uE) can carry any mixture of uC & uP & uE & uT.

• The central use of composition necessitates that the contents of each fundamental unit, especially the Units of Computation, are NOT “hidden”, but rather are accessible to be assembled on the fly and immediately used as needed

• Protocols (uE + the rules that uC use to produce, transmit, and consume uE) are the means to get the content knowledge contained in the fundamental units around the network for integration (composition) and use at the appropriate node at the appropriate time.

• This model handles particularly well distributed dynamic rule assembly and documents with mixed data and rule contents thereby allowing application “rule flow” as readily as application “data flow” e.g. • a JSP (uC) assembled web page (uE) using multiple sources (uP and others) that is sent out with the application

content (uP etc.) embedded in HTML (uT) and in Javascript (uC) to a browser (uC).• an XML document (uP) that contains elements some of which are “data” (uP) and some of which are “rules” (uC).• the Dynamic Capability Security model in these notes makes extensive use of assembling security rules (uC) from

multiple sources (using uE) on the fly to create an integrated rule set (a new uC) for immediate execution.• Gosling’s original idea of “executable” postscript messages (uC in uE) in the defunct NEWS windowing system.

11© RJD 2005 / 04 / 26

What’s the Big Deal? –cont’d

Knowledge Representation and Automation – The Holy Grail of IT

• Units of Computation, Units of Persistence, and Units of Transduction are fundamental to the knowledge representation and automation of all applications

• Units of Exchange hint at the existence of the network and may or may not be fundamental to the knowledge content of an application

• Networks of nodes and links do not need to be considered part of the fundamental knowledge content of the application, although in some applications they might be

• The 4 fundamental units are the alpha and omega of all IT architecture and design• they are either directly expressed or readily derived from the requirements, which after all is the meta- knowledge representation of the desired target application• Architectural Design establishes and Detailed System Design further evolves the details and the binding rules (time and placement) of the fundamental units• all knowledge contained in the fundamental units is fully visible unless explicitly hidden by security policy• new knowledge is formed out of composing the fundamental units together, which is a very fluid view of form and function (much better than what Frank Lloyd Wright had available to use as his physical units of construction!)• promotes separation of an explicit end user intelligible knowledge layer from infrastructure layers• can be mapped to less expressive and more restrictive models such as Object Oriented and Relational Data Models – but it should be possible to map the fundamental units back out for system refactoring or re-implementation using entirely different design and implementation techniques• like fractals, the fundamental units are self similar at all scales from the most expansive Enterprise Architecture to the darkest corners of PERL programming• the fundamental units fully exploit the potential of the von Neumann architecture that IT has avoided for the last 60 years, i.e. the interaction and transforming power of “weaving together” memory, computation, data transfer, and input-output• “rule flow” applications can be built as easily as “data flow” applications for highly flexibility

12© RJD 2005 / 04 / 26

What’s the Big Deal? –cont’d

Architecture is the answering of Application Domain Wants with IT Solutions

• Wants and Solutions have a non-linear feedback relationship:• Changes in Wants, triggers changes in Solutions.• Changes in Solutions (in particular disruptive technologies), induces changes in Wants:

“Perceived IT Technology shapes Requirements and Requirements shape desired IT Technology.”

• Therefore, any linear IT methodology is a crock (in particular the Waterfall model).• n.b. True innovations wreak disruptive havoc on both pre-established Wants and Solutions, such as

when existing business and technical models are supplanted by new ones.

• Architecture must model the application domain, the IT solution domain, and the mapping between the two:• Almost all current IT tools, techniques, and methodologies fall squarely on the side of modeling the IT solution domain – thus they are really Software Engineering tools even if they occasionally masquerade as Architectural tools, e.g. UML and MDA, OO[AD], and DBMS.• We need much better ways to enter into the application domain:

• which inevitably means using the knowledge representation appropriate to the application domain, e.g. applied mathematics for conventional engineering applications.• all of this is subject to Philip Amour’s view of software as the automation of knowledge and software creation as subject to the laws of the levels of ignorance (another source of non-linearity).

Directions in Architectural Methodology• Define an ontology of the application space (i.e. “the things assumed to exist in the domain of

discourse and the relationships between them”).• Define within the ontology the manner by which entities are reified (made into manipulatable objects) expressible by the 4 fundamental units of distributed systems.• Define the fundamental structural elements and protocols of the systems by which the entities interact.• Turn all of the above over to the Software Engineers as the top level architecture description for them to build.

13© RJD 2005 / 04 / 26

OSI NetworkLayer 3

OSI TransportLayer 4

OSI Data LinkLayer 2

OSI Physical LinkLayer 1

OSI Meta ModelLayers 1 thru 4

Architecture

Uses

ConcePTUAlDBMS Schema

User DBMS Views

InternalDBMS Schema

PhysicalDBMS Data Organization

ANSI/SPARCData Independence

Architecture

DefinesMinimumfor

M2 Metamodel CWM, UML Metamodel e.g. Class, Interface, Attribute

M3 MetaMetamodel Meta Object Facility (MOF)

M1 Model 1

Specific UML model e.g. Collection, SymbolTable, PurchareOrder, etc.

M0 “Objects” 2

Specific Object Instances e.g. specific PurchaseOrder number 4569105

OMG 4-LayerUML

Architecture

Describes and Defines

Examples of Increasing Architectural Sophistication from Conventional “Uses” Layered Architecture thru to Full Fledged Meta-Layered Architecture

In which each Meta-Layer also defines the Meta-Binding Rules (composition) for the Layer beneath it

1, 2 What many take to be the whole UML story

14© RJD 2005 / 04 / 26

Comparison of Object Oriented and Compositional Architectures

  Object Oriented Architecture Compositional Architecture

Encapsulation The most fundamental property of what makes an object and “object”

Rejected in favor of total information transparency for (re)composition as needed

Communication Rigidly formatted Method invocationse.g. CORBA, COM+, ODP

General Messages: (uE & uT)e.g. XML, MIDI, MPEG, and CORBA, COM+, ODP, etc.

Meta Level Class Inheritance “Generative” Compositional Rules

Instantiation Subclasses and Objects Composition & Transclusion

“Behavior”(Event Sequencing)

None Protocols define allowablesequences of messages

15© RJD 2005 / 04 / 26

The Architectural Relationship to Buzzwords de Jour

Question: What do the following have in common?

asynchronous programming, message-driven programming,

event-based programming, n-tiered distributed systems and

service-oriented architecture, etc.

Answer: They are all ways to dance around protocol centric architecture and issues without mentioning protocols!

API side protocol issues: Asynchronous programming, message-driven programming,

event-based programming

Structure of functionality: N-tiered distributed systems (glued together by many protocol definitions!)

Vague, but close: Service-Oriented Architecture needs to be extended into application analysis and

design in addition to deployment architecture. Note: in the PTUA architecture

every element is “service enabled”, in fact all nodes are “service only,” except for a

PTUA which is half service oriented on its backend and half User Interface oriented

on

its front end!

Totally missing: Architecture of Compositional Systems

Indeterminism: All of the above have the problem of conquering indeterminism, which generally is

avoided by software engineers like a scalded puppy running from boiling water!

But indeterminism is shared by GUI design (don’t know what the user will do next, and

that is viewed in terms of protocol design by the PTUA/M architecture), and Real Time

Systems (caveat: Real Time Systems also have the major attributes of continuous

systems which are a wholly different sort of beast!), and Data Communications

Systems (which simply involve low level “data moving” protocols). The buzzwords

“asynchronous programming” and “event-based programming” are the software

side of indeterminism stemming from the underlying protocols. In general

indeterminism can be addressed by the formal techniques of re-active systems:

state machines, Petri nets, process algebras, etc. (Danger, danger, none of

these can handle continuous systems which is what make Real Time Systems both

“real” and difficult to build, i.e. you have to borrow from the applied mathematics

of signal processing interlaced with re-active system behavior.)