Ocean Observatories Initiative OOI Cyberinfrastructure Common Operating Infrastructure Subsystem Michael Meisinger, Munindar Singh, Von Welch OOI Cyberinfrastructure

Ocean Observatories InitiativeOcean Observatories Initiative

OOI CyberinfrastructureCommon Operating

Infrastructure SubsystemMichael Meisinger, Munindar Singh, Von Welch

OOI Cyberinfrastructure Life Cycle ObjectivesMilestone Review, Release 1

San Diego, CAFebruary 23-25, 2010

OOI CI LCO Review, Feb 20102

Overview• COI Subsystem Overview • Capability Container and Messaging• Resource Management• Resource Governance & Background• Federating Facilities • Security and Identity Management• Putting it all Together


Common Operating Infrastructure

Is the integration & communication environment for all the other subsystem services

Common Operating Infrastructure

Sensing & Acquisition

Data Management

Analysis & Synthesis

Identity Management

State Management

GovernanceFramework

Resource Management

Planning & Prosecution

Exchange

Service Framework

Presentation Framework

Common Execution

Infrastructure


COI ServicesService

FrameworkResource

ManagementPresentationFramework

CapabilityContainer Exchange

IdentityManagement

GovernanceFramework

DistributedState Managemt

2940-00001 OV2 CI


Scope of Release 1• Provide a basic “capability container” for

infrastructure and application service integration• Secure reliable asynchronous messaging• Governed resource sharing and access• Support federation of facilities• Monitoring service interactions for compliance• Resource registration and basic resource life

cycle management• Basic user and external interface support• Support for multiple programming languages


Outlook: COI Release 2• Advanced resource management

– Life cycle services– User resource activation

• Advanced capability container with full federated facility support

• Interaction specification and enactment• Embedded capability containers• Advanced system operations and

monitoring

Out of scope

for release 1


Risks• High

– Capability Container integration– Common message format– Governance framework– Secure messaging– Service integration platform

• Medium– Authentication, Policy enforcement– Resource registry– Distributed state framework– Service registry

• Low– (not much because all other subsystems depend on COI)– User interface platform

Iteration 1 and 2 (Inception period) prototyping activities


Capability Container and Messaging

• Use Cases– Deploy a service in a capability container – Enroll in an Exchange Space– Send a request message to a service – Access to infrastructure services


Capability Container


Secure Reliable Messaging

CapabilityContainer

CapabilityContainer

CapabilityContainer

CapabilityContainer

CapabilityContainer

Capability(Service)

Capability(Service)

Capability(Service)

Capability(Service)

Capability(Service)


The “Exchange”

• Applications communicate through Exchange Spaces• Exchange Spaces are namespaces of “communicators”• Applications need to enroll in Exchange Spaces• Governance applies within Exchange Spaces


Exchange Space and Points

• Exchange Points are the message routing and queuing resources of Exchange Spaces

• Communicators play the role of Producer, Consumer, and Distributer (Broker)


Messaging Abstraction

CapabilityContainer

CapabilityContainer

CapabilityContainer

Send Message From: “name2”To: “name4”In: Exchg-Space1Action: “invoke-service”<Args><Data>


Common Message Format• Capability Container

– Provides message handling through interceptors

• After a service sends a message, before it receives a message

• Message signing and validation• Policy enforcement• Governance tracking

– Provides a common message format for all CI messages

• Based in ACL FIPA message structure• Content, encoding, ontology


Risk Mitigation Development• Out of the box:

– RabbitMQ AMQP message broker – Python: flexible and powerful applications– Twisted: distributed application framework– txAMQP: messaging library

• Integration– Message abstraction for services– Intercepting message handler (in and out)– Policy and governance integration (via agents)– Development console


CI Resources

“CI governed” Resource• Standard and user metadata

attributes (in OOI convention)• References to other resources• Categories

– Information resource– Physical (stateful) resource– Taskable resource


Resource Management Services

Resource Agent

Resource Registry

Resource (external)

2940-00005 OV2 COI


Services and Resources as Agents

Resource Agent

Resource Agent

Proxy Agent

CapabilityContainer

CapabilityContainer

CapabilityContainer

CapabilityContainer


Scenario• An instrument, a physical resource, is

represented by an agent to the system and its users

• Users request control of the instrument• Capabilities are projected into another

domain of authority by a proxy agent


Resource Governance


Motivating Governance• Administering collaborations

– Based on framing normative relationships among peers

– Abstracting away from low-level details• OOI, broadly: many stakeholders; many

resources; longevity of decades• Exchange spaces, narrowly: abstractions for

communicating; assembly of multiple topologies for messaging; analogous to traditional enterprise integration patterns


Elements of a Service Engagement

• Enactment: doing the domain work – what the end user cares most about

• Administration: captured via contracts– Partnerships– Rules of encounter

• Identity• Enforcement


What is Governance?Broadly, administering service engagements• IT Governance: How IT resources are

administered• SOA Governance: How services are created,

deployed, removed, …• Currently, governance is manual

– Low productivity– Poor scalability for fine-grained, real time governance

decisions– Hidden, implicit considerations yield low confidence

and poor maintainability


Why Governance?• Stakeholders using resources to best

serve individual and collective needs– Share resources in a controlled manner– Configure and reconfigure dynamically– Enable unanticipated uses for resources– Respect human organizational needs

• In a nutshell, stakeholders administer themselves


Separation of Concerns• Protocol: specifying the interactions

among autonomous parties• Policy: specifying the decision making of

each autonomous party as it participates in various protocols

• Behavior: specifying the implementation that realizes the interactions


Principles of Governance: 1• Vividness of Modeling

– Grounded in applications; modeled entities are real

• Autonomy of Participants– Stating rules of encounter; omitting policies

from specifications• Centrality of Organizations

– Modeling communities, facilities, the OOI; specifying rules of encounter; monitoring contracts; sanctioning violators


Principles of Governance: 2• Minimality of Operational Specifications

– Leaving restrictions unstated except where essential to correctness

• Institutional Actions– Creation and manipulation of commitments; granting

or denying powers, authorizations; effecting sanctions– Separation of concerns from those of operational

interactions• Reification of Representations

– Explicit: hence, inspectable, sharable, and manipulable



Exchange Space Use Case



Messaging View of Enrollment



Community Affiliation Use Case



Combined Scenario, Schematically



Risk Mitigation Development• Out of the box:

– RabbitMQ AMQP message broker – Python: flexible and powerful applications– Twisted: distributed application framework– txAMQP: messaging library

• Integration– Message abstraction for services– Intercepting message handler (in and out)– Policy and governance integration (via agents)– Development console


Security and Identity Management


Secure Messaging and Identity Management

• Identity Management (IdM) is the management and communication of user identities and attributes for use by Governance, Audit and other systems.– Federated IdM is the use of user information

from one organization in another organization.• Secure Messaging encapsulates message

authentication, integrity and confidentiality.


Scenario• User is member of organization acting as an

identity provider.• User performs one-time registration with COI.• User then routinely authenticates with COI using

identity asserted by their home organization.• After authentication, can participate in Secure

Messaging: enrolling in exchange spaces and performing operations managed by Governance.


Architecture Goals• Leverage user identities from their home

organization (identity provider).• Allow for multi-homing of users and migration of

users between organizations.• Allow for technology changes by providing for

abstraction layer between technology at user’s home institution and COI.

• Allow for trade-offs on ease-of-use versus strength of security.

• Allow for both thin (web browser) and thick (command-line) clients.


Architecture Overview


Architecture Overview


Technology Overview• Utilize InCommon as the IdM federation of

choice for U.S. higher ed. today.• CILogon builds on InCommon to support

thick clients.– Expect to be needed for next few years.

• Security messaging leverages XML Security Messaging, conceptually at least.


Putting it all together• A service gets deployed on a capability

container– Initialization: service enrolls as

“communicator” into an Exchange Space• A user application looks up the service

and sends a service request message– Look up the service in the service registry– Enroll in necessary exchange spaces/points– Send a message via the exchange


Enrolling in an Exchange Space

2940-00061 OV6 COI


Send a message

2940-00063 OV6 COI


Receive a message

2940-00062 OV6 COI


COI Technology List• Messaging

– RabbitMQ AMQP broker (with federation extensions)– Distributed IPC Facility Implementation

• Capability Container– Python, Twisted, txAMQP– Java, Spring– Open Telecom Platform (OTP) style service deployment– FIPA ACL Message Format (standard headers), DM Common Format

• Policy and Governance– Rules engine (Jess/Pyke)

• Identity Management– CIlongon– Internet2 Security infrastructure

• Resource Management– Redis Attribute Store (with DM enhancements)

• Presentation Framework– Portal framework (such as Django, Drupal)


Elaboration Plan• Elaboration Iteration 1

– Secure messaging (using IdM technologies)– Policy enforcement for resource/service requests– Integration of DM metadata model in resource registry– Distributed service state coordination via the AttributeStore

• Elaboration Iteration 2– User registration with external identities– Policy definition and enforcement– Demonstrate federated facilities– Integrated basic capability container, ready for use by – Demonstrate integration with CEI provisioning and DM

distribution, storage and inventory– Initial web user interface framework


Thanks!


Capability Container Components (1)


Capability Container Components (2)


Resource Agent Services


Policy and Governance Services


Exchange Space

• Exchange Space is comprised of– Distributed Application Facility (DAF)– Distributed IPC Facility (DIF)


Back-End Infrastructure


Exchange Points and the DIF


Message Brokers over DIF


Messaging Service Interfaces


Extra Slides


Registration Service


Authentication (thick client)


Authentication (thin client)


Secure Messaging Data Model


Documents

Ocean Observatories Initiative OOI Cyberinfrastructure Common Operating Infrastructure Subsystem Michael Meisinger, Munindar Singh, Von Welch OOI Cyberinfrastructure