Overview of Pegasus An Open-Source WBEM implementation 12 June 2001 Michael Brasher Karl Version

Overview ofPegasus

An Open-SourceWBEM implementation

12 June 2001Michael Brasher (m.brasher@bmc.com)

Karl Schopmeyer(k.schopmeyer@opengroup.org)Version 1.0

Agenda

Overview -What (and why)is Pegasus? The Pegasus Environment The Pegasus Software Architecture Pegasus Status Today Plans The Pegasus Project Pegasus and ??? A Challenge for the Future

1. Overview1. Overview

What is Pegasus?

Pegasus is an open-source reference implementation of the DMTF WBEM specifications

Pegasus is a work project of the TOG Enterprise Management Forum

Pegasus is a platform for building application management

Pegasus is a function-rich, production-quality open-source implementation designed to be used in high volume server implementations.

Why Produce Pegasus?

Demonstrate manageability concepts. Provide additional standards for WBEM Provide a working implementation of WBEM

technologies Provide an effective modular implementation Support other TOG manageability standards Base Platform for Open Group Application

management Projects

Origins of Pegasus

Pegasus was initiated in 2000 by the Open Group in collaboration with: BMC Software IBM Tivoli Systems

Key Pegasus Objectives

Open source MIT License, open availability

Portable Code designed for portability

Efficient and Lightweight Implemented in C++

Standards Based DMTF CIM/WBEM compliant

Modular and extensible Use both internal and external modularity

Pegasus Working Group Philosophy

Manageability not management The working group’s objective is not to manage systems but

to make them manageable by promoting a standard instrumentation environment

The actual management of systems is left to systems management vendors

No standards without implementation The process of implementation provides a rigorous process

for testing the validity of standards Therefore all standards must be validated by

implementation

Open Source

Code and documentation freely available Open Group and Source Forge

MIT source License Open to contributions

Portable

Designed for multi-platform, mulit-os, multi-compilers.

Platforms today UNIX (AIX, HPUX, Solaris) Linux Windows Platforms (NT, 2000, 9x) Compaq Himalaya (Tandem)

Efficient and Lightweight

Core written in C++ Designed for execution efficiency

Standards Based

Based on DMTF CIM and CIM-XML specifications

Open Group is active partner in DMTF Growth through participation in specification

growth Commitment to continue DMTF compliance

Modular and Extensible

Minimize core object broker. Maximize extensibility through plug-in components Component types

Providers Provider interfaces Clients Repositories (additional repository handlers) Manageability service extensions Protocol Adapters Modules (extend and modify core functions)

Modularity is key to doing parallel development and allowto extensibility

2. The Pegasus Environment2. The Pegasus Environment

Pegasus Architecture

ConsumersConsumersClients

CIM Server

ConsumersConsumersProviders

CIM/HTTPInteroperable*

Interoperable* CIM/HTTP In-Process

Services

StandardInterfaces

Key Interoperability Interfaces

CIM Object Manager

CIM Providers

Application

Manageability to Manager•Multiple management systems•Common open manageability

Object Manager / Providers• Multiple Providers•Encourage common providers

Management System

EnterpriseManagement

Console

ApplicationApplication

Application

Provider / Resource Interface•Protect Applications•Make application management easy

The CIM Operations

ConsumersConsumersClients

CIM Object Mgr

ConsumersConsumersProviders

CIM/HTTPInteroperable*

Interoperable* CIM/HTTP In-Process

Services

StandardInterfaces

Repository

CIMOperations

Repository

Indicators

Operations Routing

Class Operations Routed to the Class Repository

Instance Operations To Provider if Provider Qualifier exists To Instance repository if no Provider

Instance routing at Class Level Today Issues – Routing at instance level

Modularity and Extensibility

Providers Grow with DMTF provider concepts

Provider Interfaces Protocol Adapters (connectors)

Client - Xml-cim today (Soap, etc. in future) Provider, service, repository, etc.

Modules Modularize core so it can be extended and modified

through attachable modules Manageability Service Extensions

Think super providers

Building A Modular Manageability Environmnent

CoreObjectBroker

XML-CIM

Module

Module

Module

Module

Connector

Provider

Connector

Provider

Connector

CIM ClientConnector

. . .Resources Resources Resources

ServiceExtensionService

ExtensionServiceExtensionService

Extension

RepositoryRepository

Undefined

CIM ClientCIM Client

RepositoryRepository

RepositoryRepository

ServiceExtensionService

ExtensionServiceExtensionService

Extension

ManagementSystem Connector

Pegasus Manageability Environment

CIM Object Broker•Provider Registration •Service Registration•Request Routing •Securiy

Consumers

Providers

AICProvider

Apps OS Etc.

ARMProvider

Resource

Providers

Broker

Consumers Gateways Apps

XML/CIMConnector

ManagementSystem Connector

ApplicationConsumer

ClassRepository

Services core*** additional

InterfaceFor Spec

CIMOM ManagementSystem

. . .SNMPProvider

ManagementSystem

ApplicationConsumer

ApplicationConsumer

InstanceRepository

Provider Interoperability In the classical architecture, interoperability is only supported

between the client and server. In addition, the Pegasus architecture aims to support

provider/server interoperability. Goal

Write a provider once and run it under any CIM server implementation.

Provider/Server Interoperability: Participating in efforts to standardize the Provider/Server protocol. Proposing provider API standards. Writing adapters enabling Pegasus providers to run under other

CIM servers. Adapters enabling other providers to run under Pegasus

Important Provider Interfaces

SUN WBEM Provider Interface Java based Classes, etc. “similar” to Pegasus

C Provider Interface Sun has a specification here.

We will support multiple provider interfaces and language bindings.

In-Process and Out-of-process Providers Today Pegasus based on shared Library Providers Extend to

Internal Providers IPC based Providers Providers in Remotes systems

Objectives: Write Provider once and compile/link for different

environments Technique

Use connectors as basis for provider/CIMOM communication Issues

Security, discovery

Modules

The core server functions are organized into loadable modules.

Standard APIs are defined for each module. Alternative implementations can be

provided later without recompiling the Pegasus server.

Manageability Service Extensions

Super Providers Access to the Core Broker

Example Services

Indication Management service. Query engine service. Class repository service. Instance repository service.

Connectors (Protocol Adapters) Functions

Adapt to different protocols Characteristics

Protocol Encoding Security Discovery

Examples Xml-CIM Local Protocols Soap WMI

Pegasus Core

Xml-cimConnector

PegasusProvider

Connector

RemoteProvider

Xml-cimClient

Connector

SoapConnector

Xml-cimClient

JavaProvider

ServiceExtensionService

ExtensionServiceExtensionService

Extension

RepositoryRepository

Pegasus Interfaces

Common Interface base for Clients, providers,

services, connectors Based on CIM

Operations over HTTP Additional functions for

each interface Interfaces separated

from implementation

CoreObjectBroker

Provider

Connector

CIM ClientConnector

3. The Pegasus Software 3. The Pegasus Software ArchitectureArchitecture

Introduction

Major Components

Client

CIM Server Repository

ClientCIMClients

ClientClientCIMProviders

Topics

Communication. Representation of CIM Elements. The Client Interface. The CIM Object Manager. The Provider Interface. The Repository Interface.

Communication

Pathways of Communication

Client

CIM Server CIMRepository

ClientCIM Clients

ClientClientCIM Providers

Component Location

A component may be located in one of three places with respect to the CIM Server. In-process. Local out-of-process (on the same machine). Remote out-of-process (on another machine).

For example, a provider may be in-process, local, or remote.

Component Location in Pegasus Today

In-Process Local RemoteClient XProvider XRepository X

Possible Communication Mechanisms Components could potentially communicate

with the CIM Server using the following mechanisms: CIM/HTTP (remote). Proprietary TCP-based protocol (remote). Direct call (in process). Shared memory (local). Named pipes (local).

Communication Mechanisms in Pegasus

In-Process Local RemoteClient CIM/HTTPProvider Direct CallRepository Direct Call

Client Communication

Uses CIM/HTTP as sole protocol. Asynchronous socket I/O. An efficient XML parser. Fast enough to eliminate the need for a

proprietary protocol.

An Efficient XML Parser

No memory heap usage during parsing. Modifies message in place to avoid copying. Non-validating parser (“loose validation”).

HTTP Implementation

Uses asynchronous socket I/O in conjunction with message queues to achieve optimal throughput.

Provides entry points to adding web server capabilities such as putting and getting of documents (to support remote upgrade and deployment later on).

Proposals

Support out-of-process providers (local and remote).

Support out-of-process repositories (local and remote).

Location independent provider development.

Representation of CIM Elements

Representing CIM Elements in Pegasus with C++

CIM Data Types in C++

Uint8 Sint8 Uint16 Sint16 Uint32 Sint32 Uint64 Sint64 Real32 Real64

Boolean Char16 String CIMDateTime CIMReference

CIM Values in C++

CIM values (property, parameter, and qualifier values) are represented using the CIMValue class. This class: Encapsulates a union of all CIM data types. Has a type member indicating the type

currently being represented. Provides access/modifier methods overloaded

for each CIM data type.

CIM Elements in C++

CIMClass CIMInstance CIMProperty CIMMethod CIMParameter CIMQualifierDecl CIMQualifier

Class Declaration Example (Part 1)

Consider the following MOF class declaration:class Alarm{ [key] uint64 id; string message = “none”;};

Class Declaration Example (Part 2)

This class is defined in C++ as follows:CIMClass alarmClass(“Alarm”);CIMProperty id(“id”, Uint32(0));id.addQualifier(CIMQualifier(“key”, true)); CIMProperty message(“message”, “none”);alarmClass.addProperty(id);alarmClass.addProperty(message);

Class Declaration Example (Part 3)

Or more succinctly like this:CIMClass alarmClass(“Alarm”);alarmClass .addProperty(CIMProperty(“id”, Uint32(0)) .addQualifier(CIMQualifier(“key”, true))) .addProperty(CIMProperty(“message”, “none”));

Property Iteration Example:

The properties of a class may be iterated like this:CIMClass c;…for (Uint32 i = c.getPropertyCount(); i < n; i++){ CIMProperty p = c.getProperty(i);}

The Client Interface

The Client Interface

Client

CIM Server Repository

ClientCIMClients

ClientClientCIMProviders

The Client Interface

A C++ interface for interacting with the Pegasus Server (or any CIM Server).

Uses CIM/HTTP to communicate. Provides a method for each CIM operation

defined in the “CIM Operations over HTTP, V1.0” specification.

The CIM Operations GetClass GetInstance DeleteClass DeleteInstance CreateClass CreateInstance ModifyClass ModifyInstance EnumerateClasses EnumerateClassNames EnumerateInstances EnumerateInstanceNames ExecQuery

Associators AssociatorNames References ReferenceNames GetProperty SetProperty GetQualifier SetQualifier InvokeMethod

CIM Operation Example

virtual CIMInstance getInstance( const String& nameSpace, const CIMReference& instanceName, Boolean localOnly = true, Boolean includeQualifiers = false, Boolean includeClassOrigin = false, const Array<String>& propertyList =

NULL_PROPERTY_LIST);

<instance> GetInstance ( [IN] <instanceName> InstanceName, [IN,OPTIONAL] boolean LocalOnly = true, [IN,OPTIONAL] boolean IncludeQualifiers = false, [IN,OPTIONAL] boolean IncludeClassOrigin = false, [IN,OPTIONAL,NULL] string PropertyList [] = NULL )

CIM Operations

Specification

PegasusClass

Method

Client Connection Example

A client connects to the CIM Server on the host called “saturn” at port 5988 like this:Selector selector;CIMClient client(&selector);client.connect(“saturn:5988”);

GetClass Example

A client gets a class like this:CIMClass alarmClass = client.getClass(“root/cimv2”, “Alarm”);

Client Proposals

Asynchronous Interface APIs Interface independent of local/remote client

The CIM Object Manager

The Pegasus CIMOM

Client

CIM Server Repository

ClientCIMClients

ClientClientCIMProviders

Request Lifecycle (Part I)

Channel

Protocol

Encodings

Dispatcher

ProvidersProvidersProviders

Repository

1. Receive TCP Message

2. Process HTTP Request

3. Decode from XML

4. Dispatch Request

Incoming Request

Request Lifecycle (Part II)

Channel

Protocol

Encodings

Aggregator

ProvidersProvidersProviders

Repository

8. Transmit TCP Message

7. Form HTTP Response

6. Encode to XML

5. Aggregate Results

Outgoing Response

CIMOM Modules

Channel - Implements transport mechanism (e.g., TCP).

Protocol - Implement application protocol (e.g., HTTP).

Encodings - Implements encoding and decoding of messages (e.g., CIM-XML).

Dispatcher - Dispatches messages to provider(s)/repository.

Repository - Implements CIM Repository. Aggregator - Aggregates results.

Module Concept (Proposed)

The internal CIMOM architecture is being refined into a set of modules.

Each module handles part of the request lifecycle.

Alternative implementations may be provided for each module type.

New module implementations can be used to modify or refine the behavior of the CIMOM and to facilitate porting.

Module Implementations (Proposed) Channel - provide a named-pipe

implementation. Protocol - provide a proprietary binary

protocol. Encodings - provide a binary encoding

scheme. Repository - provide a relational-based

implementation. Traffic Encryption - SSL implementation. Authentication – Challange and response

Module Definition and Loading (Proposed) Modules implement one of the module

interfaces (e.g., Repository). Modules are configured as dynamic libraries. Modules listed in modules file. Modules loaded by CIMOM on startup.

Channel Module Example

A channel connects two endpoints and enables them to communicate.

Uses acceptor/connector pattern from ACE Wrappers.

Pegasus provides a TCP implementation which works on Windows and Unix.

Other implementations could be provided for named pipes, SNA, or for TCP on other platforms.

The Channel Module

Acceptor

Channel

Server

Connector

Channel

Client

1. Connect

2. Create

3. Accept

4. Create

Read/Write

The Provider Interface

The Provider Interface

Client

CIM Server Repository

ClientCIMClients

ClientClientCIMProviders

The Provider Interface

The provider interface is defined by the CIMProvider class.

The provider interface has a method for each CIM operation in the “CIM Operations over HTTP, V1.0” specification.

CIMProvider is a base provider interface (more refined provider interfaces are being developed).

CIMProvider Methods getClass() getInstance() deleteClass() deleteInstance() createClass() createInstance() modifyClass() modifyInstance() enumerateClasses() enumerateClassNames() enumerateInstances() enumerateInstanceNames() execQuery()

Associators() associatorNames() references() referenceNames() getProperty() setProperty() getQualifier() setQualifier() invokeMethod()

Example of modifyInstance() Method The provider developer derives from CIMProvider and overrides the modifyInstance() method.

class MyProvider : public CIMProvider { … }; virtual void MyProvider::modifyInstance( const String& nameSpace, CIMInstance& modifiedInstance) { // Modify the instance here! … }

Provider Registration and Loading

Providers are configured as dynamic libraries with an entry point function for creating the provider instance.

The provider() qualifier is placed on the corresponding class in the repository (this qualifier specifies the library name containing the provider implementation) .

The provider is loaded on demand (when instances of that class are requested).

Refined Provider Interfaces (Work in Progress) Refined provider interfaces are now being

developed. The CIM operations are being assigned to different types of providers: Instance provider Method provider Query provider Event provider Association provider

Provider Interface Managers (Planned) Provider Interface Managers map the base

provider interface onto one or more other provider interfaces. This scheme may be used to: Refine the base provider interface into several

provider interfaces (as mentioned above). Map the provider interface onto other

language provider interfaces (e.g., Java, Perl, Tcl).

Provider Proposals

Interoperability with SUN Wbem providers proposal

We are extending other interoperability ideas

PegasusCIMOM

SUN WBEMCIMOM

MicrosoftCIMOM

PegasusProviders

SUN WBEMProviders

Microsoft WMIProviders

The Repository Interface

The Repository Interface

Client

CIM Server Repository

ClientCIMClients

ClientClientCIMProviders

The Repository Interface

Defines the interface to manipulating the CIM repository.

Alternative implementations of this interface may be provided.

Pegasus provides a simple default implementation.

The Default Repository Implementation Each CIM object is stored in its own file. CIM objects are encoded as XML in files. Namespaces are represented using file

system directories. Single reader/writer at a time. Example: the class X (subclass of Y) which

resides in the root/cimv2 namespace would be stored in this file:

<repository_root>/root#cimv20/classes/X.Y

The Default Repository Limitations

Adequate for classes and qualifiers (which tend to be few).

Okay for a few hundred instances. Not intended to scale to thousands of

instances (uses a lot of disk space due to internal fragmentation).

Good for scenario in which a few instances come from the repository and many instances come from providers.

Alternative Repository Modules

Repository modules may be developed to achieve a highly scalable repository. Possibilities include: Basing repository on a relational database. Basing repository on Unix DB. Making the repository remote. Improving space utilization of existing

implementation by storing objects in binary format (rather than as XML).

New Default Repository Implementation (Proposed) A new repository implementation is under

consideration to improve scalability: Encode objects in binary on disk (rather than

XML). This will reduce size by three to one. Combine like objects into a single file rather

than separate files (this will reduce internal fragmentation).

Provide a fast indexing scheme for keys (disk hashing or B+-Trees). This will improve lookup time.

4. The Pegasus Implementation4. The Pegasus ImplementationTodayTodayAndAnd

TomorrowTomorrow

Status Today Phase 1 almost complete (0.98 today)

Phase 1 CIMOM including

• HTTP Server and CIM-xml Protocol• Provider interfaces and dispatch• Repository with Instances, Classes, Associations• C++ CIM Object Model representation• Admin and support functions

MOF Compiler Client Interface and test clients Test providers and beginning of some real providers Integrated unit and client server tests (regression)

Beta level code quality Needs More testing

CIM Functionality Missing From Phase 1

Events Security WQL and Query

Important Tasks Phase 1

Testing Pegasus Interoperability

More Providers for testing and Demos Better Clients Documentation Binary Release

Planned Extensions CIMOM Functions

Security Indications Threading Async CIM Operations APIs More Modularity Enhance Service Configuration Expanding Provider Characteristics Out-of-Process Providers

WBEM Functions Discovery CIMOM Object Manager

Characteristics Provider Registration and Operations

Clients Object Browser

Protocol Adapters Add protocol Adapters

Providers Test Sample

Implementations Platforms

Easier portability More platforms

Phase 2 Priority tasks

Threaded Model Basic Indications Support CIM Object Manager Capabilities Reporting Discovery Demonstration Additional Modularization

5. The Pegasus Project5. The Pegasus Project

Overview of the Project Active project of the Enterprise Management Forum of the Open

Group Produce

Pegasus open-source Implementation Core, clients, providers, repositories SDKs (Provider and Client)

Documentation for use and development Specifications for major Interfaces

Continue support and growth of Pegasus Portability New functions New Standards requirements New Providers Tools

Pegasus Status Today

Phase 1 of 4+ Phases Version 0.98 Available

Source Code available today Preliminary documentation available Multiple users evaluating today Tested on the platforms defined

Pegasus Project Phases

Phase 1 (June 2001) Goals

Model Validation Client and Provider

development Basic Environment

Core model Cim-xml Operations Class and instance

repositories Providers

Phase 2 Goals

Production Environment Additions

Threading Configuration Security Service Extensions Indications

Phase 3 Remote Providers Discovery Other extensions including

other Language Interfaces (ex. Java connector)

Participants

BMC Software Compaq Computer Focal Point Hermes Softlab Hewlett Packard IBM SIAC

The Open Group Research Institute Management Forum

Tivoli

Working Together

Open Source but coordinated Executive Committee controls strategy Single Architect Work from Proposals

Agree to objectives before doing the work Regular, Stable Releases Developers style guides Project scheduling and planning

Additional Activities

Providers Clients Growth of functionality with DMTF

Discovery Provider standardization (registration,

interfaces) Next generation interoperability

Pegasus Manageability Environment

CIM Object Broker Broker)•Provider Registration •Service Registration•Request Routing

ARMProvider

Resource

Providers

Broker

Consumers Gateways Apps

XML/CIMConnector

ClassRepository

Serviceextensions

Security

InterfaceFor Spec

. . .AICProvider

Apps

ObjectBrowser Editor

InstanceRepository

SecurityEvents

Provider SDK

RemoteProvider

Provider SDK

Queuing

Client SDK

MOFCompiler

Pegasus and Other Manageability Projects AIC – Application and Control

AIC as a Pegasus Provider ARM – Applications Response Measurement

ARM and DMTF DAP Information as Pegasus Provider

Other possible Providers JMX (Java) SNMP (mapping already defined) DMI (mapping already defined)

6. A 6. A Challenge for Challenge for all of usall of us

CIMOMs - Basic Concepts

Tool to create Management Interoperability Infrastructure for manageability Manageability interoperability

Xml-cim today, ??? Tomorrow Instrumentation Interoperability

Many providers, few CIMOMs Lots of applications – limited numbers of

providers

However

We do not make money off of infrastructure If we don’t have common interfaces we will

not have interoperability. CIM is not Easy. Creating complete and

Correct CIM environments is not easy There is a lot of work to do with a common

environment and much more with many different environments

The Alternatives

Creating a common interoperability environment Management – Manageability – xmp-cim Providers – APIs and protocols Provider building – Common object implementations

The choice Build a common structure with contributions Everybody does their own thing. (Many incompatible

and incomplete WBEM Implentations

WBEMsource Consortium

Consortium to create common WBEM manageability

In fomative stages today About 10 involved organizations today

Sun, Compaq,IBM, Tivoli, Open Group, SNIA, Caldera, Novell, Nokia, Intel

Open Group Proposing to host Announced at JavaOne 2001

WBEMsource Objectives

Create commonality between

implementations and integrate different implementations.

Create common interfaces and

APIs

Create a library of providers and tools

Create an environment of open-

source WBEM implementations

Create an environment of conformance

WBEMsourceWBEMsource

The Challenge!!!

Can we create a common WBEM infrastructure?

OR do we all go our own way?

Where to Get More Information

Pegasus is public and Open Source www.opengroup.org/management/pegasus

Pegasus WEB Site Source Code

Builds on Linux and Windows Snapshots and CVS

Binary Release Documentation Pegasus Working Group

Contributors and Users of the Code are WelcomeContributors and Users of the Code are Welcome

This presentation is on the

PegasusSite

7. 7. QUESTIONS?QUESTIONS?