Overview of Pegasus An Open-Source WBEM implementation 12 June 2001 Michael Brasher Karl Version 1.0

Agenda  Overview -What (and why)is Pegasus?  The Pegasus Environment  The Pegasus Software Architecture  Pegasus Status Today  Plans  The Pegasus Project  Pegasus and ???  A Challenge for the Future

1. Overview

What is Pegasus?  Pegasus is an open-source reference implementation of the DMTF WBEM specifications  Pegasus is a work project of the TOG Enterprise Management Forum  Pegasus is a platform for building application management  Pegasus is a function-rich, production-quality open-source implementation designed to be used in high volume server implementations.

Why Produce Pegasus?  Demonstrate manageability concepts.  Provide additional standards for WBEM  Provide a working implementation of WBEM technologies  Provide an effective modular implementation  Support other TOG manageability standards  Base Platform for Open Group Application management Projects

Origins of Pegasus  Pegasus was initiated in 2000 by the Open Group in collaboration with:  BMC Software  IBM  Tivoli Systems

Key Pegasus Objectives  Open source  MIT License, open availability  Portable  Code designed for portability  Efficient and Lightweight  Implemented in C++  Standards Based  DMTF CIM/WBEM compliant  Modular and extensible  Use both internal and external modularity

Pegasus Working Group Philosophy  Manageability not management  The working group’s objective is not to manage systems but to make them manageable by promoting a standard instrumentation environment  The actual management of systems is left to systems management vendors  No standards without implementation  The process of implementation provides a rigorous process for testing the validity of standards  Therefore all standards must be validated by implementation

Major Objectives of Pegasus  Standards Based and Compliant  DMTF CIM/WBEM  Interoperable  DMTF cim-xml Interface  Efficient and Lightweight  Implemented in C++  Portable  NT, Linux, and others planned  Modular  Replaceable modules for function change and growth  Extensible  Manageability Service extensions NOTE: THIS SLIDE GOES AWAY

Open Source  Code and documentation freely available  Open Group and Source Forge  MIT source License  Open to contributions

Portable  Designed for multi-platform, mulit-os, multi- compilers.  Platforms today  UNIX (AIX, HPUX, Solaris)  Linux  Windows Platforms (NT, 2000, 9x)  Compaq Himalaya (Tandem)

Efficient and Lightweight  Core written in C++  Designed for execution efficiency

Standards Based  Based on DMTF CIM and CIM-XML specifications  Open Group is active partner in DMTF  Growth through participation in specification growth  Commitment to continue DMTF compliance

Modular and Extensible  Minimize core object broker.  Maximize extensibility through plug-in components  Component types  Providers  Provider interfaces  Clients  Repositories (additional repository handlers)  Manageability service extensions  Protocol Adapters  Modules (extend and modify core functions) Modularity is key to doing parallel development and allowto extensibility

2. The Pegasus Environment

Pegasus Architecture Consumers Clients CIM Server Consumers Providers CIM/HTTP Interoperable* CIM/HTTP In-Process Services Standard Interfaces

Key Interoperability Interfaces CIM Object Manager CIM Object Manager CIM Providers Application Manageability to Manager Multiple management systems Common open manageability Object Manager / Providers Multiple Providers Encourage common providers Management System Enterprise Management Console Enterprise Management Console Application Provider / Resource Interface Protect Applications Make application management easy

The CIM Operations Consumers Clients CIM Object Mgr Consumers Providers CIM/HTTP Interoperable* CIM/HTTP In-Process Services Standard Interfaces Repository CIM Operations CIM Operations Repository Indicators

Operations Routing  Class Operations  Routed to the Class Repository  Instance Operations  To Provider if Provider Qualifier exists  To Instance repository if no Provider  Instance routing at Class Level Today  Issues – Routing at instance level

Key Characteristics  Open source  Available Today  Portable  Designed to build and run on wide variety of platforms  C++ core  C++ CIM Objects  C++ Operation/Provider/Service/Repository interfaces  Modular and extensible  Modular components to extend the core  Manageability service extensions to extend functionality  Light weight

Modularity and Extensibility  Providers  Grow with DMTF provider concepts  Provider Interfaces  Protocol Adapters (connectors)  Client - Xml-cim today (Soap, etc. in future)  Provider, service, repository, etc.  Modules  Modularize core so it can be extended and modified through attachable modules  Manageability Service Extensions  Think super providers

Building A Modular Manageability Environmnent Core Object Broker Core Object Broker XML-CIM Module Connector Provider Connector Provider Connector CIM Client Connector... Resources Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Repository Undefined CIM Client Repository Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension

Management System Connector Management System Connector Pegasus Manageability Environment CIM Object Broker Provider Registration Service Registration Request Routing Securiy CIM Object Broker Provider Registration Service Registration Request Routing Securiy Consumers Providers AIC Provider AIC Provider AppsOSEtc. ARM Provider ARM Provider Resource Providers Broker Consumers Gateways Apps XML/CIM Connector XML/CIM Connector Management System Connector Management System Connector Application Consumer Application Consumer Class Repository Class Repository Services core*** additional Interface For Spec CIMOM Management System... SNMP Provider SNMP Provider Management System Application Consumer Application Consumer Application Consumer Application Consumer Instance Repository Instance Repository

Provider Interoperability  In the classical architecture, interoperability is only supported between the client and server.  In addition, the Pegasus architecture aims to support provider/server interoperability.  Goal  Write a provider once and run it under any CIM server implementation.  Provider/Server Interoperability:  Participating in efforts to standardize the Provider/Server protocol.  Proposing provider API standards.  Writing adapters enabling Pegasus providers to run under other CIM servers.  Adapters enabling other providers to run under Pegasus

Important Provider Interfaces  SUN WBEM Provider Interface  Java based  Classes, etc. “similar” to Pegasus  C Provider Interface  Sun has a specification here.  We will support multiple provider interfaces and language bindings.

In-Process and Out-of-process Providers  Today Pegasus based on shared Library Providers  Extend to  Internal Providers  IPC based Providers  Providers in Remotes systems  Objectives:  Write Provider once and compile/link for different environments  Technique  Use connectors as basis for provider/CIMOM communication  Issues  Security, discovery

Modules  The core server functions are organized into loadable modules.  Standard APIs are defined for each module.  Alternative implementations can be provided later without recompiling the Pegasus server.

Manageability Service Extensions  Super Providers  Access to the Core Broker

Example Services  Indication Management service.  Query engine service.  Class repository service.  Instance repository service.

Connectors (Protocol Adapters)  Functions  Adapt to different protocols  Characteristics  Protocol  Encoding  Security  Discovery  Examples  Xml-CIM  Local Protocols  Soap  WMI Pegasus Core Xml-cim Connector Xml-cim Connector Pegasus Provider Pegasus Provider Connector Remote Provider Xml-cim Client Xml-cim Client Connector Soap Connector Soap Connector Xml-cim Client Xml-cim Client Java Provider

Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Service Extension Repository Pegasus Interfaces  Common Interface base for  Clients, providers, services, connectors  Based on CIM Operations over HTTP  Additional functions for each interface  Interfaces separated from implementation Core Object Broker Core Object Broker Provider Connector CIM Client Connector

3. The Pegasus Software Architecture

Introduction

Programming Language Support  The Pegasus core is implemented in C++ and hence client and provider interface are provided for C++.  An integrated JVM is planned to allow providers to be developed in Java.  Of course it is possible to use existing Java clients to interact with the Pegasus CIMOM.

Major Components Client CIM Server Repository Client CIM Clients Client CIM Providers

Topics  Communication.  Representation of CIM Elements.  The Client Interface.  The CIM Object Manager.  The Provider Interface.  The Repository Interface.

Communication

Pathways of Communication Client CIM Server CIM Repository Client CIM Clients Client CIM Providers

Component Location  A component may be located in one of three places with respect to the CIM Server.  In-process.  Local out-of-process (on the same machine).  Remote out-of-process (on another machine).  For example, a provider may be in-process, local, or remote.

Component Location in Pegasus Today

Possible Communication Mechanisms  Components could potentially communicate with the CIM Server using the following mechanisms:  CIM/HTTP (remote).  Proprietary TCP-based protocol (remote).  Direct call (in process).  Shared memory (local).  Named pipes (local).

Communication Mechanisms in Pegasus

Client Communication  Uses CIM/HTTP as sole protocol.  Asynchronous socket I/O.  An efficient XML parser.  Fast enough to eliminate the need for a proprietary protocol.

An Efficient XML Parser  No memory heap usage during parsing.  Modifies message in place to avoid copying.  Non-validating parser (“loose validation”).

HTTP Implementation  Uses asynchronous socket I/O in conjunction with message queues to achieve optimal throughput.  Provides entry points to adding web server capabilities such as putting and getting of documents (to support remote upgrade and deployment later on).

Proposals  Support out-of-process providers (local and remote).  Support out-of-process repositories (local and remote).  Location independent provider development.

Representation of CIM Elements Representing CIM Elements in Pegasus with C++

CIM Data Types in C++  Uint8  Sint8  Uint16  Sint16  Uint32  Sint32  Uint64  Sint64  Real32  Real64  Boolean  Char16  String  CIMDateTime  CIMReference

CIM Values in C++  CIM values (property, parameter, and qualifier values) are represented using the CIMValue class. This class:  Encapsulates a union of all CIM data types.  Has a type member indicating the type currently being represented.  Provides access/modifier methods overloaded for each CIM data type.

CIM Elements in C++  CIMClass  CIMInstance  CIMProperty  CIMMethod  CIMParameter  CIMQualifierDecl  CIMQualifier

Class Declaration Example (Part 1)  Consider the following MOF class declaration: class Alarm { [key] uint64 id; string message = “none”; };

Class Declaration Example (Part 2)  This class is defined in C++ as follows: CIMClass alarmClass(“Alarm”); CIMProperty id(“id”, Uint32(0)); id.addQualifier(CIMQualifier(“key”, true)); CIMProperty message(“message”, “none”); alarmClass.addProperty(id); alarmClass.addProperty(message);

Class Declaration Example (Part 3)  Or more succinctly like this: CIMClass alarmClass(“Alarm”); alarmClass.addProperty(CIMProperty(“id”, Uint32(0)).addQualifier(CIMQualifier(“key”, true))).addProperty(CIMProperty(“message”, “none”));

Property Iteration Example:  The properties of a class may be iterated like this: CIMClass c; … for (Uint32 i = c.getPropertyCount(); i < n; i++) { CIMProperty p = c.getProperty(i); }

The Client Interface

Client CIM Server Repository Client CIM Clients Client CIM Providers

The Client Interface  A C++ interface for interacting with the Pegasus Server (or any CIM Server).  Uses CIM/HTTP to communicate.  Provides a method for each CIM operation defined in the “CIM Operations over HTTP, V1.0” specification.

The CIM Operations  GetClass  GetInstance  DeleteClass  DeleteInstance  CreateClass  CreateInstance  ModifyClass  ModifyInstance  EnumerateClasses  EnumerateClassNames  EnumerateInstances  EnumerateInstanceNames  ExecQuery  Associators  AssociatorNames  References  ReferenceNames  GetProperty  SetProperty  GetQualifier  SetQualifier  InvokeMethod

CIM Operation Example virtual CIMInstance getInstance( const String& nameSpace, const CIMReference& instanceName, Boolean localOnly = true, Boolean includeQualifiers = false, Boolean includeClassOrigin = false, const Array & propertyList = NULL_PROPERTY_LIST); GetInstance ( [IN] InstanceName, [IN,OPTIONAL] boolean LocalOnly = true, [IN,OPTIONAL] boolean IncludeQualifiers = false, [IN,OPTIONAL] boolean IncludeClassOrigin = false, [IN,OPTIONAL,NULL] string PropertyList [] = NULL ) CIM Operations Specification Pegasus Class Method

Client Connection Example  A client connects to the CIM Server on the host called “saturn” at port 5988 like this: Selector selector; CIMClient client(&selector); client.connect(“saturn:5988”);

GetClass Example  A client gets a class like this: CIMClass alarmClass = client.getClass(“root/cimv2”, “Alarm”);

Client Proposals  Asynchronous Interface APIs  Interface independent of local/remote client

The CIM Object Manager

The Pegasus CIMOM Client CIM Server Repository Client CIM Clients Client CIM Providers

Request Lifecycle (Part I) Channel Protocol Encodings Dispatcher Providers Repository 1. Receive TCP Message 2. Process HTTP Request 3. Decode from XML 4. Dispatch Request Incoming Request

Request Lifecycle (Part II) Channel Protocol Encodings Aggregator Providers Repository 8. Transmit TCP Message 7. Form HTTP Response 6. Encode to XML 5. Aggregate Results Outgoing Response

CIMOM Modules  Channel - Implements transport mechanism (e.g., TCP).  Protocol - Implement application protocol (e.g., HTTP).  Encodings - Implements encoding and decoding of messages (e.g., CIM-XML).  Dispatcher - Dispatches messages to provider(s)/repository.  Repository - Implements CIM Repository.  Aggregator - Aggregates results.

Module Concept (Proposed)  The internal CIMOM architecture is being refined into a set of modules.  Each module handles part of the request lifecycle.  Alternative implementations may be provided for each module type.  New module implementations can be used to modify or refine the behavior of the CIMOM and to facilitate porting.

Module Implementations (Proposed)  Channel - provide a named-pipe implementation.  Protocol - provide a proprietary binary protocol.  Encodings - provide a binary encoding scheme.  Repository - provide a relational-based implementation.  Traffic Encryption - SSL implementation.  Authentication – Challange and response

Module Definition and Loading (Proposed)  Modules implement one of the module interfaces (e.g., Repository).  Modules are configured as dynamic libraries.  Modules listed in modules file.  Modules loaded by CIMOM on startup.

Channel Module Example  A channel connects two endpoints and enables them to communicate.  Uses acceptor/connector pattern from ACE Wrappers.  Pegasus provides a TCP implementation which works on Windows and Unix.  Other implementations could be provided for named pipes, SNA, or for TCP on other platforms.

The Channel Module Acceptor Channel Server Connector Channel Client 1. Connect 2. Create 3. Accept 4. Create Read/Write

The Provider Interface

Client CIM Server Repository Client CIM Clients Client CIM Providers

The Provider Interface  The provider interface is defined by the CIMProvider class.  The provider interface has a method for each CIM operation in the “CIM Operations over HTTP, V1.0” specification.  CIMProvider is a base provider interface (more refined provider interfaces are being developed).

CIMProvider Methods  getClass()  getInstance()  deleteClass()  deleteInstance()  createClass()  createInstance()  modifyClass()  modifyInstance()  enumerateClasses()  enumerateClassNames()  enumerateInstances()  enumerateInstanceNames()  execQuery()  Associators()  associatorNames()  references()  referenceNames()  getProperty()  setProperty()  getQualifier()  setQualifier()  invokeMethod()

Example of modifyInstance() Method  The provider developer derives from CIMProvider and overrides the modifyInstance() method. class MyProvider : public CIMProvider { … }; virtual void MyProvider::modifyInstance( const String& nameSpace, CIMInstance& modifiedInstance) { // Modify the instance here! … }

Provider Registration and Loading  Providers are configured as dynamic libraries with an entry point function for creating the provider instance.  The provider() qualifier is placed on the corresponding class in the repository (this qualifier specifies the library name containing the provider implementation).  The provider is loaded on demand (when instances of that class are requested).

Refined Provider Interfaces (Work in Progress)  Refined provider interfaces are now being developed. The CIM operations are being assigned to different types of providers:  Instance provider  Method provider  Query provider  Event provider  Association provider

Provider Interface Managers (Planned)  Provider Interface Managers map the base provider interface onto one or more other provider interfaces. This scheme may be used to:  Refine the base provider interface into several provider interfaces (as mentioned above).  Map the provider interface onto other language provider interfaces (e.g., Java, Perl, Tcl).

Provider Proposals  Interoperability with SUN Wbem providers proposal  We are extending other interoperability ideas Pegasus CIMOM Pegasus CIMOM SUN WBEM CIMOM SUN WBEM CIMOM Microsoft CIMOM Microsoft CIMOM Pegasus Providers Pegasus Providers SUN WBEM Providers SUN WBEM Providers Microsoft WMI Providers Microsoft WMI Providers

The Repository Interface

Client CIM Server Repository Client CIM Clients Client CIM Providers

The Repository Interface  Defines the interface to manipulating the CIM repository.  Alternative implementations of this interface may be provided.  Pegasus provides a simple default implementation.

The Default Repository Implementation  Each CIM object is stored in its own file.  CIM objects are encoded as XML in files.  Namespaces are represented using file system directories.  Single reader/writer at a time.  Example: the class X (subclass of Y) which resides in the root/cimv2 namespace would be stored in this file: /root#cimv20/classes/X.Y

The Default Repository Limitations  Adequate for classes and qualifiers (which tend to be few).  Okay for a few hundred instances.  Not intended to scale to thousands of instances (uses a lot of disk space due to internal fragmentation).  Good for scenario in which a few instances come from the repository and many instances come from providers.

Alternative Repository Modules  Repository modules may be developed to achieve a highly scalable repository. Possibilities include:  Basing repository on a relational database.  Basing repository on Unix DB.  Making the repository remote.  Improving space utilization of existing implementation by storing objects in binary format (rather than as XML).

New Default Repository Implementation (Proposed)  A new repository implementation is under consideration to improve scalability:  Encode objects in binary on disk (rather than XML). This will reduce size by three to one.  Combine like objects into a single file rather than separate files (this will reduce internal fragmentation).  Provide a fast indexing scheme for keys (disk hashing or B+-Trees). This will improve lookup time.

4. The Pegasus Implementation TodayAndTomorrow

Status Today  Phase 1 almost complete (0.98 today)  Phase 1  CIMOM including HTTP Server and CIM-xml Protocol Provider interfaces and dispatch Repository with Instances, Classes, Associations C++ CIM Object Model representation Admin and support functions  MOF Compiler  Client Interface and test clients  Test providers and beginning of some real providers  Integrated unit and client server tests (regression)  Beta level code quality  Needs More testing

CIM Functionality Missing From Phase 1  Events  Security  WQL and Query

Important Tasks Phase 1  Testing  Pegasus  Interoperability  More Providers for testing and Demos  Better Clients  Documentation  Binary Release

Planned Extensions  CIMOM Functions  Security  Indications  Threading  Async CIM Operations APIs  More Modularity  Enhance Service Configuration  Expanding Provider Characteristics  Out-of-Process Providers  WBEM Functions  Discovery  CIMOM Object Manager Characteristics  Provider Registration and Operations  Clients  Object Browser  Protocol Adapters  Add protocol Adapters  Providers  Test  Sample Implementations  Platforms  Easier portability  More platforms

Phase 2 Priority tasks  Threaded Model  Basic Indications Support  CIM Object Manager Capabilities Reporting  Discovery Demonstration  Additional Modularization

5. The Pegasus Project

Overview of the Project  Active project of the Enterprise Management Forum of the Open Group  Produce  Pegasus open-source Implementation  Core, clients, providers, repositories  SDKs (Provider and Client)  Documentation for use and development  Specifications for major Interfaces  Continue support and growth of Pegasus  Portability  New functions  New Standards requirements  New Providers  Tools

Pegasus Status Today  Phase 1 of 4+ Phases  Version 0.98 Available  Source Code available today  Preliminary documentation available  Multiple users evaluating today  Tested on the platforms defined

Pegasus Project Phases  Phase 1 (June 2001)  Goals  Model Validation  Client and Provider development  Basic Environment  Core model  Cim-xml Operations  Class and instance repositories  Providers  Phase 2  Goals  Production Environment  Additions  Threading  Configuration  Security  Service Extensions  Indications  Phase 3  Remote Providers  Discovery  Other extensions including other Language Interfaces (ex. Java connector)

Participants  BMC Software  Compaq Computer  Focal Point  Hermes Softlab  Hewlett Packard  IBM  SIAC  The Open Group  Research Institute  Management Forum  Tivoli

Working Together  Open Source but coordinated  Executive Committee controls strategy  Single Architect  Work from Proposals  Agree to objectives before doing the work  Regular, Stable Releases  Developers style guides  Project scheduling and planning

Additional Activities  Providers  Clients  Growth of functionality with DMTF  Discovery  Provider standardization (registration, interfaces)  Next generation interoperability

Pegasus Manageability Environment CIM Object Broker Broker) Provider Registration Service Registration Request Routing CIM Object Broker Broker) Provider Registration Service Registration Request Routing ARM Provider ARM Provider Resource Providers Broker Consumers Gateways Apps XML/CIM Connector XML/CIM Connector Class Repository Class Repository Service extensions Security Interface For Spec... AIC Provider AIC Provider Apps Object Browser Editor Object Browser Editor Instance Repository Instance Repository SecurityEvents Provider SDK Remote Provider Remote Provider Provider SDK Queuing Client SDK MOF Compiler

Pegasus and Other Manageability Projects  AIC – Application and Control  AIC as a Pegasus Provider  ARM – Applications Response Measurement  ARM and DMTF DAP Information as Pegasus Provider  Other possible Providers  JMX (Java)  SNMP (mapping already defined)  DMI (mapping already defined)

6. A Challenge for all of us

CIMOMs - Basic Concepts  Tool to create Management Interoperability  Infrastructure for manageability  Manageability interoperability  Xml-cim today, ??? Tomorrow  Instrumentation Interoperability  Many providers, few CIMOMs  Lots of applications – limited numbers of providers

However  We do not make money off of infrastructure  If we don’t have common interfaces we will not have interoperability.  CIM is not Easy. Creating complete and Correct CIM environments is not easy  There is a lot of work to do with a common environment and much more with many different environments

The Alternatives  Creating a common interoperability environment  Management – Manageability – xmp-cim  Providers – APIs and protocols  Provider building – Common object implementations  The choice  Build a common structure with contributions  Everybody does their own thing. (Many incompatible and incomplete WBEM Implentations

WBEMsource Consortium  Consortium to create common WBEM manageability  In fomative stages today  About 10 involved organizations today  Sun, Compaq,IBM, Tivoli, Open Group, SNIA, Caldera, Novell, Nokia, Intel  Open Group Proposing to host  Announced at JavaOne 2001

WBEMsource Objectives Create commonality between implementations and integrate different implementations. Create common interfaces and APIs Create a library of providers and tools Create an environment of open- source WBEM implementations Create an environment of conformance WBEMsource

The Challenge!!!  Can we create a common WBEM infrastructure?  OR  do we all go our own way?

Where to Get More Information  Pegasus is public and Open Source   Pegasus WEB Site  Source Code  Builds on Linux and Windows  Snapshots and CVS  Binary Release  Documentation  Pegasus Working Group Contributors and Users of the Code are Welcome This presentation is on the Pegasus Site

7. QUESTIONS?