Ingvar Bergström Senior Designer Developer Days June 2009 SMF in OpenSAF

Presentation outline SMF specification overview SMF implementation in OpenSAF Current implementation status

What is SMF ? Software Management Framework SAF standard: SAI-AIS-SMF-A Orchestring the migration from one deployment configuration to an other. Operates on AMF entities

Upgrade execution, simple view IMM Software repository Software Management Framework Campaign.xml -Admin operations -Read config -Modify config Install/remove software (available in the repository) on target nodes. ”Upgrade instruction ” SMF Adaptation commands (not in the SMF specification)

How to create the campaign.xml? From the vendor IMM Campaign builder Entity types file Current configuration Other input e.g. config changes Upgrade campaign Software bundle

The software repository (SMF definition) A logically single storage that collects all all software bundles available in the system.

The software bundle The software bundle represents a collection of independent software packages and associated files that has been delivered to the system’s repository. The software bundle is the smallest unit recognized by SMF and is represented in the information model. The content of the software bundle is described by its ETF (Entity Types File). Each software bundle contains installation and removal commands used by the upgrade to install and remove the bundle on the target nodes.

The software bundle, note that: A software bundle received from a vendor must be imported to the systems software repository before it can be used by SMF. The ETF file delivered with the bundle specifies the outer bundary values for the delivered entities. The installation/removal commands delivered with the bundle must (probably) be modified to fit the target system.

Software repository information model (in IMM) safApp=safSmfService SaAmfApplication safRepository=smfRepository OpenSafRepository SaSmfSwBundle Online/offline removal and installation commands Uri RDN:safSmfBundle=bundle1 SaSmfSwBundle Online/offline removal and installation commands Uri RDN:safSmfBundle=bundle1 SaSmfSwBundle Online/offline removal and installation commands Uri RDN:safSmfBundle=bundle1 SaSmfSwBundle Online/offline removal and installation commands Uri safSmfBundle=bundle1

The software catalog, basic information model (introduced in AMF-B.03.01) SaSmfBaseEntityType SaSmfVersionedEntityType SaSmfSoftwareEntity SaSmfSwBundle

The software catalog specialisation, AMF information model (component) SaAmfSU SaAmfSG SaAmfAppl SaAmfComp SaAmfCompType safVersion=R2 saAmfCtBundle (DN) safAmfCompType (DN) SaAmfCompType safVersion=R1 saAmfCtBundle (DN) SaAmfCompBaseType safCompType=MyServer SaSmfSwBundle safBundle=MyFirstServersafBundle=MySecondServer

The upgrade campaign structure Upgrade campaign Upgrade procedure Upgrade step Upgrade action XML syntax. Schema (.xsd) available at SA Forum home page.

Upgrade methods (procedure type) Rolling upgrade, used to update compatible entities e.g. components –Iterates the same upgrade step over a set of similar units e.g. Nodes, until the entire upgrade scope is covered. –Different versions of components must be able to coexist in the system without restrictions. –System provide service during upgrade. Single-Step upgrade, used to add/remove entities or to update incompatible entities e.g. components. –The upgrade step is applied to all units in the upgrade scope simultaneously e.g. all SU’s of a certain type. –Units provide no service.

The campaign xml In a rolling fashion For all nodes in a cluster remove bundle1 install bundle2 modify all su’s of type yy as below set attr xx = R2 Simple procedure example: targetNodeTemplate targeEntityTemplate activationUnitTemplate swRemove swAdd parent type parent type modifyOperationattributevalue 0..* 1..*

Upgrade Step actions, three sets of fixed actions Standard set of actions. Reduced set of actions, for restartable entities i.e. components. Modified set of actions, to handle OS and MW.

Upgrade Step actions, standard set Online installation of new software Lock and terminate deactivation unit Offline uninstallation of old software Modify information model Offline installation of new software Unlock and instantiate activation unit Online uninstallation of old software

Upgrade Step actions, reduced set for restartable entities Online installation of new software Modify information model Restart symmetric activation unit (e.g. component) Online uninstallation of old software

Upgrade Step actions, modified to handle upgrade of middleware and OS Online installation of new software Lock and terminate deactivation unit Offline uninstallation of old software Online installation of software where “BundleInstallOnlineScope=executionEnvironment” Modify information model Reboot node if any installed software have “BundleInstallOnlineScope=executionEnvironment” Online uninstallation of old software where “BundleRemoveOfflineScope=executionEnvironment” Offline installation of new software Unlock and instantiate activation unit Online uninstallation of old software

Interfaces NTFIMM AMF B SMF API Uses Provides SMF

Administrative API IMM SaSmfCampaign SA_SMF_CAMPAIGN_EXECUTE SA_SMF_CAMPAIGN_SUSPEND SA_SMF_CAMPAIGN_COMMIT SA_SMF_CAMPAIGN_ROLLBACK OM safApp=safSmfService SaAmfApplication saSmfCampaign=upgrade1

Execution monitoring (states) SaSmfCampaign saSmfCmpgState SaSmfProcedure saSmfProcState SaSmfStep saSmfStepState Initial Executing Failed (Final) Suspended StepUndone Completed RollingBack RollbackSuspended RollbackFailed (Final) RolledBack (Final) Initial Executing Suspending Execution Execution Suspended Execution Completed CampaignCommited (Final) ExecutionFailed (Final). RollingBack RollbackCommited (Final) RollbackFailed (Final) Initial Executing Completed Undoing Undone Failed (Final) RollingBack UndoingRollback RollbackUndone (Final) RollbackFailed (Final) Rolledback

SMF API Client Used by processes needed to be informed about the upgrade campaign progress. Register and state scope of interest, (i.e which ”labels” are of interest) Callback with ”labels” specified in the upgrade Campaign xml

SMF API, campaign callpoints libSaSmf.so Client Campaign level callbackAtInit callbackAtBackup callbackAtRollback campCommitAction/callback callbackAtCommit campWrapupAction/callback procInitAction procWrapupAction beforeLock beforeTermination afterImmModification afterInstantiation afterUnlock Procedure level Step level callbackLabel stringToPass

Alarms and notifications No alarms issued Notifications sent for campaign state changes (campaign/procedure/step level)

Upgrade, application aspects Rolling upgrade is the preferred method –Different versions of components can coexist in the system. –Backward compatible internal protocols Avoid model changes. Complicated procedures required to e.g. add an attribute to an existing class in IMM

SMF implementation in OpenSAF

SMF architecture SMF Node Director SMF Node Director Library Client Director redundancy model = 2N Node director redundancy model = noRedundancy SMF Director Controller (active) Controller (standby) Payload 1Payload n SMF Director SMF Node Director Library Client SMF Node Director Library Client MDS

SMF ”engine”, event driven SaSmfCampaign OI smfd smfUpgradeCampaign Admin operation Upgrade campaign Upgrade procedure Events (admin operations) Events Thread Process Events sent by OpenSAF mailbox service

SMF events and function calls Execute Rollback Suspend Campaign Failed StepUndone Suspended Completed RollbackSuspended RollbackFailed RolledBack Events between campaign and procedure threads Procedure Return codes: Failed Undone Completed RollbackFailed RollbackUndone RolledBack Execute() Rollback() Step Functional calls and return codes between procedure and step

SMF object implementers SaSmfCampaign SaSmfProcedure SaSmfStep IMM Upgrade procedure thread SaSmfProcedure OI SaSmfStep OI SaSmfStep SaSmfProcedure SaSmfStep Upgrade procedure thread SaSmfProcedure OISaSmfStep OI Process SMFD Upgrade campaign thread SaSmfCampaign OI Note: persistant runtime classes

SMF adaptation commands Commands called directly by SMF –smf_backup_create, create a cluster backup –smf_repository_check, is the repository accessible –smf_bundle_check is bundle available in repository –smf_node_check, check if node provides necessary resources to perform the upgrade campaign Additional commands/instructions needed to handle the system –smf_bundle_import command or instruction –smf_bundle_remove command or instruction –smf_backup_restore, restore the cluster state –smf_cluster_reboot, backup restore

SMF Bundles ETF file Bundle name Installation command (relative path) Removal command (relative path) Upgrade campaign ”bundle DN” ”bundle DN” IMM ”bundle DN” Bundle name Installation cmd (URI) Removal cmd (URI) Repository SMF specification gives full freedom regarding the software repository According to the standard the repository is any ”logically single storage” i.e could be internet. -The ETF file specifies the relative path to the install/remove scripts within the bundle. -The bundle object in IMM specifies a URI for the install/remove scripts. -The campaign referes to the bundle object in IMM. How to handle this in an OpemSAF implementation?

SMF software repository in OpenSAF (proposal for reference implementation) For an OpenSAF implementation the following assumptions are made: –The repository directory is pointed out by environment variable $SMFREPOSITORY (the directory must be accessable on each node). –For each imported software bundle a sub directory is created, using the bundle name as directory name. –The bundle is placed (unpacked) in the created directory maintaining its own internal structure

Bundles in the file system (proposal for reference implementation) ETF file Bundle name Installation command (relative path) Removal command (relative path) $SMFREPOSITORY/Bundle name/Installation cmd Upgrade campaign ”bundle DN” ”bundle DN” IMM ”bundle” Bundle name Installation cmd (full path) Removal cmd (full path) Repository smf_bundle_import (or campaign) creates the ”bundle” object in IMM smf_bundle_import creates the bundle directory and install the bundle content here

SMF backup/restore SMF will call the smf_backup_create command. The command must be adapted to the backup functionality developed/available for the specific system. Restore procedure matching the backup functionality must be created. What must be included in the backup to be able to restore the cluster to the state valid when the upgrade was started? –Execution environment –OpenSAF configuratrion –Backups created by upgrade aware entities –Repository –...

Development goal, step 1 A limited but usable SMF implementation –SW Bundle import/removal function for reference repository implementation. –Reduced SMF state machines, no rollback. Implemented campaign states are Initial, Executing, ExecutionFailed, ExecutionCompleted and CampaignCommited. –Program structure designed for easy addition of missing states. –No SMF API for upgrade aware entities –Limited implementation of upgrade campaign prerequisite check 8, ”...the expected service outage does not exceed the acceptable service outage..”. –SMF adaptation commands implemented for an OpenSAF defined reference configuration. –Limited URI support for sw bundle installation/removal commands. Commands must be located in cluster file system.

Testing UML environment. Will use the updated AMF from the ”AMF to IMM” work as soon as available. Target environment testing in an ”OpenSAF reference configuration”

SMF Implementation Status (May/June 2009) Parsing of campaign.xml (execept AMF entity types). Execution of rolling upgrade over nodes in UML environment. Creation and updating of campaign related objects in IMM (campaign/procedure/step) Move and continue execution (switchover) between controllers. Sending notifications at state change Execution of SW Bundle installation/removal commands

SMF delivery Targeting SMF-A for Release 4 of OpenSAF Will publish a partly implemented SMF working in a UML environment during august/september. Including sample campaigns and small system (AMF) configuration.

SMF in the developement tree Among other services in the services directory. The service SMFSV The director SMFSV /SMFD The node director SMFSV / SNFND The SMF service is written in C++

Thank You!