1 Discussion on Lifecycle Management and Its Impact to CDL Jun Tatemura NEC Laboratories America Feb 5, 2004

2 Motivation CDDLM should cover (from Kojo’s slides) : –Simple Job Submission (deploy and run) –Provisioning Configuration change based on new resource assignment/reservation Some components should be restarted or re-deployed –Failure Recovery Investigation of lifecycle status of components Resource reallocation if needed Reconfiguration if needed Restart some of components How do they impact to LM and CD?

3 Overview Lifecycle management of individual components –issues: reconfiguration, failure handling –impact to CDL: “attribute value lifetime” Lifecycle management of multiple, interrelated components –issues: lifecycle dependency graph, value propagation –impact to CDL

4 Lifecycle Management of Individual Components Lifecycle state chart is more complicated Issues –reconfiguration issues (provisioning, failure recovery) –failure issues (failure recovery) start stop (failure) deployedrunning failed (resource reserved) deploy remove recover[?] remove reconfigure reconfigure[?]

5 Reconfiguration Issues Do we need reconfiguration of deployed component? [Yes] –changing configuration parameters –changing content (update of software) Do we need reconfiguration of running component? [maybe] –changing configuration values without stopping the component deployed reconfigure running reconfigure deployed [2] reconfigure running [1] stop [3] start Is this alternative good enough?

6 Failure Handling Issues Who should know failures? –CDDLM knows the result of LM operations (e.g., start operation failed) and returns failure to Job Manager –What about failure while component is running? Who should be notified? (Job Manager? CDDLM?) Decision making on “recover or not” should be done by Job Manager (not CDDLM) –recovery may require resource reallocation Job ManagerCDDLM Component deploy start reconfigure start failure Job ManagerCDDLM Component deploy start deploy destroy failure

7 Impact to CDL Some configuration values can be modified when running, some require restart, others require re-deployment To manage the total system consistency, CDDLM should manage “attribute value lifetime” –deployment-time constant: the value must be constant during deployment (i.e., need re-deployment to modify) –run-time constant: the value must be constant during run-time (i.e., need restart to modify) –dynamic: the value can be modified without restarting Any modification attempt that brakes the above rule should be rejected Specifying “LAZY” would not be enough

8 Lifecycle Management of Interrelated Components A B C D E AB A must be running when B starts lifecycle dependency graph ABCDEABCDE CDDLM start ok ordering of LM operations

9 Lifecycle Management of Interrelated Components (cont’d) Lifecycle dependency should also be managed in provisioning, failure recovery cases A B C D E restart stop E stop C stop A start A start C start B stop A start A

10 Issues How CDDLM should handle configuration data dependency between multiple components? –Does CDDLM need to retrieve values from one component to process LM operations on another component? (value propagation issue)

11 Value Propagation? (1) Do we need deployment time propagation (resolution) of configuration values? If we can fix all values by resource reservation, we will not need this propagation A B C D E A CDDLM C deploy get X deploy with X X? deploying C needs X which is fixed after A is deployed

12 Value Propagation? (2) Do we need start-up time propagation (resolution) of configuration values? A B C D E A CDDLM C start get X set X start X? starting C needs X which is fixed after A is started

13 When Value Propagation is Required Value propagation is required if (and only if) there are read-only, observable attributes which values are assigned by deployment platform instead of CDDLM –e.g., It is the case if nobody (including resource managers) can tell which port number will be used by a component before it is actually deployed. Otherwise, CDDLM can fix all values based on information from Job Management and does not require value propagation Question: Do we need such attributes? [maybe]

14 Impact to CDL: Read-only Attributes CDL should support “read-only” attributes when value propagation is required –read-only deployment-time constant CDDLM should inquire values from components after they deployed –read-only run-time constant CDDLM should inquire values from components after they started –read-only dynamic It is not manageable! – such values can not be consistently propagated to others.

15 XML CDL Jun Tatemura NEC Laboratories America Feb 5, 2004

16 XML CDL should address manageability issues rather than readability (user friendliness) issues –Front-end CDL (e.g., SmartFrog) would provide user- friendly features Impacts from Lifecycle management requirements addressed –lifetime of attribute values –read-only attributes Made use of existing XML features –XML Schema for data type checking –QName (namespace) for reference

17 Attribute Sets Configuration is a set of attribute-value pairs The “extends” operation (from SmartFrog) for extensibility of configuration Every attribute set can be referred with a QName * extends overrideadditionreuse

18 Using QName for Attribute Set Name Namespace realizes a globally unique reference to specify an attribute set <attributeSet name=“myabc” extends=“tmpl:abc”.../> A template CDL provided by the application provider CDL specified by the application user

19 Attributes A pair of name and value Augmented by,,, etc. –type: type definition (XML Schema) –lifetime: deployment | runtime | (dynamic) –use: required | optional | (readonly) host xns:ipAddress deployment required ?

20 Nested Attribute Sets An attribute can have an attribute set as its value Nesting of attribute sets is flattened and linked with references (QNames) database-server cddlm:refToAttributeSet deployment required tns:database...

21 Benefit from Attribute Overriding + Augmentation Augmentation (type, lifetime,..) can be used for “requirements” from template providers –We can split type definition (given by e.g., application providers) and value definition (given by e.g., application users) –We can use XML Schema to validate attribute values host xns:ipAddress deployment required host extends The application provider can specify requirements on the attribute used in a component The application user can specify the actual value (CDDLM can validate it)

22 Component Component associates an attribute set with content (software) TBD: format for reference to content? destination of deployment should be written here? qname uri?... tns:database location-of-content content

23 Lifecycle Dependency Define System as a set of components Define “deployAfter” and “startAfter” to represent lifecycle dependency + qname *

24 Reference to Attributes Required for constraints –Value Dependency –Assertion Why I don’t use QName for Attributes? –An attribute name is unique only within an attribute set –I don’t want to define a namespace for each attribute set qname xsd:string

25 Value Dependency Value assignment to attributes derived from other attribute values TBD: data transform expression language? (XPath 2.0, MathML, or CDDLM original?) * + { | } $x+1

26 Assertion Conditions on attribute values –All conditions on fixed values must be true to procees LM operations TBD: condition expression language? (XPath 2.0, MathML, or CDDLM original? + condition + $xhost == $yhost

27 Places of Constraints * ?... ?... ?

28 CDDLM Processor Given Config/LM operations on a system, generate Config/LM operations on components CDLs are integrated and complied into five sets of data (components, lifecycle dependencies, attribute sets, dependencies, assertions) Config Operations attribute sets dependencies assertions lifecycle dependencies components CDL LM Operations (deploy, start, stop, remove,...) component SF compiled Config Operations (Get/Set) LM Operations

29 Local and Global CDDLM Processors Local CDDLM processor: required to deploy components on actual platforms Global CDDLM processor: required to deploy components in a federated environment Local CDDLM CDL LM Operations (deploy, start, stop, remove,...) platform dependant deployment Local CDDLM Local CDDLM CDL LM Operations (deploy, start, stop, remove,...) Global CDDLM Local CDDLM generates platform dependent operations Global CDDLM generates CDDLM operations to local CDDLM processors