0. Enterprise Cloud Application Orchestration in OpenStack at UBS
Session: 4966
Enterprise Cloud Application Orchestration in OpenStack at UBS
with Governance Policies
Thursday, Feb. 25th,
9:30 AM – 10:15 AM
Mandalay Bay SOUTH, Breakers L

1. TOSCA Orchestration and Management in OpenStack
Clusters, Policies, Network Function Virtualization (NFV)

2. Review: TOSCA’s Key Modeling Concepts that UBS found Interesting
Primary Goal was to Simplify Application-Centric Modeling for Cloud
Describe Topology of an Applications’ “desired state” to a Cloud Service Provider
TOSCA is an “Intent Model” that:
Goes beyond traditional Information Models (IMs)
Carries Configuration information described by Data Models (DMs)
Promote Composition of Application Services for (Cloud) Platform providers
Describe application’s (abstract) Requirements that can be fulfilled by other services’ Capabilities at design-time or run-time by:
Linking to other TOSCA Service Templates via Service Catalogs <or>
Dynamic fulfillment by Cloud Service (platform) Providers (e.g., IaaS, PaaS)
Enable granular, operational Lifecycle management
of the entire application topology or any of its composite services (including update-in-place)
Support DevOps by associating different Repositories, Policies, Artifacts, Configurations (data)
Update-in-place of individual (micro)services and/or sub-topologies.
Encourage Portability via normative Node (resource) and Relationship types
Growing set of Normative IaaS and PaaS types in the base specification
Encourage Profile specifications for adding “domain-specific” type systems
Move away from describing exactly HOW everything should be done
Permit service providers to match their platform service Capabilities that best suits the applications requirements

3. TOSCA in Open Source & Open Standards
Heat-Translator
Senlin
Tacker
TOSCA Template Validation
and Translation to other DSLs
Clustering / Policy
with Monitoring
Virt. Network Function (NFV) Manager
(Software-Defined Networks)
TOSCA Blueprint,Type & Workflow Designer
alien4cloud
Service Orchestration & Management
Seaclouds
Open, Multi-Cloud Managemnt
CERN
Indigo-DataCloud
Data/computing platform targeted at scientific communities
OPNFV
Parser
Deployment Template Translation
TOSCA is technology agnostic, OpenStack was chosen place for a reference impl. works with traditional scripting solutions Ansible, Chef, Puppet, etc.
The TOSCA impl. in OS is Heat Translator in OpenStack has support for OpenStack APIs today, but is able to translate and support other APIs as well
in fact, the Translator and Parser are reusable libraries in Pypi that can support plug-ins for new APIs
The TOSCA Service Templates and types can be realized with ANY target platform API by a TOSCA Orchestrator (either directly or directly by translation as we do in OpenStack via Heat)

4. OpenStack Update: New Services utilizing TOSCA in OpenStack
Already able to model & orchestrate TOSCA IaaS and PaaS patterns in OpenStack
using the Heat Orchestration Service and the TOSCA Heat-Translator
Logical
PaaS Patterns
via Heat / Heat-Translator
IBM Bluemix …
Software Services
Application Support
Patterns
App
App Catalog
(Murano)
Container
(Magnum)
Cluster-Policy
(Senlin)
NFV
(Tacker) …
IaaS Patterns
via Heat / Heat-Translator
Shared
Image
(Glance)
Database
(Trove)
Identity
(Keystone) …
Metering
(Ceilometer)
Core
Compute
(Nova)
Storage
(Cinder, Swift)
Network
(Neutron)
Baremetal
(Ironic)
Physical
Software-Defined Systems
and Hardware Patterns
IaaS, PaaS and SaaS is supported using the TOSCA Heat-Translator and the Heat Orchestration services
New OpenStack Services moving to use TOSCA for modeling and orchestration:
Senlin (Cluster-Policy) & Tacker (Network Function Virtualization)

5. OpenStack Senlin Project
Cluster – Policy Service

6. Senlin – Clustering and Policy service for OpenStack Clouds
Profiles – describe supported Container types and how to deploy them using their corresponding OpenStack services’ APIS; including:
Linux Containers (Magnum) e.g., Docker, Stacks (Heat), VMs (Nova), Bare-metal (Ironic)
Policies - framework to monitor and enforce various Operational policies on Cluster
TOSCA
Service Templates
Senlin Policies
(TOSCA)
Supported Senlin Profiles
Placement
Scaling
Load
Balancing
Deletion
Performance …
Senlin Engine
Linux
(Magnum)
Stacks
(Heat)
Load Profile +
VMs
(Nova)
Bare-metal
(Ironic)
Deploy using APIs
Topology of
Cluster / Containers
Register Alarms
from Policies
Monitoring
(Ceilometer)
Senlin plans to use TOSCA Policy definitions which can be associated with Clusters and Containers described by TOSCA Service Templates

7. TOSCA Attaching Operational Policies to Applications
Normative Operational Policies for: Placement (Affinity), Scaling and Performance
with Rules that are evaluated to execute Automatic and Imperative Triggers 2
Policy
Event (Type)
Condition (Rule)
Action (Trigger)
my_cluster 1
Policy
Event (Type)
Condition (Rule)
Action (Trigger)
app_service
Compute
Capabilities
Container
...
Lifecycle
create
configure
backend_app
Compute 3
Policy
Event (Type)
Condition (Rule)
Action (Trigger)
web-app
Compute
my_database
Compute
Policies can be attached to:
Interfaces for specific Operations to monitor lifecycle events on a resource (service)
Nodes – facilitates resource (service) level monitoring
Groups – facilitates composite service / application monitoring
Abstract
A key feature of any Cloud infrastructure is to provide auditing capabilities for compliance with security, operational and business processes.
In this talk we provide an overview of the recent enhancements made in OpenStack projects to support API and security auditing using the DMTF Cloud Auditing Data Federation (CADF) standard.
We will describe how auditing is seamlessly enabled for Nova, Glance, Swift, Cinder, Neutron and Keystone and illustrate what is audited, where it is stored, what the records contain and how this supports compliance.
We will finish by presenting some possible future directions such as extending the use of CADF beyond audit to facilitate event correlation and federation across multiple tiers.
Enables monitoring of Utilization, Health and Scaling at any level 7 7

8. 1..N Triggers can be declared
<policy_name>:
type: <policy_type_name>
description: <policy_description>
properties: <property_definitions>
targets: [ <list_of_valid_target_resources> ]
triggers:
<trigger_symbolic_name_1>:
event_type: <event_type_name>
schedule:
start_time: <iso 8601 date-time>
end_time: <iso 8601 date-time>
# Identify node (resource) to monitor
target_filter:
node: <node_template_name> | <node_type>
# (optional) reference to a related node
# via a requirement
requirement: <requirement_name>
# (optional) Capability within node to monitor
capability: <capability_name>
# Describes an attribute-relative test that
# causes the trigger’s action to be invoked.
condition: <constraint_clause>
action:
# implementation-specific operation name
<operation_name>:
inputs: <list_of_parameters>
implementation: <script> | <service_name>
1..N Triggers can be declared
TOSCA Policy Definition:
Event
Normative TOSCA Event Type
That describes an event based upon a Resource “state” change.
Or a change in one or more of the resources attribute value.
Optional schedule
Condition
Identifies:
the resource (Node) in the TOSCA model to monitor.
Optionally, identify a Capability of the identified node.
Describe the attribute (state) of the resource to evaluate (condition)
Action
Describes:
An Operation (name) to invoke when the condition is met
within the declared Implementation
Optionally, pass in Input parameters to the operation along with any well-defined strategy values. 8 8

9. Senlin Example : TOSCA Topology for Kubernetes Pod, Docker Containers
Kubernetes “redis” example:
TOSCA Service Template
Cluster Scaling Policy
Event =
tosca.resource.utilization
Condition =
utilization > 80 %
Action = Senlin.scale()
redis-master-pod
Cluster.Kubernetes
...
capabilities:
loadBalancer:
properties: ...
members:
[ master-container,
sentinel-container ]
Kubernetes Pod
modeled as a
TOSCA Cluster (Group Type)
built-in LoadBalancer Capability
“Attach”
Container Scaling Policy
Event =
tosca.resource.utilzation
Condition =
utilization > 90 %
Action = Senlin.scale()
Associated via
TOSCA “MemberOf” Relationships (implied)
“Attach”
Docker Containers
modeled as
TOSCA Container Node Types
master-container
sentinel-container
Container.App.Kubernetes
Container.App.Kubernetes …
There was some assumptions made that the “cpu” was related to CGROUPS or “Control Groups” within Linux… from Wikipedia…
One of the design goals of cgroups is to provide a unified interface to many different use cases, from controlling single processes (by using nice, for example) to whole operating system-level virtualization (as provided by OpenVZ, Linux-VServer or LXC, for example). Cgroups provides:
Resource limitation: groups can be set to not exceed a configured memory limit, which also includes the file system cache[6][7]
Prioritization: some groups may get a larger share of CPU utilization[8] or disk I/O throughput[9]
Accounting: measures how much resources certain systems use, which may be used, for example, for billing purposes[10]
Control: freezing the groups of processes, their checkpointing and restarting[10]
Senlin scaling cluster / containers independently with Magnum service
OpenStack Tokyo 9 9

10. OpenStack Tacker Project
Network Function Virtualization (NFV)
Orchestrator / Manager Service
for use in Cable and Telco industries and any Cloud network provider

11. OpenStack Heat Translator
OpenSource related to ETSI NFV and OASIS TOSCA
TOSCA
Service Templates
OPNFV
Parser
OpenStack Heat Translator
TOSCA NFV
Models
NSD
VNFD
VNFFGD
Tacker
NFV Orchestrator
ETSI NFV Descriptor
tosca-parser
reference
implements
ETSI NFV
Standards:
NFVO / VNFM / Catalog
TOSCA
Policies
heat-translator
Cluster-Policy
(Senlin)
Orchestrate using
Heat or Nova APIs

12. Why Network Function Virtualization (NFV) uses TOSCA
TOSCA Service Templates can describe these various levels of composition
NFV Information model has such a layered structure:
Network Services (NSDs) are composed of
Virtual Network Functions (VNFDs), Virtual Links (VLDs),
and sometimes actual Physical Network Functions (PNFDs), etc.
VNFDs are composed of Virtualization Deployment Units (VDUs), with VLDs, etc.
NSD
VNFD
VNFD
VLD
Any node in a TOSCA topology can be an abstraction
of another layer or sub-topology
VNFD
VDU
VLD
VNFD
Using the TOSCA, NFV multi-layer, Information Models can be described by bringing together multiple TOSCA service templates

13. Thank You Your Feedback is Important!
Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your smartphone,
laptop or conference kiosk.

14. Open Tech @ IBM – Learn More!

15. OASIS TOSCA
Backup Materials

16. What is TOSCA? Who participates?
TOSCA is an important new open cloud standard, that is enabling a unique eco-system, supported by a large and growing number of international industry leaders…
Associated Companies
TOSCA Version 1.0 Specification (XML) - OASIS Standard (Nov 2013)
TOSCA Simple Profile v1.0 Specification (YAML) – OASIS Standard (target March 2016)
Government and Corporate Awareness:
OASIS: participant companies & orgs., participants spanning 65+ countries
International Standards & Research: ETSI NFV liaison, EU FP7, etc.
Industry Analysts: Forrester names TOSCA as a top four cloud open standard (Mar 2014)
Multi-company Interoperability Demonstrated:
EuroCloud 2013, Open Data Center Alliance 2014, OSCON 2015

17. Topology
Primarily, TOSCA is mainly used to describe the topology of the deployment view for cloud applications
Defining Node templates to describe components in the topology structure
Defining Relationship templates to describe connections, dependencies, deployment ordering

18. Lifecycle TOSCA models have a consistent view of state-based lifecycle
have Operations (implementations) that can be sequenced against state of any dependent resources
fits into any Management Framework or Access Control System
Lifecycle Customization
Standardize Resource Lifecycle
Standardize Relationship Lifecycle
source_resource
Type_A
my_relationship
ConnectsTo
target_resource
Type_B B
Lifecycle.Configure.NFV
my_resource_name
My_Resource_Type
Lifecycle.Standard A
create
Lifecycle.Standard
my_pre_config
create
configure
start
stop
delete
Lifecycle.Configure
configure
Operations
start
pre_config_source
post_config_source
add_source
remove_source
pre_config_target
post_config_target
add_target
remove_target
Operations
stop
my_pre_delete
delete
Operations
Create new Lifecycles or
Augment existing (via subclassing)
Parameters and Policies can be supplied to operations to affect resource behavior (state)
Workflow - TOSCA is developing workflow to allow handling complex state changes, configurations, etc.

19. Choice of Providers with different Infrastructures
Portability from Standardization
TOSCA enables applications flexible movement between different cloud infrastructures
TOSCA
Portable
Cloud
Application
TOSCA Service Template
Storage
Compute1 DB
Compute2
App
Network
Scaling
Policy
Avoids binding to a specific implementation by expressing Application Requirements independently from Cloud Capabilities
Standardization enables Portability of services across different Clouds
Compose from a robust set of Normative Types for different domains (including IaaS, PaaS, NFV)
Declarative model spanning infrastructure, platform and application services
Manipulate the orchestration declaratively instead of dealing with disparate cloud APIs (leave to the TOSCA Orchestrator)
Multi VIM Support
TOSCA Orchestration
Choice of Providers with different Infrastructures

20. Applying a TOSCA Scaling Policy to Resources in a Cluster
using Senlin and the Kubernetes “redis” example (Pod, Docker Containers)
TOSCA Scaling Policy Definition
Policy Target
In this case, a Kubernetes Pod
A TOSCA resource type (tosca.groups.placement type)
Normative Properties
TOSCA provides base properties used for scaling
my_scaling_policy:
type: tosca.policies.scaling
properties: # normative TOSCA properties for scaling
min_instances: 1
max_instances: 10
default_instances: 3
increment: 1
# target the policy at the “Pod”
targets: [redis-master-pod]
triggers:
# Symbolic trigger name
resize_compute:
event: tosca.events.resource.utilization
target_filter:
node: master-container
requirement: host
capability: Container
condition: utilization greater_than 80%
action:
# map to SENLIN::ACTION::RESIZE operation
resize: # operation name
inputs:
strategy: BEST_EFFORT
# Invoke Senlin webhook (service)
Implementation: Senlin.webhook()
TOSCA normative event type
(name) that would map to
domain-specific names
(e.g., OpenStack Ceilometer)
Trigger Name
Could be used to reference an externalized trigger
Map Event, Condition to target Monitoring Service
Map event to OpenStack notifications for resource (e.g., Nova)
Register alarm with the target monitoring service (e.g., Ceilometer, Monasca, etc.)
Monitor Resource
Find Resource (node) named “master-container” in TOSCA topology.
Locate its “host” Container
Attach an alarm | alert | event to identified Container resource and monitor for Condition
Parameters
optionally provide input parameters, for example “strategy” with value of “BEST_EFFORT”
Action
Invoke “resize” operation
Within Senlin “webhook” (service)
Based upon the Senlin “scaling_out_policy_ceilometer.yaml” 20 20