Overview and update for the E3C Wednesday, 3 August, 2016 OASIS TOSCA - Cloud Portability, Lifecycle Management and more! Presenting on behalf of the TOSCA TC: Matt Rutkowski IBM STSM, Cloud Open Technologies Editor, Co-Chair, Simple Profile WG,

Agenda What is TOSCA? What Makes TOSCA Unique? Key Modeling Concepts Topology, Composition, Portability, Lifecycle (management), Policy TOSCA’s Growing Eco-System in open source & standards Work Group Activities & What’s Next ·       how do you est interop between clouds ·       what are existing implementations of your standard ·       pros/cons ·       where are each at in development/adoption – where are you going ·       who’s involved, etc.

What is TOSCA? An important Open standard, that is enabling a unique Cloud eco-system supported by a large and growing number of international industry leaders… TOSCA uses a domain-specific language (DSL) to define interoperable descriptions of : Cloud applications, services, platforms, infrastructure and data components, along with their relationships, requirements, capabilities, configurations and operational policies… …thereby enabling portability and automated management across cloud providers regardless of underlying platform or infrastructure thus expanding customer choice, improving reliability and time-to-value while reducing costs. Current DSL is in YAML, legacy is XML

TOSCA Milestones and Participation TOSCA Version 1.0 Specification approved as an OASIS Standard published Nov 2013, XML format TOSCA Simple Profile v1.0 Specification (YAML format) TC-approved, implementable; June 2016; TOSCA Simple Profile v1.1 Spec (TC-approved Public draft, June2016) Supports Domain-Specific Profile Specifications: Network Function Virtualization (NFV) Profile v1.0 (Revision 3 approved) Government and Corporate Awareness: OASIS: 600+ participant organizations. 5000+ participants spanning 65+ countries TOSCA Committee: 175+ people 45+ companies/organizations International Standards & Research: ISO/IEC JTC 1 liaison, EU FP7, ETSI NFV liaison, etc. Multi-company Interoperability Demonstrated: EuroCloud 2013, Open Data Center Alliance 2014, OSCON 2015, OpenStack Summit 2016 (Indigo DataCloud) Associated Companies Only contributors, reviewers, implementers, users, and supporters of the TOSCA Standard within OASIS are listed

What Makes TOSCA Unique? TOSCA is an Intent Model which is declarative (integration points for imperative) incorporates both Data and Information Model features and concepts … Intent Model Adds: TOSCA is can work with imperative scripts (e.g., Ansible, Chef, Bash, Ant, etc.) Topology Composition Requirements - Capabilities State (Nodes, Relationships) Lifecycle (Management) Policy TOSCA can include other data models (e.g., JSON, YANG) Information Models Typically, used to model a constrained domain that can be described by a closed set of entity types, properties, relationships and operations. Structure Format interfaces Types, Relationships Properties Operations Data Models Typically, describe the structure (format), enabling manipulation (via interfaces) of the data stored in data management systems assuring integrity. … but brings unique orchestration concepts focus in Lifecycle mgmt. and State

TOSCA Key Modeling Concepts Topology Composition Portability Lifecycle Policy

Topology – Define Topology with Nodes and Relationships TOSCA is used first and foremost to describe the topology of the deployment view for cloud applications and services Node templates to describe components in the topology structure Relationship templates to describe connections, dependencies, deployment ordering Nodes - are the resources or components that will be materialized or consumed in the deployment topology Groups Create Logical, Management or Policy groups (1 or more nodes) Tier (Group Type) source_resource Node_Type_A target_resource Node_Type_B connect_relationship ConnectsTo Requirement Capability Relationships express the dependencies between the nodes (not the traffic flow) Requirement - Capability Relationships can be customized to match specific source requirements to target capabilities

Topology – Relationships based on Containment and Connectivity Example: a simple, 2-Tier Cloud application expressed in a TOSCA Service Template TOSCA Service Template (container) Application Tier (container) Database Tier (container) Web Server (container) DB Server (container) Connectivity Containment Web App Database PHP Module Service Templates provide the “container” to exchange and reuse topologies: Reusable models extend investments by making it easy to compose more valuable and complex apps from existing apps Determines dependency boundaries to maximize parallelism of deployments Models (dependencies) can be validated by automation to ensure application-aware, policy-aligned configuration, deployment and operational semantics

TOSCA Key Modeling Concepts Topology Composition Portability Lifecycle Policy

Composition – different service templates can be “wired” together Enabling the description of complex, multi-tier (hybrid) Cloud applications Application Tier (container) Logging/Monitoring Tier (ELK) Database Tier (container) logstash elasticsearch kibana mongo_db Database paypal_pizza store WebApplication SoftwareComponent SoftwareComponent SoftwareComponent Capabilities ConnectsTo Capabilities ConnectsTo log_endpoint search_endpoint Requirements Requirements search_endpoint nodejs WebServer Requirements search_endpoint Container Container mongo_dbms Container DBMS logstash_server elasticsearch _server kibana_server HostedOn HostedOn HostedOn ConnectsTo app_server Compute Compute Compute Compute Capabilities Capabilities Capabilities mongo_server Compute Container Container Container collectd rsyslog Example: Connect a Logging / Monitoring Service composed of ElasticSearch, LogStash and Kibana (ELK)

Composition – Substitution of Abstract Services Abstract nodes in one TOSCA topology can be substituted with another topology Cloud Application (Topology) Java Application Monitoring Service (Abstract) Monitoring Service (Topology) Collector Web Application Server Analytics Service (Abstract) Analytics Service (Topology) Monitoring Framework SQL Datastore Analytics Engine Logger Service Template 1 Hadoop Service Template 2 Orchestrators can “substitute” for abstract nodes… … as long as all declared “requirements” are met: Monitoring Service can be substituted in Cloud Application Analytics Service can be substituted in Monitoring Service Service Template 3

TOSCA Key Modeling Concepts Topology Composition Portability Lifecycle Policy

Portability – TOSCA Service Templates are Portable Templates include (or reference) all necessary configuration and Infrastructure requirements TOSCA Service Template TOSCA Meta-Model Normative Types Nodes Properties Attributes Relationships Capabilities Interfaces (Operations) Groups Policies Requirements TOSCA’s defines Normative Types for different domains, for example: Application, IaaS Types are part of “core” specification e.g., Web Server, Database, Compute, Block Storage, Network Cloud Application’s declarative modelled from these normative types … … Can be understood by any Cloud Provider Storage Compute1 DB Compute2 App Network Scaling Policy based upon composed from unfulfilled Application Requirements can be exported for Orchestrators to fulfill Interfaces TOSCA applications, using normative types, are portable to different Cloud infrastructures

Portability – TOSCA Orchestrators find “Best Match” during deployment Example: TOSCA applications are portable to different Cloud infrastructures by expressing application Requirements… independently from cloud provider Capabilities… TOSCA Service Template Storage Compute1 DB Compute2 App Network Scaling Policy Application Requirements Requirements TOSCA Orchestration & Optimization Automatic Matching Infrastructure Capabilities Capabilities Cloud Provider A Cloud Provider B Cloud Provider C Orchestrators concern themselves dealing with disparate cloud APIs 14

TOSCA Key Modeling Concepts Topology Composition Portability Lifecycle Policy

Lifecycle – TOSCA has Standardized Lifecycle & States TOSCA models have a consistent view of state-based lifecycle Relationship Lifecycle Node Lifecycle Nodes have their own Lifecycle Operations which are invoked in order to achieve a target state source_resource Type_A my_relationship ConnectsTo target_resource Type_B B my_resource_name My_Resource_Type A Relationships also have their own Lifecycle Operations to configure or allocate and de-configure or deallocate Node related resources Lifecycle.Standard create configure start stop delete Lifecycle.Configure pre_config_source post_config_source add_source remove_source pre_config_target post_config_target add_target remove_target Operations Implementations (e.g., imperative scripts) can be bound to operations. Operations The Orchestrator moves the nodes through their Lifecycle States by executing their Lifecycle Operations in topological order Orchestrators can work to deploy nodes in parallel based upon node relationships

TOSCA Key Modeling Concepts Topology Composition Portability Lifecycle Policy

Policy – Focus on Operational Policies v1.0 includes the groundwork for Placement (Affinity), Scaling and Performance Policies Orchestrators can evaluate Conditions based on Events that trigger Automatic or Imperative Actions 2 Policy Type Event, Condition Action my_scaling_group Scaling 1 Policy Type Event, Condition Action my_app_1 Compute Capabilities Container ... Lifecycle create configure backend_app Compute web-app Compute my_database Compute 3 Policy Type Event, Condition Action Abstract A key feature of any Cloud infrastructure is to provide auditing capabilities for compliance with security, operational and business processes. In this talk we provide an overview of the recent enhancements made in OpenStack projects to support API and security auditing using the DMTF Cloud Auditing Data Federation (CADF) standard. We will describe how auditing is seamlessly enabled for Nova, Glance, Swift, Cinder, Neutron and Keystone and illustrate what is audited, where it is stored, what the records contain and how this supports compliance. We will finish by presenting some possible future directions such as extending the use of CADF beyond audit to facilitate event correlation and federation across multiple tiers. Policies can be declared independently and ttached to various points in your models That can be attached to Interfaces or specific Operations, Nodes and Groups of Nodes “Policies are non-functional Requirements independent of nodes” 18 18

TOSCA’s Growing Eco-system Reference by other Standards Open Source OpenStack

TOSCA’s Growing Eco-system – Use in open source and standards http://ariatosca.org// Multi-Cloud Orchestration (Amazon, Azure, VMware, OpenStack) Open Sourced from Cloudify Topology, Type & LCM Design http://alien4cloud.github.io/ alien4cloud Service Orchestration & Management http://getcloudify.org/ Data/computing platform targeted at scientific communities http://information-technology.web.cern.ch/about/projects/eu/indigo-datacloud www.seaclouds-project.eu/media.html Open, Multi-Cloud Management Parser Deployment Template Translation https://wiki.opnfv.org/display/parser/Parser Note: ETSI NFV ack. TOSCA can be used as an input model/format Indigo DataCloud Project Goal is to provide a sustainable European infrastructure for science, spanning multiple computer centers and existing public clouds. Leverage OpenStack components as a means of deploying TOSCA cloud stacks OpenStack Heat Project provides robust tools to convert alternative orchestration languages to OpenStack Heat’s HOT templates Heat-Translator: Provides a framework for translation orchestration models to Hot TOSCA-Parser: Production quality tool for parsing TOSCA templates https://wiki.openstack.org/ Heat-Translator (IaaS, App Orchestration) Tacker (Network Function Orchestration)

What’s Next? - TOSCA Work Group Activities Interoperability (Conformance) Goal: Conformance test suite for v1.0; includes tests for each section of Simple Profile v1.0 specification. Each test is a TOSCA Service Templates with metadata describing test using the OASIS Test-Assertion (TAG) Standard Work underway to publish in new GitHub repo., announcement (target ~Sept 2016) Container (Clustering) Goal: Finish new Cluster capability definitions, Data Cluster use cases. for Simple Profile v1.2 Instance Model Goal: new schema for an Instance Model (reuse existing schema where possible) Discussing API potentially enabling capture, export and management of deployed application Monitoring Goal: Create normative event types for basic operational events Focus on events types for Health, Scaling & Performance Support basic “Red-Yellow-Green” and Percentage-based monitoring for dashboards Network Function Virtualization (NFV) Expanded Scope: include Software-Defined Network (SDN) use cases Goal: Complete v1.0 Specification, v1.0 Public Review Draft 3 Published (17 March 2016) Can model complete ETSI MANO specification: Network Services, Virtual Network Functions (NFV)s, Virtual Links, with Forwarding Paths, Orchestration demonstrated with OpenStack Tacker Project, multi-VNF use cases for next release

TOSCA Resources – Learn More TOSCA Technical Committee (TC) Public Page (TC approved updates on documents, strategy, and more) https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca OASIS TOSCA LinkedIn Group: (latest news, community and eco-system updates, etc. Join now to stay informed!) https://www.linkedin.com/groups/8505536 OASIS YouTube Channel, TOSCA Playlist https://www.youtube.com/user/OASISopen , http://bit.ly/1BQGGHm TOSCA Simple Profile in YAML v1.0 (TC-approved Committee Spec, implementable; 12, June 2016) http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/cs01/TOSCA-Simple-Profile-YAML-v1.0-cs01.zip TOSCA Simple Profile for NFV v1.0 (latest public review draft, 17 March 2016) http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html Contact the Technical Committee Co-Chairs (also part of TOSCA TC Panel): Paul Lipton, paul.lipton@ca.com; John Crandall, jcrandal@brocade.com Today’s TOSCA TC Panel also Included: Karsten Beins, Karsten Beins@ts.fujitsu.com; Lowell Higley, Lowell.Higley@ca.com; Chris Lauwers, lauwers@ubicity.com; Derek Palma, dpalma@vnomic.com; Matt Rutkowski, mrutkows@us.ibm.com Find out more about TOSCA through these links and contacts.

Questions? Q&A

What’s Next? - TOSCA Simple Profile Version 1.0 Approved Committee Specification, 12 June 2016 Target Fall 2016 for full OASIS Standard http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/cs01/TOSCA-Simple-Profile-YAML-v1.0-cs01.html Version 1.1 Approved Public Draft 01, June 2016 Metadata (completed) now supported in all Types (Node, Relationship, Capability, Data, etc.) Conformance Testing metadata Group Type (completed) Expanded Group Type to allow management of member resources (i.e., Lifecycle) Has its own Capabilities and Requirements Policy Definition (completed) Event-Condition-Action model Includes Event Filters and Triggers Workflow (completed) Intermix declarative with Imperative (e.g., Ansible, Chef, Ant, Bash) Preserve investment in existing scripts for complex installations / configurations Version 1.2 Target Sept. 2016, public draft. Improve import of Service Templates Using template naming / versions (expressions) to be used with Catalogs / Repostiories Allow Composition of Group Type Provide use cases using Clusters Differentiate from Abstract Node Types Cluster Type (75% completed) Add support for Cluster normative type; based upon new Group Type Will support new normative LoadBalancer , Scalable and Router Capability Types Data Clusters (e.g., Cassandra, MongoDB, etc.) – In-Progress

TOSCA “Hello World” Illustrative, Single Server tosca_definitions_version: tosca_simple_yaml_1_0 description: > Template for deploying a single server with predefined properties and input parameter topology_template: inputs: cpus: type: integer description: Number of CPUs for the server. constraints: - valid_values: [ 1, 2, 4, 8 ] node_templates: my_server: type: tosca.nodes.Compute capabilities: host: properties: num_cpus: { get_input: cpus } disk_size: 10 GB mem_size: 512 MB os: architecture: x86_64 type: linux distribution: rhel outputs: server_address: description: IP address of server instance. value: { get_attribute: [server, private_address] } Complete Source: https://github.com/openstack/heat-translator/blob/master/translator/toscalib/tests/data/tosca_single_server.yaml