DMaaP Edge Deployments ONAP Dublin 2/14/2019 Version 1

Motivation for DMaaP Edge Deployment Technical Reasons… …Timing Network considerations: Keep traffic localized if it isn’t needed anywhere else Inter-site transport bandwidth may be a constraint Inter-site communication may add to message delivery latency Security considerations: Limit network access to Central Assume Central has most important (i.e. service-wide) resources Scaling considerations: Avoid need to scale Central linearly as Edge deployments increase Adopt an “offloading” approach: encourage Edge mS to analyze, filter, aggregate, etc A Central shared resource may become overloaded and add to delivery latency Possible to have different data retention policies at Edge and Central Resiliency considerations: A Central failure may not directly impact Edge operations Central failover implementation can be transparent to Edge clients Dublin 5G Use Case 3GPP PM Mapper microservice Familiarity with AT&T ECOMP Extensive reliance on DMaaP Edge deployments (though not k8s) More use cases Tackle this while we have dev resources for ONAP DMaaP OOM Edge Deployment Complexity of DMaaP provisioning DMaaP = Data Movement as a Platform Data (files and messages) Movement (between 2 or more ONAP components, usually an application Platform – service that other components can rely on. Intention is to make life easier than rolling your own.

Design Studio (DCAE-DS) Policy PM Configuration Casablanca 3GPP PM Mapper microservice (Dublin perf3gpp domain specific interactions) Dublin Message Router Data Router 3GPP PM XML VES perf3gpp pmMeasResults DMaaP file metadata VES sample XML sample DCAE 3GPP PM Mapper XML data create PM event(s) DFC Source XML data is converted to VES PM events but content (e.g. counter names, units) are unchanged. metadata 3GPP PM XML Validation: - XML schema compliance file metadata Mapping config XSD sample Source: 5G_UC_for_Dublin_Bulk_PM (DCAE_Dec13).pptx from Mark Scott Design Studio (DCAE-DS) Mapper config Policy updates needed? PM Dictionaries PM Configuration Service design selects counters for specific NF and PM dictionary version. Instantiate new or update existing 3GPP PM Mapper

DMaaP Use Cases <= Casablanca Dublin Fallback Future Central ONAP 3) Inter-site Transport a) Edge DMaaP 1) Intra-site Transport 2) Localized Transport 3) Inter-site Transport b) No Edge DMaaP 3) Inter-site Transport c) Regional DMaaP presence Central ONAP Central ONAP Central ONAP Central ONAP Central ONAP Sub1 Sub1 Sub1 Sub3 Sub1 Sub3 Sub1 Sub3 DMaaP DMaaP DMaaP DMaaP DMaaP Pub1 Pub1 Pub1 Pub1 Pub1 Regional ONAP Edge ONAP Edge ONAP Edge DMaaP Pub1 Reference: https://wiki.onap.org/display/DW/VES+Component Sub1 Reference: https://wiki.onap.org/display/DW/SDNC+APIs Pub/Sub2 & Pub/Sub3 are based on current thinking of 5G Use Case. Though still under discussion, they help justify the option. Option 3b is included for completeness, but is counter to slide 3 reasons. Sub2 Sub2 Sub2 DMaaP DMaaP Small Compute Edge Pub2 Pub2 Pub3 Pub2 Pub3 Sub2 Examples Pub1: VES Sub1: SDN-C Pub2: 5G DFC Sub2: 5G PM Mapper Pub3: PM Mapper Sub3: VES Pub2 Pub3

Background: Single Kubernetes site in Casablanca Site Central1 Site Central2 Out of Scope Site Edge1 Site Edge2 Site EdgeN Casablanca deployment of K8S in one site. How is scope expanded in Dublin? Central sites are assumed to be in different cities. i.e. geodiversity/ site redundancy Edge sites are assumed to be in different cities too. The same city might have a Central and Edge site, and they might be in the same datacenter and physically very close, but they are still logically separate sites. Out of Scope Out of Scope Out of Scope

Dublin: Geo-redundant Central and Multi-site Edge Site Central1 Site Central2 Region=central K8S Node C1A K8S Node C1B K8S Node C1C K8S Node C2A K8S Node C2B K8S Node C2C Region=kubernetes cluster name Site Edge1 Site Edge2 Site EdgeN Region=E2 Region=En Region=E1 Assumes a central multi-site K8S Assumes separate K8S instances at Edge We’ll have some naming convention for Sites. Current thinking: Release = kubernetes cluster name K8S Node EnA K8S Node EnB K8S Node EnC K8S Node E1A K8S Node E1B K8S Node E1C K8S Node E2A K8S Node E2B K8S Node E2C

Background: Casablanca DMaaP Deployment Site Central1 K8S Node 1 K8S Node 2 DR Node K8S Node 3 postgres mariadb ZK postgres Kafka NFS DR Prov Bus Controller Assumes a single K8S cluster since there is 1 site Simple, single pod services (by default) Some horizontal scaling NOTE: Bus Controller uses common Postgresql Chart, which provides redundant PV MR

DMaaP Components – High Level Characteristics C=Casablanca D=Dublin Component Central Deployment Persistent Volume Geo-redundant High Availability Edge Deployment Message Router C C - NFS D Data Router Prov C - MariaDB No Data Router Node Bus Controller C - Postgres Mirror Maker Furthermore: Bus Controller logically views all DMaaP components (deployed by same “Operator”) as a single instance. Creates a requirement for sites and components to be registered with Bus Controller DR Prov logically views all DR Nodes as part of a single instance Creates a requirement for DR Nodes to be registered with DR Prov Mirror Maker intended to replicate messages between different kafka instances DMaaP components rely on (central) AAF C=Casablanca, D=Dublin Geo-redundant = a second site designated to support services if primary site fails High Availability = other measures beyond site redundancy such as DB sync, ensembles/clusters Edge Deployment = placement of a component pod in the Edge site Conclusion: DMaaP sites do not deploy and operate independently.

Dublin: DMaaP Geo-redundant Deployment Site Central1 Site Central2 Region=central K8S Node C1A K8S Node C1B K8S Node C1C K8S Node C2A K8S Node C2B K8S Node C2C ZK Mirror maker ZK Bus Controller ZK pg NFS Kafka Kafka mariadb mariadb Kafka MR NFS NFS pg DR Prov NFS DR Prov NFS DR Node DR Node Bus Controller Site Edge1 Site Edge2 Site EdgeN Region=E2 Region=En Region=E1 Assumes a central multi-site K8S Pod anti-affinity used to distribute pods across nodes, but not all nodes. Node anti-affinity used to distribute pods across sites. TBD. K8S Node E1C K8S Node E2A K8S Node EnA K8S Node EnC K8S Node E1A K8S Node E1B K8S Node E2B K8S Node E2C K8S Node EnB

Dublin: Edge Deployment Overide config file: dmaap-edge.yaml Site Central1 Site Central2   dmaap:     dmaap-message-router:       enabled: true     dmaap-bus-controller:       enabled: false     dmaap-dr-prov:     dmaap-dr-node: Region=central K8S Node C1A K8S Node C1B K8S Node C1C K8S Node C2A K8S Node C2B K8S Node C2C ZK Mirror maker ZK Bus Controller ZK pg VES NFS Kafka Kafka mariadb mariadb Kafka MR NFS NFS pg DR Prov NFS DR Prov NFS DR Node DR Node Bus Controller Site Edge1 Site Edge2 Site EdgeN Region=E2 Region=En Region=E1 Assumes distinct kubernetes deployments at each edge site Selective DMaaP components get deployed at edge. Same Helm Charts, but different configuration override file for edge deployment Issue: how to discover the edge site kubernetes names? (needed for setting dcaeLocation value) Issue: how do components register with central Bus Controller? K8S Node E1A K8S Node E1B K8S Node E1C K8S Node E2A K8S Node EnA K8S Node EnC K8S Node E2B K8S Node E2C K8S Node EnB Mirror maker ZK ZK Mirror maker Mirror maker ZK ZK ZK ZK NFS Kafka NFS Kafka ZK Kafka NFS Kafka Kafka ZK ZK Kafka MR MR NFS NFS NFS MR NFS NFS NFS Kafka Kafka Kafka DR Node PM Mapper PM Mapper DFC PM Mapper DR Node DR Node NFS NFS DFC NFS DFC

Deployment Ordering Central K8S Deployment Central DMaaP Deployment Use k8s cluster name as the Release.  e.g. "central" Deploy aaf Deploy aai Deploy dmaap Deploy dcae Deploy VES perf3gpp via dcae Edge K8S Deployment Register Edge K8S Deployment in AAI (?) Add dcaeLocation (for new Edge K8S) to DMaaP Bus Controller Edge DMaaP Deployment Update dmaap-edge.yaml configuration override file with values from central Use k8s cluster name as the Release.  e.g. "edge1" deploy dmaap deploy PM Mapper via dcae DMaaP depends on AAF DMaaP and DCAE depend on AAI: Assume sites are registered in AAI and there is some API for retrieving info. DCAE deployment of microservices depends on DMaaP