1. DMaaP Edge Deployments ONAP Dublin
2/14/2019
Version 1

2. Motivation for DMaaP Edge Deployment
Technical Reasons…
…Timing
Network considerations:
Keep traffic localized if it isn’t needed anywhere else
Inter-site transport bandwidth may be a constraint
Inter-site communication may add to message delivery latency
Security considerations:
Limit network access to Central
Assume Central has most important (i.e. service-wide) resources
Scaling considerations:
Avoid need to scale Central linearly as Edge deployments increase
Adopt an “offloading” approach: encourage Edge mS to analyze, filter, aggregate, etc
A Central shared resource may become overloaded and add to delivery latency
Possible to have different data retention policies at Edge and Central
Resiliency considerations:
A Central failure may not directly impact Edge operations
Central failover implementation can be transparent to Edge clients
Dublin 5G Use Case
3GPP PM Mapper microservice
Familiarity with AT&T ECOMP
Extensive reliance on DMaaP Edge deployments (though not k8s)
More use cases
Tackle this while we have dev resources for ONAP DMaaP
OOM Edge Deployment
Complexity of DMaaP provisioning
DMaaP = Data Movement as a Platform
Data (files and messages)
Movement (between 2 or more ONAP components, usually an application
Platform – service that other components can rely on. Intention is to make life easier than rolling your own.

3. Design Studio (DCAE-DS) Policy PM Configuration
Casablanca
3GPP PM Mapper microservice (Dublin perf3gpp domain specific interactions)
Dublin
Message Router
Data Router
3GPP PM XML
VES perf3gpp pmMeasResults
DMaaP
file metadata
VES sample
XML sample
DCAE
3GPP PM Mapper
XML data
create PM event(s)
DFC
Source XML data is converted to VES PM events but content (e.g. counter names, units) are unchanged.
metadata
3GPP PM XML
Validation:
- XML schema compliance
file metadata
Mapping config
XSD sample
Source: 5G_UC_for_Dublin_Bulk_PM (DCAE_Dec13).pptx from Mark Scott
Design Studio (DCAE-DS)
Mapper config
Policy
updates needed?
PM Dictionaries
PM Configuration
Service design selects counters for specific NF and PM dictionary version.
Instantiate new or update existing 3GPP PM Mapper

4. DMaaP Use Cases <= Casablanca Dublin Fallback Future Central ONAP
3) Inter-site Transport
a) Edge DMaaP
1) Intra-site Transport
2) Localized Transport
3) Inter-site Transport
b) No Edge DMaaP
3) Inter-site Transport
c) Regional DMaaP presence
Central ONAP
Central ONAP
Central ONAP
Central ONAP
Central ONAP
Sub1
Sub1
Sub1
Sub3
Sub1
Sub3
Sub1
Sub3
DMaaP
DMaaP
DMaaP
DMaaP
DMaaP
Pub1
Pub1
Pub1
Pub1
Pub1
Regional ONAP
Edge ONAP
Edge ONAP
Edge
DMaaP
Pub1 Reference:
Sub1 Reference:
Pub/Sub2 & Pub/Sub3 are based on current thinking of 5G Use Case. Though still under discussion, they help justify the option.
Option 3b is included for completeness, but is counter to slide 3 reasons.
Sub2
Sub2
Sub2
DMaaP
DMaaP
Small Compute Edge
Pub2
Pub2
Pub3
Pub2
Pub3
Sub2
Examples
Pub1: VES
Sub1: SDN-C
Pub2: 5G DFC
Sub2: 5G PM Mapper
Pub3: PM Mapper
Sub3: VES
Pub2
Pub3

5. Background: Single Kubernetes site in Casablanca
Site Central1
Site Central2
Out of Scope
Site Edge1
Site Edge2
Site EdgeN
Casablanca deployment of K8S in one site. How is scope expanded in Dublin?
Central sites are assumed to be in different cities. i.e. geodiversity/ site redundancy
Edge sites are assumed to be in different cities too.
The same city might have a Central and Edge site, and they might be in the same datacenter and physically very close, but they are still logically separate sites.
Out of Scope
Out of Scope
Out of Scope

6. Dublin: Geo-redundant Central and Multi-site Edge
Site Central1
Site Central2
Region=central
K8S Node C1A
K8S Node C1B
K8S Node C1C
K8S Node C2A
K8S Node C2B
K8S Node C2C
Region=kubernetes cluster name
Site Edge1
Site Edge2
Site EdgeN
Region=E2
Region=En
Region=E1
Assumes a central multi-site K8S
Assumes separate K8S instances at Edge
We’ll have some naming convention for Sites. Current thinking: Release = kubernetes cluster name
K8S Node EnA
K8S Node EnB
K8S Node EnC
K8S Node E1A
K8S Node E1B
K8S Node E1C
K8S Node E2A
K8S Node E2B
K8S Node E2C

7. Background: Casablanca DMaaP Deployment
Site Central1
K8S Node 1
K8S Node 2
DR Node
K8S Node 3
postgres
mariadb ZK
postgres
Kafka
NFS
DR Prov
Bus Controller
Assumes a single K8S cluster since there is 1 site
Simple, single pod services (by default)
Some horizontal scaling
NOTE: Bus Controller uses common Postgresql Chart, which provides redundant PV MR

8. DMaaP Components – High Level Characteristics
C=Casablanca
D=Dublin
Component
Central Deployment
Persistent Volume
Geo-redundant
High Availability
Edge Deployment
Message Router C
C - NFS D
Data Router Prov
C - MariaDB No
Data Router Node
Bus Controller
C - Postgres
Mirror Maker
Furthermore:
Bus Controller logically views all DMaaP components (deployed by same “Operator”) as a single instance.
Creates a requirement for sites and components to be registered with Bus Controller
DR Prov logically views all DR Nodes as part of a single instance
Creates a requirement for DR Nodes to be registered with DR Prov
Mirror Maker intended to replicate messages between different kafka instances
DMaaP components rely on (central) AAF
C=Casablanca, D=Dublin
Geo-redundant = a second site designated to support services if primary site fails
High Availability = other measures beyond site redundancy such as DB sync, ensembles/clusters
Edge Deployment = placement of a component pod in the Edge site
Conclusion: DMaaP sites do not deploy and operate independently.

9. Dublin: DMaaP Geo-redundant Deployment
Site Central1
Site Central2
Region=central
K8S Node C1A
K8S Node C1B
K8S Node C1C
K8S Node C2A
K8S Node C2B
K8S Node C2C ZK
Mirror
maker ZK
Bus Controller ZK pg
NFS
Kafka
Kafka
mariadb
mariadb
Kafka MR
NFS
NFS pg
DR Prov
NFS
DR Prov
NFS
DR Node
DR Node
Bus Controller
Site Edge1
Site Edge2
Site EdgeN
Region=E2
Region=En
Region=E1
Assumes a central multi-site K8S
Pod anti-affinity used to distribute pods across nodes, but not all nodes.
Node anti-affinity used to distribute pods across sites. TBD.
K8S Node E1C
K8S Node E2A
K8S Node EnA
K8S Node EnC
K8S Node E1A
K8S Node E1B
K8S Node E2B
K8S Node E2C
K8S Node EnB

10. Dublin: Edge Deployment
Overide config file: dmaap-edge.yaml
Site Central1
Site Central2
  dmaap:
    dmaap-message-router:
      enabled: true
    dmaap-bus-controller:
      enabled: false
    dmaap-dr-prov:
    dmaap-dr-node:
Region=central
K8S Node C1A
K8S Node C1B
K8S Node C1C
K8S Node C2A
K8S Node C2B
K8S Node C2C ZK
Mirror
maker ZK
Bus Controller ZK pg
VES
NFS
Kafka
Kafka
mariadb
mariadb
Kafka MR
NFS
NFS pg
DR Prov
NFS
DR Prov
NFS
DR Node
DR Node
Bus Controller
Site Edge1
Site Edge2
Site EdgeN
Region=E2
Region=En
Region=E1
Assumes distinct kubernetes deployments at each edge site
Selective DMaaP components get deployed at edge.
Same Helm Charts, but different configuration override file for edge deployment
Issue: how to discover the edge site kubernetes names? (needed for setting dcaeLocation value)
Issue: how do components register with central Bus Controller?
K8S Node E1A
K8S Node E1B
K8S Node E1C
K8S Node E2A
K8S Node EnA
K8S Node EnC
K8S Node E2B
K8S Node E2C
K8S Node EnB
Mirror
maker ZK ZK
Mirror
maker
Mirror
maker ZK ZK ZK ZK
NFS
Kafka
NFS
Kafka ZK
Kafka
NFS
Kafka
Kafka ZK ZK
Kafka MR MR
NFS
NFS
NFS MR
NFS
NFS
NFS
Kafka
Kafka
Kafka
DR Node
PM Mapper
PM Mapper
DFC
PM Mapper
DR Node
DR Node
NFS
NFS
DFC
NFS
DFC

11. Deployment Ordering Central K8S Deployment Central DMaaP Deployment
Use k8s cluster name as the Release.  e.g. "central"
Deploy aaf
Deploy aai
Deploy dmaap
Deploy dcae
Deploy VES perf3gpp via dcae
Edge K8S Deployment
Register Edge K8S Deployment in AAI (?)
Add dcaeLocation (for new Edge K8S) to DMaaP Bus Controller
Edge DMaaP Deployment
Update dmaap-edge.yaml configuration override file with values from central
Use k8s cluster name as the Release.  e.g. "edge1"
deploy dmaap
deploy PM Mapper via dcae
DMaaP depends on AAF
DMaaP and DCAE depend on AAI: Assume sites are registered in AAI and there is some API for retrieving info.
DCAE deployment of microservices depends on DMaaP