A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system Patrick Schäfer, 08.11.2017, Munich.

A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system Patrick Schäfer, , Munich Advisor: Martin Kleehaus

Outline Motivation Research Questions Prototypical Implementation
Modeled & Runtime Architecture Problem Statement Research Questions Prototypical Implementation Operating Environment Discovery Concept Implementation Live Demonstration Limitations Schäfer, Patrick - Master's Thesis final presentation © sebis

Motivation Modeled Enterprise Architecture
Business Layer S 4 S 8 S 1 S 3 S 6 S 9 S 2 S 5 S 7 Application Layer VM VM VM VM Technology Layer Schäfer, Patrick - Master's Thesis final presentation © sebis

Motivation Runtime architecture
Necessity of knowledge about runtime architecture Detecting deviations from modeled architecture Compliance violations / SLA violations Semi-automated root cause analysis Semi-automated Impact analysis State of the art solutions Application Runtime Architecture Discovery A1 A2 V A3 A4 A5 Business Layer S 1 S 4 S 2 S 5 S 6 S 3 S 7 S 9 S 8 Application Layer Technology Layer S 1 S 4 S 2 S 5 S 6 S 3 S 7 S 9 S 8 Discovered Applications Architecture discovery VM Discovered Technology Layer Schäfer, Patrick - Master's Thesis final presentation © sebis

Motivation Problem statement
Current discovery solutions don‘t own knowledge about the entire architecture including dependencies between and within the architectual layers Consequences comparison of modeled and runtime architecture not possible across all layers root cause analysis is challenging and time consuming Incomplete impact analysis Objective a system, that owns knowledge about all realtime architecture components as well as their inter- and intralayer dependencies S 1 S 4 S 2 S 5 S 6 S 3 S 7 S 9 S 8 Discovered Application Layer Discovered Technology Layer ? A1 A2 V A3 A4 A5 Business Layer Schäfer, Patrick - Master's Thesis final presentation © sebis

Research questions 1 How to discover the Microservice Architecture by applying distributed tracing? What are component relationship types and how to discover them automatically? How to discover concurrency and synchronization? How to recognize changes in the Microservice Architecture? How to provide a smart user interface for adding business semantic on top of business services? 2 3 4 5 Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Operating Environment
Required components of the prototypes operating environment: Instrumented Services Service Repository Eureka Zipkin Server Zipkin Server Service Service Service API Gateway Client Service Service Service Eureka Repository Instrumented Distributed System Legend: Required Component Instrumented Service Optional Component Distributed Tracing Spans Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Discovery Concept: Architecture Model
Business Layer Process Processes Activities Activity 1 Activity 2 Application Layer Services Service 1 Service 2 Service-Instances Instance 1.1 Instance 1.2 Instance 2 Technology Layer Hardware Hardware 1 Hardware 2 entities of output architecture entities are: Components, Revisions and Relations Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Discovery Concept: Architecture Model - Relations
Implicit meaning of relations Business Layer Process Processes owns owns Activities Activity 1 is followed by Activity 2 is executed by is executed by Application Layer uses Services Service 1 Service 2 is composed of is composed of is composed of Service-Instances Instance 1.1 Instance 1.2 Instance 2 runs on runs on Technology Layer Closer look relations – every combination of component types => different implicit meaning Hardware Hardware 1 Hardware 2 Schäfer, Patrick - Master's Thesis final presentation © sebis

Relation classification: Interlayer- and Intralayer-Relations Business Layer Process Processes Activities Activity 1 Activity 2 Application Layer Services Service 1 Service 2 Service-Instances Instance 1.1 Instance 1.2 Instance 2 Technology Layer Hardware Hardware 1 Hardware 2 Hierarchical Interlayer-Relation Non-Hirarchical Intralayer-Relation Schäfer, Patrick - Master's Thesis final presentation © sebis

Relation classification: Direct and Indirect Relations Business Layer Process Processes Activities Activity 1 Activity 2 Application Layer Services Service 1 Service 2 Service-Instances Instance 1.1 Instance 1.2 Instance 2 Technology Layer Hardware Hardware 1 Hardware 2 Direct Relation Indirect Relation Schäfer, Patrick - Master's Thesis final presentation © sebis

Relation classification: Direct and Indirect Relations Business Layer Process Processes Activities Activity 1 Activity 2 Application Layer Services Service 1 Service 2 Service-Instances Instance 1.1 Instance 1.2 Instance 2 Technology Layer Hardware Hardware 1 Hardware 2 Indirect Relation Direct Relation Path of Indirect Relation Schäfer, Patrick - Master's Thesis final presentation © sebis

Indirect Relations are necessary Activity 1 Activity 2 API Gateway Service Service 1 Service 2 Service 3 Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Discovery Concept: Business Layer
Modelling via Web UI: Process and Activity entities Process-flow Added to the architecture model via REST API Processes Activities Services Instances Hardware entities of output architecture entities are: Components, Revisions and Relations Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Discovery Concept: Application / Technology Layer
Detection of Services, Instances, Hardware Interlayer-Relations of these types In Realtime by analysing Distributed Tracing Detects new entities only Cyclic by processing Eureka API Detects new, updated or removed entitites Processes Activities Services Instances Hardware entities of output architecture entities are: Components, Revisions and Relations Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Discovery Concept: Service intralayer Relations
Detection of relations between Services in Realtime by analysing Distributed Tracing Spans Processes Activities Services Instances Hardware entities of output architecture entities are: Components, Revisions and Relations Distributed Tracing Span production* *Source: Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Discovery Concept: Business / Application Relations
Detection of Interlayer-Relations between Activities and Services in Realtime Set of regular expressions mapping to activities Set defined via Web UI and REST API Distributed Tracing Spans applied to ruleset Processes Activities Services Instances Hardware entities of output architecture entities are: Components, Revisions and Relations Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Implementation: System Components
Microservice Architecture Prototype Business Layer Modelling Webapplication Relation Ruleset Architecture- discovery service REST API Eureka Repository Architecture Visualization Distributed Tracing Websocket Bridge Kafka Stream entities of output architecture entities are: Components, Revisions and Relations Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Implementation: Environment
Zipkin Server replaced by Prototype Prototype extends Zipkin implementation Zipkin Server Features stay complete and unmodified Zipkin Server Service Service Service API Gateway Service Client Service Service entities of output architecture entities are: Components, Revisions and Relations Eureka Repository Instrumented Distributed System Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Implementation: Environment
Zipkin Server replaced by Prototype Prototype extends Zipkin implementation Zipkin Server Features stay complete and unmodified Prototype and Zipkin Server Service Service Service API Gateway Service Client Service Service entities of output architecture entities are: Components, Revisions and Relations Eureka Repository Instrumented Distributed System Schäfer, Patrick - Master's Thesis final presentation © sebis

Live Demonstration Scenario: Business Process
Start Login Select Provider Search Route Book Route Logout Ende Schäfer, Patrick - Master's Thesis final presentation © sebis

Live Demonstration Scenario: Microservice Architecture
Accounting Core Service Login Ende Start Select Provider Search Route Book Route Logout DeutscheBahn Mobility Service Zuul API Proxy Travel- companion Service Maps Helper Service DriveNow Mobility Service Business Core Service Schäfer, Patrick - Master's Thesis final presentation © sebis

Accounting Core Service Start DeutscheBahn Mobility Service Login Select Provider Zuul API Proxy Travel- companion Service Maps Helper Service Search Route Book Route DriveNow Mobility Service Logout Business Core Service Ende Schäfer, Patrick - Master's Thesis final presentation © sebis

Business Core Service Start DeutscheBahn Mobility Service Login Select Provider Zuul API Proxy Travel- companion Service Maps Helper Service Search Route Book Route DriveNow Mobility Service Logout Accounting Core Service Ende Schäfer, Patrick - Master's Thesis final presentation © sebis

Live Demonstration Scenario: Microservice Architecture - Deployment
Eureka Zuul API Proxy Travel- companion Service Travel- companion Service DeutscheBahn Mobility Service DeutscheBahn Mobility Service Config Service Business Core Service DriveNow Mobility Service DriveNow Mobility Service Maps Helper Service Maps Helper Service Prototype Accounting Core Service Host 0 ( ) Host 1 ( ) Host 2.1 ( ) Host 2.2 ( ) Schäfer, Patrick - Master's Thesis final presentation © sebis

Limitations Requires Zipkin-compatible instrumentation as well as Eureka Amount of metainformation depends on instrumentation Raw communications like Database-Access is not detected Virtual Hosts are not detected Business layer is modelled Schäfer, Patrick - Master's Thesis final presentation © sebis

B.Sc. Patrick Schäfer

Zuul API Proxy Travel- companion Service Business Core Service DriveNow Mobility Service DeutscheBahn Mobility Service Accounting Core Service Maps Helper Service Synchronous communication Asynchronous communication Schäfer, Patrick - Master's Thesis final presentation © sebis

Relation classification: Direct and Indirect Relations Business Layer Process Processes Activities Activity 1 Activity 2 Application Layer Services Service 1 Service 2 Service-Instances Instance 1.1 Instanz 1.2 Instanz 2 Technology Layer Hardware Hardware 1 Hardware 2 Indirect Relation Direct Relation Path of Indirect Relation Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Implementation: Zipkin Server Extension
Collektor MySQL Storage Prototypical Implementation Implementation: Zipkin Server Extension Span Span Span Span REST Collektor Kafka Collektor … Architecturediscoveryservice REST API MySQL DB Zipkin UI entities of output architecture entities are: Components, Revisions and Relations Schäfer, Patrick - Master's Thesis final presentation © sebis

Prototypical Implementation Implementation: Zipkin Server Extension
Collektor Architecture Discovery Storage (Storage-Proxy) Prototypical Implementation Implementation: Zipkin Server Extension Span Span Span Span REST Collektor Kafka Collektor … Architecturediscoveryservice MySQL Storage Architecturediscoveryalgorithm Datamodel REST API MySQL DB Zipkin UI Webapplication Stream REST API MySQL DB entities of output architecture entities are: Components, Revisions and Relations Schäfer, Patrick - Master's Thesis final presentation © sebis

A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system Patrick Schäfer, 08.11.2017, Munich.

Similar presentations

Presentation on theme: "A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system Patrick Schäfer, 08.11.2017, Munich."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system Patrick Schäfer, 08.11.2017, Munich.

Similar presentations

Presentation on theme: "A prototypical tool to discover architecture changes based on multiple monitoring data sources for a distributed system Patrick Schäfer, 08.11.2017, Munich."— Presentation transcript:

Similar presentations

About project

Feedback