Dimitrios Makrakis SITE University of Ottawa CEG 4190-Winter 2018 Flow Analysis Instructor: Dimitrios Makrakis.

Dimitrios Makrakis SITE University of Ottawa CEG 4190-Winter 2018 Flow Analysis Instructor: Dimitrios Makrakis

Dimitrios Makrakis SITE University of Ottawa Previous : Requirement Analysis The Network Analysis Process –Requirement Analysis Gather Requirements Develop Service Metrics to Measure Performance –capacity, delay, RMA Characterize Behavior Develop Performance Thresholds Distinguish Between Service Requirements

Dimitrios Makrakis SITE University of Ottawa Requirements Analysis

Dimitrios Makrakis SITE University of Ottawa

Dimitrios Makrakis SITE University of Ottawa Previous : Requirement Analysis The Network Analysis Process –Requirement Analysis Gather Requirements Develop Service Metrics to Measure Performance –capacity, delay, RMA Characterize Behavior Develop Performance Thresholds Distinguish Between Service Requirements –Flow Analysis Establish Flow BoundariesEstablish Flow Boundaries Identify Backbone/Composite FlowIdentify Backbone/Composite Flow Develop FlowspecDevelop Flowspec –Capacity Plan –Service Plan

Dimitrios Makrakis SITE University of Ottawa Flow Analysis Flow analysis is the process of characterizing traffic flows passing through the network. The intend is not to show every flow in the network. Only show those that have the greatest impact on the network architecture and design. –Generally these are often a small fraction of the total set of flows for a network. We will look at: –individual, composite, and critical flows. –Source and sink models –Flow specifications.

Dimitrios Makrakis SITE University of Ottawa Flow Analysis

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Background Flows Data Sources and Sinks Flow Models Flow Boundaries Flow Distributions The Flow Specification

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Background Background –We have established the user, application, host and the network requirements, established requirements specifications –Further analyse these requirements based on their end-to-end characteristics –We will discuss the concepts of flow and flow specification Use with best effort service for capacity planning Use with specified services for service planning

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Background Introduce flow concepts –Data sources and sinks –Flow models –Flow distributions Identify, size, and choose flows Develop cumulative performance specifications for each flow

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Definition Flows –Also known as traffic flows or data flows A flow is –A set of application and protocol information that has some common attributes, such as Source address Destination address Information type Options Routing etc. and is transmitted during a single session of an application.

Dimitrios Makrakis SITE University of Ottawa Routing through Network QoS Parameters, Directionality Flow Example Network Source/Destination Addresses, Port Numbers, Information Type User, Server or Other Device

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Flows are: –End-to-end, source to destination (between applications/hosts) They can be directly linked to an application, device, or network –Can also be examined on a link-by-link or network-by-network basis –Flow analysis is where performance requirements, services, and service metrics are combined with location information to show where service is needed in the network.

Dimitrios Makrakis SITE University of Ottawa Flow Analysis Perspective Flow Analysis can provide: –An end-to-end perspective on requirements –Some insight to the degree of hierarchy and redundancy needed –Useful information for choosing an interconnection strategy Switching? Routing? Hybrid mechanisms?

Dimitrios Makrakis SITE University of Ottawa Routing through Network QoS Parameters, Directionality Flow Example Network Source/Destination Addresses, Port Numbers, Information Type User, Server or Other Device

Dimitrios Makrakis SITE University of Ottawa Representing Flows Flow 1: (Bidirectional) 100 Kbps Upstream 500 Kbps Downstream Flow 2: (Unidirectional) 100 Kbps, 40 ms Upstream Flow 3: 250 Kbps Flow 4: 1 Mbps Flow 3 & 4: (A set of 2 unidirectional flows)

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Types of Flows

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Types of Flows There are 3 types of flows –Individual flow Flow for a single session of an application. When an individual flow has guaranteed requirements, it is generally kept with that individual flow. –Composite flow Combination of individual flows from multiple applications. Most flows in a network are composite. –Backbone flow Really a composite flow that is defined by its location in the network (i.e. backbone ). At this stage of our design we should not be considering network structure.

Dimitrios Makrakis SITE University of Ottawa An individual flow is the flow for a single session of an application –Derived from the requirements specification or –Estimated A composite flow is the combination of individual flows that share the same path, link or network –Used in capacity planning of the network Flow Analysis: Individual & Composite

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Backbone A backbone flow is formed by composite flows when the network achieves a certain degree of hierarchy Has implications on capacity planning –Useful for routing and addressing plans

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Critical Flows Critical flows are flows that are more important than others. They are higher in performance or have strict requirements. –mission critical, rate-critical, real-time, interactive, high performance. How do we identify and determine these flows? The first step is to identify the flow.

Dimitrios Makrakis SITE University of Ottawa Identifying and Developing Flows From an application perspective: –Focus on a particular application, application group, device, or function (e.g. video conference, storage). –Develop a “profile” of common or selected applications that can be applied over a user population. –Choose the top N (e.g. 3, 5, 10) applications to be applied across the entire network.

Dimitrios Makrakis SITE University of Ottawa Process for Developing Flows Identify Flows, Flow Requirements, and Locations from Requirements Specification Determine where sources and sinks are located Apply flow models where needed Combine performance requirements for flows into flow specification

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Location Maps 67 40 Servers (4) 45 22 14 22 2 88 Storage Servers (2) North Campus LAN Central Campus LAN South Campus LAN Devices of same type same location are combined; number Special Device Bldg A Bldg B Bldg C Bldg D Bldg E Bldg F

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Location Maps Example Data Migration Application “X” – staging data from user devices. Capacity: 100 Kbps. Delay: unknown. Reliability: 100%. Application “Y” – migrating data between servers. Capacity: 500 Kbps. Delay: unknown. Reliability: 100%. Application “Z” – migration to remote (tertiary) storage. Capacity: 10 Kbps. Delay: N/A. Reliability: 100%.

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Location Maps Example Data Migration Application 1 – staging data from user devices. Capacity: 100 Kbps. Delay: unknown. Reliability: 100%. Application 1 – migrating data between servers. Capacity: 500 Kbps. Delay: unknown. Reliability: 100%. Application 2 – migration to remote (tertiary) storage. Capacity: 10 Kbps. Delay: N/A. Reliability: 100%.

Dimitrios Makrakis SITE University of Ottawa 67 40 Servers (4) 45 22 14 22 2 88 Storage Servers (2) App 1 flows from user devices to temp servers App 1 flows from special devices to temp servers App 1 flows between servers Flow Analysis: Maps Application1 Bldg A Bldg B Bldg C

Dimitrios Makrakis SITE University of Ottawa Flows– Performance Measures 67 40 Servers (4) 45 Bldg A Bldg B Bldg C F1 F2 F3 F4 To North Campus Server F1, F2, F3 – 100 Kb/s, 100% F4 – 500 Kb/s, 100%

Dimitrios Makrakis SITE University of Ottawa Flows – Expansion of Bldg. C 67 40 Bldg A Bldg B F1 F2... To North Campus Server Flow Aggregation Point F3 F4 Combination of F1, F2, F3 Bldg C

Dimitrios Makrakis SITE University of Ottawa Choosing the Top N Applications Decision can be based on a number of factors, ranging from degree of usage, number of users, number of devices / servers, or performance requirements. –Web browsing –E-mail –File transfer –Word processing –Database transactions

Dimitrios Makrakis SITE University of Ottawa Data Sources and Sinks A data source is a device or group of devices that primarily produces data that the network will carry. A data sink primarily accepts data from the network, acting as a data repository. Almost all devices on a network will both produce and accept data. Some devices will act as either sources or sinks.

Dimitrios Makrakis SITE University of Ottawa Data Sources and Sinks Data sources: –Computing servers –Mainframes –Parallel systems –Computing clusters Data sinks: –Groups of disks/tape devices –Video editing or display devices Gives an indication of where flows are consolidated or generated Create flow models to help with the design process

Dimitrios Makrakis SITE University of Ottawa Data Source and Sink Symbols Source Sink Network

Dimitrios Makrakis SITE University of Ottawa 67 40 Servers (4) 45 22 14 22 2 88 Storage Servers (2) App 1 flows from user devices to temp servers App 1 flows from special devices to temp servers App 1 flows between servers Flow Analysis: Maps Application1

Dimitrios Makrakis SITE University of Ottawa Data Source and Sink Map 67 40 Servers (4) 45 22 14 22 2 88 Storage Servers (2) App 1 flows between servers

Dimitrios Makrakis SITE University of Ottawa Flow Models Another approach is to map flows to general, well known flow models Flow models are characterized by their –Directionality (some flows are asymmetric like DSL) –Degrees of hierarchy and interconnectivity. –Apply only to a single application or multiple applications Many flows have different requirements in each direction

Dimitrios Makrakis SITE University of Ottawa Flow Models Concepts Most flows are asymmetrical –Access and transmission technologies can be optimized for asymmetrical flows Digital Subscriber Loop (xDSL), ATM Hierarchy describes the degree of concentration of flows Hierarchy, is a result of the logical grouping of users, hosts, network, names, addresses etc.

Dimitrios Makrakis SITE University of Ottawa Well Known Flow Models Peer-to-peer Client-server Cooperative computing –Hierarchical client-server Distributed computing For each of these models we can study their flow characteristics and leverage this to create our flow models to: –Determine the directionality and hierarchy of flows –Identify critical flows – the most important ones impacting the backbone flows

Dimitrios Makrakis SITE University of Ottawa Hierarchical Design

Dimitrios Makrakis SITE University of Ottawa Peer-to-Peer Individual Flows

Dimitrios Makrakis SITE University of Ottawa Peer to Peer Flows Users and applications are roughly similar No obvious directionality No obvious hierarchy Example: early Internet where FTP and telnet were primarily used to transfer information We cannot distinguish flows in this model, therefore either all of the flows or none of the flows are critical. The flows are equivalent and can be described by a single specification, one profile.

Dimitrios Makrakis SITE University of Ottawa Peer-to-Peer Flows Default model All users need equal access to each other for an application Example: A tele-services application (e.g. teleconferencing) Most likely to be best effort Used in the capacity planning

Dimitrios Makrakis SITE University of Ottawa Client-server Model The most generally applicable model Has flow directionality and hierarchy Two way communications: –Request – small –Response – bigger –Asymmetric, mainly towards the client –May even be considered unidirectional (in practice) –Hierarchical

Dimitrios Makrakis SITE University of Ottawa Client-server flow model

Dimitrios Makrakis SITE University of Ottawa Client-server: Concepts Multicast at some layer in the network should be considered to optimize the flows Example: The Internet has evolved to become a client-server model – ftp servers, gopher, Archie, web-sherfing, email,…

Dimitrios Makrakis SITE University of Ottawa Cooperative Computing Flow Model More hierarchical Multiple applications work together and share information to accomplish a task or Multiple client-server applications are managed by a higher-level application Critical flows: server-to-server, server-to- manager, server-to-client Server – source or sink? - More information about application needed to decide

Dimitrios Makrakis SITE University of Ottawa Cooperative Computing Flow Potential Critical Flow Critical Flows Critical Flow

Dimitrios Makrakis SITE University of Ottawa Web services model Regional Server Web Clients CDN Mirror Local Server

Dimitrios Makrakis SITE University of Ottawa Distributed Computing Flow Model The most specialized flow model Flows may be primarily –Between a task manager and its computing nodes or –Between the computing nodes Computing nodes: –Loosely coupled – little or no information transfer between nodes –Closely coupled – frequent information transfer between nodes

Dimitrios Makrakis SITE University of Ottawa Distributed Computing Flow Model Tasks: –Having a coarse granularity Each task dedicated to a single computing node –Having a fine granularity A task is subdivided between several computing nodes and computing is done concurrently

Dimitrios Makrakis SITE University of Ottawa Distributed Computing Flow Model Compute Node Compute Node Compute Node Request Response Interaction

Dimitrios Makrakis SITE University of Ottawa Cyclone ™

Dimitrios Makrakis SITE University of Ottawa Computing Cluster Example When a task has a coarse granularity and nodes are loosely coupled –Computing cluster Asymmetric flow, mainly node to task manager Flows are not synchronized (independent of each other) Critical flows: node to manager

Dimitrios Makrakis SITE University of Ottawa Parallel Processing Example When a task has a fine granularity and nodes are closely coupled –Simplified parallel processing system Each task is subdivided between several computing nodes Nodes work concurrently exchanging information with neighbor nodes Flows may have the most stringent performance characteristics of any of the models Critical flow: between nodes

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Distributed computing model –Critical flow: between computing nodes –Multicasting should be considered for optimizing flow performance –Flow characteristics will vary between the computing cluster and parallel system models –The degree of granularity and degree of coupling is important in determining the flow.

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Flow Boundaries –Separations between large portions of the system, used to indicate where flow consolidations and hierarchies occur –Usually applied to separate geographic areas LAN/WAN LAN/MAN MAN/MAN Campus/Campus Building/Building Floor/Floor

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Flow Boundaries –Most frequently used boundaries: LAN/WAN Campus/Campus Building/Building

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Flow Boundaries –Logical boundaries can be used to separate between Backbones Flow concentration points (e.g. a network access point in the Internet) WANs, where service providers are likely to be used Specialized areas having specific requirements

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Flow Distributions –Used to determine where backbone flows are located in the design –Show when flows stay in one region of the network or transit one or more regions of the network –Regions for flow distributions: LANs, MANs and WANs –Flow distribution: within a LAN/between LANs

Dimitrios Makrakis SITE University of Ottawa Flow distributions

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Flow distributions –80/20 rule 80% of the flow of traffic stays within a LAN 20% is across a WAN Therefore, WAN capacity is approximately 25% of LAN –10 Mbps Ethernet LANs connected through 1.544/2.048 Mbps T1/E1 circuits –Today, 80/20 rule is less applicable –80/20, 50/50, 20/80: All possible

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts The flow specification –When we have Flows identified Each application’s requirements listed We can then determine how to combine the requirements for each flow

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts The flow specification – three forms: –A one-part (unitary) flowspec, used for capacity planning of best-effort flows when there are no specified flows –A two-part flowspec containing both best-effort and specified flows –A multipart flowspec, providing more detail on individual components of the specified flows

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts The flowspec algorithm –To develop the flowspec algorithm List the characteristics of each of the flows Apply an algorithm to bring these characteristics together into a specification for the combined flows –Combines Reliability Capacity Delay characteristics for each of the flows in order to describe the expected overall performance required

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts The flowspec algorithm – conditions: –Only capacity requirements are used in best- effort calculations –Use all characteristics (if available) in calculations for specified flows –When guaranteed delay and/or reliability requirements are indicated, they will be used individually in the calculations for the flowspec –Capacities generated by the unitary, two-part, and multipart flowspecs are base line capacities only, and not the performance modifiers

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Unitary flowspec –Capacity planning Best-effort (BE) environment

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Two-part flowspec –Capacity and Service Planning Predictable Flows R P, D P (Min Values) Best-effort environment

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Concepts Multipart flowspec –Capacity and Service Planning Guaranteed Environment, Requirements are listed Individually for each guaranteed flow for a set of values. (C i, R i, D i) Predictive Environment R P, D P, Best-effort environment

Dimitrios Makrakis SITE University of Ottawa Example

Dimitrios Makrakis SITE University of Ottawa Type 1: Flows between high-performance computing devices. Compute servers that are high-performance devices for this network. The first type of flow consists of traffic flows between these devices. Flows are: –approximately 10% of the time synchronized between pairs of devices. –At other times, these devices may draw data from the local data store of another computing device, from the storage server at the Main Engineering building, or request a computing or data management task from a device. Most computing tasks are controlled from Main Engineering. Type 2: Data migration from computing to storage at Main Engineering. These flows may be from each compute server as its task is completed, from the local data store at each compute server at a specific time, or from the compute server at Main Engineering at the completion of a combined (multiserver) task. Type 3: Data migration to external server. Final data sets, or extracts of final data sets, are pushed from the Main Engineering storage server to a storage server at a building where external (Internet) access is managed. Data at this external access server are used to feed other campuses and for remote (off-site) data archival. Type 4: Data archival and Internet access. These are flows from the external access server to users on the Internet, as well as flows to an off-site archival server. Example

Dimitrios Makrakis SITE University of Ottawa Example Flow IDCapacity (Mb/s)Delay (ms) F1 – Flow Type 1 Synch Files320100 Update16001000 Final16010 5 Results1600100

Dimitrios Makrakis SITE University of Ottawa Example Flow IDCapacity (Mb/s)Delay (ms) F1 – Flow Type 2 Update Files8010 4 Final16010 5 Results16010 4

Dimitrios Makrakis SITE University of Ottawa Example Flow IDCapacity (Mb/s)Delay (ms) F11760100 F21760100 F31760100 Results16010 4

Dimitrios Makrakis SITE University of Ottawa END

Dimitrios Makrakis SITE University of Ottawa Example The best example is that presented in the book Sect. 4.10 pages 195-203. In particular it is important to understand the values in the table in figure 4.48 Flow IDCapacity (Mb/s)Delay (ms) F1 – Flow Type 2 Update Files8010 4 Final16010 5 Results16010 4

Dimitrios Makrakis SITE University of Ottawa Example The best example is that presented in the book Sect. 4.10 pages 195-203. In particular it is important to understand the values in the table in figure 4.48 Flow IDCapacity (Mb/s)Delay (ms) F1 – Flow Type 1 Synch Files320100 Update16001000 Final16010 5 Results1600100 This is the Max Value because the 3 application flows do not occur at the same time This is the Min, Value as the algorithm states for predictable flows

Dimitrios Makrakis SITE University of Ottawa Example (contd.) Flow IDCapacity (Mb/s)Delay (ms) Results F1 –Type 11600100 Results F1 –Type 216010 4 Results for F11760100 This is the Min Value of the two as the algorithm states for predictable flows This is the Sum of Capacities as the algorithm states for predictable flows

Dimitrios Makrakis SITE University of Ottawa Example (Contd.) Flow IDCapacity (Mb/s)Delay (ms) F4 – Flow Type 11600100 F4 – Flow Type 2 Update32010 4 Final64010 5 Results Flow Type 264010 4 This is only 1xFlow Type 1 because flow is from Storage Server to Compute Server in M.E. where it is distributed to Bldgs. A, B, & C This is only 4xFlow Type 2 because flow is from the 4 Compute Servers (Bldgs. A,B,C & M.E.) to the Storage Server in M.E.

Dimitrios Makrakis SITE University of Ottawa Points to be Careful About It is important to list carefully the flow specification since it affects the design values. –The example in the book is one where at times it is not clear why certain capacities are summed and others are not. This has more to do with the details of how the applications execute (timing), model of the flow (client / server), and use of interim servers. –Building flow maps can be deceptive. The example in the book shows F4 as an aggregation of the 3 flows from Bldgs. 1-3 when in reality it also includes a fourth flow that is internal to the M.E. Building.

Dimitrios Makrakis SITE University of Ottawa Summary – Flow Analysis Guidelines The following set of slides are guidelines for the procedure for Flow Analysis.

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Applying the flow models Establishing flow boundaries Applying flow distributions Combining flow models, boundaries, and distributions Developing the flow specification Prioritizing Flows

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Applying the flow models –Application map – done –Flow model – done –The locations of data sources and sinks – identified –Therefore, flows and flow directions are identified See example 6.1 (McCabe)

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Establishing flow boundaries 1.Identify data sources and sinks 2.Determine the flow model 3.Place the flow boundaries 4.Identify flows crossing these boundaries 5.For flows crossing boundaries, determine whether they are critical flows 6.If step 5 applies give special consideration to critical flows crossing boundaries

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Applying flow distributions –Flow distributions tell us the expected relative traffic volumes of a flow when it crosses flow boundaries (e.g. 80/20) –Guidelines: Peer-to-peer – no particular flow distribution Client-server – 50/50 when clients must cross a boundary to get a server, otherwise 80/20 Cooperative computing - 50/50 when traffic from different servers must cross a boundary to get a server, otherwise 80/20

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Distributed computing flow distributions –Computing cluster and computing nodes are remote from the task manager 50/50 –Parallel computing and the traffic between computing nodes is across flow boundaries 20/80 These are only guidelines to indicate starting points and will be changed according to –Applications’ requirements –The distribution of users –The user behavior patterns Flow distributions are used to determine whether a backbone flow will be generated or not

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Combining flow models, boundaries, and distributions –Indicates the possibility and location of backbone flows –Backbone flows imply hierarchy in a network

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Developing the flow specification –Helps in making technology choices –Helps in requesting, configuring, and verifying services –Provides capacity and service plans –Used to generate a service plan (capacity + delay + reliability requirements) –Helps to determine how to meet delay and reliability requirements

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines To developing the flow specification we need –A listing of applications (applications portfolio) and their requirements –Category of each application – low/high performance, specified, or best effort –The applications map –Potential user and application modifiers for performance –Flow models, boundaries, distributions, and data sources and sinks for our environment

Dimitrios Makrakis SITE University of Ottawa Flow Analysis: Guidelines Prioritizing flows –Flowspec enables us to prioritize flows –This can be done according to Number of users affected by each flow The performance characteristics of each flow Distances and locations

Dimitrios Makrakis SITE University of Ottawa Example The best example is that presented in the book Sect. 4.10 pages 195-203. In particular it is important to understand the values in the table in figure 4.48 Flow IDCapacity (Mb/s)Delay (ms) F1 – Flow Type 1 Synch Files320100 Update16001000 Final16010 5 Results1600100

Dimitrios Makrakis SITE University of Ottawa CEG 4190-Winter 2018 Flow Analysis Instructor: Dimitrios Makrakis.

Similar presentations

Presentation on theme: "Dimitrios Makrakis SITE University of Ottawa CEG 4190-Winter 2018 Flow Analysis Instructor: Dimitrios Makrakis."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Dimitrios Makrakis SITE University of Ottawa CEG 4190-Winter 2018 Flow Analysis Instructor: Dimitrios Makrakis.

Similar presentations

Presentation on theme: "Dimitrios Makrakis SITE University of Ottawa CEG 4190-Winter 2018 Flow Analysis Instructor: Dimitrios Makrakis."— Presentation transcript:

Similar presentations

About project

Feedback