0. Fundamental Cloud Architectures
Cloud Computing
Lecture Note 9
Fundamental Cloud Architectures
오상규
정보통신대학원

1. Workload Distribution Architecture
Purpose
To evenly distribute the workload among the available IT resources for optimization of IT resource utilization – no over-utilization nor under-utilization
Architecture
Horizontal scale out/in of identical IT resources with load balancing mechanism
Various type of sophisticated load balancing algorithms
Applied to any independent IT resource: physical/virtual networks, storages, servers, etc. in general
Other cloud mechanisms affecting workload distribution
Cloud
Consumer A
Cloud
Consumer B
Cloud
Consumer C
Audit monitor – determining whether to apply depending on the type and geographical location of the IT resources
Cloud usage monitor – carrying out runtime workload tracking and data processing
Hypervisor – distributing workload (VM allocation) between hypervisors
Logical network perimeter – determining its boundary depending of how and where workloads are distributed
Resource cluster – applying only to those types of active/active or N-way active IT resource clusters
Resource replication – determining where to replicate based on runtime workload distribution demands
Workload Distribution (Load Balancer) for VMs
VM0
VM1
VMX
VM0
VM1
VMY
Hypervisor0
Hypervisor1
PMX
PMY
Cloud Z

2. Resource Pooling Architecture – 1/4
Purpose
To best utilize the given types of IT resources and to enhance allocation efficiency
Architecture
Grouping of identical IT resources together under a single IT resource pool and maintaining them ready to be allocated for efficient and fast resource allocation
Traditional mechanism to maintain free resources for possible allocation
Typical cloud resource pools
Physical server pool – composed of networked physical servers ready for immediate allocation with operating system and other standard software applications already installed
Virtual server pool – a virtual server image template pre-configured (specifying required OS, CPUs, amount of memory, etc.) by the cloud consumer during provisioning rather than a group of actual instances
Storage pool – composed of file-based or block-based storage structures that contain empty and/or filled cloud storage devices which are actually implemented as files on underlying OS or Hypervisors
Network pool – composed of different pre-configured network connectivity devices HERE
CPU pool – composed of individual processing cores ready to be allocated to virtual servers
Memory pool – composed of logically partitioned physical memory on each physical server
Resource pool structure
A larger resource pool can be created from a group of different resource pools – e.g., a dedicated pool consisted of multiple sub-pools: CPU pool, memory pool, storage pool and network pool.
The structure of resource pools can be hierarchical in a way that a resource pool (child pool) can be derived from another (parent pool) and each child pool is allocated a cloud consumer.

3. Resource Pooling Architecture – 2/4
Resource pools can be nested in a way that larger pools are divided into smaller pools that individually group the same type of IT resources together and such nested resource pools are allocated to different departments or groups in the same cloud consumer organization.
Other cloud mechanisms related to resource pool architecture
Audit monitor – monitoring on resource pool usage to ensure compliance with privacy and regulation requirements, especially when pools contain cloud storage devices or data loaded into memory
Cloud usage monitor – keeping track of runtime usages for various types of pooled IT resources
Hypervisor – associating proper resource pools for VMs to access and hosting resource pools directly when necessary
Logical network perimeter – logically organizing and isolating resource pools
Pay-per-use monitor – collecting usage & billing information on various resource pools allocated to each individual cloud consumer
Remote administration system – interfacing with backend systems and programs in order to provide resource pool administration features via front-end portal
Remote management system – supplying cloud consumers with the tools and permission management options for administrating of resource pools
Resource replication – generating new instances of IT resources for resource pools

4. Resource Pooling Architecture – 3/4
Dedicated pools and hierarchical structure
Dedicated Resource Pool A (Parent Pool of B and C)
Virtual Server Pool
CPU Pool
Storage Pool
Memory Pool
Network Pool
Dedicated Resource Pool B (Child Pool of A – Sibling Pool of C)
Virtual Server Pool
CPU Pool
Memory Pool
Dedicated Resource Pool C (Child Pool of A – Sibling Pool of B)
CPU Pool
Memory Pool
Network Pool

5. Resource Pooling Architecture – 4/4
Nested pool structure
Dedicated Resource Pool A
Virtual Server Pool
CPU Pool
Storage Pool
Memory Pool
Network Pool
Nested Resource Pool A-1
Virtual Server Pool
CPU Pool
Storage Pool
Memory Pool
Network Pool
Nested Resource Pool A-2
Virtual Server Pool
CPU Pool
Storage Pool
Memory Pool
Network Pool

6. Dynamic Scalability Architecture – 1/2
Purpose
To dynamically scale up/down or out/in IT resources based on predefined scaling option
Dynamic scaling architecture
Runtime status monitored in real-time by automated scaling listener (ASL) agents
Any runtime status reported to VIM when going out of predefined thresholds
Scaling up/down or out/in determined automatically by VIM according to predefined scaling policy
Scaling action performed dynamically in cooperation with VIM, Hypervisors, and ASL agents
Automated dynamic scaling out example (vs. scaling up)
Cloud
Consumer
The amount of requests from the cloud consumer increases.
ASL agent detects that the VM gets overloaded.
ASL agent reports the runtime status to VIM.
VIM decides what to do based on predefined user scaling policy.
VIM triggers VM resource replication from the current hypervisor to another hypervisor.
VM resource is replicated to another physical server.
VIM initiates a new VM instance creation to the new hypervisor.
A new VM instance is created from the replica.
The overloaded requests are finally served. 1
Request Increase
Automated Scaling Listener 3
Report 9
Request Served
VMX 2
VM Overloaded
VM0
VM1
VMX
VM0
VM1
VM Creation 8
VM Replication 6
Hypervisor0
Hypervisor1 5
Replication
Command 7
VM Instance Creation Command
PMX
PMY
VIM 4
Check Out Scaling Policy
Cloud Z

7. Dynamic Scalability Architecture – 2/2
Dynamic scaling mechanism
Dynamic resource allocation mechanism enables variable utilization as dictated by usage demand fluctuations since unnecessary IT resources are efficiently reclaimed without requiring manual interaction in addition to allocating new IT resources when required.
The automated scaling listener is configured with workload thresholds that dictate when new IT resources need to be allocated to or when allocated IT resources need to be deallocated based on the terms of a given cloud consumer’s provisioning contract.
Types of dynamic scaling
Dynamic horizontal scaling: IT resource instances are scaled out or in to handle fluctuating workload. The automated scaling listener monitors requests and triggers resource allocation or deallocation.
Dynamic vertical scaling: IT resource instances are scaled up or down when there is a need to adjust the processing capacity of a single IT resource. The memory or processing cores of a virtual server can be dynamically added or removed depending on the current load and predefined scaling policy.
Dynamic relocation: dynamic vertical scaling may trigger dynamic IT resource relocation when more processing capacity is required – e.g., a VM can be dynamically relocated to another physical server when more processing power (cores) or IO capacity is needed, but no more resource is available on the current physical server.
Other cloud mechanisms related to dynamic scalability architecture
Cloud usage monitor – keeping track of runtime usages in response to dynamic fluctuations
Hypervisor – creating or removing virtual server instances
Pay-per-use monitor – collecting usage & billing information in response to the scaling of IT resources

8. Elastic Resource Capacity Architecture
Purpose
To dynamically provision processing cores and memory capacity for virtual servers
Dynamic scaling up/down architecture
Runtime status monitored in real-time by automated scaling listener (ASL) agents
Workflow logic deployed via intelligent automation engine script for dynamic allocating or reclaiming of CPU cores and RAM – vertical scaling
Any runtime status reported to VIM when going out of predefined thresholds to trigger elastic resource de/allocation via intelligent automation engine script
Cloud
Consumer
Scaling up/down automatically by running intelligent automation engine script on VIM
Dynamic scaling up example (vs. scaling out)
The amount of requests from the cloud consumer increases.
ASL agent detects that the VM gets overloaded.
ASL agent reports the runtime status to VIM to trigger the intelligent automation engine.
A corresponding intelligent automation engine script automatically runs and signals the hypervisor.
The hypervisor adds more processing cores to the VM from CPU resource pools.
The overloaded requests are finally served. 1
Request Increase
Automated Scaling Listener 3
Report 6
Request
Served
VMX 2
VM Overloaded
VIM
Intelligent Automation Engine
VM0
VM1
VMX + CPU 5
Add Cores
VMX
Hypervisor 4
Signal to Scale Up PM
Cloud

9. Service Load Balancing Architecture
Purpose
To evenly distribute the workload among the available IT resources for optimization of IT resource utilization
A specialized variation of the workload distribution architecture that is geared specially for horizontally scaling cloud service implementation
Service load balancing architecture
Redundant deployments of cloud services with a load balancing system added for dynamic distribution of workload
Multiple instances of each cloud service generated as part of a resource pool that responds to fluctuating request volumes more efficiently, depending on the anticipated workload and processing capacity of host server environments
Cloud
Consumer A
Cloud
Consumer B
Cloud
Consumer C
Load balancer mechanism implemented as either external (independent from service instances) or built-in components (main gateway on the primary server (VMX) with software load balancing and redistribution mechanisms embedded)
Other related cloud mechanisms
Cloud usage monitor – keeping track of runtime usages in response to dynamic fluctuations
Resource cluster – applying only to those types of active/active or N-way active IT resource clusters
Resource replication – determining where to replicate based on runtime workload distribution demands
VM0
VM1
VMX
VM0
VM1
VMY
Hypervisor0
Hypervisor1
PMX
PMY
Cloud Z

10. Cloud Bursting Architecture
Purpose
To dynamically scale out or “burst out” on-premise IT resources into a cloud whenever predefined capacity thresholds have been reached
Cloud bursting architecture
The duplicated system of a on-premise system pre-deployed redundantly in a cloud environment while remaining inactive until cloud bursting occurs
Resource replication mechanism and automated scaling listener activated to synchronize on-premise and cloud-based systems and to monitor real-time service traffics for possible re-direction
Requests to the service re-directed to the cloud-based system when the predefined capacity thresholds have been reached by automated scaling listener (burst out) and directed back to the on- premise system when the service load goes under the predefined capacity thresholds (burst in)
Flexible scaling architecture for cloud-based IT resources to be used as a contingency backup or standby system in preparation for workload bursting
Consideration
Replication overhead – possible performance degradation due to resource replication, especially with low possibility of workload bursting
Synchronization overhead – drastic performance degradation due to bi-directional resource synchronization when a part of service requests are redirected to the cloud-based IT system – highly recommended for WORK workload only
Cloud
Consumer 1
Request Increase 2
Overloaded
Automated Scaling Listener 3
Request Redirection
Cloud
Resource Replication
On-Premise System

11. Elastic Disk Provisioning Architecture
Purpose
To dynamically provision storage system in order to ensure that the cloud consumer is granularly billed for the exact amount of storage actually in use
Elastic disk provisioning architecture
Thin-provisioning technology for the dynamic allocation of storage space and runtime usage monitoring system for collecting accurate usage data for billing purpose
Thin-provisioning software module on virtual servers to process dynamic storage allocation via the hypervisor
Pay-per-use monitor to keep track of and to report granular billing-related disk usage data
Resource replication for possible conversion of dynamic thin-disk storage into static thick-disk storage
Pay-per-use Monitor
Thin-provisioned VM
300GB Provisioned
Thin-provisioned VM
Pay-per-use Model
Thick-provisioned VM
300GB Statically Allocated
Thick-provisioned VM
Pay-per-allocation Model
Applications
Applications
Applications
Applications
Host OS
Host OS
Host OS
Host OS
100GB
100GB
100GB
100GB
100GB
100GB
100GB
100GB
100GB
100GB
100GB
100GB
Hypervisor
Hypervisor
Thin Provisioning Module
1TB
1TB
1TB
1TB
1TB
1TB
1TB
1TB
Physical Storage Pool
Physical Storage Pool
Physical Server
Physical Server

12. Redundant Storage Architecture
Purpose
To provide highly available storage system via redundant storage device architecture and storage service gateway mechanism
Redundant storage architecture
Two storage devices with the same capacity – the primary and the secondary storage devices
Data requests directed to the primary storage device by storage service gateway
The primary and the secondary storage devices synchronized for possible service failover: data is the only IT resource or component that can not be instantly replaced – real-time or periodic synchronization is required
Data requests re-directed to the secondary storage device when the primary storage device is not available due to network failure, hardware failure, security breaches, etc.
Consideration
Replication/synchronization overhead – possible drastic performance degradation due to significant data replication overhead, especially when the secondary storage device locates on the remote site
Check pointing mechanism – minimizing the amount of data reversely synchronized after storage service failover and costing extra overhead – the secondary storage device is now the primary storage device and requires its backup device in order to keep continuously providing high availability
Cloud
Consumer
Storage Service Gateway 2
Requests Redirected 3
Device Recovered 1
Device Fail
1TB
1TB
1TB
1TB
1TB
1TB
Reverse Replication
No Replication
Data Replication
Primary Storage Device
Secondary Storage Device

13. Cloud Computing
End of Lecture Note
오상규
정보통신대학원