Dynamically Scaling Applications in the Cloud Presented by Paul

Problem The cloud offers o Near infinite computing capabilities o Near infinite storage o On-demand access to new VMs o Pay-as-you-go model But how do you build an application to automatically take advantage of those resources? [1]

Overview of scaling options [1]

Horizontal Scaling VM Replication o Duplicate servers and create load balancers to distribute incoming request o Most common approach to scaling o Will look in more detail... Network Scalability o As an application scales up, the bandwidth requirements for intercloud communication and app requests increases, [1] assesses the possibility of requesting network resources in conjunction with cloud resources Network as a Server (NaaS) [1]

Vertical Scaling VM resizing (live) o Taking a VM with given processing power, and upgrading it with more resources while it's still running VM replacement o Replacing less powerful VM with more powerful VM (shutting down original VM) Might require reboots, and application tolerance [1]

"Rule of thumb" CPU-Intensive o Best to load balance and split computation among many instances Network intensive o Better to use single CPU heavy instance More network intensive o DNS-based load balancing on powerful CPUs [1]

Decisions, decisions 1 Cloud to rule them all? o Do you want to use multiple cloud providers? o Does part of your application require running on private cloud? How much control over automated scaling would you like? o If you don't want any control, maybe PaaS is better option o Do you simply want horizontal scaling, and will handle load balancing on your own? o How much $$$ do you have?

Methods for Scaling Automatic VM Scaling Dynamic Workload-pattern Matching Whole Application Scaling Non-scalable Load Balancing (e.g. Elastic Load Balancing) DNS-based Load Balancing [1]

Basic: Automatic VM Scaling Services that scale based on predefined VM- related performance metrics Offered by Amazon and Rightscale o Amazon Autoscaling comes with Cloudwatch  Set conditions based on Cloudwatch variables, react to latency, CPU speed, etc. What you might imagine for scaling - like a state machine o If conditions a,b,c are met, do x,y,z [1]

The "Controller"

"Non-Scalable" Load Balancers Amazon offers load balancing service (Elastic Load Balancing) o The reason it's "non-scalable" is this is a single load balancer Could be used in conjunction with horizontal scaling o Scale up/down the number of instances, configure load balancer to balance across running instances

Elastic Beanstalk Automatically scale up your application depending on services it's using, and beanstalk controller parameters o Only pay for underlying AWS resources Can upload.NET, PHP/Python, and Java apps for integration with Elastic Beanstalk Closer in function to a PaaS o Still have access to AWS instances, can interact with elastic beanstalk controller

Whole Application Scaling [3]

Whole Application Scaling [3]

Dynamic Workload-pattern Matching [2] Scalability Controller

Dynamic Workload-pattern Matching [2]

Final Observations You must first decide how many clouds you want to use Scaling is still an art o Systems for automated scaling still in their infancy At the core, there is always a scaling controller o Either you configure and operate the controller, or the IaaS provider does (e.g. Elastic Beanstalk) You pay for automation o Either with your time or your money o More customized automation == more complexity

Sources [1] L. M. Vaquero, L. Rodero-Merino, and R. Buyya. "Dynamically scaling applications in the cloud," SIGCOMM Comput. Commun. Rev., 41: [2] P. Marshall, K. Keahey, and T. Freeman, "Elastic site:Using clouds to elastically extend site resources," Cluster Computing and the Grid, IEEE International Symposium on, vol. 0, pp , [3]R. Buyya, R. Ranjan, and R. Calheiros, “Intercloud: Utility-oriented federation of cloud computing environments for scaling of application services.” in ICA3PP 2010: The 10th International Conference on Algorithms and Architectures for Parallel Processing, 2010, pp. 19–24.

Links for papers [1] nCloud2011.pdf [2] e_ccgrid_2010.pdf [3]