1. Adaptive software in cloud computing Marin Litoiu mlitoiu@yorku.ca York University Canada

2. © Marin Litoiu, EU-Canada Future Internet Workshop

3. Content  Elasticity  Business Driven Elasticity  Architectures –Model based feedback loops –Strategy trees based feedback loops  Open Issues

4. © Marin Litoiu, EU-Canada Future Internet Workshop Elasticity  Traditionally, we sized applications for typical workloads Wasted capacity (Cost) Lost revenue Lost customers (lost revenue) *Berkley report  The promise: adapt at runtime

5. © Marin Litoiu, EU-Canada Future Internet Workshop Business Driven Elasticity  Elasticity achieves business goals –Minimize cost (and/or increase revenue) –Includes energy savings –Meet SLO  By monitoring the goals through sensors  Response times, utilization, profit, cost etc..  And by changing control inputs (actuators) –Hardware resource allocation ( CPUs, memory, storage…) –Software resources (licences, threads, replicas…)  Subject to policies (constraints) –Hardware and software capacity limits 5

6. © Marin Litoiu, EU-Canada Future Internet Workshop Adaptive Feedback Loops

7. © Marin Litoiu, EU-Canada Future Internet Workshop Cloud Landscape SaaS Management PaaS Management IaaS Management SaaS (Salesforce, Google, IBM) PaaS (Google, IBM) IaaS (Amazon, IBM) CPU Hardware Execution Environment Programming Environment Application Simple services (OpenID) Virtual machine CPU Access Services Service Models: IaaS, PaaS, SaaS Deployment Models: private, community, hybrid, public

8. © Marin Litoiu, EU-Canada Future Internet Workshop …hence Layered Adaptation PaaS IaaS SaaS Sensors Processors, memory, disk utilization Processors throughput and response times Actuators VM to processor allocation VM settings ( memory, CPU ratio) VM storage Network bandwidth Major Goals: hardware utilization Constraints: capacity Sensors VM Utilization, response time, throughput Container utilizations, throughput, response times.... Actuators Number of VMs, licenses Allocation of containers to VMs Container settings (threads, caching)... Major Goals: Reduce cost Increase revenue Constraints: capacity Sensors: - Service (application) QoS -response time -throughput Actuators Deployment topology Parameter tuning Major Goals QoS (response time, throughput) Constraints : SLA

9. © Marin Litoiu, EU-Canada Future Internet Workshop 1. Model Based Adaptive Loops

10. © Marin Litoiu, EU-Canada Future Internet Workshop Adaptation at PaaS Layer  Goal: Profit = Revenue-Cost –Revenue = proportional with the number of applications –Cost = Price per VM running + price licences thus at a given moment the goal is minimum cost for the given applications  Constraints: SLAs, capacity limits, etc PaaS Sensors VM Utilization, response time, throughput Container utilizations, throughput, response times.... Goals: platform profit Actuators Number of VMs, licenses Allocation of containers to VMs Container settings (threads, caching)...

11. © Marin Litoiu, EU-Canada Future Internet Workshop PaaS Optimization and Control…. Minimize COST in PaaS -Across N applications -Subject to -SLA -application integrity -processing capacity -memory -licence constraints By asking IaaS to slice the physical resources into virtual resources

12. © Marin Litoiu, EU-Canada Future Internet Workshop Results (1): SaaS The cost is low when traffic is low Response time is kept below a target The application uses less physical machines when traffic is low multitier interactive applications

13. © Marin Litoiu, EU-Canada Future Internet Workshop Results(2): PaaS  Results are for multi tier applications, that can scale horizontally and vertically  FO: full optimization  IO: incremental optimization  It is more efficient to redeploy ALL applications periodically (similar to disk defragmentation)

14. © Marin Litoiu, EU-Canada Future Internet Workshop 2. Policy Based Adaptive Loops

15. © Marin Litoiu, EU-Canada Future Internet Workshop Further Challenges  Centralized versus decentralized adaptation –Geographically distributed clouds  Coordination among different layers –Sensors and actuators  Global versus local optimization  Accurate models for different layers

16. © Marin Litoiu, EU-Canada Future Internet Workshop References  Litoiu M, Woodside M., Wong J., Ng J., Iszlai G., “A Business Driven Cloud Optimization Architecture”, Proceedings of ACM SAC 2010, Sierre, Switzerland, March 24-29, 2010  Simmons B., Litoiu M., “Towards a Cloud Optimization Architecture with Strategy Trees,” Proceedings of IEEE I2TS 2010, Rio de Janeiro, Brazil, Dec 2010.  Ghanbari H., Litoiu M., Simmons B., Barna C., “Feedback-based Optimization of a Private Cloud,” IEEE Conference on Utility and Cloud Computing ( UCC 2010), December, Chennai, India, 2010.  Zheng T., Litoiu M., Woodside M., “Integrated Estimation and Tracking of Performance Model Parameters and their Trends,” 2 nd ACM/Spec International Conference on Performance Engineering, Karlsruhe, March 14-16, 2011.