N ORTH D AKOTA S TATE U NIVERSITY D EPARTMENT OF C OMPUTER S CIENCE © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Autonomic Self-Testing Of the Cloud, In the Cloud CS-Industry Consortium Outreach: IBM Presenter: Tariq M. King April 21, 2010

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Outline  Motivation  Background  Position Paper (ICSTW ‘10 )  Migrating AST to the Cloud  Work-In-Progress  Conclusion 2

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Motivation  Cloud services deliver computing resources to users via the web, e.g., business process, data store  Generally characterized by:  High availability and performance requirements  Dynamic behavior through service discovery, redirection, load balancing, on-line updates  Tailor-made services can be built from existing services, while hiding underlying complexity  How can we test stateful, adaptive, remote services in the cloud without excessive degradation of QoS? 3

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Background  Cloud offers Everything-as-a-Service (XaaS)  Software-as-a-Service – programs designed to help users perform a task, e.g., Google Apps  Platform-as-a-Service – development, testing, and execution environments to support applications, e.g., Microsoft Windows Azure.  Infrastructure-as-a-Service – computer hardware including servers, network equipment, and the software to support them e.g., Amazon CloudWatch  Trend towards virtualization for providing XaaS 4 CLOUD COMPUTING

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Background  Autonomic Computing [IBM 2003 ] aims to reduce the burden of managing complex systems  Systems dynamically adapt to changes in their environment, i.e., four self-* features  King et. al [ 2007 ] introduce an implicit self-test characteristic  Regardless of self-* features implemented, system should validate adaptations at runtime 5 AUTONOMIC COMPUTING Self- Configure Self- Optimize Self- Heal Self- Protect Self- Test

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Self-* is realized through Monitor-Analyze-Plan- Execute (MAPE) loops in Autonomic Managers  M onitor – polls state of managed resource  A nalyze – determines if state is undesirable  P lan – formulates a plan to address the problem  E xecute – implements the plan on the resource Managed Resource Background 6 AUTONOMIC COMPUTING Monitor AnalyzePlan Execute Knowledge Effector SymptomsChange RequestChange Plan Sensor Effector Sensor Autonomic Manager

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Autonomic Test Managers apply the concept of MAPE to runtime testing of adaptive changes  Monitor, intercept, and validate change requests generated by AMs  AST is supported by: » Two General Strategies » Detailed Workflows » Various Algorithms » Object-Oriented Designs Managed Resource Autonomic Self-Testing 7 TEST MANAGERS Monitor AnalyzePlan Execute Knowledge Effector Sensor Effector Sensor Autonomic Manager Monitor AnalyzePlan Execute Knowledge Autonomic Manager Test Manager

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Safe Adaptation with Validation  tests changes in-place, directly on managed resource  Replication with Validation  tests changes using copies of managed resources  testing can occur “transparently” on a separate node TESTING NODE SYSTEM NODE AST (cont’d) 8 VALIDATION STRATEGIES Autonomic Self-ManagementAutonomic Self-Testing Managed ResourceCopy of Managed Resource

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Combine the development of an autonomic self- test harness for a cloud service, with the delivery of Test Support as-a-Service (TSaaS)  Each cloud provider develops a self-test harness for validating changes to their hosted services and underlying infrastructure  Sub-procedures of automated tests are modeled as operations realized on a separate test bed and exposed via a TSaaS API for cloud partners Migrating AST to the Cloud 9 OVERVIEW

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Emphasis is on the use of Replication with Validation in the cloud to address high availability requirements This model can then be encoded in test scripts for automated tools, and linked to a self-test harness During the development of a cloud service, a test model is generally created to validate its behavior Local Test Bed Developing the Self-Test Harness 10 X-as-a-Service Monitor AnalyzePlan Execute Knowledge Test Model Cloud ConsumerCloud Partner XaaS Test Manager

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Exposing Test Support as-a-Service 11 Hypervisor Test Support as-a-Service Partner Test Bed Automated test setup, input, and assertion operations from the test model are used as the basis for developing TSaaS Partners can use TSaaS during development of tailor made applications and services, and for automating regression tests Virtual Test Environment XaaS Virtualization is used to reduce the overhead of maintaining copies and enhance testing (on-demand testing, snapshots) Local Test Bed X-as-a-Service Monitor AnalyzePlan Execute Knowledge Test Model Cloud ConsumerCloud Partner XaaS Test Manager

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Application View 12 SYSTEM-WIDE SELF-TESING  Partners generally develop applications by extend- ing existing services (A,B) that are remotely hosted  An adaptation manager is used to enable dynamic behavior, and a test manager for validation  Now using TSaaS APIs partners can perform integration testing of stateful remote services

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Work-In-Progress 13  Goal: Proof of Concept TSaaS with Virtualization  Steps:  Define a method and detailed workflows for TSaaS  Acquire necessary hardware and software resources  Apply the method in the context of a problem-based scenario in the healthcare domain  Platform and Tool Support:  Windows Server 2008 R2 – WMI and Hyper-V API  Visual Studio 2010 – Service Development (Azure) PROTOTYPE PLAN

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Scenario 14 PATIENT-CENTERED HEALTHCARE  While abroad you take ill and are admitted to a local hospital. A cloud-based solution provides the admitting doctor at the hospital with services for:  Locating and accessing your medical history using a variety of sources, e.g., hospitals, clinics, pharmacies  Consulting with other specialists in real-time for a second opinion on your condition (AV, Chat, File)  Scheduling an appointment with another specialist on your behalf, or with your regular physician  Filing medical claims with your insurance

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Such a System Can Be… 15  Mission Critical  In the presence of medical emergencies  Self-Configuring  New hospitals, clinics, pharmacies, specialists, and services can come up or existing ones go obsolete  Automatically realize communication based on user needs and device capabilities  Self-Optimizing  Balance workloads to maintain throughput...and more giving rise to the need for AST in the Cloud

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Conclusion 16  Presented a high-level view of Autonomic Self- Testing of Cloud Applications and Services  Our approach seeks to improve the testability of the cloud, and enable automated testing in the cloud to cross system boundaries  Plans to build a prototype of TSaaS and apply it in the context of patient-centered healthcare  Investigation will include using TSaaS during development, and for automated regression testing

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP Acknowledgements 17  Annaji Sharma Ganti  Dr. Kenneth Magel  Mohammed Akour  Akanksha Jaidev  Dr. Scott Tilley and the STITC Reviewers

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP References 18  IBM Autonomic Computing Architecture Team, “An Architectural Blueprint for Autonomic Computing, IBM, Hawthorne NY, Technical Report, June  T. M. King, A. E. Ramirez, R. Cruz, and P. J. Clarke, “An integrated self-testing framework for autonomic computing systems,” JCP, vol. 2, no. 9, pp. 37–49,  T. M. King, “A self-testing approach for autonomic software,” Ph.D. dissertation, Florida International University, Miami, FL, USA, August  A. Weiss, “Computing in the Clouds,” netWorker, vol. 11,no. 4, pp. 16–25,  W. K. Chan, L. Mei, and Z. Zhang, “Modeling and Testing of Cloud Applications,” in APSCC ’09: Proceedings of the 2009 IEEE Asia- Pacific Services Computing Conference. Los Alamitos, CAA, USA: IEEE Computer Society, 2009.

© NDSU S OFTWARE T ESTING R ESEARCH G ROUP 19 Questions? ¿Preguntas? 問題 Sawwal вопросы 質問 domande ερωτήσεις The End.