1. Performance Testing Methodology for Cloud Based Applications

2. Cloud Performance Challenges
Scalability
Elasticity
Failover/DR Tests
Multi-tenancy
Availability of wide range of IP addresses for load simulation

3. Traditional On-Premise Application Testing in QA cloud
Traditional On-Premise Application Testing in QA cloud – absolute no change in approach barring the below points to note in performance testing.
Factors
Key Points to Note
Earlier performance benchmarks available from the on-premise /staging environments
Performance test results from cloud need to be compared with earlier available benchmarks. If this is not available, then it might be worth considering to do a performance bench marking exercises from production like staging environment. Remember the cloud and virtualized environment can result in a 05-10% performance degrade . So factor in this for translating the results to on-premise apps.
Factor in the performance testing tool setup
Enterprises might have performance testing tools with them and hence might have the load injection infrastructure within their network else they might be availing load testing tool and infrastructure services externally. You need to factor in these carefully.
Monitoring physical and virtual resources essential
In virtualized environment OS monitors such as CPU, memory utilization, disk & network activity reported by guest OS. This does not give the complete picture and hence it is essential to monitor the underlying physical system resources as well. Example Genilogix Monitor Bridge for VMware is an add-in for LR allows VMWareESX server stats from multiple hosts to be collated agentlessly.

4. Performance Testing in the Cloud
Existing Apps being re-engineered for the cloud:
Check for existence of performance benchmarks when on- premise.
If there are no performance bench marks, then carryout a performance benchmarking of the apps on-premise.
Go to “New App built for the Cloud”
New Apps built for the cloud:
Validate the NFRs for the applications being hosted on the cloud.
Assess the Cloud – Private/Public/ Hybrid
Do a tool fitment based on the technology stack, cloud setup and strategy.
Setup the test tool in the cloud, identify all dependencies, port openings between cloud service providers, where tool and applications are hosted.
Carryout a proof of concept for the identified tool.
Test the performance of the cloud platform.
If clouds involved are Hybrid then validate for performance – private and public separately.
In case of Hybrid cloud perform inter-cloud connectivity.
Test web services performance individually at component level, check for compliance to NFRs
Then test for the services performance together while integrating them.
Finally, test for performance for end to end business scenarios holistically.
Characteristics to Test: Scalability, Elasticity, Accessibility, DR, Virtual Failover Testing, Performance
© 2011 Infosys Technologies Limited

5. Performance Testing Methodology for cloud based applications1
Assess Non-functional Requirements 2
Understand Usage Patterns 3
Test Strategizing & Test Planning 4
Workflow Traversal &Test Script Preparation 5
Cloud Environment Readiness 6
Test Data Population 7
Automated Test Suite Generation for Non-functional Testing 8
Test Execution, Reporting & Publishing 9
Certification
© 2011 Infosys Technologies Limited

6. Performance Testing Methodology for Cloud Based Applications- Performance, Elasticity, Availability & Resilience
Step 1: NFR Analysis
Step 2: Test Strategy & Planning
Step 3: Test Preparation
Step 4: Test Script Preparation
Step5: Test Execution, Reporting & Certification
Activities:
Understand Business Needs
Elicit and analyze NFRs– spanning application, data, infrastructure (Performance, Elasticity, Availability, Compliance, & Regulations)
Understand underlying
Cloud Architecture
Cloud Deployment Model
App Design, Technology Stack
3rd party tools/middleware integration
Hardware Configs
Compliance Regulations
Availability zones & regions, if any
Document NFRs
Review and Sign off NFR doc
Activities:
Identify critical business transactions & scenarios
Analyze usage patterns, geo distribution,
Prepare workload model
Identify different types of testing to be done covering performance , elasticity, availability and resilience
Identify key performance metrics to be measured, SLAs to be monitored
Identify performance and monitoring test tools, diagnostics provided by cloud service provider
Prepare performance test strategy and test plan
Review and Sign off performance test strategy and plan
Activities:
Create Traversal Workflow documents for identified business critical, resource intensive business transactions that need to performance tested
Ensure as part of traversal workflows the exception scenarios as well are being accounted for.
Review and Sign off traversal workflows
Activities:
Ensure environment readiness for testing
Install and configure test tools, establish connectivity to cloud in which application will be hosted
Using the identified performance test tool create test scripts for performance
Setup test data required for validation
Perform dry runs of test scripts against the cloud instances.
Review and Sign off of test scripts
Activities:
Carry out performance tests – performance , elasticity, scalability and resilience as per agreed traversal workflows and as per defined strategy
In case of hybrid cloud scenarios execute and measure performance - isolated and then together
Collate performance metrics from all layers of the cloud setup. Ensure the monitoring is not only accounted for virtual systems but as well as for physical resources
Prepare analysis and publish test report
Identify bottlenecks, make recommendations for improving performance
Deliverables:
Non-Functional Requirements Document
Deliverables:
Performance Test Strategy & Plan Document
PoC, ROI Report
Deliverables:
Traversal Workflow Documents
Deliverables:
Performance Test Scripts
Deliverables:
Performance Test Reports
Performance Defect
Tuning Recommendations