Test Environments Arun Murugan – u Rohan Ahluwalia – u Shuchi Gauri – u

More…….. I have developed a great piece of code. I am doing some Unit Testing and my code works great. There is no error in it. No Bugs... Hooray!!! My job is to verify and validate the piece of software this developer has made. I have so many test cases to run. Hey!!!!! I am this cool software over which these two people work. Let’s see what happens..

Manager, This software is just not working. I think developers don’t know what they have given. It just doesn’t work. Oh ok, I’ll write a mail to the Project Manager But PM, everything is working fine here. Ok ok.. Let’s just go to the tester and see what’s wrong. More……..

Crisis!!!! What will happen to the project. Does not work May be it’s an environment issue. Lets compare the environments We have SP2 here We have SP1 here It works

I am so glad that the problem has been solved. We wasted one whole day to figure it out. Yes, we should now have a checklist for environment configuration and control it.

Test Environments Topics Covered: Development and management of test environments. How many environments? How to manage differences in test environments from development through to test, acceptance and production environments. Issues associated with not being able to replicate the operational (production) environment. Potential issues arising from testing with real production data and non production data.

Development & Management of Test Environments Establishing the right testing environment is an essential component of an effective and efficient software testing process. Key components of the environment are: Software testing work processes. Software testing and test management tools (software and hardware). Competent testing staff. Process improvement methods. Operating systems, and Environment. Test data. Test environments.. need to be controlled through various levels of management. are driven by business needs. enhance productivity of testers. are responsibility of Senior IT management. should be as close as possible to the production environment.

Development & Management of Test Environments contd.. Role of management in creating test environment: Decide policy for testing Type of support provided by management for software testing. Processes and tools. Level of “risk appetite” the management has. The testing environment to a large degree enables testers to perform a role that will both identify early in the project potential gaps, and during the project determine the magnitude of those gaps. The testing environment must help testers identify: Gaps between software needs and delivered software. Gaps between software specifications and delivered software.

Development & Management of Test Environments contd.. There is a need to manage and control differences between test environments: To uncover issues consistently across all the environments and not just some. To enhance predictability. To avoid surprises/shocks. To keep the customer’s confidence in the project and staff.

Test environment throughout development lifecycle Important to have environment throughout development lifecycle as close as production. Test environment would ensure proper validation of the project. Test environment for development should be separate from production, so that no changes are required on fly. Levels: Hardware, Software/OS versions, environment to ensure proper configuration management. Using test support tools to manage a testing environment. E.g. defect tracking systems, automation tools. A quality process should always check the environments to ensure their closeness to the production environment. Minimum 3:  1 – Development  2 – Testing  3 – Production

How to manage diff. b/w environments Organizations prefer to maintain separate environments for development, testing, acceptance and production. Under development, the team continually implements new features and makes changes to improve performance. Under production, stability is required and changes are done only on the basis of bug fixing. Therefore, we need a common logical design across different physical environments. Steps: Ensure same h/w and s/w components are installed across all the environments. Create a configuration checklist. Get this checklist reviewed from QA as and when required. Use Version control or any change management/control process to manage the changes in the environment. Compare the version mapping at all times. If versions differ, deploy same versions across all environments.

Issues associated with not being able to replicate the production environment Reliability is a concern as real scenario might be quite different. Huge business and market risk. Handling huge volumes of data could be unexpected. There might be new integration issues in production environment while integrating with other 3 rd party software interfaces. Hardware performance may depreciate on production environment. There might be some real life scenarios that cannot be tested on any environment apart from production and thus may cause shocks. Some scenarios might occur more frequently in production environment than in testing environment. Low predictability and confidence among the staff and customers.

Issues in Testing with Real Production Data Data Confidentiality and data security issues. Data acquisition & integrity issues. Cannot be played around with as and when required. May not be able to satisfy all testing objectives. Cannot be controlled. May require special resources, e.g. DBA, better hardware. Issues in testing with non-production data Incomplete and incorrect data. It is impossible to create test data keeping all the test situations in mind. Non-production data maybe much cleaner than the production data resulting in a less robust system. There is no way to ensure completely that the non production data will test the system for real data too. The tester needs to first determine the expected results.

References Managing Multiple environments from Development to Production [ nfigs_11.htm] Effective methods for Software testing [William E. Perry]