The Importance of Data in Functional Testing

Slides:



Advertisements
Similar presentations
Configuration management
Advertisements

Designing Reusable Frameworks for Test Automation
Test process essentials Riitta Viitamäki,
Programming Types of Testing.
Software Quality Assurance Inspection by Ross Simmerman Software developers follow a method of software quality assurance and try to eliminate bugs prior.
Modified from Sommerville’s originalsSoftware Engineering, 7th edition. Chapter 21 Slide 1 Software evolution.
Software evolution.
Chapter 9 – Software Evolution and Maintenance
Software Testing Verification and validation planning Software inspections Software Inspection vs. Testing Automated static analysis Cleanroom software.
Software evolution. Objectives l To explain why change is inevitable if software systems are to remain useful l To discuss software maintenance and maintenance.
MCTS Guide to Microsoft Windows Vista Chapter 4 Managing Disks.
Project monitoring and Control
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 22 Slide 1 Software Verification, Validation and Testing.
Construction, Testing, Documentation, and Installation Chapters 15 and 16 Info 361: Systems Analysis and Design.
TESTING LEVELS Unit Testing Integration Testing System Testing Acceptance Testing.
Software Testing and Quality Assurance Practical Considerations (4) 1.
Chapter 1: Fundamental of Testing Systems Testing & Evaluation (MNN1063)
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 21 Slide 1 Software evolution.
1 Object-Oriented Analysis and Design with the Unified Process Figure 13-1 Implementation discipline activities.
Test Automation Steffen Goerlitz Barry Lange Mitchell Meerman Harry Schultz Trevor Spees.
Software Testing History, Trends, Perspectives – a Brief Overview.
Software Development Module Code: CST 240 Chapter 6: Software Maintenance Al Khawarizmi International College, AL AIN, U.A.E Lecturer: Karamath Ateeq.
Laurea Triennale in Informatica – Corso di Ingegneria del Software I – A.A. 2006/2007 Andrea Polini XVII. Verification and Validation.
Tutorial 4 IT323.  Q1. As a software project manager in a company that specializes in the development of software for the offshore oil industry, you.
Better Data: Better Testing
Information Systems Development
Methodologies and Algorithms
Configuration Management
Software Configuration Management
Software Configuration Management
Software Project Configuration Management
Gmail Helpline Numberfix Gmail.
CSC 480 Software Engineering
Cisco Data Virtualization
Where Agile Business Meets Agile Development
THIS IS TO EVIDENCE YOUR WORK AND GET THE BEST GRADE POSSIBLE
Chapter 18 Maintaining Information Systems
Configuration Management
Chapter 8 – Software Testing
The Systems Engineering Context
Chapter 6: Design of Expert Systems
Systems Analysis and Design: What is it?
Software Engineering (CSI 321)
Information Systems Development
Completing the tasks for A452 with….
Engineering Processes
Introduction to Software Testing
FOUNDATIONAL CONCEPTS
Testing and Test-Driven Development CSC 4700 Software Engineering
Software Testing and Maintenance Maintenance and Evolution Overview
The Automation of the U.S. Budget Appendix Volume
Regulatory Oversight of HOF in Finland
Chapter 9 – Software Evolution and Maintenance
 Multi-Currency Wallet  Cross-Platform Pairing  Internal Cryptocurrency Exchange User-friendliness  QR Scanning  No Fees.
Chapter 13 Quality Management
Course: Module: Lesson # & Name Instructional Material 1 of 32 Lesson Delivery Mode: Lesson Duration: Document Name: 1. Professional Diploma in ERP Systems.
Internal QA in Open Source Development
JOINED AT THE HIP: DEVSECOPS AND CLOUD-BASED ASSETS
CS310 Software Engineering Dr.Doaa Sami Khafaga
Project Management How to access the power of projects!
Chapter 8 Software Evolution.
Bringing more value out of automation testing
Lecture 06:Software Maintenance
20. The Software-Quality Landscape
Zach Wahl and Tatiana Baquero Project Performance Corporation (PPC)
Ensuring the data you have collected is properly verified is vital
that focus first on areas of risk.
Chapter 7 Software Testing.
Software Testing Strategies
Presentation transcript:

The Importance of Data in Functional Testing James Lyndsay Workroom Productions

Data is important to functional testing. A system is programmed by its data. Functional testing suffers if data is poor. Good data is vital to reliable test results. Good data can help keep testing on schedule.

Problems caused by poor data The following problems can be caused by poor data; Unreliable test results Degradation of test data over time Increased test maintenance budget Reduced flexibility in test execution Obscure results and bug reports Larger proportion of problems can be traced to poor data Less time spent hunting bugs Confusion between developers, testers and business Requirements problems can be hidden in inadequate data Simpler to make test mistakes Unwieldy volumes of data Business data not representatively tested Inability to spot data corruption caused by bugs Poor database integrity

1) Recognising types of data 2) Organising the data Topics 1) Recognising types of data 2) Organising the data 3) Involving the Business 4) Data Load and Maintenance 5) Testing the data

Types of Data Environmental data tells the system about its technical environment. Setup data tells the system about the business rules. Input data information input by day-to-day system functions. Fixed input data is available at the start of the test. Consumable input data forms the test input.

1) Permutations 2) Partitioning 3) Clarity Organising the Data Good data increases data reliability, reduces data maintenance time and can help improve the test process. Good data assists testing, rather than hinders it. Good data is based on; 1) Permutations 2) Partitioning 3) Clarity

Data Permutations 1 Permutations are familiar from test planning Utility Customer Care / Billing Example: A customer can have one of three products (1,2,3). They may be billed Monthly or Quarterly, be High or Low value, and their last bill was either Paid or Unpaid. There are 3x2x2x2 = 24 combinations - and so the number of possible permutations climbs rapidly as system complexity increases. By requiring that the list holds not all possible combinations, but all possible pairs, the list can be reduced. All possible pairs; M1, M2, M3. Q1, Q2, Q3. H1, H2, H3. L1, L2, L3. P1, P2, P3. U1, U2, U3. MH, ML, QH, QL, MP, MU, QP, QU. HP, LP, HU, LU. The following six permutations contain all the pairs; Customer Account Product Care Bill 1 M 1 H P 2 M 2 L P 3 M 3 H U 4 Q 1 L U 5 Q 2 H U 6 Q 3 L P

Data Permutations 2 Permutation is appropriate when: Fixed input data consists of many rows Fields are independent You want to do many tests without loading / you do not load fixed input data for each test. Permutation helps because: Achieves good test coverage without having to construct massive datasets Can perform investigative testing without having to set up more data Can be used to test other data - particularly setup data Permutation is familiar from test planning. Reduces the impact of functional/database changes

Partitioned data Data load/reload can be inconvenient. Data can be partitioned into: 1) Safe area Used for enquiry tests, usability tests etc. No test changes the data, the area can be trusted. Many testers can use simultaneously 2) Change area Used for tests which update/change data. Data must be reset or reloaded after testing. Used by one tester at a time. 3) Scratch area Used for investigative update tests and those which have unusual requirements. Existing data cannot be trusted. Used at own risk! Data can be partitioned by machine / database / instance. Can also be partitioned by disciplined use of text / value fields.

Clarity Developers and the Business don’t need to understand test data / data requirements - so some of them won’t. We can make our data clearer by using available free text fields; Customer Name Account Product Care Bill HP1 Monthly M 1 H P LP2 Monthly M 2 L P HU3 Monthly M 3 H U LU1 Quarterly Q 1 L U HU2 Quarterly Q 2 H U LP3 Quarterly Q 3 L P Clarity helps because: Improves communication within and outside the team Reduces test errors caused by using the wrong data Helps when checking data after input Helps in selecting data for investigative tests

Data and the Business ‘The Business’ is good at looking at data; Easier to understand than tests Can be compared with existing systems Advantages Helps focus when requirements are vague Helps UAT Increases trust and understanding Helps early user identification of problems Disadvantages Data creep Vague requirements can lead to vague data Incomplete data can lead to incomplete testing

Using the system you're trying to test Ways of loading data Using the system you're trying to test Manually entered Automated tool, making keystrokes into application Using a data load tool All new data, created for testing Old data, selected for testing / filtered and migrated Complete set, migrated and loaded, identical but for personal details Not loaded Already set up for testing / Left in the system You are working on the live system

Frequency of data load At the start of testing With each release First thing Monday Whenever I want Before every test NB: Straw poll of ‘When does the data usually get loaded’ also came back with the following answers; Before I ever got involved The developers left it there The last testers / tests left it Whenever we get enough time After we've found out what shape the database is When we know what it means

Data Maintenance When and why? Replacing consumed data Repairing broken data Responding to change - database schema, code, requirements New test requirements Problems: Sizeable task - can be a substantial fraction of overall test maintenance Prone to error Performed by more than one group

Solutions to Loading and Maintenance headaches 1) Automate data load and maintenance where possible 2) Control / measure data change 3) Recognise and prepare for problems 4) Use good data

TEST THE DATA Testing the data Systems are programmed by their data . . . If the functionality of your system is at all affected by the setup data; TEST THE DATA 1) Does the current setup data affect the application appropriately? 2) Will changes made to the setup data have the desired effect? To incorporate into functional testing, you need; organised data good load/maintenance business involvement

Testing the data 2 Use a known dataset to: Show functionality correctly applied Assess effect of changes in setup data Aims: Improve overall testing Help get the data right before live operation Increased confidence in re-configuration Help pinpoint bugs in live operation

Non-functional testing Data verification Further data issues Operational Profiles Non-functional testing Data verification

1) Recognising types of data Topics: conclusion 1) Recognising types of data Environmental Data Setup Data Input Data - Fixed and Consumable 2) Avoiding common problems Recognise the problems Use good data Involve the Business Automate loading and maintenance Test your data

Plan the data for maintenance and flexibility Summary Plan the data for maintenance and flexibility Know your data, and make its structure and content transparent Use the data to improve understanding throughout testing and the business Test setup data as you would test functionality James Lyndsay Workroom Productions www.workroom-productions.com jdl@workroom-productions.com