1. Several definitions: “Testing is the process of establishing confidence that a program or system does what it is supposed to.” by Hetzel 1973 “Testing is the process of executing a program or system with the intent of finding errors.” by Myers 1979 “Testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results.” by Hetzel 1983 What is Software Testing 0 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

2. - One of very important software development phases - A software process based on well-defined software quality control and testing standards, testing methods, strategy, test criteria, and tools. - Engineers perform all types of software testing activities to perform a software test process. - The last quality checking point for software on its production line What is Software Testing 1 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

3. - Uncover as many as errors (or bugs) as possible in a given timeline. - Demonstrate a given software product matching its requirement specifications. - Validate the quality of a software testing using the minimum cost and efforts. - Generate high quality test cases, perform effective tests, and issue correct and helpful problem reports. Major goals: uncover the errors (defects) in the software, including errors in: - requirements from requirement analysis - design documented in design specifications - coding (implementation) - system resources and system environment - hardware problems and their interfaces to software Testing Objectives 2 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

4. Think of all the programs running on all the computer systems in the world now! How many of those programs do you rely on to perform correctly every day? When software fails, how would it affect you or the people you care about? 3 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

5. Some software failures Infamous Software Error Case Studies Disney’s Lion King, 1994 –1995 Intel Pentium Floating-Point Division Bug, 1994 NASA Mars Polar Lander, 1999 Patriot Missile Defense System, 1991 The Y2K (Year 2000) Bug, circa 1974 4 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

6. Disney’s Lion King, 1994 –1995: 1994, the Disney company released its first multimedia CD-ROM game for children “The Lion King Animated Storybook”. December 26, the day after Christmas, Disney’s customer support phones began to ring, and ring, and ring. Soon the phone support technicians were swamped with calls from angry parents with crying children who couldn’t get the software to work. Intel Pentium Floating-Point Division Bug, 1994: October 30, 1994, Dr. Thomas R. Nicely of Lynchburg (Virginia) College traced an unexpected result from one of his experiments to an incorrect answer by a division problem solved on his Pentium PC. 5 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

7. NASA Mars Polar Lander, 1999: On December 3, 1999, NASA’s Mars Polar Lander disappeared during its landing attempt on the Martian surface. A Failure Review Board investigated the failure and determined that the most likely reason for the malfunction was the unexpected setting of a single data bit. Patriot Missile Defense System, 1991 U.S. Patriot missile defense system is a scaled-back version of the Strategic Defense Initiative (“Star Wars”) program proposed by President Ronald Reagan. A small timing error in the system’s clock accumulated to the point that after 14hours, the tracking system was no longer accurate The Y2K (Year 2000) Bug, circa 1974: 1970s a computer programmer—let’s suppose his name was Dave—was working on a payroll system for his company One method he used was to shorten dates from their 4-digit format, such as 1973, to a 2-digit format, such as 73. Because his payroll program relied heavily on date processing, Dave could save lots of expensive memory space.Problems that might occur when the current year hit 2000 and his program began doing computations on years such as 00 and 01 6 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

8. 7 Y2K (1999) Cost: $500 billion Disaster: One man’s disaster is another man’s fortune, as demonstrated by the infamous Y2K bug. Businesses spent billions on programmers to fix a glitch in legacy software. While no significant computer failures occurred, preparation for the Y2K bug had a significant cost and time impact on all industries that use computer technology. Cause: To save computer storage space, legacy software often stored the year for dates as two digit numbers, such as “99″ for 1999. The software also interpreted “00″ to mean 1900 rather than 2000, so when the year 2000 came along, bugs would result. 7 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

9. Some other failures Case Histories WW III Almost Medical Killing Machine AT&T Line Goes Dead Ariana Rocket Goes Boom Mars Climate Crasher Lessons to Learn 8 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

10. Jerry Gao Ph.D.7/2002 Software Testing Process Unit test Integration test Validation test System test System engineering Requirements Software Design Code & Implementation V&V Targets 9 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

11. Unit Test (Component Level Test) Unit testing: Individual components are tested independently to ensure their quality. The focus is to uncover errors in design and implementation, including - data structure in a component - program logic and program structure in a component - component interface - functions and operations of a component Unit testers: developers of the components. Operations and Functions with I/O White-box interface input output Internal logic, data, structure outpu t input interface operation Black-box Jerry Gao Ph.D.7/2002 10 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

12. Jerry Gao Ph.D.7/2002 Integration Testing Integration test: A group of dependent components are tested together to ensure their the quality of their integration unit. The focus is to uncover errors in: - Design and construction of software architecture - Integrated functions or operations at sub-system level - Interfaces and interactions between them - Resource integration and/or environment integration Integration testers: either developers and/or test engineers. Component #1 Operations and Functions with I/O input interface operation Component #2 Operations and Functions with I/O outpu t interface operation 11 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

13. Jerry Gao Ph.D.7/2002 Function Validation Testing Validation test: The integrated software is tested based on requirements to ensure that we have a right product. The focus is to uncover errors in: - System input/output - System functions and information data - System interfaces with external parts - User interfaces - System behavior and performance Validation testers:test engineers in ITG or SQA people. System (Operations & Functions & Behavior) User interface UserExternal interfaces 12 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

14. Jerry Gao Ph.D.7/2002 System Testing System test: The system software is tested as a whole. It verifies all elements mesh properly to make sure that all system functions and performance are achieved in the target environment. The focus areas are: - System functions and performance - System reliability and recoverability (recovery test) - System installation (installation test) - System behavior in the special conditions (stress and load test) - System user operations (acceptance test/alpha test) - Hardware and software integration and collaboration - Integration of external software and the system System testers:test engineers in ITG or SQA people. When a system is to be marketed as a software product, a testing process called beta testing is often used. 13 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

15. What is a ”bug”? Error - a human action that produces an incorrect result Fault - a manifestation of an error in software also known as a defect, bug, issue, problem, anomaly, incident, variance, inconsistency, feature, … if executed, a fault may cause a failure Failure - deviation of the software from its expected delivery or service Failure is an event; fault is a state of the software, caused by an error. 14 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

16. 15 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

17. What is a fault? A software error is present when the program does not do what its end user reasonably expects it to do (Myers, 1979) There can never be an absolute definition for bugs, nor an absolute determination of their existence. The extent to which a program has bugs is measured by the extent to which it fails to be useful. This is a fundamentally human measure. (Beizer, 1984) 16 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

18. 17 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

19. V -model 18 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

20. V-model is easy to understand Simple generic model Easy to understand and explain Describes the test levels and phases Shows how testing activities are related to other development activities Matches to familiar waterfall model 19 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

21. V-model says… It is good to have requirements laid out first. It is good to have plans and specifications done before implementation. It is good to complete one task before proceeding to the next that depends on the output of the first task. It is good to test each intermediate product. It is good to use each development specification as a basis for test process. It is much easier to find faults in small units than in large entities. Therefore, test small units first alone before putting them together. Testing of large units can be carried out more easily when the smaller parts are already tested. 20 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

22. 21 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

23. 22 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

24. 23 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

25. 24 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

26. 25 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

27. Levels, types, and phases – not a same thing Test level – a test level is a group of test activities that focus into certain level of the test target. E.g. unit test, integration test, system test, acceptance test Test levels can be seen as levels of detail and abstraction Test type – A test type is a group of test activities aimed at evaluating a system for a number of associated quality characteristics. E.g. functional test, performance test, stress test,usability test Test phase – Test phases are used to describe temporal parts in testing process that follow sequentially each other, with or without overlapping 26 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

28. - Due to the testing time limit, it is impossible to achieve total confidence. - We can never be sure the specifications are 100% correct. - We can never be certain that a testing system (or tool) is correct. - No testing tools can copy with every software program. - Tester engineers never be sure that they completely understand a software product. - We never have enough resources to perform software testing. - We can never be certain that we achieve 100% adequate software testing. Software Testing Limits 27 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

29. Limitations of testing It is impossible to test a program completely Testing can show the presence of defects but cannot show the absence of defects You can report found defects and what you have tested and how All bugs cannot be found All found bugs will not be fixed It’s really not a bug It’s too risky to fix, there is not enough time There is more important tasks to do Testing does not create quality software or remove defects The critique is focused on the product, not the developer 28 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

30. Test Issues in Real World Software testing is very expensive. How to achieve test automation????? When should we stop software testing? Test criteria, test coverage, adequate testing. Other software testing: GUI Testing Object-Oriented Software Testing Component Testing and Component-based Software Testing Domain-specific Feature Testing Testing Web-based Systems Jerry Gao Ph.D.7/2002 29 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

31. SDLC Model A framework that describes the activities performed at each stage of a software development project. 30 DACOE,CSE Prepared by:Prof. A.N.Mandale 9/20/2018

32. V-Shaped SDLC Model A variant of the Waterfall that emphasizes the verification and validation of the product. Testing of the product is planned in parallel with a corresponding phase of development 31 DACOE,CSE Prepared by:Prof. A.N.Mandale9/20/2018