1. Trend Quality Assurance Edward Tsai 蔡木本 趨勢科技全球軟體研發部協理 April 22, 2003

2. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Contents  Software quality assurance overview  Testing skill and debug tools  Testing concepts.  Testing methodology.  Testing tools.  The importance of QA Automation  Software testing in enterprise environment

3. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Overview  What is the quality?  What is QA role in Trend?  How many different type of testing strategy?  What is the typical software life-cycle?  What is the QA challenge and future?  What is Trend QA organization?

4. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line What is the quality?  2 level of quality:  Level 1: product defect rate and reliability.  Level 2: product quality, process quality, customer satisfaction.  Quality from professional views:  Conformance to requirement.  Fitness for use.  For example:  Cadillac and Chevrolet car, Which has good quality?

5. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line What is QA role in Trend?  The purpose of QA engineers:  To discover all possible errors in development cycle.  To discover every conceivable fault or weakness in product.  To simulate customer’s focus and need.  To measure the quality level of product.  To delivers the quality report to service team.

6. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line How many different type testing strategy?  Black-Box testing:  Derived from the functional design spec, without touching the internal program structure.  White-box (or Glass-box) testing:  Require knowledge of the internal program structure and derived from the internal design spec or code.  Trendmicro:  Black-box testing and will migrate to Gray-box testing.

7. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line What is the typical software life-cycle?  Concept.  Requirement.  Design.  Implementation or coding.  Test.  Operation and maintenance.

8. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line DOT-U model 1 2 3 4 5 6 7 8 9 10 11 Requirements specification Requirements verification Functional design specification Functional Design verification Internal design specification Internal design verification Products simulation Usability test Code and spec modification System and Acceptance validation Function validation Code and spec modification Code and spec modification Code and spec modification Code Code verificationUnit validationIntegration validation 1

9. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line What is Trendmicro software life-cycle?  Requirement.  High-Level design/Planning.  Detailed design.  Construction.  Pre-release.  Closure.  Service.

10. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line What is QA challenge/future?  Professional discipline requiring trained, skilled people.  Advances in technology/tools to create complex software in GUI, Client-Server, Intranet/Internet architecture.  Multi-platform, multi-language.  Many/complex compatibility software testing.  Keep on process improvement.  Test automation is not mature yet.

11. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Defect Distribution 

12. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line The cost of defect  Cost Time AlphaBetaReleaseRelease

13. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Tester-to-Developer Ratios  The lesson:  The importance of testing is now recognized.  Vendors are willing to pay for proper testing.  A dramatic shift:  Historically, for Mainframes : 1: 5-10.  At Microsoft, 1992 : 2:3.  At Microsoft, 1995 : 1:1.  At Microsoft, 2000 : 2:1.  At Microsoft, 2005 : ?  TrendMicro, 2000 : 1 : 2  TrendMicro, 2003 : 1 : 1

14. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line What is the Trend QA organization?

15. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Testing skill and debug tools  Testing methodology.  Test case design.  Testing tools.

16. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Testing methodology  Verification testing:  Reference to Dotted-U Model.  Verification methods : Inspections, walkthroughs, Technical reviews, others.  The verification tool : Checklists.  Include: Requirement, Architecture, Plans, Design Spec, code, Documentation.  Validation testing:  A process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirement.  Include: Unit testing, integration testing, Usability testing, Function testing, Regression testing, System testing, Acceptance testing (Alpha, Beta testing).

17. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Testing case design  Go through all testing synopsis at every checking point.  Task-Oriented Functional Test (TOFT)  Forced-Error Test (FET)  Boundary test  Volume Test  Stress Test  Performance Test  Security Test.  Configuration Compatibility Test  Documentation Test  Trend Standards in Product

18. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Test Coverage  Test coverage has three components:  Requirements coverage.  Function coverage.  Logic coverage.

19. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Example

20. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Example  Input a Directory name to install a product in NT system.  Task-Oriented Functional Test (TOFT) Input a legal directory name with length between 1 to 255 character.  Forced-Error Test (FET) Read privilege only directory, Non-existed drive, more than 255 characters, Not enough disk space.  Boundary test Input a space or null character, Input one character, Input 255 characters, Input 266 characters. Memory leak after install complete.  Volume Test Install it with 30 MB free disk space.  Stress Test Install it with CPU utilization more than 70%, Free visual Memory left less than 1 MB,...  Performance Test Memory usage, CPU utilization, Any network traffic generated,...

21. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Example  Security Test. Does it can be installed from every users or Administrator only.  Configuration Compatibility Test N/A  Documentation Test The description are properly and correctly  Trend Standards in Product Does it install under c:\programs files\TrendMicro\PC-cillin 98?

22. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Testing tools  Testing documents database:  Test manager, TCMT.  Bugs tracker system:  PVCS Tracker 7.0.  Automation tools:  Visual test 4.0, Robot J, ATP. Winrunner.  Source code Coverage:  PureCoverage.  System tools:  Loadrunner, Load test, Winbench.  Debug tools:  Boundchecker, Quantity, Purify, SoftIce.

23. RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line Q & A