Software Self-Testing

Slides:



Advertisements
Similar presentations
Operating Systems Components of OS
Advertisements

Configuration management
1 A Test Automation Tool For Java Applets Testing of Web Applications TATJA Program Demonstration Conclusions By Matthew Xuereb.
Using the Self Service BMC Helpdesk
Automating with Open Source Testing Tools Corey McGarrahan rSmart 01-July-08.
Test Automation An Approach to Automated Software Regression Testing Presented by Adnet, Inc Feb 2015.
Test Automation Framework Ashesh Jain 2007EE50403 Manager Amit Maheshwari.
Automation Testing Presentation Phil Hunter Phil Hunter - Automation Presentation 1.
SOFTWARE PRESENTATION ODMS (OPEN SOURCE DOCUMENT MANAGEMENT SYSTEM)
Documentation Testing
Presentation Outline  Project Aims  Introduction of Digital Video Library  Introduction of Our Work  Considerations and Approach  Design and Implementation.
15 Chapter 15 Web Database Development Database Systems: Design, Implementation, and Management, Fifth Edition, Rob and Coronel.
Introduction to z/OS Basics © 2006 IBM Corporation Chapter 8: Designing and developing applications for z/OS.
Tutorial 8 Sharing, Integrating and Analyzing Data
Chapter 7 Managing Data Sources. ASP.NET 2.0, Third Edition2.
An ide for teaching and learning prolog
Introduction to the new mainframe © Copyright IBM Corp., All rights reserved. Chapter 7: Designing and developing applications for z/OS.
© 2004 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice Automation Fundamental Concepts &
Introduction and simple using of Oracle Logistics Information System Yaxian Yao
Manage Engine: Q Engine. What is it?  Tool developed by Manage Engine that allows one to test web applications using a variety of different tests to.
Software Quality Assurance QA Engineering, Testing, Bug Tracking, Test Automation Software University Technical Trainers SoftUni Team.
COMPUTER SOFTWARE Section 2 “System Software: Computer System Management ” CHAPTER 4 Lecture-6/ T. Nouf Almujally 1.
An Approach to Test Autonomic Containers Ronald Stevens (IEEE Computer Society & ACM Student Member) August 1, 2006 REU Sponsored by NSF.
What is QTP ► QTP stands QuickTest Professional ► It is an automated testing tool provided by HP/Mercury Interactive ► QTP integrates with other Mercury.
Microsoft Access Lecture -13- By lec. (Eng.) Hind Basil University of Technology Department of Materials Engineering 1.
4 - 1 Copyright © 2006, The McGraw-Hill Companies, Inc. All rights reserved.
Capture and Replay Often used for regression test development –Tool used to capture interactions with the system under test. –Inputs must be captured;
1Copyright © 2011 Pearson Education, Inc. Publishing as Prentice Hall. Exploring Microsoft Access 2010 by Robert Grauer, Keith Mast, Mary Anne Poatsy Chapter.
DBSQL 14-1 Copyright © Genetic Computer School 2009 Chapter 14 Microsoft SQL Server.
Informix IDS Administration with the New Server Studio 4.0 By Lester Knutsen My experience with the beta of Server Studio and the new Informix database.
Configuration Management (CM)
SOFTWARE TESTING Scope of Testing  The dynamic Indian IT industry has always lured the brightest minds with challenging career.

Event Management & ITIL V3
FotoGazmic Software (From left to right: Chad Zbinden, Josey Baker, Rob Mills, Myra Bergman, Tinate Dejtiranukul)
1 What’s Next for Financial Management Line of Business (FMLoB)? AGA/GWSCPA 6 th Annual Conference Dianne Copeland, Director, FSIO May 8, 2007.
CHAPTER TEN AUTHORING.
Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.
CS 390 Unix Programming Summer Unix Programming - CS 3902 Course Details Online Information Please check.
Software Research, Inc. Setting the Standard for Software Testing Windows Solution.
DB2 Universal Database Confidential | July 2012 | India Software Lab Click to add text © 2012 IBM Corporation An End to End Windows Automation Framework.
MSE Presentation 1 By Padmaja Havaldar- Graduate Student Under the guidance of Dr. Daniel Andresen – Major Advisor Dr. Scott Deloach-Committee Member Dr.
M4 Alpha Release August 9 th February 8th, 2006 M4 Alpha Release Modified Agenda Review M4 Objectives, General Concepts and Design Review M4 Development.
What is WinRunner ► WinRunner is Mercury’s legacy automated testing tool ► It is similar to QTP in functionality ► WinRunner integrates with other Mercury.
P6 BTEC Level 3 Subsidiary Diploma in ICT. Automation The end user of a spreadsheet may be proficient in using the software, but the more that you automate.
Access Chapter 1: Intro to Access Objectives Navigate among objects in Access database Difference between working in storage and memory Good database file.
Software Engineering1  Verification: The software should conform to its specification  Validation: The software should do what the user really requires.
July, 2008 Impati – Software Test Solutions. July, Contents Testing Service Overview and Approach Test Services and Industries Key Services Offering.
Survey of Tools to Support Safe Adaptation with Validation Alain Esteva-Ramirez School of Computing and Information Sciences Florida International University.
Software Quality Assurance and Testing Fazal Rehman Shamil.
Unit 17: SDLC. Systems Development Life Cycle Five Major Phases Plus Documentation throughout Plus Evaluation…
Chapter 5 Introduction To Form Builder. Lesson C Objectives  Use sequences to automatically generate primary key values in a form  Create lists of values.
T EST T OOLS U NIT VI This unit contains the overview of the test tools. Also prerequisites for applying these tools, tools selection and implementation.
By SPEC INDIA. TestComplete – An Introduction An automated testing tool by SmartBear – TestComplete is a comprehensive product with umpteen features and.
5/29/2001Y. D. Wu & M. Liu1 Content Management for Digital Library May 29, 2001.
 Software reliability is the probability that software will work properly in a specified environment and for a given amount of time. Using the following.
Definition CASE tools are software systems that are intended to provide automated support for routine activities in the software process such as editing.
Software Quality Assurance
Chapter 13 & 14 Software Testing Strategies and Techniques
Applied Software Implementation & Testing
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 2 Database System Concepts and Architecture.
Introduction to Software Testing
Achieve Next level of Automation with Robotic Test Automation
Using JDeveloper.
Testing RESTful Web APIs
Course: Module: Lesson # & Name Instructional Material 1 of 32 Lesson Delivery Mode: Lesson Duration: Document Name: 1. Professional Diploma in ERP Systems.
Data validation in Statistical Office of the Republic of Serbia
Overview Activities from additional UP disciplines are needed to bring a system into being Implementation Testing Deployment Configuration and change management.
Best Practices in Higher Education Student Data Warehousing Forum
Presentation transcript:

Software Self-Testing This work has been supported by the European Social Fund within the project «Support for Doctoral Studies at University of Latvia». Software Self-Testing Edgars Diebelis Prof. Dr. sc. comp. Jānis Bičevskis

Self-Testing Ability to execute stored test examples to ensure that software functionality is correct. Testing approach Manual testing; Automated testing with testing tools; Self-Testing.

Content Problem statement. Concept of Self-Testing. Implementation of Self-Testing. Comparison of the Self-Testing Concept with Conventional Testing Support Tools. Efficiency Measurements of Self-testing. Conclusions.

Problem Statement „Computing systems’ complexity appears to be approaching the limits of human capability” (Kephart, J., O., Chess, D., M.). IBM Autonomic Computing Manifest, 2001 outlined four key features that characterise autonomic computing. Smart Technology approach, 2007 identified seven types of the smart technology; self-testing is one of the type of the smart technology.

Concept of Self-Testing

Self-Testing Functionality Self-testing contains two components: Test cases of system’s critical functionality to check functions, which are substantial for system using; Built-in mechanism (software component) for automated software testing (regression testing) that provides automated running of test cases and comparing test results with standard values.

Test storage file (XML) Test storage mode Test storage file (XML) 1. 2. 3.

Test storage file (XML) Self-Testing mode Test storage file (XML) Self-Testing tool 1. 2. 4. 6. 3. 5.

Test Points Test point is a programming language command in the software text. Test point ensures that: particular actions and field values are saved when storing tests; the software execution outcome is registered when tests are executed repeatedly. By using test points, it is possible to repeat the execution of system events.

Test Point Types Test point types: input field test point; comparable value test point; system message test point; SQL query result test point; application event test point; test execution criteria test point; self-testing test points; etc. 10 10

Implementation of Self-Testing

Build of Self-Testing Self-Testing tool modules: Self-Testing module; Self-testing test management module. 12 12

Self-Testing in use

Comparison of the Self-Testing Concept with Conventional Testing Support Tools

Testing Tools Selection Automated Testing Institute (ATI) opinion. “ATI Automation Honors ” awards. Since May 2009, the ATI has been publishing its magazine Automate Software Testing. Website has a list of 716 automated testing tools. ATI annual conference on automated testing (Verify/ATI). ~8000 registred users.

Testing Tools TestComplete FitNesse Ranorex T-Plan Robot the best commercial automated functional testing tool in 2010 FitNesse the best open source code automated functional testing tool in the NET sub- category in 2010 Ranorex the best commercial automated functional testing tool in the NET and Flash/Flex sub-categories in 2010 T-Plan Robot the best open source code automated functional testing tool in the Java sub- category in 2010 Rational Functional Tester in 2009 and 2010, it was a finalist among the best commercial automated functional and performance testing tools HP Unified Functional Testing Software the best commercial automated functional testing tool in 2009 Selenium the best open source code automated functional testing tool in 2009 and 2010

Criteria for Comparison Testing method (TM) Test automation approach (TAA) Test automation framework (TAF) Testing level Test recording and playback Desktop applications testing Web applications testing Services testing Database testing Testing in production environment System user can create tests Simultaneous running of several tests Performing simultaneous actions Identifying the tested object Test result analysis Test editing Screenshots Control points Object validation Object browser Test log Test schedule planner Identification of the end of command execution Plug-ins and extensions etc.

Comparison

Comparison II 20 20

Self-Testing further development New Test automation frameworks. Test editor and log. Object validation and object browser. Load, stress and other testing levels. Web applications and services testing. Additional platforms. Plug-ins and extensions. Integration with external environment testing.

Self-Testing advantages White box testing. Testing in production environment. Users without in-depth IT knowledge to define and run test cases. Testing external interfaces. Perform system testing without specific preparation for running the test.

Efficiency Measurements of Self-testing

Efficiency Measurements Retrospective analysis of incident notifications in a real project. It is not possible to apply and compare two different concepts in the same conditions. Analysis of incident notifications (1,171 in total) in the CSAS in the period from July 2003 to 23 August 2011. Subjective opinion; however, the high number of incident notifications and the statistics do reflect trends. 24 24

Statistics of Incident Notifications Type of Incident Quantity % of total Hours Duplicate 68 5.81 23.16 0.47 User error 43 3.67 67.46 1.37 Unidentifiable bug 178 15.2 1011.96 20.52 Identifiable bug 736 62.85 3293.74 66.79 Improvement 102 8.71 241.36 4.89 Consultation 44 3.76 293.92 5.96 Total: 1171 100 4931.6 25 25

Bugs Unidentifiable by the Self-testing Bug type Quantity % of total External interface bug 5 2.81 Computer configuration bug 12 6.74 Data type bug 7 3.93 User interface bug 25 14.04 Simultaneous actions by users Requirement interpretation bug 41 23.03 Specific event 83 46.63 Total: 178 100 26 26

Bugs Identifiable by the Self-testing Test point Quantity % of total Hours File result test point 59 8.02 150.03 4.56 Input field test point 146 19.84 827.14 25.11 Application event test point 105 14.27 364.24 11.06 Comparable value test point 28 3.8 93.53 2.84 System message test point 11 1.49 58.84 1.79 SQL query result test point 387 52.58 1799.96 54.65 Kopā: 736 100 3293.74 27 27

Conclusions

Conclusion self-testing approach offers not only options equal to those offered by other globally recognized testing support tools; moreover, self-testing additionally offers options that other testing tools do not possess: testing external interfaces; testing in production environment; testing with the white-box method; possibility for users without IT knowledge to capture tests. Testing support is part of systems developed and it is available throughout the entire life cycle of software.

Conclusion II Self-testing changes the testing process by considerably broaden the role of the developer in software testing. Self-testing requires additional work to include the self-testing functionality in the software and to develop critical functionality tests. Self-testing saves time by repeated (regression) testing of the existing functionality. Implementation of the self-testing functionality is useful in incremental development models, in particular in systems that are gradually improved and maintained for many years.

Thank You!