Download presentation
Presentation is loading. Please wait.
Published byMelvin Evelyn Modified over 10 years ago
1
PREDICT Model for Test Automation
2
Does it sound familiar to you? Organization has procured test automation tools Management expectations are high Multiple test automation project initiatives started Enthusiasm, commitment and motivation declined with time Script maintenance had become a significant challenge Test automation tools are back on shelf
3
Testing and test automation requires same skills Capture/Replay is a successful test automation approach Test automation is an integral part of the over all test process Test automation is effective in solving regression testing issues Every test and test type can be automated Test automation development is software development Test automation projects can be delivered by personnel with strong project management skills Spot Quiz
4
Essential building blocks for successful test automation solutions Test automation solution approach
5
PREDICT Model Rationalize Estimate Design Implement Consolidate Transition Plan
6
PREDICT Model Structured methodology Multi stage process consisting of seven components Plan Rationalize Estimate Design Implement Consolidate Transition Step-wise approach Easy to apply
7
PREDICT Model - Plan Rationalize Transition Plan Estimate Design Implement Consolidate
8
Need for test automation project What do you consider to start a project ? Test related checks Objective Scope Documentation Test ware Organization related checks Investments Expectations Resource availability Technology related checks Stability of SUT Test tool suitability
9
Need for test automation project Are you getting bang for your buck ? Costs Licensing Framework Implementation Maintenance factor Benefits Saved manual effort per test run Number of test runs Re-usability factor Amortizing benefits Life cycle of the application/product Opportunities for test automation
10
Plan of approach Introduction Objective Pre-requisites Set-up pilot project Activities Planning Deliverables Organization Resources
11
PREDICT Model - Rationalize Rationalize Transition Plan Estimate Design Implement Consolidate
12
How do we rationalize requirements ? 5 P factors –Project –Process –Product –People –Price Test suite attributes Building trade-offs based on project context
13
Project Objective and scope Clear objectives in terms of time, money and quality Strategy Test automation strategy fitment with manual testing strategy Build Build frequency Configuration management Bad experiences User interface Stability Cycle time Regression possible
14
Process Quality documentation Manual test scripts, documentation Development approach Phases clearly defined, how early can test automation start Test practice New builds tested before delivering to QA Regression moments Naming conventions User Interface conventions
15
Product Stability Base platform stability Testability Degree of interaction between the tool and product under test Test tool integration Integration between test management tool and configuration management Test oracles Interaction with non UI components,database Automation framework Availability of frame work to reuse
16
People Organization readiness Management expectations Resource availability Machine, Licenses and people with right skills Non- programming resources Framework inclined towards non-programming resources QA population Ratio of development to QA Collaboration Collaboration between QA and Dev Technical skill set Test engineers with technical skills
17
Price Return on Investment Opportunities of reusing test automation investment Reusability of Investment Are there any products which are tested in a similar way Business Plan Is there a business plan to realize benefits of Test Automation
18
Test suite attributes Maintainability Reusability Reliability Usability Portability
19
Which is best test automation regime ? IT DEPENDS ON THE PROJECT CONTEXT
20
PREDICT Model - Estimate Rationalize Transition Plan Estimate Design Implement Consolidate
21
Test automation estimation challenges Test organization maturity Scope of requirements Framework Skill level Test case complexity Domain complexity Test tool Testability
22
Estimation techniques Delphi Gather estimates from each team member Ask high and low estimator to explain estimates Repeat twice, then use average Three point method Have team estimate best, worst case and expected case Use expected case Variances between best and worst useful to gauge estimate accuracy
23
Work Break down Structure
24
Risk categories Technical Test tool Application/product Communication
25
Risk management
26
PREDICT Model - Design Rationalize Transition Plan Estimate Design Implement Consolidate
27
Test suite design issues Testability Test suite design patterns
28
Testability considerations Tools Design Naming Conventions Exception & Error Handling Timing issues Logging Visual Cue Processes & Documentation
29
Record/Play back Functional de-composition Key-word Driven User Interface-driven Model-based Test automation development approaches
30
Advantages Useful in determining how the tool interacts with the application under test Provides initial ideas on how to develop test scripts Useful while Playing around with the tool Disadvantages Test scripts contains hard-coded values Un-reliable scripts : Any small change in the application results in not being able to replay Maintenance costs are astronomical Not viable & cost-effective Record & Playback
31
Reduces test scripts into their fundamental tasks Driver Scripts : Initialization, Calling scripts in a desired order Generic scripts : General,Application specific,Navigation,Data verification,Reporting,logging etc… Business Function Scripts : Re-usable Business functionality,Makes calls to Generic Scripts Test Script : Test Logic, Makes calls to Business Function Scripts and Generic Scripts Test data and logic are separated Hierarchical structure is employed : Modular design Each test script unique data file Functional decomposition
32
Test Director Reports Test Scripts Generic Library Business Library Test Scripts Environment Configuration Stored ProcObject Repository Technical Architecture
33
Functional decomposition: Analysis Advantages Modular design : Reduces redundancy and duplication of efforts Test data input & output stored in a single data file : Easy maintenance Scripts can be developed whilst application development is in progress : Generic Scripts Single point maintenance is possible using Business Function scripts Scripts can return TRUE (or) FALSE to the calling scripts. This makes it easy for Error Recovery & un-attended test execution Assemble tests on demand Disadvantages Proficiency in the TSL Complex test data file management
34
The test script is represented as a set of key-words in a spread sheet Examples of Key-words are start_test, Initial window,Url,Input,Verify etc.. Driver Script Performs the initialization as required Driver Script Reads and processes the file name(s) of the test scripts Driver Script Matches the Key Words contained in the file Driver Script invokes Business Function & Generic scripts Report the return values back The entire process is data-driven Key-word driven
35
Advantages Preserves most of the advantages of the Functional decomposition method Very similar to documenting manual test script Minimal programming knowledge is required to write and execute test scripts Return on investment can be achieved quicker than other methods in most of the cases Re-usability significantly increases over a period of time Test script development is possible with out a functioning application Disadvantages Investment in initial frame work Training might become tedious Key-word driven - Analysis
36
Generic Scripts Keyword Driven Test Cluster Business Function Script4 Reports Driver Script Business Function Script1 Business Function Script2 Business Function Script3 GUI Map Technical Architecture
37
Abstraction of test structure: Test set Test case Test step Test action Key-words are identified based on the application under test Test steps are created using user interface Test set management (Creation/execution) Test set Test case Test step Test action User interface driven
38
Automation Framework Recording Engine Script Generation Module APPLICATION User Interface Test Test Script XML Object Repository Object Creation Module Execution Engine Win Runner/QT P/Silk Test/… Library Structure Technical Architecture
39
Advantages Preserves most of the advantages of the previous methods Insulates tests both from the changes of the application and failings of test automation tools Zero-programming required Return-on-investment is very high Re-usability significantly increases over a period of time Robustness and maintainability is high Partly application independent Minimal learning curve Challenges High initial investment in creating framework Functioning application needed User interface, Model driven - Analysis
40
Test Case Generator (In key words) Keyword Translator engine (Test tool specific) Automated Test Scripts Test driver generator Behavior modeler Requirements (UML) Win Runner Silk Test Other test tools Model Driven
41
PREDICT Model - Implement Rationalize Transition Plan Estimate Design Implement Consolidate
42
Test automation implementation issues Team structure Work allocation Coding guide-lines and check lists Peer reviews Test runs Integration Configuration management
43
Work allocation
44
Coding guidelines and checklist
45
Reviews
46
Building integrations Test Script1 Test Script2 Test Script3 Call Test Script1() Call Test Script2() Call Test Script3()
47
Configuration management : VSS
48
Test execution Test suite execution check-lists are used Test Execution A separate test environment is prepared Fresh test data in test DB Fresh test data in Application DB Tests are executed Test suite execution issues & root cause analysis is documented Report Test Results Update test scripts if needed
49
Test execution log
50
PREDICT Model - Consolidate Rationalize Transition Plan Estimate Design Implement Consolidate
51
Update scripts and test data Complete documentation (User Manual, installation Guide and maintenance guide) Freeze & archive test suite Professionally packaged shippable test suite Easy distribution mechanism Activities in consolidation
52
User manual
53
Professional packaging
54
PREDICT Model - Transition Rationalize Transition Plan Estimate Design Implement Consolidate
55
Acceptance test runs performed Knowledge transfer to the user groups Gain feedback from the field Prepare for maintenance and sustenance Up-gradation plans Discussion for next project Transition process
56
Successful test automation projects require mature test automation leadership skills Test automation projects should be treated like a development project Understanding PREDICT Model Challenges, issues, tips and tricks in applying PREDICT model Wrap up
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.