Download presentation
Presentation is loading. Please wait.
Published byJordon Overland Modified over 9 years ago
2
FreeNest Brand Basic Material for testing course
3
Testing Course Agenda Learn basic concepts to under stand testing process and work Requirement engineering and management Agile methods and testing Test design & execution Test Planning & Management Test Automation Continous Integration & Testing Understand how and why testing is important
4
-This material if for general training for Test Design and management -Material is more supportive in class room -Material will be updated during courses -FreeNest Portable Project Platform is used to demonstrate things only in practice. This is not limiting usage for material for other training environments (I hope )
5
ISTQB http://www.rbcs-us.com/images/documents/The-ISTQB- Advanced-Syllabus.pdf
6
Verification & Validation Validation = Are we building the right product? Verification = Are we building the product right?
7
Requirement Engineering We take branding seriously, The FreeNest Brand is our pride and joy. It is the message we want to send to the developer teams, customers and stakeholders. These guidelines must be followed in all FreeNest’s print and electronic communications, marketing and informing. Documentation defines logo guidelines, typography and the use of colors.
8
System Requirements?
9
Different points of view on the product Seller Customer Developer Hacker
10
Different levels of requirements Customer Business Stakeholder System Design Components Implementation
11
Requirements in different levels http://www.sysml.org/ UM L OM T System Engineering Google: SysML, UML, Systems engineering, Software Design, Code Software Engineering - Architecture SysML.NET, JAVA, C++ Software Engineering - Design http://www.sysml.org/ UML
12
What is feature? Feature X * n Calory Counter: Player can measure calories during training session. This can be seen as exercise result in web service eg. Facebook application Energy usage
13
Is product a combinatio of features? Calory Counter Drum Metronome Table Drum Mode Standby Mode MIDI Support Touch Screen with single tap
14
Is product a combination of features Core Software Calory Counter Drum Metronome Table Drum Mode Standby Mode MIDI Support Touch Screen with single tap
15
Customer/Business Requirements Calory Counter Drum Metronome Table Drum Mode Simple Training Mode MIDI Support Touch Screen with single tap Customer Type 1 Customer Type 2 Customer Type 3Customer Type 4 Who are our target customers?
16
Customer strategy Drum Metronome Table Drum Mode Simple Training Mode MIDI Support Touch Screen with single tap Customer Type 1 Customer Type 2 Customer Type 3Customer Type 4 What is our key customer? Primary Target Calory Counter Secondary Target
17
Features and roadmap Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1User Story #1 User Story #2 User Story #3 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1 User Story #1 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1User Story #1 User Story #2 User Story #3 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1 User Story #1 User Story #2 User Story #3 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1 Requirement USE CASE #2 USE CASE #1 Requirement USE CASE #1 Release 0.1 Release 1.2 Feature: Simple Training Mode Feature: Table Drum mode Feature Touch Screen with single tap Release 1.0 TIME TO MARKET!! For Target Group 3 CORE/Platform Software Development TIME TO MARKET!! For Target Group 2 TIME TO MARKET!! For Target Group 1 Release 1.1
18
Release plan for large system Unstable Released Edge Stable Released Day 1 Day 2 Day … Quartal Release Quartal Release Monthly Release Monthly Release 6 month Release 6 month Release Day 1 Day 2 Day … Day 1 Day 2 Day … Monthly Release Monthly Release Monthly Release Monthly Release Fix Released Unit & Testing Integration Testing System Acceptance Testing
19
SW Project vs Open Source Open Source – Crowd Sourcing SW Relase tested without coordination by group of volunteers Release tested by customer Field Testing Test Group
20
Discussion Do some study about Ubuntu release testing process?
21
V-Model for testing Customer Requirements System Requirements Sub System Requirements Component RequirementsUnit Testing Integration Testing System Testing Acceptance Testing
22
Developer vs Tester??? Customer Requirements System Requirements Sub System Requirements Component RequirementsUnit Testing Integration Testing System Testing Acceptance Testing
23
Verification & Validation FEATURES Use Cases User Story Requirement Validation Verification Validation = Are we building the right product? Verification = Are we building the product right? Validation VS verification?
24
Testing in brief? Test Case FEATURE 1 deliver ready for test ? Nope! Desing & implementation Acceptance Testing FEATURE 2 FEATURE 3 FEATURE 4
25
Testing Orientation Black Box Testing Grey Box Testing White Box Testing ”System Testing”- perspective ”Code level testing”- perspective http://en.wikipedia.org/wiki/Software_testing
26
Testing Orientation Black Box Testing White Box Testing ”System Testing”- perspective ”Code level testing”- perspective http://en.wikipedia.org/wiki/Software_testing
27
Development Process (Waterfall) Requirement Gathering Design Implementation Verification Maintenance Validation Milestone 1Milestone 2Milestone 3 Milestone 4 Milestone 5
28
Development Process (Agile) User Story Y Sprint Task1 Task2 Task3 Sprint Task4 Task5 Task6 User Story X Design Implementation Verification Design Implementation Verification Product Backlog Design Implementation Verification Design Implementation Verification Design Implementation Verification Design Implementation Verification Design Implementation Verification Design Implementation Verification User Story Z Task7 Task8 Task9 User Story Z
29
Testing levels We take branding seriously, The FreeNest Brand is our pride and joy. It is the message we want to send to the developer teams, customers and stakeholders. These guidelines must be followed in all FreeNest’s print and electronic communications, marketing and informing. Documentation defines logo guidelines, typography and the use of colors.
30
How to test? What should be tested? How?
31
How to test? What should be tested? How ?
32
Verification of implementation Unit Testing Static Analyze Dynamic Analyze
33
What should be tested? Input ? Output?
34
Class model ? http://www.ibm.com/developerworks/rational/library /content/RationalEdge/sep04/bell/
35
xUnit Framework Implementation xUnit Framework
36
Component/ Unit Testing Developer Test Class Attributes TestMethodCall Class Attributes Methods Test Method Call Method Call result xUnit Test Framework MethodCountValues( int x,int y) z = x+y+1 Return z a=1; b= 2 C=Class.TestCountValues(1,2) C <> 3 = Fail C = 3 PASS
37
Test Driven Development? DEFINE TEST CASE FIRST!!! DEFINE TEST CASE FIRST!!! IMPLEMENT CODE AGAINST TESTS IMPLEMENT CODE AGAINST TESTS
38
Ideal project team and unit testing Integration Team Implemented Software Component #1 Implemented Software Component #1 Implemented Software Component #2 Implemented Software Component #2 Implemented Software Component #3 Implemented Software Component #3 Implemented Software Component #4 Implemented Software Component #4 TESTS Developer #1 Developer #2 Developer #3 Developer #4 Integration test engineer Component Sandbox
39
Integration Testing
40
How to test ? What should be tested? How ? http://prosentti.vero.fi/veropros_tietojen_syotto2011.asp
41
Other Services Data Base Integrate early! Avoid Big Bang Web Service Hardware Mobile Application 10 % 50 % 90 % 99% 10 % 99%+90%+10%+50%+10% =?
42
Integration Testing with stubs Test target (Component, part of solution) Test target (Component, part of solution) Messages/Events Logs Interfaces STUB/MOCK Component #1 STUB/MOCK Component #1 STUB/MOCK Component #2 STUB/MOCK Component #2 Target Control interface Execution Environment Execution environment control interface Logs
43
Integration Testing with stubs Messages/Events Logs Interfaces STUB/MOCK Component #1 STUB/MOCK Component #1 STUB/MOCK Component #2 STUB/MOCK Component #2 Target Control interface Execution environment control interface Logs
44
How to test? What should be tested? How ?
45
System Testing in Large What should be tested? How ?
46
System Testing in Large What should be tested? How ? Sales DB Cabin Reservation DB DB ? CRM DB Application & Gateways Application & Gateways Application & Gateways Application & Gateways
47
Acceptance Testing What should be tested before so customer could be so happy ? How ?
48
Test design & execution Discuss about reasons for test design? Why we need to do design? Stupid work ! I wan’t to progress!??
49
What is test desing? IDEAL REQ-Y REQ-Z REQ-O REQ What I should check ?
50
Test Case ? Verify drum track player pause mode functionality. Do this with IFDK software release X and playing song ”Show must go on by Freddy Mercury” Test should be done using android emulator environment and using your hands, ears and eyes” Verify drum track player pause mode functionality. Do this with IFDK software release X and playing song ”Show must go on by Freddy Mercury” Test should be done using android emulator environment and using your hands, ears and eyes” Pre State: Android emulator is running Release X is installed on emulator Test Case Steps: 1. Open drum kit player application 2. Select song ”Show must go on” 3. Start to play 4. Press Pause and check song is paused 5. Check memory usage from system application 6. Press Play 7. jump to 4 several time (<10) 8. Listen song to the end 9. Exit player using ”exit button” End State: IFDK Kit in main screen mode Pre State: Android emulator is running Release X is installed on emulator Test Case Steps: 1. Open drum kit player application 2. Select song ”Show must go on” 3. Start to play 4. Press Pause and check song is paused 5. Check memory usage from system application 6. Press Play 7. jump to 4 several time (<10) 8. Listen song to the end 9. Exit player using ”exit button” End State: IFDK Kit in main screen mode Test Case Name: Test Case Id: Test Case owner/writer: Date comments Test Case Name: Test Case Id: Test Case owner/writer: Date comments If Pause is working result is PASS. If Pause mode failed result is FAIL Verify what? Using configuration? With tools? Verify what? Using configuration? With tools? What is verdict? Add Information about case Define pre-state Define Steps Define end-state Define pre-state Define Steps Define end-state
51
Why we need test design ???? Stupid work! This takes ages! This Test Case documentation is old as soon I have changed some implementation? Why you need to do so hard documentation? Give me a one good reason!
52
Agile Thinking? We have to automate all tests!! No sense to create test case documentation ? Who does automation without a design?
53
Checklist vs Test Case? Check UI is workingWorking? Check color changeWorking? Check Counter value after 50 loginsWorking? Check disable mode for counterWorking? Check UI is workingWorking? Check color changeWorking? Check Counter value after 50 loginsWorking? Check disable mode for counterWorking? Checklist can be working great in small team! What happens if team is disbanded to other projects? And you are new maintainer for this project?
54
Sources for test design Test Design Method Negative Testing Requirement based test design Defect based test design Design based test desing Functional test design NonFunctional test design Specification based Testing design Test Case Step Test Case Step Customer's Idea Brainstorm Intitution Exploratory Customer's Idea Brainstorm Intitution Exploratory Test Engineer’s Daily Job? Test Engineer’s Daily Job? MixedMixed MixedMixed Write
55
Mechanical route to design Create a Test Case! Functional? Your Sources For Test Case Functional Test Case: ● Verify functionality of XXXX Functional Test Case: ● Verify functionality of XXXX Non-Functional Test Cases ● Verify Stability of XXXX ● Verify Performance of XXX ● Verify Security of XXXX ● Verify Usability of XXXX ● Verify Scalability of XXXX ● etc... Non-Functional Test Cases ● Verify Stability of XXXX ● Verify Performance of XXX ● Verify Security of XXXX ● Verify Usability of XXXX ● Verify Scalability of XXXX ● etc... Non-Functional? Which Type? ● Requirement ● Use Case ● Feature ● User Story ● Customer's Idea ● Brainstorm ● Intitution ● Exploratory Check also..... ● Correct functionality path ● Miss-usage of functionality ● Boundary Check Check also..... ● Correct functionality path ● Miss-usage of functionality ● Boundary Check Check also..... ● Check Possiblity to automated testing? Check also..... ● Check Possiblity to automated testing? Regression Test Case?? Write a Case Yläotsikko
56
What is Testing level What is Testing level Check different sources & strategy for Test Case design Check different sources & strategy for Test Case design Functional Test Case Non-Functional Test Cases Acceptance Test Case Field Test Case Interoperability Test Case Conformance Test Case Regression Test Case Choose Case Type WRITE A Test Case! WRITE A Test Case!
57
Test Driven Development in all testing levels? Design Tests Implement Code Design Draft System Acceptance Test Cases Design Draft System Acceptance Test Cases Define System Architecture & Design?
58
Release Management http://en.wikipedia.org/wiki/Configuration_management http://en.wikipedia.org/wiki/Release_management
59
Release Management Version 0.1 Version 0.2 Version 0.3 Version 0.2.1 Version 0.2.2.1 Version 0.2.2 Version 0.4 Trunk Customer 1 Version 0.2.3 Version 0.2.2.2 Version 0.2.2.3
60
Release & Configuration Management Version 0.1 Version 0.2 Version 0.3 Version 0.2.1 Version 0.2.2.1 Version 0.2.2 Version 0.4 Trunk Customer 1 Version 0.2.3 Version 0.2.2.2 Version 0.2.2.3 Feature s Release 1.0 Feature s
61
Release Management and integration testing IFDK Application Facebook Web Service HW Component Calore Meter Enabled Drum Stick HW Component Calore Meter Enabled Drum Stick Calore Meter SW Component 10% tested Calore Meter SW Component 10% tested Data Base Schema Design Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d First System Test With all components First System Test With all components
62
Test Planning & Management Why Test Management is needed ?
63
What should be tested first? Test Case 1 Test Case 2 V2.0 Test Case 3 V2.0 P1 P3 P4
64
What is a test target Test Case 1 Test Case 2 V2.0 Test Case 2 V2.0 Test Case 3 V2.0 Test Case 3 V2.0 Test Plan V 0.1 PASS / FAIL Executes Product RELEASE/BUILD VERSION 1.1 Product RELEASE/BUILD VERSION 1.1 Product RELEASE/BUILD VERSION 1.3 Product RELEASE/BUILD VERSION 1.3 Product RELEASE/BUILD VERSION 1.2 Product RELEASE/BUILD VERSION 1.2
65
Who executes test cases and when Test Case 1 Test Case 2 V2.0 Test Case 2 V2.0 Test Case 2 V2.0 Test Case 2 V2.0 Test Plan V 0.1
66
Group of Test Cases = Test Plan Test Case 1 Test Case 2 V2.0 Test Case 3 V2.0 Test Case 1 Test Case 2 V2.0 Test Case 3 V2.0 Test Plan V 0.1
67
Test plan life cycle Test Plan 1.0 Test Plan 1.0 Week 0 Week 26 Week 52 REL 0.1 REL 0.2 REL 0.5 REL 1.0 Test Plan 2.x Test Plan 2.x Routine Test Needs Still Work Effort Needs Still Work Effort
68
Test Case Pool and version control Test Case 1 Test Case 2 V1.0 Test Case 3 V1.0 Test Case 2 V2.0 Test Case 2 V2.0 Test Case 2 V3.0 Test Case 4 Draft
69
Test Management Using Testlink How to write test case? Creating a test plan TestLink is a web based Test Management tool. The application provides Test specification, Test plans and execution, Reporting, Requirements specification and collaborate with well-known bug trackers.
70
Execution of Test Case Test Engineer daily job ?
71
Different Roles in Testing Project Manager Test Designer Test Manager System Test Engineer System Test Engineer Test Automation Engineer Test Automation Engineer Acceptance Test Engineer Acceptance Test Engineer Developer
72
Test Designer ? Desiger for test cases (has substance knowledge) Test Case Step1 Step2 Step3 Test Case Step1 Step2 Step3 Create/Define
73
Tester ? Executes testing Creates reports and defect/bug/incident reports Test Case Step1 Step2 Step3 Test Case Step1 Step2 Step3 Execute PASS / FAIL ? Any bugs? PASS / FAIL ? Any bugs? Report
74
Test Manager ? Has control ower testing process and resources This role exists in large projects and product lines Testing Strategy Management Master Test Planning
75
Test Engineer ?
76
Example Product verification & validation organization Feature Component Feature Component Project Manager Designer/Coder Integration Test Engineer Integration Test Engineer Test Manager System Testing Feature Unit/Integration Testing System Acceptance Testing System Test Engineer System Test Engineer Test Automation Engineer Test Automation Engineer Acceptance Test Engineer Acceptance Test Engineer Validation Verification Regression Testing Integration Testing Functional System Testing Acceptance Testing Unit Testing Component Integration Testing Load, Stress, performance Load, Stress, performance
77
Agile Team and testing Acceptance Test Engineer Acceptance Test Engineer Developer / Tester Scrum Master Developer Product Owner
78
Large Agile organization
79
Testlink in brief Database Feature Implementation Note Verdict Error Database FAIL TestCase Test Plan/Suite SourceCode Version Control TESTLINK Bug Report
80
What we can do with Testlink Tool
81
Selecting Configuration Version 0.4 Version 0.2.2.2 Version 0.2.2 Test Plan Test Cases For Features Test Plan Test Cases For Features Tested Release/configuration Error/De fect Report Error/De fect Report Error/De fect Report Error/De fect Report Error/De fect Report Error/De fect Report
82
Test Case SUT/DUT LOG FILE Indications Events NOTIFICATIONS SUT = System Under Test DUT = Device Under Test ENVIRONMENT/ EMULATOR TOOLS TEST CASE
83
Agile methods and testing We take branding seriously, The FreeNest Brand is our pride and joy. It is the message we want to send to the developer teams, customers and stakeholders. These guidelines must be followed in all FreeNest’s print and electronic communications, marketing and informing. Documentation defines logo guidelines, typography and the use of colors.
84
Test Automation We take branding seriously, The FreeNest Brand is our pride and joy. It is the message we want to send to the developer teams, customers and stakeholders. These guidelines must be followed in all FreeNest’s print and electronic communications, marketing and informing. Documentation defines logo guidelines, typography and the use of colors.
85
Test Automation Frameworks
86
Why Test Automation Framework? There has to be common way to communicate with test environment. Test Automation Framework provides it Test Team communicates with common language Test Automation Framework provides a way to Control testing environment with common way
87
STAF as foundation for test automation The Software Testing Automation Framework (STAF) is an open source, multi-platform, multi- language framework designed around the idea of reusable components, called services STAF removes the tedium of building an automation infrastructure, thus enabling you to focus on building your automation solution The STAF framework provides the foundation upon which to build higher level solutions, and provides a pluggable approach supported across a large variety of platforms and languages. http://staf.sourceforge.net/
88
Test Environment Example TED = Test Environment Device Test Environment Devices are ”lego” bricks of for test scenario building. TED’s can be used as service intefaces to DUT (Design/Device under test) TED service has to be programmed to use DUT API/interface Test Automation needs support for virtual ”eyes” and ”arms”. TED’s can be also used such a way All TED ’s are communicating using STAF Network
89
TED ”virtual hand” STAF Service HW interface Service examples: Press Key Connect Power Read Led State Rasberry PI? http://www.raspberrypi.org/
90
Using STAF/STAX in PMR testing STAX STAF Service Virtual Phone STAF Service Base Station Host PC/Virtual machine STAF Service HW control STAF Service Web Cam SW Observer TED DUT TED Contoller TED Infrastructure TED STAF Service STAF Service Network Host PC/Virtual machine
91
STAX STAF Service Virtual Phone DUT TED STAF Service STAF Service Network Host PC Virtual machine #Presettings Case_Passed=False PMRRadioA.Select DMO_Mode PMRRadioB.Select DMO_Mode PMRRadioA.Select DMO_Channel( 10) PMRRadioB.Select DMO_Channel (10) #Case start PMRRadioB ResetDMOCallCounter PMRRadioB CountDMOCall Do while call_count < 100 { PMRRadioB MakeDMOCall(5, 5) call_count ++ } If (PMRRadioB.CallCounter = 100) { Case_Passed= True } # Case End STAX executes test script using PMRRadio virtual phone service interface Virtual phone implementation is DUT specific
92
Ideas? There is need for hardware to control keypad with machineThere is need for hardware to control keypad with machine It’s just a one small part of whole test automationIt’s just a one small part of whole test automation How to control screen? You will need another test tool?How to control screen? You will need another test tool?
93
Code Analyzing
94
Code Complexity Example tool CCCC https://wiki.jenkins- ci.org/display/JENKINS/CCCC+Plugin http://sourceforge.net/projects/codeanalyze- gpl/?source=recommended
95
Static Code Analyze 10 CLS: PRINT TAB(15); "Polysons': PRINT 20 PI=3. 14159: TP=PI*2 30 INPUT "Number of Sides';SD 40 INPUT "Size (5 to 80)';SZ 50 SP=TP/SD 60 SCREEN 1 70 FOR A=0 TO TP+SP/2 STEP SP 90 X=SZ*SIN(A)+100 100 Y=SZ OS(A)+96 110 IF A=0 THEN 130 120 LINE (X0, Y0)-(X, Y) 130 X0=X:Y0=Y 140 NEXT 150 GOTO 150 ? http://www.atarimagazines.com/creative/v10n5/158_Simple_screen_graphics_wi.php
96
Static Code Analyze – Heat Map http://www.statsvn.org/demo/ruby/
97
Code Coverage
98
Code Coverage – Branch Coverage The percentage of branches that have been exercised by a test suite. 100% branch coverage implies both 100% decision coverage and 100% statement coverage. http://booster911.hubpages.com/hub/BranchDecisionTesting
99
Code Analyze – Cyclomatic Complexity ? http://en.wikipedia.org/wiki/Cyclomatic_complexity
100
Build server ? http://buildbot.twistedmatrix.com/waterfall
101
Jenkins as build server Version Control Version Control Build Server Build Server Test report Execute Unit Tests for specific components If Unit Tests are passed – Jenkins will build whole product package SVN GIT Mercurial Etc. http://www.youtube.com/watch?v=Ago7hpp6xZI&feature=related Youtube Jenkins CI
102
Simple continuous integration Build service Source Code Repository Source Code Repository Build Job 1 Build Job 2 Build Job 3 1. Code commit 2. Trigger/Hook Build Slave 3. Execute Build 4. Fetch source TestBuild Build Report 5. Generates 6. Generates 7. Create report 8. Show 9. Inform developer
103
Continuous Integration and code analyze? Code Checker Builder Make / Ant Gcc Javac Builder Make / Ant Gcc Javac Integration Tester Unit Tester Complexity Analyze Complexity Analyze Heat Map Source Code Interface Tester
104
Test Bench Advanced Continous Integration Build service Source Code Repository Source Code Repository Build Job 1 1. Code commit 2. Trigger/Hook Build Slave 3. Execute Build 4. Fetch source TestBuild Build Report 7. Generates 8. Create report 9. Show 10. Inform developer 8 Analyze Code Analyze Code Analyze 6 Executes Build Slave
105
Multiple release targets Build service Build Slave Source Code Repository Source Code Repository Commits Windows Linux Mac
106
Build Service and FreeNest FreeNest Instance GIT Repository (master) GIT Repository (master) Jenkins CI server Jenkins CI server Developer Workstation Developer Workstation GIT (local) GIT (local) Push to master Build Slave (optional) Build Slave (optional)
107
Cloud as scalable CI enviroment Build Slave instance Build Slave instance Build Slave instance Build Slave instance Build Slave instance Build Slave instance Test Target Cloud API Cloud API Create, start, stop and destroy
108
Error Management
109
Concepst Failure - Fault, Defect, Bug - Incident, Failure, Error Example forum thread: http://www.allinterview.com/showanswer s/36257.html http://www.allinterview.com/showanswer s/36257.html ISTQB syllabus
110
Error/Bug/Defect Report ● Defect/Burg/Error ID ● Reporter ● Time ● Founded where ● Which way? ● Test Case ● Test Setup/Configuration ● Describe scenario? ● Attachements? Picture/Log/etc.. ● Defect/Burg/Error ID ● Reporter ● Time ● Founded where ● Which way? ● Test Case ● Test Setup/Configuration ● Describe scenario? ● Attachements? Picture/Log/etc..
111
● Defect/Burg/Error ID ● Reporter ● Time ● Founded where ● Which way? ● Test Case ● Test Setup/Configuration ● Describe scenario? ● Attachements? Picture/Log/etc.. ● Defect/Burg/Error ID ● Reporter ● Time ● Founded where ● Which way? ● Test Case ● Test Setup/Configuration ● Describe scenario? ● Attachements? Picture/Log/etc.. Error Database Error Database Id1 Id2 Id1 Id3 Id1
112
Nature of bug Id1 Id2 Id4 Id3 Location? Priority? Severity? Where is the bug is found?Which one should fixed first? How fatal is the bug? Id2 ID N Target? When the bug should be fixed and verified ?
113
Error Reporting? ● TEST CASE ID XXXXX ● Step1 ● Step2 ● Step3. ● Step4. ● TEST CASE ID XXXXX ● Step1 ● Step2 ● Step3. ● Step4. INCIDENTS Bug/Defect Report Bug/Defect Report Error Database System Under Test EXECUTE TEST ! Test Engineer Writes report and provides evidences Collects INCIDENTS Can lead to
114
Error Management and unit testing Implementation Error Database Bug Report Bug Report I should create new unit tests
115
Error Management and integration testing Implementation Integration Test Plan Integration Test Plan Error Database Bug Report Bug Report Bug Report Bug Report
116
Example of error process
117
Error Management Implementation Integration Test Plan Integration Test Plan Hardware Platform Integration Test Plan Platform Integration Test Plan System Integration Test Plan System Integration Test Plan
118
Week 0 Week 26 Week 52 REL 0.1 REL 0.2 REL 0.5 REL 1.0 Defect Found Defects Found Future ?
119
Bugzilla as example What is Bugzilla? Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe. http://www.bugzilla.org/ http://www.bugzilla.org/installation- list/
120
Reporting and metrics
121
CMMI Process Framework CMMI – covers ”error management” in several process areas SP3.2 Analyze Verification results Typical Work Products: Trouble reports - Analyze the verificationd ata on defects - Record all results of the analysis in a report - Provide infromation on how defects can be resolved (including verfiocation methods, criteria, and verification environment) and initiate corrective action Project Monitoring and Control SG2 Manage Corrective Action to Closure SP2.1 Analyze Issues SP2.2 Take corrective Action SP2.3 Manage corrective Action SG2 Validate Product or Product Components SP2.2 Analyze Validation Results. Change request management & Configuration Management process SG2 Track and Control Changes SP2.1 Track Change Requests SP2.2 Control Configuration Items
122
Sources for error report CRM Field Testing Testing Process Customer Feedback / Customer Feedback / Error Report Change Request? Change Request? N x Incidents
123
Test & Error Management Test Management Database Test Management Database Feature Implementation Feature Implementation TestCase Bug Report Error Database Error Database PASS FAIL TestCase Test Plan/Suite SourceCode Version Control SourceCode Version Control BUGZILLA TESTLINK VC + Implementation
124
Change Management Sometimes founded defect during testing can lead to change requests Bug? Change Request? Not Clear Requirements Not Clear Requirements Feature ?
125
Other issue trackers JIRA – Commercial Requisite Pro – Commercial Rational Synergy - Commercial Mantis – Open Source
126
Regression Testing Testing of a previously tested program following modification to ensure that defects have not been introduced or uncovered in unchanged areas of the software, as a result of the changes made. It is performed when the software or its environment is Changed. (ISTQB Syllabus)
127
Regression testing reason for test automation ? Test Case #1 Test Case #1 Test Case #2 Test Case #2 Test Case #4 Test Case #4 Test Case #3 Test Case #3 Manual Regression Test Plan Test Automation Solution Test Cases which are possible to automate?
128
Test Plan & Regression Test Case #1 Test Case #1 Test Case #3 Test Case #3 Test Case #5 Test Case #5 Test Case #7 Test Case #7 Test Case #9 Test Case #9 Test Case #2 Test Case #2 Test Case #4 Test Case #4 Test Case #6 Test Case #6 Test Case #8 Test Case #8 Test Case #10 Test Case #10 TEST PLAN 1 Test Case #1 Test Case #1 Test Case #3 Test Case #3 Test Case #7 Test Case #7 Test Case #4 Test Case #4 Test Case #8 Test Case #8 REGRESSION TEST PLAN REGRESSION TEST PLAN Test Case #5 Test Case #5 Test Case #9 Test Case #9 Test Case #2 Test Case #2 Test Case #6 Test Case #6 Test Case #10 Test Case #10 Test Case #9 Test Case #9 Test Case #10 Test Case #10 Test Case #11 Test Case #11 Test Case #12 Test Case #12 Test Case #13 Test Case #13 New TCases Added! New TCases Added!
130
Team Work 1 Setup test management solution Testlink 1.9.8 Linux / Windows MySQL
131
Team Work 2 Setup Issue Management solution Bugzilla, Mantis, JIRA, TRAC, Redmine
132
Team Work 3 Integration of Test Management & Issue Managment Testlink + Your Selection
133
Team Work 4 Set up CI environment Jenkins + SVN/GIT/Mercurial Every member of team will add own project in version control tree Create Job for all different Bonus about static coverage analyze tool installation
134
Team Work 5 Install and deploy Robot Framework Create test scripts for selected target
135
Team Work 6 Do Test Design for selected target Design and write 10 Test cases. Functionality, Performance, Stability, Release
136
Team Work 7 Create Test Planning for selected target Do Master Test Plan Do Resource Plan In Testlink Project
137
Team Work 8 Usablility testing: Read a selected Usability Report Analyze current state of solution Write report about changes what you can find according report
138
Team Work 9 Create performance tester image and plan result gathering and do some performance testing
139
Team Work 10 Create a video of test case and embend this video as test case in testlink !
140
Team Work Learn how Ubuntu as a product is tested? Make presentation about process and show it as a presentation.
141
Functional Requirement Non-Functional Requirement LOAD SECURITY MAINTENANCE IMPLEMENTATION Non-Functional Requirement Agile Epic / Theme SCALING FUNCTION SAFETY Non-Functional Requirement RECOVERY? NON FUNCTIONAL PERFORMANCE? Main { …. } Main { …. } ?
142
Team Work Stake Holders? Customer / User ?Business Request ? Use Case NON FUNCTIONAL PERFORMANCE? SCALING USABILITY UC Acceptance Criteria? RECOVERY? NON FUNCTIONAL User Story ? User Story Addition + SCENARIO ? Traditional Feature ?
143
Pet Capsule Concept Pet Capsule Service Service 30€/day Food, drinks and internet connection included Live Cam feed Web portal login Bring Cat to safe house Get Cat back home
144
What is The FreeNest Brand? We take branding seriously, The FreeNest Brand is our pride and joy. It is the message we want to send to the developer teams, customers and stakeholders. These guidelines must be followed in all FreeNest’s print and electronic communications, marketing and informing. Documentation defines logo guidelines, typography and the use of colors.
145
Logo The main logo consists of two elements, the leaf and the text laying horizontally. You can also use the logo with the FreeNest text under the leaf, if you need. Don’t do this often, the main logo should be the horizontal one.
146
Logo You can also use the logo with dark background and white text. You can use the logo with the “Product Platform” text under the FreeNest if the logo itself does not open up in the context.
147
Logo Do … observe the free space around tho logo. use the logo with black text when the background color is lite. use the logo with white text when the background color is dark. use white, black or neutral backgrounds. respect our logos. keep the logo readable. Don’t … squash or stretch the logo. alter the proportion of the logo. alter the layout of the logo. put anything over the logo. change the colors of the logo.
148
Brand colors TMNT and Leaf Green are the main colors of the brand. You can find the colors from the FreeNest logos leaf. Logos typography has the color called 666. Use these green colors with light or dark background, but do not use TMNT color behind the logo. TMNT Hex: #5f8029 CMYK: C67 M30 Y100 K15 Leaf Green Hex: #8ec03d CMYK: C52 M0 Y90 K0
149
Brand colors Kissa can be used as a background color for some elements. It must also be used as a text color if the background is gray (Toolbox for example). Gandalf is meant to be used as a text color if the background is light (Steven Seagull). Steven Seagull is a main background color for everything in FreeNest. It can be also used as a text color if the background is dark (Kissa for example). Gandalf Hex: #454545 CMYK: C65 M55 Y52 K53 Kissa Hex: #1a1a1a CMYK: C76 M67 Y0 K0 Steven Seagull Hex: #fafafc CMYK: C2 M2 Y1 K0
150
Brand colors Sad Cloud, Happy Cloud and Hazy Morning are meant to be used for other visual elements in layouts, buttons, images, illustrations etc. Use them carefully, do not let the blue shades dominate the green shades! Sad Cloud Hex: #3b5975 CMYK: C81 M57 Y34 K21 Happy Cloud Hex: #7abaf5 CMYK: C53 M16 Y0 K0 Hazy Morning Hex: #c5dff7 CMYK: C26 M6 Y0 K0
151
Web colors Great success, Doing It Wrong and Fatal error are utility colors. They are used to give feedback for the user. Great Success means that everything’s good. Doing It Wrong is a notice colour for noticing the user. Fatal Error is used for errors. Fatal Error Hex: #ad2a20 Doing It Wrong Hex: #ffc30f Great Success Hex: #7fb34b
152
Web colors 666 is the darkest shade you can use anywhere. 666 is near black. If there is necessary to use black, this color is the one. Toolbox and Team Board are used as a background for some elements (a toolbox or web site element background for example). 666 Hex: #060606 Team Board Hex: #c8c8c8 Toolbox Hex: #ebebeb
153
Typography Roboto Condensed, Roboto Condensed Italic Roboto Bold Condensed, Roboto Bold Condensed Italic Roboto Thin, Roboto Thin Italic Roboto Light, Roboto Light Italic Roboto Regular, Roboto Regular Italic Roboto Bold, Roboto Bold Italic Roboto Black, Roboto Black Italic The Roboto family The brand font is called Roboto. Always maintain the consistency in FreeNest software, web media, print media etc! If it’s not possible to use Roboto in web, use Arial.
154
Typography Using Roboto In body text you should use Thin or Light. Light can be used if Thin gets too thin and stuffy. This text is written with Roboto Light. For highlighting etc. you can use Bold or Black. Never use Italic for anything else than just short quotations and such. See the text colors in the Colors section to select right color for the right background when writing text. FreeNest (Roboto Thin and Roboto Black)
156
FreeNest Brand
157
What is The FreeNest Brand? We take branding seriously, The FreeNest Brand is our pride and joy. It is the message we want to send to the developer teams, customers and stakeholders. These guidelines must be followed in all FreeNest’s print and electronic communications, marketing and informing. Documentation defines logo guidelines, typography and the use of colors.
158
Logo The main logo consists of two elements, the leaf and the text laying horizontally. You can also use the logo with the FreeNest text under the leaf, if you need. Don’t do this often, the main logo should be the horizontal one.
159
Logo You can also use the logo with dark background and white text. You can use the logo with the “Product Platform” text under the FreeNest if the logo itself does not open up in the context.
160
Logo Do … observe the free space around tho logo. use the logo with black text when the background color is lite. use the logo with white text when the background color is dark. use white, black or neutral backgrounds. respect our logos. keep the logo readable. Don’t … squash or stretch the logo. alter the proportion of the logo. alter the layout of the logo. put anything over the logo. change the colors of the logo.
161
Brand colors TMNT and Leaf Green are the main colors of the brand. You can find the colors from the FreeNest logos leaf. Logos typography has the color called 666. Use these green colors with light or dark background, but do not use TMNT color behind the logo. TMNT Hex: #5f8029 CMYK: C67 M30 Y100 K15 Leaf Green Hex: #8ec03d CMYK: C52 M0 Y90 K0
162
Brand colors Kissa can be used as a background color for some elements. It must also be used as a text color if the background is gray (Toolbox for example). Gandalf is meant to be used as a text color if the background is light (Steven Seagull). Steven Seagull is a main background color for everything in FreeNest. It can be also used as a text color if the background is dark (Kissa for example). Gandalf Hex: #454545 CMYK: C65 M55 Y52 K53 Kissa Hex: #1a1a1a CMYK: C76 M67 Y0 K0 Steven Seagull Hex: #fafafc CMYK: C2 M2 Y1 K0
163
Brand colors Sad Cloud, Happy Cloud and Hazy Morning are meant to be used for other visual elements in layouts, buttons, images, illustrations etc. Use them carefully, do not let the blue shades dominate the green shades! Sad Cloud Hex: #3b5975 CMYK: C81 M57 Y34 K21 Happy Cloud Hex: #7abaf5 CMYK: C53 M16 Y0 K0 Hazy Morning Hex: #c5dff7 CMYK: C26 M6 Y0 K0
164
Web colors Great success, Doing It Wrong and Fatal error are utility colors. They are used to give feedback for the user. Great Success means that everything’s good. Doing It Wrong is a notice colour for noticing the user. Fatal Error is used for errors. Fatal Error Hex: #ad2a20 Doing It Wrong Hex: #ffc30f Great Success Hex: #7fb34b
165
Web colors 666 is the darkest shade you can use anywhere. 666 is near black. If there is necessary to use black, this color is the one. Toolbox and Team Board are used as a background for some elements (a toolbox or web site element background for example). 666 Hex: #060606 Team Board Hex: #c8c8c8 Toolbox Hex: #ebebeb
166
Typography Roboto Condensed, Roboto Condensed Italic Roboto Bold Condensed, Roboto Bold Condensed Italic Roboto Thin, Roboto Thin Italic Roboto Light, Roboto Light Italic Roboto Regular, Roboto Regular Italic Roboto Bold, Roboto Bold Italic Roboto Black, Roboto Black Italic The Roboto family The brand font is called Roboto. Always maintain the consistency in FreeNest software, web media, print media etc! If it’s not possible to use Roboto in web, use Arial.
167
Typography Using Roboto In body text you should use Thin or Light. Light can be used if Thin gets too thin and stuffy. This text is written with Roboto Light. For highlighting etc. you can use Bold or Black. Never use Italic for anything else than just short quotations and such. See the text colors in the Colors section to select right color for the right background when writing text. FreeNest (Roboto Thin and Roboto Black)
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.