1. EuroSTAR 2006 Making the Most of Test Design
Torbjörn Ryber
Keep IT Simple
(c) Torbjörn Ryber september 2006

2. (c) Torbjörn Ryber september 2006
(c) Torbjörn Ryber september 2006

3. (c) Torbjörn Ryber september 2006
(c) Torbjörn Ryber september 2006

4. Testers are Complainers…
Testers are often seen as complainers. Unfortunately this is too often the truth
We demand to participate early in the project. Then we complain about the quality of requirements
We can do better!
I have met testers that use only one technique – and that technique is really only a format template everything else is resisted.
I have met a tester that wanted no contact at all with people – give me the requiremnets and i will enter my room and present to you the end results.
(c) Torbjörn Ryber september 2006

5. (c) Torbjörn Ryber september 2006
What Can we NOT Change?
The way requirements are handled
How the project is run
What development process is used (or not)
The deadline (at least most of the time)
So leave it alone!
Leave it or get disliked by the rest
Focus on what is important and possible to do
Have more fun
Don´t get burned out
(c) Torbjörn Ryber september 2006

6. What Can we Change? Ourselves!
Our knowledge of testing
Our knowledge of development
How we adapt to a situation
The end result
Do it!
We need to know a lot, both on testing and on development.
Agile, UP, test-first, databases, etc
testdesign
(c) Torbjörn Ryber september 2006

7. (c) Torbjörn Ryber september 2006
Some Wisdom…
”Give me the power to change what I can change, peace of mind to accept what I cannot and skill enough to tell one from the other”
I often see testers trying to change the whole project – they invariably fail. I have tried it myself – and, of course, failed.
(c) Torbjörn Ryber september 2006

8. What does a tester have to know?
Practical considerations – executing tests
Asking the right questions – test design
Quality assurance – the models you create
(c) Torbjörn Ryber september 2006

9. (c) Torbjörn Ryber september 2006
Executing tests
Practical stuff…
(c) Torbjörn Ryber september 2006

10. (c) Torbjörn Ryber september 2006
How do I ask questions?
Graphical User Interface
Easy to ask
Hard to test
Harder to automate
Read a file
Interface may already exist
Create your own interface
Build your own
Ask the developer ?
(c) Torbjörn Ryber september 2006

11. (c) Torbjörn Ryber september 2006
How do I see the answers?
Grapical User Interface
Instant answer
Harder to automate
Reports
Maybe not enough!
File
May already exist
Easy to check
Create your own
SQL-query
Ask developer – demand testability ?
(c) Torbjörn Ryber september 2006

12. (c) Torbjörn Ryber september 2006
Know Your Oracles
Manually from requirements
Older tested/verified version of same system
Current other systems where we can assume data is correct
Formulas in MS Excel or other tools
Heuristics
(c) Torbjörn Ryber september 2006

13. How do I test this on time?
Regelverk testas för sig
Filformat in: Display ut med svar och delresultat
Objektorienterat tankesätt
Tråkig attityd hos testare av dialogen – vi gör vårt så gör du ditt!
(c) Torbjörn Ryber september 2006

14. Strategy: parallel testing GUI and Rules
Fördelar: kan köra parallella tester dialog/regelverk
Snabbare att köra testfallen i regelverket via fil istället för manuellt
Dialogtesten fokus på att alla fält användes korrekt: typ integrationstest
Förbättring: agil automatisering: läsa från fil och ställa alla frågor i rad och automatiskt utvärdera svaren.
Demo: Raktprototyp
(c) Torbjörn Ryber september 2006

15. Creative planning solved it!
Alt1: time to short for development and testing
Alt2: longer time period for both development AND testing
Vi hade aldrig hunnit klart i tid med ett traditionellt sätt att jobba
Tidigt engagemang= gott om tid för inlärning och samtidig testdesign
(c) Torbjörn Ryber september 2006

16. Philosophy and Testing
Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability…
(Karl Popper: Conjectures and Refutations p 48)
A requirement is testable if it possible to show that it is not true. (If you can´t – it´s a Barnum statement)
More difficult tests give more information
These tests demand better test design
Barnum Statement: like astrology – too general, cannot say it is wrong
Problem: testers are not good enough regarding design techniques!
(c) Torbjörn Ryber september 2006

17. Test Design – asking the right questions
Analyse, create model
Cover the model with (basic) test cases
Add test data for deeper tests
Finally the whole system and some creative thinking
Syfte:
Ge en överblick till kursens upplägg samt hur de olika delarna i testdesignen hänger samman
Förmedla:
Snabbgenomgång av tågordningen vid testdesign.
Kommunicera ut vilka delar vi kommer att gå igenom samt i vilken ordning:
Komplettera med testdata
Bygg paket, kedjor och genomförandeplan
Skapa grundläggande testfall
Övergång:
Då startar vi med rutan ”Komplettera med testdata”…
(c) Torbjörn Ryber september 2006

18. Good Quality Assurance
Good Test Design finds more defects than test execution!
Test design is verification of the requirements
Test execution is validation of the actual code
Agile Modeling – Good Enough Modeling. Scott Ambler
Exploring Requirements: Jerry Weinberg: Many different models on the same information gives you different views
(c) Torbjörn Ryber september 2006

19. (c) Torbjörn Ryber september 2006
Model examples H
(c) Torbjörn Ryber september 2006

20. All Science is Based on Models
All models are wrong but some are useful. (Gerald Weinberg)
A good model is easy to understand
A good model is a true simplification of reality
Too little information makes the model incomplete
Too much information makes the model hard to understand
Quality Assurance of the models are of utmost importance
Excel is not big eough – hey ever considered breaking in pieces
Excel sheet from the ceiling to the floor – something is wrong – been doing it for five years…get a new job decoratong christmas trees
(c) Torbjörn Ryber september 2006

21. Test Design Techniques
All techniques use some kind of model
Focus on
Data
Flows
Logic
Combinatorics
Overall tests
(c) Torbjörn Ryber september 2006

22. Data – is a part of all techniques
Equivalence partitioning
Numeric
Other types : text, lists, time dependent, relative
Boundary Values
Only one vaue for each partition – are you SURE that you have selected the correct partitions and NO other factors will affect your tests? Will you get a ”warm fuzzy feeling” or are you beeing sensible when selecting two or three values? Maybe it is about risk
(c) Torbjörn Ryber september 2006

23. (c) Torbjörn Ryber september 2006
Flows
Variations
Business processes
Use Case Flows
Control and Data Flows
Design Technique
Create Flow Graph
Cover branches, decisions, paths etc
(c) Torbjörn Ryber september 2006

24. (c) Torbjörn Ryber september 2006
Event-Driven
Real-time systems (telephone, dialogue, ATM)
Design Technique
Create State Graph/table
Cover nodes, links, paths
(c) Torbjörn Ryber september 2006

25. (c) Torbjörn Ryber september 2006
Dialogues
Concurrent users?
Robustness?
Time-Outs?
(c) Torbjörn Ryber september 2006

26. (c) Torbjörn Ryber september 2006
Insurance policy: states
Syfte:
…forts…
Förmedla:
Varje statusläge/tillstånd togs fram
Förflyttningarna mellan tillstånden ritades ut
Orsak till förflyttning noterades
(Ingen UML-notation – Tillstånden är ritade med ovaler)
Övergång:
Tillståndsgrafen kompletterades med statistik…
(c) Torbjörn Ryber september 2006

27. (c) Torbjörn Ryber september 2006
Driving a train
(c) Torbjörn Ryber september 2006

28. (c) Torbjörn Ryber september 2006
Logic
Variations
Set of Rules
Conditions
Formulas
Design techniques
Decision table
Decision Tree
Exploring Req: Jerry Weinberg
(c) Torbjörn Ryber september 2006

29. Combinatoric Explosion
Common Problem: too many combinations
Design Technique
Elementary Comparison (MCDC)
Pairwise Testing – allpairs.exe
(c) Torbjörn Ryber september 2006

30. Advanced testing – the whole picture
Syntax – standard over the whole system?
Data Cycle – covered in CRUD?
Time Cycle – regular events that need to be tested (yearly, monthly etc)
Soap Opera
Non-functional – what should WE do?
(c) Torbjörn Ryber september 2006

31. How do we get the respect we (think we) deserve?
By adding value - And showing that we add value
”Testers light the way ”
- James Bach
Effective and efficient test design adds value in all phases
Whatever there is – we will test it and inform of our findings
Now let´s get to the point: W-model
But it is not automatically a given fact that we get respect once we deserve it – But then it it fairly easy to showing that we add value
(c) Torbjörn Ryber september 2006

32. Testing is Like Martial Arts
Don´t come to EuroSTAR with the wrong attitude. There is always something to learn. I have met people that have attended several years but not seem to have listened.
You must master a number of basic techniques and when in need a true master creates/adapts his own technique to fit the situation
If you believe you already know everything – you will never learn!
You must open your mind. (Tomita Sensei: 8th Dan)
(c) Torbjörn Ryber september 2006

33. Your Success Depends on YOU!
Enhance your competence
Learn how to model
Learn test design techniques
Read, take classes, be creative
Be flexible – adapt to the situation
Testing is advanced stuff – you need skill to succeed
Change yourself – not others
(c) Torbjörn Ryber september 2006