1. Testing The JCOP Framework
Paul Burkimsher
October 2010

2. What is the JCOP Framework?
You should know that by now… ;-)
It’s been around a while but is of course being maintained.
The problem with maintenance is you can fix one bug and introduce another.
Same for upgrading…
=> Need for Regression testing
Does the original bit still work like it did yesterday?

3. Regression testing Is boring, repeating the same tests every day
so we automate it.
We have taken a very top-down approach. We pretend to be a user clicking on buttons, manipulating the DEN, declaring devices…
We write Python programs which fire up PVSS panels, press buttons, type in field values…

4. Write Python programs which fire up PVSS panel and press buttons, type in field values…

5. How do we do it?
Use a GUI testing tool called “Squish” from a German company.
We chose Squish because ETM chose Squish for their own GUI testing…

6. Squish is powerful I agree with the salesman  : It is flexible
You can run Squish on Windows and test
Fw components also running on Windows or
Fw components running remotely on Linux
It supports several programming languages. I chose to use Python (I’m not a Java, Perl or Tcl fan)
It is easy to use
You can record mouse movements and keystrokes to get your initial scripts

7. Powerful, but Squish itself is being developed intensively
Not ever so stable (presumably they test it :-P )
PVSS is changing underneath. ETM try very hard to keep the look and feel the same, but their implementation underneath changes 
If the computer clicks on qt_go button and its name changes to qt_start, you’re screwed.
Qt is changing underneath (and PVSS is built on top of Qt)
… a lot of running fast to just stay still 

8. How does it work?
I defined a sequence of Test Suites (Python programs):
suite_createCleanPut (“ProjectUnderTest”)
suite_installFwInstallationTool
suite_installFwComponents
suite_exerciseAnalogDigitalComponent
suite_exercise CaenComponent
tst_createCrateSy127; tst_createCrateSy403
tst_createCrateSy527; tst_createCrateSy2525
suite_exerciseWienerComponent
suite_exerciseTrendEditorNavigator

9. How does it work – II ? AUT (Application Under Test) uses Qt Network
Background Python Script or
Python Script
in IDE
Special qt.dll
get interpreted by
Squish Runner
(interpreter - client)
Squish server

10. How do you know its worked?

11. It gets to the end without breaking?
You can put assertions in the Python code
You can inspect the output

12. What is the output of the DEN?
DPs and hence .dpl files
We compare the produced .dpl file with an “expected” .dpl file.
NB You have to remove the timestamps and Dp Ids from the .dpls first.

13. How do we communicate results?
Produce log files at every stage.
Squish produces an XML summary of the assertion results (and provides an html translator)
We create an with this info and send it to all the JCOP authors whose components are being tested

14. Nightly

15. (cont)

16. Further work More tests
More software version combinations (PVSS 3.9 etc)
Fully automate the test across to Linux
Code coverage in the scripts

17. Questions ?