Testing The JCOP Framework Paul Burkimsher October 2010

What is the JCOP Framework? You should know that by now… ;-) It’s been around a while but is of course being maintained. The problem with maintenance is you can fix one bug and introduce another. Same for upgrading… => Need for Regression testing Does the original bit still work like it did yesterday?

Regression testing Is boring, repeating the same tests every day so we automate it. We have taken a very top-down approach. We pretend to be a user clicking on buttons, manipulating the DEN, declaring devices… We write Python programs which fire up PVSS panels, press buttons, type in field values…

Write Python programs which fire up PVSS panel and press buttons, type in field values…

How do we do it? Use a GUI testing tool called “Squish” from a German company. We chose Squish because ETM chose Squish for their own GUI testing…

Squish is powerful I agree with the salesman  : It is flexible You can run Squish on Windows and test Fw components also running on Windows or Fw components running remotely on Linux It supports several programming languages. I chose to use Python (I’m not a Java, Perl or Tcl fan) It is easy to use You can record mouse movements and keystrokes to get your initial scripts

Powerful, but Squish itself is being developed intensively Not ever so stable (presumably they test it :-P ) PVSS is changing underneath. ETM try very hard to keep the look and feel the same, but their implementation underneath changes  If the computer clicks on qt_go button and its name changes to qt_start, you’re screwed. Qt is changing underneath (and PVSS is built on top of Qt) … a lot of running fast to just stay still 

How does it work? I defined a sequence of Test Suites (Python programs): suite_createCleanPut (“ProjectUnderTest”) suite_installFwInstallationTool suite_installFwComponents suite_exerciseAnalogDigitalComponent suite_exercise CaenComponent tst_createCrateSy127; tst_createCrateSy403 tst_createCrateSy527; tst_createCrateSy2525 suite_exerciseWienerComponent suite_exerciseTrendEditorNavigator

How does it work – II ? AUT (Application Under Test) uses Qt Network Background Python Script or Python Script in IDE Special qt.dll get interpreted by Squish Runner (interpreter - client) Squish server

How do you know its worked?

It gets to the end without breaking? You can put assertions in the Python code You can inspect the output

What is the output of the DEN? DPs and hence .dpl files We compare the produced .dpl file with an “expected” .dpl file. NB You have to remove the timestamps and Dp Ids from the .dpls first.

How do we communicate results? Produce log files at every stage. Squish produces an XML summary of the assertion results (and provides an html translator) We create an email with this info and send it to all the JCOP authors whose components are being tested

Nightly email

Email (cont)

Further work More tests More software version combinations (PVSS 3.9 etc) Fully automate the test across to Linux Code coverage in the scripts

Questions ?