CSC Multiprocessor Programming, Spring, 2012 Chapter 12 – Testing Concurrent Programs Dr. Dale E. Parson, week 12

Testing considerations Test for safety. Nothing bad ever happens. Test for liveness. Something good eventually happens Test code can introduce timing and synchronization changes. Performance testing and benchmarks. Throughput, Responsiveness (Latency), Scalability

Testing for correctness Validating invariants and precondition -> postcondition satisfaction can be automated. Design by Contract constraints should remain visible on a per-thread basis. From any individual thread’s perspective, objects maintain their constraints. Thread.getState useful for debugging, not for testing or application logic. Try to have more active threads than CPUs to increase probability of timing indeterminacy.

Variety in testing Test resource management. Vary the load and the client-side demand to stress the system. Test on a variety of platforms (CPU and O.S.). Add trace hooks when they do not hurt performance or skew CPU load noticeably. Use configuration parameters that you can vary to bring out nondeterminacies. Thread pool size, buffer sizes, concurrent connections.

Comparing multiple algorithms Use configuration to set factories, such as plug-in factory methods, to load alternative classes, e.g., alternative library implementations of queues, lists, sets, maps. Vary these configuration parameters for performance testing. Use internal time measurement library methods to profile major components and data flows. Get real time and CPU time(s).

Pitfalls for testing and measurement Garbage collection can introduce non-linear jumps in numbers. Shut it off with –verbose:gc. Bad idea  Run enough processing to average out its effects Dynamic (Just In Time) compilation is another source of non-linear jumps in numbers. Again, shutting it off is a bad idea. Again, solution is to hammer the typical application paths through the code, and try to traverse them all. A coverage tool is useful for the latter.

Server mode JVM Prefer –server to –client JVM for production and testing on multiprocessor systems. Make sure that any benchmark does not get optimized away as dead code. Use any computed result somehow, and call System’s currentTimeMillis and nanoTime around the running code of interest. Code inspections are useful, even by yourself the next morning.

Static Analysis Tools Tools are under development. Invoking Thread.run, unreleased lock, empty synchronization block, double-checked locking (reading state before and after acquiring lock), starting a thread from a constructor, notify errors (notify after no state change), calling (a)wait while not in a loop that tests a lock predicate or without a lock held, misuse of a Lock or Condition (using their intrinsic locks), sleeping or waiting while holding a lock, spinning on (non-volatile) variables. Commercial tools.