1.
RV-Monitor: Efficient Parametric Runtime Verification with Simultaneous Properties
Qingzhou Luo, Yi Zhang, Choonghwan Lee, Dongyun Jin, Patrick O’Neil Meredith, Traian Florin Serbanuta and Grigore Rosu
University of Illinois at Urbana-Champaign
Runtime Verification, Inc.

2. Motivation RV can significantly increase system reliability
However, RV still not very used in production
Why?
Lack of specifications
Not easy to write specifications to check against
High monitoring overhead
Many RV systems incur large overhead when checking multiple properties simultaneously in real world software

3. Our Contributions Large collection of formal Java API specifications
A comprehensive set (179 in total) of formal specifications covering io, network, lang, util packages
Scalable Runtime Verification
Monitoring all 179 specifications simultaneously
Large scale evaluation
- Compare with other state-of-art RV tools

4. Java API Specifications
(Informal) Specifications for Java API Usage
If map is modified while an iteration over the set is in progress ... the results of the iteration are undefined.
Unintuitive, ambiguous and hard to monitor
We manually formalized properties implied in such Java API documentation text covering
Four packages (only): io, lang, util and net
Tried to mine them, but the didn’t work well

5. Formal Specifications From Java API
Separated specification-implying text
- If the map is modified while an iteration over the set is in progress ... the results
of the iteration are undefined.
is modified: put(...)/clear()/…
iteration … in progress: hasNext()/next()
undefined: error
Wrote specifications using various formalisms
Extended regular expressions (most common)
FSM, LTL, CFG, SRS (Turing complete)
Classified formal specifications as
Error, Warning, Suggestion

8. Reducing Monitoring Overhead
Most RV Systems are not capable of monitoring multiple properties simultaneously
Monitor lookup overhead
Memory / Garbage collection overhead
Proposing techniques to reduce RV overhead
Global Weak Reference Table (GWRT) with Caches
Combining Indexing Trees
Fine grained locking

9. Indexing Trees
An indexing tree is a multi-level map that, at each level, indexes each concrete object of the parameter binding. One can retrieve the related monitor instance by searching for corresponding concrete object at each level.
Map_UnsafeIterator(Map m, Collection c, Iterator i) {
creation event getC after(Map m)
returning(Collection c) ....
event getI after(Collection c) returning(Iterator i) …
event modifyM before(Map m) ...
  event modifyC before(Collection c) ...
event useI before(Iterator i) ...
ere : getC (modifyM | modifyC)* getI useI*
(modifyM | modifyC)+ use
@match { print("Map was modified while being iterated"); } }

10. Global Weak Reference Table (GWRT)
Problem:
when constructing indexing tree , multiple weak references for a single object are created.
Solution:
A GWRT for each parameter type that stores only one weak reference for each distinct object.

11. Combining Indexing Trees
- Problem:
Monitoring a real program causes millions of parameter and monitor instances and thus the size of indexing trees becomes quite large
Solution:
Combine indexing trees originating from the same specification, if they share the same prefix

12. Combining Indexing Trees
Before
After

13. Fine-grained Locking Problem:
JavaMOP uses a single global lock throughout all the operations for handling an event.
Solution:
Remove the global lock, and instead use fine-grained locking given that each of the GWRTS and indexing trees can be processed independently

14. (Re)Implementation

15. Evaluation Runtime Overhead
Comparison with previous JavaMOP with all 179 specifications
Comparison with MOPBox on Collection_UnsafeIterator, the most heavily used spec
Running on all DaCapo benchmarks
Violations Finding
Exceptions thrown / thread unsafe object / ...

16. Evaluation – Runtime Overhead

17. Evaluation – Violation Finding

18. Ineffectual Approaches
Combining Indexing Trees across Specifications
Wasted space for uncommon parts
Enhanced Indexing Cache
Hit ratio is already high
Indexing Tree Cleaning by GWRT
Costs more than cleaning by indexing tree themselves, although GWRT is more effective
Statistics-Based Indexing Tree Cleaning
Saving by partial cleanup is relatively small

19. Conclusion
Collection of 179 formal specifications covering most commonly used Java packages; assistance tool
RV-Monitor: new monitoring system, outperforms old JavaMOP when monitoring multiple properties; new JavaMOP implemented on top
Comprehensive evaluation, showing that new optimizations pay off
Future?
Parallelize: indexing, monitoring
Static analysis: do not instrument unnecessarily