The Java Query Language Darren Willis, David J. Pearce and James Noble Victoria University of Wellington, New Zealand

What is the Java Query Language? Prototype extension to Java –Provides first-class query constructs –Supports standard database optimisations –Also experimented with caching optimisations Key Idea: –By changing loops into queries we allow the system to perform more aggressive optimisations

JQL Syntax Query Expression: For loop query: for(Robot r : robots | !r.isDead()) { r.method(); } List f(List students) { return [ Student s : students | s == jim ]; }

For each query, JQL constructs pipeline on-the-fly: –Selecting good join ordering and join types improves performance –E.g. hash join versus nested-loop join r = [String x:words, Integer y:lengths | x.length().equals(y) && x.length() > 1]

Understanding Program Loops Many loops in Java can be converted into JQL queries: for(Robot r : robots) { if(!r.isDead()) { r.method(); } } for(Robot r : robots | !r.isDead()) { r.method(); }

for(Robot r : robots) { if(!r.isDead()) { r.method(); } } for(Robot r : robots) { r.out.close(); } for(Robot r : robots) { if(!r.isDead()) { for(Robot dead : deadRobots) { if(!= dead.team) { r.doSomething();} }}} (Level 1 Filter) (Level 2 Filter) (Unfiltered)

ZK = 45KLoc RSSOwl= 46KLoc Freemind = 70KLoc Robocode = 23KLoc, Chungles = 4KLoc SoapUI = 68KLoc

Part 2 Caching and Incrementalisation

Robocode

Caching + Incrementalisation class Battle { private List robots;... public List robots() { return robots; } } void scan() { for(Robot r : battle.robots()) { if(r!=null && r!=this && !r.isDead() && …) {... }}

class Battle { private List robots; private List liveRobots;... public List robots() { return robots; } public List aliveRobots() { return liveRobots; }} void scan() { for(Robot r : battle.aliveRobots()) { if(r!=this && …) {... }} Caching + Incrementalisation

Incrementalisation When caching we must incrementalise: class Battle { private List robots, aliveRobots;... public void addRobot(Robot r) { robots.add(r); if(!r.isDead()) { aliveRobots.add(r); }} public void robotDied(Robot r) { aliveRobots.remove(r); }}

Caching + Incrementalisation in JQL xs=[ Student s : students | s == jim ]; Key Idea: –Cache + Incrementalise queries’ result set –Thus, subsequent evaluations can reuse it Issues: –Don’t want to cache everything! –Don’t want to exhaust memory! Problems: –How to correctly incrementalise? –What about methods?

Prototype Implementation Cache Manager –Decides when to start/stop caching queries –Tracks object updates and incrementalises cached results Object Updates: –E.g. Object added to collection, field updated, etc. –Re-evaluate query pipeline using just this object Caching Heuristics: –Always on - every query always cached –Ratio – enable caching when query/update ratio high enough

Prototype Implementation Dependency Tracking: –How to know when object update occurs? –How to know if object update affects cache? –(the latter requires knowledge of the object graph) Our (Simplistic) Approach: –Use Aspect/J to monitor object updates –Only track fields –Ignore object graph; instead prevent multi-level indirection in queries! –Prevent caching if methods used in query!

Further Reading Caching and Incrementalisation in the Java Query Language. Darren Willis, David J. Pearce and James Noble. In Proc OOPSLA, 2008 (to appear). Efficient Object Querying in Java. Darren Willis, David J. Pearce and James Noble. In Proc ECOOP, 2006.

Conclusion Short Story: –Many loops can be turned into queries –Queries can be optimised using database style optimisations –Queries could also be optimised using caching and incrementalisation –Such optimisations make real programs go faster