1. Aspect-oriented Application-level Scheduling for J2EE Servers Kenichi Kourai, Hideaki Hibino, and Shigeru Chiba Tokyo Institute of Technology

2. Is AOP useful for anything except logging? The visitor pattern And what else?

3. Our assumption AOP is useful for performance tuning  Fixing unexpected performance problems at the last stage of software development  Without major architectural changes

4. Kasendas: our case study A river monitoring system  Collects and reports water levels of major rivers in Japan  Developed by an outside corporation independently Using JBoss, Tomcat, Struts, and Seasar2 12,000 lines of code 8.8 man-month chart generation data collection data generator data generator DB Kasendas water levels water-level update (emulator)

5. The map of Japan

6. Current water levels in Tokyo

7. Chart of recent changes of water levels

8. A performance problem in the first release No quality of service (QoS)  Kasendas failed periodic data collection under heavy workload The data collection must finish within 15 seconds  Triggered every 15 seconds Chart generation interferes with the data collection  A heavy-weight task time 15 seconds deadline miss collection time many requests Kasendas

9. Aspect-oriented application-level scheduling QoSWeaver  Enables scheduling at the application level A thread periodically calls a scheduler's method The method causes the thread to yield its execution voluntarily  according to a scheduling policy  Separates scheduling code into an aspect scheduler object thread A thread B aspect

10. Development cost Less than 1 man-month by our student  To find the cause of instability (1 week)  To develop a scheduling policy by trial and error (less than 2 weeks)  To test and modify the policy (1 week) For comparison  0.9 man-month to modify potential performance improvement No bottleneck analysis No guarantee

11. Technology The inside of QoSWeaver

12. Preemptive scheduling at the application level Long-running threads should be preempted by higher-priority threads  Scheduling code should be inserted everywhere in applications Inserting scheduling code at several points is not enough Non-preemptive Preemptive t t chart generation data collection

13. Profile-based pointcut generator Automatically generates pointcuts for inserting scheduling code  Manually defining pointcuts is difficult There are many candidates Scheduling code should be called at regular intervals  Based on execution profile Information on all join points (method calls)  Time stamps, caller's methods, and callee's methods Pointcuts are generated from a sequence of these join points

14. Algorithm of pointcut generation Overview  The algorithm divides execution profile into time slots by a given interval  It generates pointcuts to select only one join point for each time slot (if possible) Developers generate pointcuts for various parameters They choose the best set of pointcuts experimentally time slot (10 ms) time join point

15. Algorithm by example Pointcut candidates  circle, diamond, triangle, hexagon, square, and star 1st iteration 2nd iteration 3rd iteration Initial state See the paper for the detailed algorithm

16. Applied scheduling policy Proportional-share scheduling  For two groups A high-importance task: periodic data collection A low-importance task: chart generation  Policy To keep the ratio of the number of threads  1:1 was the best in our environment The number of low-importance threads is limited to 1  while a high-importance thread is running highlow

17. Developed aspect (pointcuts) lowImportance execution(void PlaceChartCreatePseudActionImpl.execute(..)) highImportance execution(void CollectorImpl.doObtain()) controlPoint (withincode(Range CategoryPlot.getDataRange(ValueAxis)) ANDAND call(Range Range.combine(Range, Range))) OROR : generated by the pointcut generator (17 pairs) method execution for chart generation method execution for periodic data collection written in GluonJ [Chiba et al.'05]

18. Developed aspect (advice) Advice calls methods of the PSScheduler class  A support class Creates a scheduler object Implements a real scheduling policy 150 lines of code lowImportance PSScheduler.startLowImportance(); $_ = proceed($$); PSScheduler.endLowImportance(); highImportance PSScheduler.startHighImportance(); $_ = proceed($$); PSScheduler.endHighImportance(); controlPoint PSScheduler.yield();

19. Developed scheduler Methods  startHighImportance() Sets yield flags to all but one low-importance thread  yield() Calls wait() if the yield flag is set  endHighImportance() Resets yield flags of all low-importance threads Calls notify() to all suspended threads scheduler object high-importance thread low-importance threads 1. startHighImportance 2. yield 4. endHighImportance 6. return 3. suspend 5. wakeup

20. Our experience: aspects Aspects allowed us to change a scheduling policy without modifying Kasendas  Initial policy Low-importance threads call sleep() except within the library  Ineffective because the library consumed CPU time  Second policy Low-importance threads call sleep() anywhere  Insufficient because many threads woke up at the same time  Final policy Low-importance threads call wait()  Effective because the thread execution is controlled properly

21. Our experience: pointcut generator The pointcut generator enabled us to select appropriate pairs of pointcuts  It selected 17 pairs out of 803 candidates The total number of join points was 248,661  The pointcut generator could re-generate pointcuts when Kasendas was modified Kasendas was released several times  Appropriate pointcuts changed We did not need to modify the pointcut definition in our aspect

22. Experiments We measured execution performance  For comparison Original Kasendas Kasendas tuned with admission control  Limits the number of running threads  Only for new requests  Already running threads are not suspended halfway  Apache JMeter to generate heavy workload Concurrently sent 40 requests to the low-importance task Servers: Sun Fire V60x, Linux 2.6.8, Sun JVM 1.4.2, Jboss 4.0.2 Client: Sun Fire B100x, Solaris 9, Sun JVM 1.4.2 requests Kasendas

23. Time for collecting water levels The data collection must finish within 15 seconds  Triggered every 15 seconds  Original 29.5 seconds  QoSWeaver 5.3 seconds  Admission control 10.1 seconds Unstable

24. Number of running low-importance threads The scheduling policies should reduce the number to 1 as soon as possible  when a high-importance thread starts running  QoSWeaver 1.9 seconds  Admission control Often fail to reduce the number to 1 12.0 seconds

25. Performance impacts against the low-importance task Overhead by calling a scheduler  No high-importance thread  QoSWeaver 6.6% Performance degradation  Suspending low-importance threads during periodic data collection  QoSWeaver 19%

26. Limitation QoSWeaver may cause deadlocks  For applications using synchronization  Web applications would not often include synchronization code Kasendas did not use thread synchronization  Using wait(timeout) could prevent deadlocks instead of wait() synchronized (o) { } 1. A thread is suspended 2. Other threads are blocked cannot wake up!

27. Related work Re-QoS [Tesanovic et al. '05]  Adapts applications to the real-time systems By admission control Bossa [Aberg et al. '03]  Enables developers to change the OS scheduler By using DSL and AOP ALPS [Newhouse et al. '06]  Controls UNIX processes by a scheduler process Not applicable to applications in Java

28. Conclusion AOP is useful for performance tuning  QoSWeaver AOP makes application-level scheduling realistic The profile-based pointcut generator automatically generates pointcuts  Kasendas Our scheduling policy could fix the performance problem at the last stage of the development Our experiences show that aspects and the pointcut generator were helpful

30. Distribution of selected join points

31. Average profiled intervals

32. Average real intervals

33. Differences by parameters

34. Difference of scheduling overhead