1. Concurrent Mark-Sweep Presented by Eyal Dushkin GC Seminar, Tel-Aviv University 30.12.14

2. Outline of talk  Introduction  Tricolour Invariants (Reminder)  Mostly Concurrent Mark & Sweep  Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

3. Introduction  During the course we have seen different types of garbage collection algorithms, particularly of types: STW (stop the world), Incremental and Parallel.  Each of which has its own tradeoffs with respect to:  Throughput  Pause time  Space overhead  Scalability and portability  And more …  We’ll present the concurrent mark-sweep paradigm.

4. Incremental Collection ( Roman Kecher Lec.09 ) 4  First, on a Uniprocessor Time

5. Incremental Collection ( Roman Kecher Lec.09 ) 5  On a Multiprocessor:

6. Incremental Collection ( Roman Kecher Lec.09 ) 6  On a Multiprocessor:  Can also be parallelized:

7. Mostly Concurrent Collection ( Roman Kecher Lec.09 ) 7

8. 8  Can also be incremental:

9. Concurrent Collection ( Roman Kecher Lec.09 ) 9

10. 10  Can also be incremental:

11. Our Focus

12. Tricolour Abstraction – Reminder ( Pavel Lec.02 )  A convenient way to describe object states:  Initially, every node is white.  When a node is first encountered (during tracing), it is colored grey.  When it has been scanned, and its children identified, it is colored black.  At the end of each sweep, no references from black to white objects.  All the white objects are unreachable = garbage.

13. Grey Wavefront ( Roman Kecher Lec.09 ) 13 Roots

14. Grey Wavefront ( Roman Kecher Lec.09 ) 14 Roots

15. Grey Wavefront ( Roman Kecher Lec.09 ) 15 Roots

16. Grey Wavefront ( Roman Kecher Lec.09 ) 16 Roots

17. Grey Wavefront ( Roman Kecher Lec.09 ) 17 Roots

18. Grey Wavefront ( Roman Kecher Lec.09 ) 18 Roots

19. Grey Wavefront ( Roman Kecher Lec.09 ) 19 Roots

20. Grey Wavefront ( Roman Kecher Lec.09 ) 20 Roots

21. Grey Wavefront ( Roman Kecher Lec.09 ) 21 Roots

22. The Lost Object Problem  Direct and transitive.

23. Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1423  D1 // Insertion  Write(X, b, Read(Y, a)) Roots XY a Z

24. Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1424  D1 // Insertion  Write(X, b, Read(Y, a)) Roots XY a Z b

25. Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1425  D2 // Deletion:  Write(Y, a, null) Roots XY a Z b

26. Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1426  D2 // Deletion:  Write(Y, a, null) Roots XY Z b

27. Lost Object – Direct ( Roman Kecher Lec.09 ) 27  D3:  scan(Y) Roots XY Z b

28. Lost Object – Direct ( Roman Kecher Lec.09 ) 28 Roots XY Z b

29. Lost Object – Transitive ( Roman Kecher Lec.09 ) 29 Roots PQ c RS d

30. Lost Object – Transitive ( Roman Kecher Lec.09 ) 30  T1 // Insertion:  Write(P, e, Read(R, d)) Roots PQ c RS d e

31. Lost Object – Transitive ( Roman Kecher Lec.09 ) 31  T2 // Deletion:  Write(Q, c, null) Roots PQ RS d e

32. Lost Object – Transitive ( Roman Kecher Lec.09 ) 32  T3:  scan(Q) Roots PQ RS d e

33. Lost Object – Transitive ( Roman Kecher Lec.09 ) 33 Roots PQ RS d e

34. Tricolour Invariants – Reminder  The strong tricolour invariant: There are no pointers from black objects to white objects. root YX Z

35. Tricolour Invariants – Reminder  The strong tricolour invariant: There are no pointers from black objects to white objects.  The weak tricolour invariant: All white objects pointed to by a black object are reachable from some grey object through a chain of white objects. root YX Z

36. Barriers for Grey Mutator Weak or Strong?

37. Barriers for Black Mutator Weak or Strong?

38. Barriers – to be continued …

39. Introduction Tricolour Invariants (Reminder)  Mostly Concurrent Mark & Sweep  Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

40. Mostly Concurrent Mark & Sweep Initialisation:  Triggering is a critical decision.  Once a collection cycle has started, we would like to:  Minimize its impact on mutator throughput (efficiency).  Complete the cycle before mutator exhausts memory (pause time).

41. Mostly Concurrent Mark & Sweep Mutator Allocation: New(): CollectEnough() ref  allocate() if ref = null errror "out of memory" return ref atomic collectEnough(): while behind() if not markSome() return

42. Mostly Concurrent Mark & Sweep Let’s dive deep into the markSome method: markSome(): if isEmpty(worklist) scan(Roots) if isEmpty(worklist) sweep() return false ref  remove(worklist) scan(ref) return true

43. Mostly Concurrent Mark & Sweep Additional Methods: shade(ref): if not isMarked(ref) setMarked(ref) add(worklist, ref) scan(ref): for each fld in Pointers(ref) child  *fld if child ≠ null shade(child)

44. Mostly Concurrent Mark & Sweep Additional Methods: revert(ref): add(worklist, ref) isWhite(ref): return not isMarked(ref) isGrey(ref): return ref in worklist isBlack(ref): return isMarked(ref) && not isGrey(ref)

45. Mostly Concurrent Mark & Sweep Recall: Black and Grey mutators. markSome(): if isEmpty(worklist) scan(Roots) if isEmpty(worklist) sweep() return false ref  remove(worklist) scan(ref) return true

46. Mostly Concurrent Mark & Sweep Recall: Black and Grey mutators. markSome(): if isEmpty(worklist) scan(Roots) if isEmpty(worklist) sweep() return false ref  remove(worklist) scan(ref) return true

47. Mostly Concurrent Mark & Sweep Termination:  Depends on the mutator’s colour:  If black, no need to rescan.  If grey, need to rescan and sync with mutator’s thread.

48. Mostly Concurrent Mark & Sweep Allocation:  Depends on the mutator’s colour:  If black, could not be allocated as white under the strong invariant.  What about the weak invariant? Shall we allocate it grey?  If grey, then it’s non trivial:  White – might affect pause time and throughput.  Black – increase floating garbage quantity.  Yellow – hybrid method (Vechev et al [2006]).

49. Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep  Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

50. Concurrent marking and sweeping  So far, marking-sweeping proceeding in series.  This of course is not always the case.  Lazy sweeping?

51. Recap: Lazy Sweeping ( Pavel Lec.02 ) atomic collect() markFromRoots() for each block in Blocks if not isMarked(block) add(blockAllocator, block) else add(reclaimList, block)

52. Recap: Lazy Sweeping ( Pavel Lec.02 ) lazySweep(sz): repeat block  nextBlock(relaimList, sz) if block != null sweep(start(block), end(block)) if spaceFound(block) return untill block = null allocSlow(sz) allocSlow(sz): block  allocateBlock() if block != null initalise(block, sz)

53. Concurrent marking and sweeping  We may also trigger concurrent sweeping from the previous marking phase.  Even if a gc cycle has been already running the next one …  How could we advise the marker and sweeper to distinguish between different white nodes?

54. Concurrent marking and sweeping  Example - Roots block

55. Concurrent marking and sweeping Roots block

56. Concurrent marking and sweeping Roots block

57. Concurrent marking and sweeping  Gc cycle is ended …  Now what? Roots block

58. Concurrent marking and sweeping How could we advise the marker and sweeper to distinguish between different white nodes?  We could use a new color: purple (Lamport [1976]).  But this means we need to: 1. Wait until all markers and sweeper have finished. 2. Changing colors – white to purple, black back to white. 3. Shade all roots. 4. Start the markers and sweepers again.

59. Concurrent marking and sweeping A Caveat: Changing the colors – white to purple, black back to white consumes time. Elegant solution (Lamport): don’t change colors, change perspective

60. Lamport Solution roots X YZ W

61. Lamport Solution roots X YZ W

62. Lamport Solution roots X YZ W

63. Lamport Solution roots X YZ W

64. Lamport Solution roots X YZ W

65. Lamport Solution roots X YZ W

66. Lamport Solution roots X YZ W

67. Lamport Solution roots X YZ W

68. Lamport Solution roots X YZ W

69. Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

70. On-the-fly marking (Heuristic)  So far we have assumed that the mutator threads are all stopped at once.  On-the-fly collection never stops the mutator threads at once.  An alternative is to sample the roots of each mutator thread separately, concurrently with other mutator threads working.  But it’s not as simple as it seems (if it seems) …

71. On-the-fly marking (Heuristic)  Usually mostly-concurrent collectors are using a black mutator with a deletion barrier.  Avoiding stw phase, some mutator's roots are neither black, nor grey.  As a result, we have black and white thread stacks. Thread Stack 1 x Thread Stack 2 Y

72. Barriers for Black Mutator (again) Weak or Strong? Merely Read Barrier?

73. On-the-fly marking (Heuristic)  Read barriers does not apply on stack operations, particularly the deletion barrier is not applied.  One can use stack operations barriers ( Visit Lecture no.6 by Oleg Dobkin ). Thread Stack 1 x Thread Stack 2 Y

74. On-the-fly marking (Heuristic)  Write(X,b,Y) Thread Stack 1 x Thread Stack 2 Y

75. On-the-fly marking (Heuristic)  Write(X,b,Y) Thread Stack 1 x Thread Stack 2 Y b

76. Doligez-Leroy-Gonthier [1993]  Let’s separate data allocated solely on a single thread and not shared with other threads.  In addition we have a global heap that allows sharing of objects among threads.  Private objects may move to global heap, if needed.

77. Doligez-Leroy-Gonthier [1993]

80. On-the-fly marking (Heuristic) This solves the first problem: Thread Stack 1 x Thread Stack 2 Y

81. On-the-fly marking (Heuristic) Thread Stack 1 x Thread Stack 2 Y

82. On-the-fly marking (Heuristic) Thread Stack 1 x Thread Stack 2 Y b

83. Doligez-Leroy-Gonthier [1993]

86. Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping On-the-fly marking  Abstract Concurrent Collection  Summary

87. Abstract Concurrent Collection  We present an abstract framework for concurrent garbage collection [Vechev et al, 2005, 2006, 2007].  The correctness of a concurrent collector rely on the cooperation between the collector and the mutator.  Thus it’s reasonable to introduce a log event structure to append and synchronize the events of both.

88. Abstract Concurrent Collection - Preliminaries src old fld

89. Abstract Concurrent Collection - Preliminaries src new fld

90. Abstract Concurrent Collection - Preliminaries

91. Abstract Concurrent Collection - CMS

92. Abstract Tracing Algorithm – Reminder ( Yarden Lec.04 ) atomic collectTracing(): rootsTracing(W) scanTracing(W) sweepTracing() rootsTracing(R): for each fld in Roots ref ← *fld if ref ≠ null R ← R + [ref] scanTracing(W): while not isEmpty(W) src ← remove(W) (src) ← (src)+1 if (src) = 1 for each fld in Pointers(src) ref ← *fld if ref ≠ null W ← W + [ref]

93. Abstract Tracing Algorithm – Reminder ( Yarden Lec.04 )

94. The Concurrent Version shared log  () collectTracingInc(): atomic rootsTracing(W) log  () repeat scanTracingInc(W) addOrigins() until ? atomic addOrigins() scanTracingInc(W) sweepTracing() initialization termination

95. The Concurrent Version scanTracingInc(W): while not isEmpty(W) src  remove(W) if ρ (src) = 0 for each fld in Poinrters(src) atomic ref  *fld log  log T if ref ≠ null W  W + [ref] ρ (src)  ρ (src) + 1

96. The Concurrent Version addOrigins(): atomic origins  expose(log) for each src in origins W  W + [src] New(): ref  allocate() atomic ρ (ref)  0 log  log N return ref atomic Write(src, i, new): if src ≠ roots old  src[i] log  log W src[i]  new

97. Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping On-the-fly marking Abstract Concurrent Collection  Summary

98. Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping On-the-fly marking X Abstract Concurrent Collection  Summary

99. Discussion  What are CMS advantages?  Minimize Pause Time/Low Latency  Shared Resources between mutator and collector  What are CMS downsides?  Fragmentation  Promotion Failures  CPU Intensive  When would you use CMS?  Short GC times  Large sets of living objects

100. CMS in Practice  Is being used in JVM (not by default)  -XX:+UseConcMarkSweepGC  It has several flags, I (we) can now understand:  -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode  In practice, the vast majority of CMS are an implementation of mostly-concurrent algorithms with small stw phases.  It does minimize pause time.

101. CMS in Practice

103.  Additional nice metrics and benchmarks: http://blog.mgm-tp.com/2013/12/benchmarking-g1-and- other-java-7-garbage-collectors/

104. Summary  We’ve reviewed CMS algorithm –  Mostly concurrent  Mark and Sweep concurrently  On the fly  No free-lunch  It minimizes pause time, but at the expense of throughput and fragmentations.

105. Thank You! Questions and Feedback?