1. Real-time Garbage Collection Presented by Boris Dogadov 20/01/15 1

2. Roadmap What is Real-time Garbage Collection (RTGC) Work-based Time-based Slack-based Tax and Spend Summery and Conclusion 2

3. Is it real? ?How is real-time gc possible in production software? ?Why use gc language in the real-time software? ?What does real-time gc even mean? Real RT-GC GC What is Real-time Garbage Collection (RTGC) 3

4. It is not a myth! Java-based Synthesizer What is Real-time Garbage Collection (RTGC)4.NET Micro Framework Java Platform, Micro Edition Wi-Fi coffee maker NetDuino Arm JAviator (w/ Salzburg)

5. Real-time systems What is Real-time Garbage Collection (RTGC)5 Introduce constrains on response time, to defined events. ‘ Any information processing system which has to respond to externally generated input stimuli within a finite and specified period ’ Hard Deadline misses results in total failures Firm Infrequent deadline misses are tolerable (results are useless) Soft Deadline misses degrade QoS (results usefulness degrades with time)

6. Real-time Garbage Collection What is Real-time Garbage Collection (RTGC)6 Challenges Bounding the Mutator time (minimum application time slack) Bounding the Collector time (maximum pause time) Space constrains

7. Roadmap What is Real-time Garbage Collection (RTGC) Work-based real-time collection Time-based Slack-based Tax and Spend Summery and Conclusion 7

8. Copying GC - Reminder Work-based real-time collection8 From SpaceTo Space Roots A C A D C B E

9. Work based GC - Baker Work-based real-time collection9 Baker (1978) Classic copying GC Introduces space and time bounds analysis Given fixed size objects The bounds are very impractical

10. Real time GC – Properties Work-based real-time collection10 The most common parameter for RT-GC was application Pause Time For example 100ms pause time is not suitable for applications with 50ms response time This is not enough, why?

11. Real time GC – Properties Work-based real-time collection11 Most common parameter for RT-GC was application Pause Time For example 100ms pause time is not suitable for applications with 50ms response time

12. Real time GC – Properties Work-based real-time collection12 Most common parameter for RT-GC was application Pause Time For example 100ms pause time is not suitable for applications with 50ms response time This is not enough, why? Very frequent short pauses – not different from one long pause

13. Blelloch and Cheng Work-based real-time collection13 Luckily of all of us, Blelloch and Cheng first introduce a new term “Minimum mutator utilization” (2001)

14. Blelloch and Cheng Work-based real-time collection14 Luckily of all of us, Blelloch and Cheng first introduce a new term “Minimum mutator utilization” (2001) At any time window (up to some granularity t), portion of mutator work has a low- bound t is the pause time

15. Blelloch and Cheng Work-based real-time collection15 Luckily of all of us, Blelloch and Cheng first introduce a new term “Minimum mutator utilization” (2001) At any time window (up to some granularity t), portion of mutator work has a low- bound t is the pause time Example Mouse tracking requires response time of 50ms Mouse tracking handling code requires 1ms Sufficient to have minimum utilization time of 2% at granularity of 50ms

16. Blelloch and Cheng Work-based real-time collection16 Assumptions Memory accesses are sequentially consistent

17. Blelloch and Cheng Work-based real-time collection17 Assumptions Memory accesses are sequentially consistent Every New (with n fields) is followed by n Invocations of InitSlot()

18. Blelloch and Cheng Work-based real-time collection18 Assumptions Memory accesses are sequentially consistent Every New (with n fields) is followed by n Invocations of InitSlot() Object cannot be used before all InitSlot() were completed

19. Blelloch and Cheng Work-based real-time collection19 Assumptions Memory accesses are sequentially consistent Every New (with n fields) is followed by n Invocations of InitSlot() Object cannot be used before all InitSlot() were completed New() cannot interfere with other New()

20. Blelloch and Cheng Work-based real-time collection20 Assumptions Memory accesses are sequentially consistent Every New (with n fields) is followed by n Invocations of InitSlot() Object cannot be used before all InitSlot() were completed New() cannot interfere with other New() Read/Write can interfere with New

21. Blelloch and Cheng Work-based real-time collection21 Assumptions Memory accesses are sequentially consistent Every New (with n fields) is followed by n Invocations of InitSlot() Object cannot be used before all InitSlot() were completed New() cannot interfere with other New() Read/Write can interfere with New Writes are atomic

22. Blelloch and Cheng Work-based real-time collection22 From Space To Space allocated copy stack free sharedStack free toBot toTop fromTop fromBot

23. Mutator: Allocate(n) Work-based real-time collection23 allocated copy stack free sharedStack free toBot toTop fromTop fromBot

24. Mutator: Allocate(n) Work-based real-time collection24 allocated copy stack free sharedStack free toBot toTop fromTop fromBot

25. Mutator: Allocate(n) Work-based real-time collection25 allocated copy stack free sharedStack free toBot toTop fromTop fromBot

26. Mutator: New(n) Work-based real-time collection26 From Space To Space

27. Mutator: New(n) Work-based real-time collection27 From Space To Space FA = null null

28. Mutator: New(n) Work-based real-time collection28 From Space To Space FA = null null CopyCount null

29. Mutator: New(n) Work-based real-time collection29 From Space To Space FA = r null CopyCount null

30. Mutator: New(n) Work-based real-time collection30 From Space To Space FA = r null CopyCount = 0 null

31. Mutator: InitSlot(v) Work-based real-time collection31 From Space To Space FA = r null CopyCount = 0 null v

32. Mutator: InitSlot(v) Work-based real-time collection32 From Space To Space FA = r null v CopyCount = 0 null v

33. Mutator: InitSlot(v) Work-based real-time collection33 From Space To Space FA = r null v CopyCount = 0 null v v

34. Mutator: InitSlot(v) Work-based real-time collection34 From Space To Space FA = r null v CopyCount = 0 null v v v

35. Mutator: MakeGray Work-based real-time collection35 Work Stack From Space To Space FA = null X Y Z Item1 Item2 XYZ

36. Mutator: MakeGray Work-based real-time collection36 Work Stack From Space To Space FA = null X Y Z Item1 Item2 XYZ TAS

37. Mutator: MakeGray Work-based real-time collection37 Work Stack From Space To Space FA = null X Y Z Item1 Item2 XYZ TAS

38. Mutator: MakeGray Work-based real-time collection38 Work Stack From Space To Space FA = null X Y Z Item1 Item2 XYZ TAS CopyCount = 0 null

39. Mutator: MakeGray Work-based real-time collection39 Work Stack From Space To Space FA = null X Y Z Item1 Item2 XYZ TAS CopyCount = 3 null

40. Mutator: MakeGray Work-based real-time collection40 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ TAS CopyCount = 3 null

41. Mutator: MakeGray Work-based real-time collection41 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ TAS CopyCount = 3 null P

42. Mutator: CopyOneSlot(p) Work-based real-time collection42 Work Stack From Space To Space FA = r X Y Z XYZ CopyCount = 3 null P Item1 Item2

43. Mutator: CopyOneSlot(p) Work-based real-time collection43 Work Stack From Space To Space FA = r X Y Z XYZ CopyCount = -3 null P 3 Item1 Item2

44. Mutator: CopyOneSlot(p) Work-based real-time collection44 Work Stack From Space To Space FA = r X Y Z XYZ CopyCount = -3 null P 2 z -3 Item1 Item2

45. Mutator: CopyOneSlot(p) Work-based real-time collection45 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ CopyCount = -3 null P 2 z -3 Z

46. Mutator: CopyOneSlot(p) Work-based real-time collection46 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ CopyCount = -3 null Z P 2 z -3 Z

47. Mutator: CopyOneSlot(p) Work-based real-time collection47 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ CopyCount = 2 null Z P 2 z -3 Z 2

48. Mutator: CopyOneSlot(p) Work-based real-time collection48 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ CopyCount = 2 null Z P 2 z -3 Z 2 2

49. Mutator: Write (GC is on) Work-based real-time collection49 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ CopyCount = 3 null V

50. Mutator: Write (GC is on) Work-based real-time collection50 Work Stack From Space To Space FA = r X Y Z Item1 Item2 XYZ CopyCount = 3 null Y V

51. Mutator: Write (GC is on) Work-based real-time collection51 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 3 null Y V

52. Mutator: Write (GC is on) Work-based real-time collection52 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 3 null Y V

53. Mutator: Write (GC is on) Work-based real-time collection53 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 3 null Y V Wait until the object is copied

54. Mutator: Write (GC is on) Work-based real-time collection54 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 3 null Z Y V Wait until the slot is copied Z

55. Mutator: Write (GC is on) Work-based real-time collection55 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = -2 null Z Y V Wait until the slot is copied Z 1

56. Mutator: Write (GC is on) Work-based real-time collection56 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 1 null Y Z Y V Z Wait until the slot is copied

57. Mutator: Write (GC is on) Work-based real-time collection57 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 1 null Y Z Y V V Z

58. Mutator: Write (GC is on) Work-based real-time collection58 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 3 null V Z Y V V Z

59. Mutator: Write (GC is on) Work-based real-time collection59 Work Stack From Space To Space FA = r X V Z Item1 Item2 XYZ CopyCount = 3 null V Z Y V V Z

60. Reminder: Rooms (Lec. 8 Elinor Ainy) Work-based real-time collection60 All ‘Popers’ wait until the pop-room is available Pop fixed (usually) amount of work Wait until everyone finished poping

61. Reminder: Rooms (Lec. 8 Elinor Ainy) Work-based real-time collection61 All ‘Popers’ wait until the pop-room is available Pop fixed (usually) amount of work Wait until everyone finished poping Enter the Push room Push the work

62. Reminder: Rooms (Lec. 8 Elinor Ainy) Work-based real-time collection62 All ‘Popers’ wait until the pop-room is available Pop fixed (usually) amount of work Wait until everyone finished poping Enter the Push room Push the work The last thread checks if the stack is empty The collection is off

63. Mutator: Collect(k) Work-based real-time collection63

64. Mutator: Sync() Work-based real-time collection64

65. Mutator: CollectorOn() Work-based real-time collection65

66. Mutator: CollectorOff() Work-based real-time collection66

67. Blelloch and Cheng - Analysis Work-based real-time collection67 Space Bounds 2 * (R(1 + 2 / K) + N + 5 * P *D) words P := #Processors R := Maximum reachable memory used N := #Reachable object D := Maximum depth of any object K := Mutator copying Time Bounds C*K time for each Mutator operation

68. Blelloch and Cheng - Analysis Work-based real-time collection68 Performance 51% overhead compared to stop-the-world collector Minimum Mutator utilization 10% for k = 2 15% for k = 1.2 Maximum pause time 3-4 ms

69. Blelloch and Cheng – Cons Work-based real-time collection69 Worst case execution time Is very far from average case The problem is in write – why?

70. Blelloch and Cheng – Cons Work-based real-time collection70 Worst case execution time Is very far from average case The problem is in write – why? Big difference between regular writes and during-GC writes

71. Blelloch and Cheng – Cons Work-based real-time collection71 Worst case execution time Is very far from average case The problem is in write – why? Big difference between regular writes and during-GC writes

72. Blelloch and Cheng – Cons Work-based real-time collection72 Worst case execution time Is very far from average case The problem is in write – why? Big difference between regular writes and during-GC writes Causes over-provisioning

73. Roadmap What is Real-time Garbage Collection (RTGC) Work-based real-time collection Slack-based real-time collection Time-based algorithm Tax and Spend Summery and Conclusion 73

74. Slack-based GC - Henriksson Slack-based real-time collection74 Brooks style concurrent copying collector New scheduling approach Introduce a new real-time task Collector task Mutators do not perform any mutator work any longer RT tasks GC task Low priority task Task priority

75. Henriksson Slack-based real-time collection75 GC is not running during real-time tasks Low-priority tasks perform some of GC work Special high-priority GC task

76. Reminder: C oncurrent Copying Collector (Lec. 11 Hanan Rofe Haim) Slack-based real-time collection76 Not concurrent: The collector ran while the mutator was halted. Concurrent: The collector must, while the mutator is running, be able to: 1. Copy objects from From-space to To-space. 2. Update pointers to new location of objects.

77. Slack-based GC - Henriksson Slack-based real-time collection77 Brooks style concurrent copying collector New object are allocated at the top of ToSpace Old objects are evacuated to the bottom of ToSpace Out of memory? - perform a flip To Space Evacuated objects Allocated objects

78. Slack-based GC - Henriksson Slack-based real-time collection78 As allocation proceeds, Evacuation must also proceed In order to be able to perform a flip To Space Evacuated objects Allocated objects

79. Henriksson Slack-based real-time collection79

80. Henriksson Slack-based real-time collection80 To Space Evacuated objects Allocated objects

81. Henriksson Slack-based real-time collection81 If RT task scheduled just before a flip, H must be big enough to hold all task allocations. Requires developer input To Space Evacuated objects Allocated objects

82. Henriksson Slack-based real-time collection82 Pros. Much better bounds for worst-case scenarios Real-time threads run without GC interference Cons. Requires developer’s input Maximum memory footprint Maximum new allocations per task Fragile under overload

83. Roadmap What is Real-time Garbage Collection (RTGC) Work-based real-time collection Slack-based real-time collection Time-based algorithm Tax and Spend Summery and Conclusion 83

84. Metronome Time-based real-time collection84 Bacon et.al. 2003 Incremental mark and sweep On-demand compaction

85. Metronome Time-based real-time collection85 Divides the processor time into short quanta Maintains minimum Mutator utilization percentage

86. Metronome - Scheduling Time-based real-time collection86 Alarm thread Heartbeat mechanism The highest priority Decides whether to suspend the Mutator and wake GC GC thread Runs not more than a quantum Goes to sleep if there is not work or quantum exceeded Wait for the Mutators to stop, via a handshake

87. Metronome - Problems Time-based real-time collection87 Quanta and fragmentation Problem: Allocating big objects (usually arrays) Solution: Use arraylets

88. Metronome - Problems Time-based real-time collection88 Read barrier overhead (to-space invariant. Brooks) Problem: About 20% overhead Solution: Make it 4% overhead

89. Metronome - Analysis Time-based real-time collection89 Partial Mutator time

90. Metronome - Analysis Time-based real-time collection90

91. Metronome - Analysis Time-based real-time collection91 Pros Short pause time Well defined Mutator utilization High Mutator utilization Cons Complex tuning Clock speed Input parameters Works best on uniprocessors and small multiprocessor Has to stop all the Mutators

92. Roadmap What is Real-time Garbage Collection (RTGC) Work-based real-time collection Slack-based real-time collection Time-based algorithm Tax and Spend Summery and Conclusion 92

93. Tax and Spend 93 Auerbach et al [2008] Combine three approaches Work based (Blelloch and Cheng) Slack based (Henriksson) Time based (Metronome)

94. Tax and Spend 94 Underlying GC properties Incremental Both Mutators and Collector perform Collector’s work Concurrent Collector can run with the Mutator Parallel For multiprocessors support

95. Tax and Spend 95 Introduce tax bank Taxes Mutators performing collectors workCollectors Mutators

96. Tax and Spend – time based taxation Tax and Spend96 Each Collector (parallel collectors) Contributes to the shared tax credit Available tax credit defines the tax rate implied on Mutators

97. Tax and Spend – work based taxation Tax and Spend97 Each Mutator Is scheduled according to the minimum mutator utilization parameter For each N units of allocation perform c*N units for GC work Tax rate depends on the credit available Have the same tax rate Simple rules work best at practice If there is enough credit Mutator may choose to not perform any work May have unique utilization ratio

98. Tax and Spend - Metronome Time-based real-time collection98 Make it concurrent and parallel Introduce taxing system Tax the mutators

99. Tax and Spend - Metronome Tax and Spend 99 Synchronizing all together – Epochs mechanism Reach global consensus like install particular write barrier Or notifying the collector that each mutator has drained it’s store buffer A single atomic monotonic global epoch number Per thread local epochs Always less than or equal to global epoch Each time a thread reaches a safe-point: Uses global epoch number as its new local epoch

100. Tax and Spend - Metronome Tax and Spend100 Synchronizing all together – Last-Man-Out mechanism Phase Marking Sweeping Finalizing … Global atomic “current workers” counter The last one switches the phase if needed Epochs are not efficient here Taxed mutators are not distinguished from others

101. Tax and Spend - Metronome Tax and Spend101 Scheduling

102. Tax and Spend - Metronome Tax and Spend102 Performance

103. GPU Garbage Collection 103 What????

104. GPU Garbage Collection 104 GPU is a loafer – Let is collect garbage GPUs 100-1000 cores Variuous types of cores Used for all sort of interesting tasks, CUDA, OpenCL

105. GPU Garbage Collection 105 Dr. Ronald Veldema and Prof. Michale Philippsen Introduce Mark&Sweep on GPU Wrapper around CUDA Allocator/Collector written in CUDA

106. GPU Garbage Collection 106

107. GPU Garbage Collection 107 Martin Maas, Philip Reames, Jeffrey Morlan, Krste Asanovic, Anthony D. Joseph, John Kubiatowicz Explore the previous idea further Offload the Garbage Collection to GPU

108. GPU Garbage Collection 108 1.Maintain a reference graph 2.Offload it to GPU for marking 3.CPU performs the sweeping

109. GPU Garbage Collection 109 There is an experimental working version! Variation of Jikes RVM (Research Virtual Machine) https://github.com/preames/gpu-garbage-collection

110. 110