Complutense University, Madrid. Spain. * IMEC, Heverlee. Belgium. SEMINARIO 5-Marzo-2004 Garbage Collectors Refinement for New Dynamic Multimedia Applications on Embedded Systems Garbage Collectors Refinement for New Dynamic Multimedia Applications on Embedded Systems José M. Velasco, David Atienza, Francky Catthoor*, Francisco Tirado, Katzalin Olcoz, José M. Mendias

Seminario 5 Marzo Motivation  New application domains of embedded systems: Mobile terminals ( all sort of multimedia applications) Mobile terminals ( all sort of multimedia applications) AutomotiveAutomotive Game processors, Game processors, PDA, MP3 Player, etc.PDA, MP3 Player, etc.  Java is present everywhere. Java is a General Purpose OOP Language !!!!!! Java is a General Purpose OOP Language !!!!!!

Seminario 5 Marzo Java Popularity  Portability  JVM  OOP  Reusable Code  Dynamic Class Loading  Security  Multi-Thread  Automatic Memory Management

Seminario 5 Marzo Manual MM versus Automatic MM C++ malloc ( ) Free ( )  Destroy object Java Java { Class1 Object = new Class1 Class1 Object = new Class1 }  Take reference out from stack }  Take reference out from stack

Seminario 5 Marzo Automatic Memory Management Stack Pointer 2 Pointer 1 Heap Object 2 Object 1

Seminario 5 Marzo Automatic Memory Management Stack Pointer 1 Heap Unreacheable Object 1

Seminario 5 Marzo Automatic Memory Management Stack Pointer 3 Pointer 1 Heap Object 3 GARBAGE Object 1

Seminario 5 Marzo Manual MM versus Automatic MM Alive Internal Frag. External Frag. Alive Dead Internal Frag. C ++ Java

Seminario 5 Marzo Java

10 Virtual Machine Organization Compiler Subsystem Bytecodes.class JVM Dynamic class loading:.class Thread and synchronization Subsystem Run-Time Subsystem Memory Managment Subsystem

Seminario 5 Marzo Execution Profile

Seminario 5 Marzo A Mono-Application Virtual Machine JVM App. 1 App. 2 App. 3 Memory JVM JVM

Seminario 5 Marzo Automatic Memory Management  Allocation. It is the process of assigning resources of memory when requested by the program.  Garbage Collection. It is the automatic recycling of this dynamically allocated memory when it is no longer needed.  A lot of GC strategies  Reference Count, Mark&Sweep, M&S compact, Copying, Generational, Older First, Connectivity, Incremental,... All of them focused in one Objective.  Simplicity, performance, real time...All of them focused in one Objective.  Simplicity, performance, real time...  Why GC Refinements are important? Performance. Memory Footprint. Real Time requirements. Energy consumption. Tradeoff

Seminario 5 Marzo Presentation Overview  Garbage Collection Distinguishing Garbage.Distinguishing Garbage. Reclaiming Garbage.Reclaiming Garbage. AllocationAllocation Heap Organization.Heap Organization. Write BarriersWrite Barriers  The goal: JVM on Embedded SystemsJVM on Embedded Systems Concurrent Multimedia applicationsConcurrent Multimedia applications  Our proposal Multi-Application JVMMulti-Application JVM Custom GCsCustom GCs Adaptive techniquesAdaptive techniques  Future Work

Seminario 5 Marzo Distinguishing Garbage Phase.

Seminario 5 Marzo Distinguishing Garbage Phase. Reference Count. 2 StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Reference Count. 1 StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Reference Count. 0 StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Reference Count StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Reference Count StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Tracing Collection. StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Tracing Collection. StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Tracing Collection. StackHeap

Seminario 5 Marzo Distinguishing Garbage Phase. Tracing Collection. StackHeap

Seminario 5 Marzo Cache Behavior during Tracing Collection Cache Misses Misses Misses Misses

Seminario 5 Marzo Reclaiming Garbage Phase

Seminario 5 Marzo Reclaiming Garbage Phase. Tracing Collection. StackHeap Free lists

Seminario 5 Marzo Free Lists Pool

Seminario 5 Marzo Free Lists -- Fragmentation Pool

Seminario 5 Marzo Semispace Copying Policy Stack Pointer 2 Pointer1Semispace

Seminario 5 Marzo Semispace Copying Policy Stack Pointer 2 Pointer1Semispace

Seminario 5 Marzo Semispace Copying Policy Stack Pointer 2 Pointer1Allocation Semispace Free Semispace

Seminario 5 Marzo BumpPointer

Seminario 5 Marzo BumpPointer

Seminario 5 Marzo BumpPointer

Seminario 5 Marzo BumpPointer

Seminario 5 Marzo  Mark&Sweep. Free List.Free List. Fragmentation. Fragmentation. Poor Locality. Poor Locality. Small Heap size?. Small Heap size?.  Copying Policy Bump Pointer.Bump Pointer. Available memory / 2. Available memory / 2. High Cost for copying. High Cost for copying. Big Heap Size?. Big Heap Size?.

Seminario 5 Marzo Execution Mode  Stop-the-World. Parallel.  Concurrent.  Incremental Program Program ProgramProgram Program ProgramGC GC GC GCGCGC ProgramProgramProgram ProgramProgramProgram

Seminario 5 Marzo Barriers Collected SubHeap SubHeap Not Collected SubHeap Remember

Seminario 5 Marzo Heap Organization Copying policy Mark&SweepMark&Sweep Large Objects Heap

Seminario 5 Marzo Mark&Sweep

Seminario 5 Marzo SemiSpace Copying Policy

Seminario 5 Marzo Copy + Mark&Sweep

Seminario 5 Marzo Generational Garbage Collection Strategy Generational Collection. Weak Generational Hypothesis.Weak Generational Hypothesis. Java Objects Lifetime. Java Objects Lifetime. AdultYoungQuasi-Inmortal 60% 60%10%30% Java Distribution Gauss Distribution

Seminario 5 Marzo Generational Collection  The memory is divided into multiple subheaps with varying scavenge frequencies  Objects begin their life in the higher collection frequency area.  The objects that survive a number of collections move to a less frequently collected area.  This collector avoids to copy repeatedly the inmortal or quasi-inmortal objects. Mature Nursery Two generations Heap

Seminario 5 Marzo Generational – Mature  Mark&Sweep Policy

Seminario 5 Marzo Generational – Mature  Copying Policy

Seminario 5 Marzo Pauses Distribution through Execution Time

Seminario 5 Marzo Minimum Program Utilization

Seminario 5 Marzo Java on Embedded Systems Minimum Heap Size  Max Number of living ObjectsMinimum Heap Size  Max Number of living Objects Copying  x2 Copying  x2 Mark&Sweep  + Frag Mark&Sweep  + Frag Soft Real Time  PerformanceSoft Real Time  Performance Pause Time  MPU Pause Time  MPU Write Barriers Write Barriers  Incremental  Concurrent Tracing  Cache Misses Tracing  Cache Misses Mono-Application Virtual MachineMono-Application Virtual Machine

Seminario 5 Marzo Embedded Systems  Multimedia Applications  Internal Dynamism Unpredictable data behavior.Unpredictable data behavior.  External Dynamism Several applications working concurrently.Several applications working concurrently. Different scenarios of available memory.Different scenarios of available memory. How can we handle these dynamic systems?

Seminario 5 Marzo Our proposal

Seminario 5 Marzo A Multi-Application Virtual Machine JVM App. 1 App. 2 App. 3 Min Min x 3 Min x 1.5 Memory

Seminario 5 Marzo A Multi-Application Virtual Machine JVM App. 1 App. 2 App. 3 Min x 2.5 Min Min x 2 Memory

Seminario 5 Marzo A Multi-Application Virtual Machine JVM App. 1 App. 2 App. 3 Min Min x 1.5 Min x 4 Memory

Seminario 5 Marzo Overview of Our Approach In this paper we propose a new method that allows the design of custom GCs for new dynamic applications (e.g. multimedia), taking into account the application behavior and the constraints of the final embedded device. - Analysis (mainly profiling) of the application - Characterizing DM behaviour pattern

Seminario 5 Marzo Overview of Our Approach Application Design Search Space  All possible decisions to select an optimal GC This Search Space is formalized as a set of orthogonal trees with basic interdependencies a set of orthogonal trees with basic interdependencies

Seminario 5 Marzo Overview of Our Approach Application We propose a new system-level method to explore the vast design search space of possible garbage collector mechanisms, Our long term Goal is to develop an automatic tool to traverse the trees and produce a tailor made GC for a specific multimedia application with a tradeoff between performance, memory footprint, real time requirements and energy consumption. Our long term Goal is to develop an automatic tool to traverse the trees and produce a tailor made GC for a specific multimedia application with a tradeoff between performance, memory footprint, real time requirements and energy consumption.

Seminario 5 Marzo Overview of Our Approach Application GCRefinements As a result of our Refinements As a result of our Refinements a set of modules that form a custom GC a set of modules that form a custom GC These modules are linked to the Java Virtual Machine that will run over the final device

Seminario 5 Marzo Overview of Our Approach Application GCRefinements Linked Modules Linked Modules The custom GC can be a mixed hardware/ Software implementation. The custom GC can be a mixed hardware/ Software implementation. - Write Barriers. - Real-Time Restrictions. -Hardware Design Tool.

Seminario 5 Marzo Overview of Our Approach Application GCRefinements Linked Modules Linked Modules Dedicated Hardware And finally, a convenient custom GC design for a specific embedded system without a complex engineering process.

Seminario 5 Marzo Overview of Our Approach – Matisse Project (IMEC) Application GCRefinements Linked Modules Linked Modules Dedicated Hardware MicroProcessorFPGA

Seminario 5 Marzo Summary of our proposed approach Main objective: - New design method to design custom GCs for new dynamic applications (e.g. multimedia), taking into account the application behavior and the constraints of the final embedded device. Phases: - Analysis (mainly profiling) of the application - taking into account the application behavior and the constraints of the final embedded device. - Systematic exploration of possible GC solutions from structured (orthogonalized) design space, taking into account the application behavior and the constraints of the final embedded device. - Implementation of promising GC solutions and empirical evaluation of them.

Seminario 5 Marzo Our Design Space  Design Search Space of Orthogonal Decisions for Garbage Collectors AllocationAllocation Distinguishing Garbage.Distinguishing Garbage. Reclaiming Garbage.Reclaiming Garbage. Heap Organization.Heap Organization. Write BarriersWrite Barriers  Any Combination of decisions in Orthogonal trees  Valid Solution  All possible solutions in the search space should be spanned by a combination of leaves in the orthogonal trees.

Seminario 5 Marzo The different zones distinguished by our custom GCs A-ZoneB-ZoneC-Zone

Seminario 5 Marzo Experimental Results.

Seminario 5 Marzo Experimental Results.

Seminario 5 Marzo Adaptive Techniques

Seminario 5 Marzo Minimum Heap Size  Different Benchmarks Minimum Heap Size  Different Benchmarks

Seminario 5 Marzo Appel Collector

Seminario 5 Marzo Adaptive Management of Reserved Space

Seminario 5 Marzo Dynamic Management of Nursery Organization. Jack.

Seminario 5 Marzo Dynamic Strategies  Dynamic “Worst Case” strategy.  Dynamic “Average” strategy.

Seminario 5 Marzo Mispredictions

Seminario 5 Marzo Nursery Threshold

Seminario 5 Marzo Experimental Results  Metrics: Amount of Copied memory.Amount of Copied memory. Number of Collections.Number of Collections. Global Collection Time.Global Collection Time. Execution Time.Execution Time.

Seminario 5 Marzo Amount of copied memory Jess Jack

Seminario 5 Marzo Summary

Seminario 5 Marzo Reduction in the number of collections. Jess

Seminario 5 Marzo Summary

Seminario 5 Marzo Total Collection Time SpeedUp

Seminario 5 Marzo Summary

Seminario 5 Marzo Execution Time SpeedUp

Seminario 5 Marzo Summary

Seminario 5 Marzo Conclusions Generational Collector that Dynamically:Generational Collector that Dynamically: Readjust the reserved space in the nursery. Readjust the reserved space in the nursery. Changes the nursery threshold that triggers a Full Heap Collection. Changes the nursery threshold that triggers a Full Heap Collection. Novel Field of Research.Novel Field of Research. Can be used jointly with others strategies based in Copying Policy.Can be used jointly with others strategies based in Copying Policy.

Seminario 5 Marzo Current work  Incremental Copy

Seminario 5 Marzo Write Barriers  Remembered Sets LOS Root Set

Seminario 5 Marzo Write Barriers  Remembered Sets LOS Root Set

Seminario 5 Marzo Is Not Complete !!! Is Not Complete !!!

Seminario 5 Marzo Is Not Complete !!!

Seminario 5 Marzo Adaptive  Incremental

Seminario 5 Marzo Future Work – BrainStorming Heap CompressionHeap Compression Selective Selective  Object fields  References  Generations Compiler AnalysisCompiler Analysis Objects  Used Fields Objects  Used Fields Fixed Regions  Stack Fixed Regions  Stack Data Connectivity Data Connectivity Hardware SupportHardware Support Write Barriers Write Barriers Compression Compression Tracing Tracing On-Line ProfilingOn-Line Profiling Class behavior Class behavior Data Connectivity behavior Data Connectivity behavior Gehringer & Chang Mushroom (Wolczko&Williams) L1 L2Co-processorReferenceCounting L1Nursery TaggedMemoryFixed regions regions