CLR: Garbage Collection Inside Out Maoni Stephens FUN421 Software Design Engineer Microsoft Corporation

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Basics New Heap Begins with New Generation Accessible References Keep Objects Alive Compacts Referenced Objects Objects Promoted to Older Generation New Allocations Rebuild New Generation

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Generational GC Three generations Most objects die in gen0 Gen1 holds in the in flight data Gen2 holds the long lived data Large object heap Gen0 and Gen1 – ephemeral generations

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

GC Heap Segments Unit of VirtualAlloc When CLR is loaded, initial segments are allocated Additional segments reserved as needed Committed/Decommitted as needed in the segment Deleted when not in use VM hoarding feature in next CLR 1.1 SP and CLR 2.0 – STARTUP_HOARD_GC_VM startup flag

Generations And Segments Start from the ephemeral segment Ephemeral segment becomes a gen2 segment Newly reserved segment becomes ephemeral Many gen2 segments, only one ephemeral

Before GC #0 Before GC #1 Before GC #2 Before GC #100 Before GC #500 Gen0 Before GC #0 Before GC #1 Gen1 Gen0 Before GC #2 Gen2 Gen1 Gen0 Before GC #100 Gen2 Gen1 Gen0 Before GC #500 Gen2 Gen1 Gen0

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Allocation Cheap lock on UP; lock free on MP Moving a pointer forward Clearing the memory for new objects Register for finalization if applicable Objects allocated together are close together

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Collection When Allocation. Not enough space in gen0. Induced GC. By calling System.GC.Collect(). System pressure.

Collection How Suspend managed threads GC Resume managed threads Two phases of GC Mark Compact

Roots Stack Handle table Statics Older generation(s) Finalizer queue

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Large Object Heap For objects that are 85,000 bytes or larger Compacting large objects costs a lot So we only sweep (freelist) Many LOH segments Segments are handled similarly to small object heap segments Collection happens during gen2 GCs

Collection – Cost GC takes time – “% time in GC” counter If objects die in gen0 (survival rate is 0) it’s the ideal situation The longer the object lives before being dead, the worse (with exceptions) Gen0 and gen1 GCs should both be relatively cheap; gen2 GCs could cost a lot LOH – different cost model Temporary large objects could be bad Should reuse if possible

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Concurrent GC Why do we have concurrent GC How it’s done For interactive applications How it’s done Trade some CPU and memory for shorter pause time Only for gen2 Done on a concurrent GC thread How do you get concurrent GC On by default Can be turned off via hosting or config

Server GC Why do we have server GC – for server apps that Have a fairly consistent number of requests Require high scalibility and high throughput How it’s done One thread for each CPU, running at highest priority One ephemeral segment per CPU How do you get server GC Only on via config or hosting Hosts like ASP.NET and SQLCLR use server GC In CLR 1.1 it’s in mscorsvr.dll; in CLR 2.0 it’s in mscorwks.dll

WKS GC SVR GC Where it runs On the user thread that triggered GC On the GC threads running at highest priority On a multi proc machine One small object heap + One LOH N small object heaps + N LOHs On a uni proc machine WKS GC + concurrent GC (if not turned off) WKS GC + concurrent GC OFF

Agenda Basics Generations Segments Allocation Collection Large Object Heap Different Flavors of GC Pinning Tools and Resources

Pinning Why do we need pinning How objects get pinned Interop with unmanaged code How objects get pinned Use of GCHandle of type GCHandleType.Pinned Allowed by language syntax, eg. fixed in c# Args get pinned by the interop frame Fragmentation problem

Fragmentation Problem Caused By Pinning Before GC X Gen1 Gen0 start P After GC X Gen2 P Gen1 start Gen0 start After N more GCs Gen2 P Gen1 P Gen0 start Gen1 start

Without demotion Gen1 start Gen0 start With demotion Gen1 start P Gen0 start With demotion Gen2 Gen1 start P Gen0 start

Without Segment Reuse Before GC After GC Seg0 Seg1 Seg2 Seg3 Eph Seg0 Gen2 Seg0 Seg1 Seg2 Seg3 Gen1 Eph P Gen0 Gen2 Seg0 Seg1 Seg2 Seg3 Old Eph P Gen1 New Eph

With Segment Reuse Before GC After GC Seg0 Seg1 Seg2 Seg3 Eph Seg0 Gen2 Seg0 Seg1 Seg2 Seg3 Gen1 Eph P Gen0 Gen2 Seg0 Seg1 Seg2 P Old Eph Old Seg3, New Eph Gen2 Gen1

What The User Code Can Do To Help Best patterns Pinning for a short time is cheap. Create pinned buffers at the beginning that will stay in regions that are very compacted Create pinned buffers that stay together instead of scattered around Techniques Allocate a pinned big buffer (byte array on the LOH), give out a chunk at a time Allocate a pool of small buffers and hand one out when needed

void M(byte[] b) { // if the buffer is already in gen2 it’s unlikely to move. if(GC.GetGeneration(b) == GC.MaxGeneration || // avoid copying if the buffer is too big b.Length > 8 * 1024) RealM(b); } // GetBuffer will allocate one if none is free. byte[] TempBuffer = Pool.GetBuffer(); RealM(TempBuffer); CopyBackToUserBuffer(TempBuffer, b); Pool.Release(TempBuffer);

Managed Caches Design patterns Cache tuning Don’t do it in the finalizer Use combination of weak and strong references – could convert strong to weak after certain period Cache tuning Hit rate Cost to add an item Cost to find an item Frequency of cleanup If not implemented carefully could easily cause more gen2 collections than needed

// sweep the cache for dead objects and compacts the list. static void sweep_WeakRef() { // don't bother if no new collection happened since // we swept last if (GC.CollectionCount(0) != gen0_count) gen0_count = GC.CollectionCount (0); else return; lock (list) { for (int i = 0; i < WeakRef_fill;) { if (list[i].Target == null) { // install the last element in the free slot. WeakRef_fill--; if (i != WeakRef_fill) { list[i] = list[WeakRef_fill]; list[WeakRef_fill] = null; } i++;

Why We Don’t Allow Many Customized Settings Only allow very few settings specified via hosting API or config Advantages for GC to do the tuning for you GC has intimate knowledge of memory when applications run GC is tested on many platforms + many configurations

Looking Ahead Challenges on 64-bit More common as people start using 64-bit machines VM is practically unlimited so physical memory is the limit Servers like the SQL Server tend to not allow paging Gen2 could take a long time

Tools And Resources http://blogs.msdn.com/maoni/ CLRProfiler .NET CLR Memory performance counters The SoS Debugger Extension for windbg/cdb vadump Task Manager

© 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.