Instructor: Haryadi Gunawi Dynamic Memory Allocation Free-Space Management e.g., malloc(), kmalloc() CSAPP Book Chapter 9.9 Instructor: Haryadi Gunawi REAL EXAMPLES / NUMBERS NOTE: The slides that are marked with the above label are slides with real examples/numbers. Slides that don’t have the label only focus on the high-level concepts (conceptual numbers for illustrations only).

Q1: How malloc() decides what address to return? int main() { int size = 4; int *p = malloc(size); int *q = malloc(size); int *r = malloc(size); printf(”p points to %08x \n", p); printf(”q points to %08x \n", q); printf(”r points to %08x \n", r); } // Run in CSIL linux.cs.uchicago.edu Logical View 2^n Stack (unused) Q1: How malloc() decides what address to return? Q2: Why there is a gap? 0x50 -- 0x30 = 0x20 bytes = 32 bytes (but I asked for 4 bytes) Heap R = 0160f050 Q = 0160f030 P = 0160f010 Code (all hex numbers)

Stack (unused) P = 0160f010 Heap R = 0160f050 Q = 0160f030 (Horizontal view of the heap) Heap P = 0160f010 R = 0160f050 Q = 0160f030 Start of heap Code

Assumptions Made in This Lecture [IMPORTANT!!] Memory is word addressed A box = a word = integer size = 4 bytes malloc(X)  malloc (X words)  malloc (X * 4 bytes) But in actual C code, malloc(Y)  Y bytes A block = a block of words allocated by malloc() 1 word (4 bytes) An allocated block (4 words) 16 bytes A free block (3 words) 12 bytes Free word Allocated word

Alignment/padding In CSIL example (first slide): 8-word alignment Returned pointer is always 32-byte aligned E.g. 0x…10, … 0x….30, …. 0x50 Last five bits are always 10000 In this lecture: 2-word alignment Returned pointer by malloc must be 8-byte aligned E.g. the last 3 bits of the pointer must be always 000 Ex: 0x…00, …08, …10, … 18, Why alignment/padding? Unit of memory line (low-level DRAM) Aligned  1 line access No-aligned  2 lines accesses  worse performance R = 0x 0160f050 Q = 0x 0160f030 P = 0x 0160f010 2 words My data is here P0x…04 | My data Is here |

Another allocated block Allocation Example p1 = malloc(4) An allocated block p2 = malloc(5) Another allocated block p3 = malloc(6) free(p2) A free block Another free block p4 = malloc(2)

How to manage free blocks? Method 1: Implicit list using length—links all blocks Method 2: Explicit list among the free blocks using pointers Method 3: Segregated free list Different free lists for different size classes Method 4: Blocks sorted by size Can use a balanced tree (e.g. Red-Black tree) with pointers within each free block, and the length used as a key (For this class, we just cover the first two)

Outline Basic concepts Implicit free lists (I will just present the concept) Please read the book for pointer arithmetic and code samples

Managing free and allocated blocks Need block headers Keep the size and status (free/allocated) of a block in the word preceding the block. This word is often called the header field or header Requires an extra word for every block p0 p0 = malloc(4) 5 block size payload

Method 1: Implicit List For each block we need both size and allocation status p1 = malloc(3) a Payload padding Size p1 1 word Payload Optional padding Size a An allocated block a = 1: Allocated block a = 0: Free block Size: block size Payload: application data (in allocated blocks only) Format of allocated and free blocks:

Detailed Implicit Free List Example Padding is needed for this block Unused Start of heap 8/0 16/1 32/0 16/1 0/1 byteSize / alloc? P=malloc(2 words) Q=malloc(3 words) Double-word aligned REAL EXAMPLES / NUMBERS Allocated blocks: shaded Free blocks: unshaded Headers: labeled with size in bytes/allocated bit

Forms an “implicit” list Unused Start of heap 8/0 16/1 32/0 16/1 0/1 Word#: 0 1 2 3 4 5 6 7 8 9 10 11 13 15 16 17 “Implicit” List of Free blocks Pos: w1 Sz: 8 bytes Pos: w7 Sz: 32 bytes …

Outline Basic concepts Implicit free lists How to find a free block? Which free block to return? Policies: Best fit Worst fit First fit Next fit

Best fit Example: malloc (15 bytes) Pros/cons: Best fit: Search the list, choose the fully-fit or with fewest bytes left over Example: malloc (15 bytes) There are 3 free blocks Return which block? (the “size” below does not include header size) Pros/cons: (+) Keeps fragmentation low—usually improves memory utilization (-) Slow (must iterate through all free blocks) Pos: .. Sz: 5 “Implicit” List of Free blocks Pos: .. Sz: 10 Pos: .. Sz: 30 Pos: .. Sz: 20 20

Worst fit Example: malloc (15 bytes) Pros/cons:; Worst fit: Search the list, choose the largest free block Example: malloc (15 bytes) Return which block? Pros/cons:; (+) Keeps large free blocks available (-) Slow (must iterate through all free blocks) Pos: .. Sz: 15 “Implicit” List of Free blocks Pos: .. Sz: 10 Pos: .. Sz: 30 Pos: .. Sz: 20 20

First fit Example: Pros/cons First fit: Search list from beginning, choose first free block that fits Example: Malloc(15 bytes) Return which block? Pros/cons (+) Faster than best/worst fit (-) Still slow: can take linear time in total number of blocks (allocated and free) Most early parts of the heap have been allocated (-) In practice it can cause “splinters” at beginning of list Pos: .. Sz: 15 “Implicit” List of Free blocks Pos: .. Sz: 10 Pos: .. Sz: 30 Pos: .. Sz: 20

Next fit Example: Pros/cons Next fit: Like first fit, but search list starting where previous search finished Example: Malloc(15 bytes), return which one? Malloc(10 bytes), return which one? Pros/cons (+) Faster than first fit: avoids re-scanning unhelpful blocks (-) Some research suggests that fragmentation is worse Pos: .. Sz: 10 Pos: .. Sz: 5 Pos: .. Sz: 20 “Implicit” List of Free blocks Pos: .. Sz: 10 Pos: .. Sz: 20

Policies Which policy should I use? Depends … No single best policy for all workloads Must know your customer’s malloc()/free() patterns In systems, best performance is based on empirical evaluation (not theoretical)

Outline Basic concepts Implicit free lists (Cont’d) For SIMPLICITY of presentation, in the following slides, we will *not* show the block header

Allocating in Free Block Allocating in a free block: splitting Since allocated space might be smaller than free space, we might want to split the block 4 4 6 2 malloc(4 words) p A free block 4 2 A new free block allocated

Freeing a Block Simplest/naïve implementation: Need only clear the “allocated” flag But can lead to “false fragmentation” Malloc(5)?  error! Oops! There is enough free space, but the allocator won’t be able to find it (or must check the next block  more complex) 4 2 p free(p) 4 2

Freeing and Coalescing Join (coalesce) with next/previous blocks, if they are free Coalescing with next block 4 4 4 2 2 p logically gone free(p) 4 2 6

Coalescing But how do we coalesce with previous block? 4 4 4 2 2 p How to merge with this free block? free(p) Must iterate from beginning O(N) not O(1)/constant-time No “back” pointer to previous block So far, single, one-directional linked list

Bidirectional Coalescing Boundary tags [Knuth73] Add block footer: replicate block header at “bottom” (end) of free blocks Allows us to traverse the “list” backwards 4 6 a a Format of allocated and free blocks Payload and padding Size Size REAL EXAMPLES / NUMBERS Header Boundary tag (footer)

Constant Time Coalescing Allocated Case 1 p Allocated Free Case 2 Case 3 Free Case 4 Free Block being freed free(p) p Allocated (aargh, you flip the orientation again?) Yes, above is a vertical view of the heap

Coalescing (Case 1) Pos: p-2words p-8bytes allocated block Becomes an m2 m1 1 n 1 Becomes an unallocated block free (p) n 1 m2 1 allocated block m2 1 This is called Pointer arithmetic Pos: p – 1 word + n bytes p – 4 byte + n bytes prevFooter = (void*) p – 8; nextHeader = (void*) p + (n-4) REAL EXAMPLES / NUMBERS

Coalescing (Case 2) allocated Non- allocated m1 1 m1 1 n+m2 m1 1 n 1 m1 1 n 1 free (p) n 1 m2 Non- allocated m2

Coalescing (Case 3) Non-allocated Allocated m1 n+m1 m2 1 m1 n 1 n+m1 m2 1 m1 n 1 free (p) n 1 m2 1 Allocated m2 1

Coalescing (Case 4) Non-allocated Non-allocated m1 n+m1+m2 m1 n 1 n+m1+m2 m1 n 1 free (p) n 1 m2 Non-allocated m2

Outline Basic concepts Implicit free lists Explicit free lists

Disadvantages of implicit free list? Method 1: Implicit free list using length—links all blocks Method 2: Explicit free list among the free blocks using pointers No need to hop to allocated blocks Malloc(5) Not fit, Go next Allocated, Go Next Fit! 4 4 6 2 4 2 6 Not fit, Go next Fit! REAL EXAMPLES / NUMBERS

Block format Maintain list(s) of free blocks, not all blocks Payload is unused anyway. Let’s use it for optimization. Allocated block (same as before) Size Next free addr Prev free addr Free Size 1 Payload and padding Important note: Not size (implicit) It is pointer to next free block (explicit) Size 1 Maintain list(s) of free blocks, not all blocks Goal of malloc() is to manage free space, so it’s about the free list Luckily we track only free blocks, so we can use payload area REAL EXAMPLES / NUMBERS

Explicit Free Lists Logically: Physically: free blocks can be linked in any order (Explicit pointers allow different orderings of free blocks) More powerful 4 bytes Free (A) 4 bytes Free (B) 6 bytes Free (C) 4 6 Forward (next) links Back (prev) links A B C

Allocation (and splitting) conceptual graphic Before A free block After (with splitting) allocated P = malloc(…)

Freeing? Insertion policy: Where in the free list do you put a newly freed block? One possible policy: LIFO (last-in-first-out) Insert freed block at the beginning of the free list

Freeing With a LIFO Policy (Case 1) conceptual graphic Before free( ) Root Insert the freed block at the root of the list After Root

Freeing With a LIFO Policy (Case 2) Next adjacent block is also free Before free( ) Root Splice out successor block, coalesce both memory blocks and insert the new block at the root of the list Root After

Freeing With a LIFO Policy (Case 3) conceptual graphic Before free( ) Root Previous adjacent block is also free Splice out predecessor block, coalesce both memory blocks, and insert the new block at the root of the list After Root

Freeing With a LIFO Policy (Case 4) conceptual graphic Before free( ) Root Both adjacent blocks are free Splice out predecessor and successor blocks, coalesce all 3 memory blocks and insert the new block at the root of the list Root After

Free-block Insertion policy LIFO Pro: simple and constant time Con: studies suggest fragmentation is worse than address ordered Another policy? Address-ordered policy Insert freed blocks so that free list blocks are always in address order: addr(prev) < addr(curr) < addr(next) Pro: studies suggest fragmentation is lower than LIFO Con: requires search Solution: segregated free list (next slide, see book)

Other Free-space management solutions Method 3: Segregated free list Different free lists for different size classes Method 4: Blocks sorted by size Can use a balanced tree (e.g. Red-Black tree) with pointers within each free block, and the length used as a key

Outline Basic concepts Implicit free lists Explicit free lists EXTRA: RECAP – What are we doing?

Constraints Applications Allocators Can issue arbitrary sequence of malloc and free requests free request must be to a malloc’d block Allocators Can’t control number or size of allocated blocks Must respond immediately to malloc requests i.e., can’t reorder or buffer requests Must allocate blocks from free memory i.e., can only place allocated blocks in free memory Must align blocks so they satisfy all alignment requirements 8-byte (x86) or 16-byte (x86-64) alignment on Linux boxes Can manipulate and modify only free memory Can’t move the allocated blocks once they are malloc’d i.e., compaction is not allowed A low-level memory management (Unlike high-level data structures)

Performance Goal: Throughput Given some sequence of malloc and free requests: R0, R1, ..., Rk, ... , Rn-1 Goals: maximize throughput and peak memory utilization These goals are often conflicting Throughput: Number of completed requests per unit time Example: 5,000 malloc calls and 5,000 free calls in 10 seconds Throughput is 1,000 operations/second Malloc() and free() must be fast

Performance Goal: Peak Memory Utilization Given some sequence of malloc and free requests: R0, R1, ..., Rk, ... , Rn-1 Def: Aggregate payload Pk malloc(p) results in a block with a payload of p bytes After request Rk has completed, the aggregate payload Pk is the sum of currently allocated payloads Def: Current heap size Hk Assume Hk is monotonically nondecreasing i.e., heap only grows when allocator uses sbrk Def: Peak memory utilization after k+1 requests Uk = ( maxi<=k Pi ) / Hk

Internal Fragmentation For a given block, internal fragmentation occurs if payload is smaller than block size Caused by Overhead of maintaining heap data structures Padding for alignment purposes Explicit policy decisions (e.g., to return a big block to satisfy a small request) Depends only on the pattern of previous requests Thus, easy to measure Block Internal fragmentation Payload Internal fragmentation Must be space efficient

External Fragmentation Occurs when there is enough aggregate heap memory, but no single free block is large enough Depends on the pattern of future requests Thus, difficult to measure p1 = malloc(4) p2 = malloc(5) p3 = malloc(6) free(p2) Error! Oops! Can’t proceed! p4 = malloc(6) Must have good allocation policy

EXTRA

The malloc Package #include <stdlib.h> void *malloc(size_t size) Successful: Returns a pointer to a memory block of at least size bytes aligned to an 8-byte (x86) or 16-byte (x86-64) boundary If size == 0, returns NULL Unsuccessful: returns NULL (0) and sets errno void free(void *p) Returns the block pointed at by p to pool of available memory p must come from a previous call to malloc or realloc Other functions calloc: Version of malloc that initializes allocated block to zero. realloc: Changes the size of a previously allocated block. sbrk: Used internally by allocators to grow or shrink the heap

Implicit Lists: Summary Implementation: very simple Allocate cost: linear time worst case Free cost: constant time worst case even with coalescing Memory usage: will depend on placement policy First-fit, next-fit or best-fit Not used in practice for malloc/free because of linear-time allocation used in many special purpose applications However, the concepts of splitting and boundary tag coalescing are general to all allocators

Explicit List Summary Comparison to implicit list: Allocate is linear time in number of free blocks instead of all blocks Much faster when most of the memory is full Slightly more complicated allocate and free since needs to splice blocks in and out of the list Some extra space for the links (2 extra words needed for each block) Does this increase internal fragmentation? Most common use of linked lists is in conjunction with segregated free lists Keep multiple linked lists of different size classes, or possibly for different types of objects

Dynamic Memory Allocation Allocator maintains heap as collection of variable sized blocks, which are either allocated or free Types of allocators Explicit allocator: application allocates and frees space E.g., malloc and free in C Implicit allocator: application allocates, but does not free space E.g. garbage collection in Java, ML, and Lisp Will discuss simple explicit memory allocation today

Constraints Applications Allocators Can issue arbitrary sequence of malloc and free requests free request must be to a malloc’d block Allocators Can’t control number or size of allocated blocks Must respond immediately to malloc requests i.e., can’t reorder or buffer requests Must allocate blocks from free memory i.e., can only place allocated blocks in free memory Must align blocks so they satisfy all alignment requirements 8-byte (x86) or 16-byte (x86-64) alignment on Linux boxes Can manipulate and modify only free memory Can’t move the allocated blocks once they are malloc’d i.e., compaction is not allowed

Performance Goal: Throughput Given some sequence of malloc and free requests: R0, R1, ..., Rk, ... , Rn-1 Goals: maximize throughput and peak memory utilization These goals are often conflicting Throughput: Number of completed requests per unit time Example: 5,000 malloc calls and 5,000 free calls in 10 seconds Throughput is 1,000 operations/second

Performance Goal: Peak Memory Utilization Given some sequence of malloc and free requests: R0, R1, ..., Rk, ... , Rn-1 Def: Aggregate payload Pk malloc(p) results in a block with a payload of p bytes After request Rk has completed, the aggregate payload Pk is the sum of currently allocated payloads Def: Current heap size Hk Assume Hk is monotonically nondecreasing i.e., heap only grows when allocator uses sbrk Def: Peak memory utilization after k+1 requests Uk = ( maxi<=k Pi ) / Hk