1 Dynamic Memory Allocation: Basic Concepts Andrew Case Slides adapted from Jinyang Li, Randy Bryant and Dave O’Hallaro

2 Topics Basic concepts & challenges Implicit free lists

3 Why dynamic allocator? We’ve discussed how exactly systems handle two types of data allocation so far:  Data segments (global variables in.data/.bss)  Stack-allocated (local variables) Not sufficient!  What about data whose whose size is only known during runtime?  Need to control lifetime of allocated data?  E.g. a linked list that grows and shrinks

4 Dynamic Memory Allocation Heap (via malloc ) Program text (.text ) Initialized data (.data ) Uninitialized data (. bss ) User stack 0 Holds local variables Holds global variables Grown/shrunk by malloc library, holds dynamically- allocated data

5 Example usage: a simple linked list typedef struct item_t { int val; item_t *next; }item_t; item_t *head = NULL; void insert(int v) { item_t *x; x = (item_t *)malloc(sizeof(item_t)); if (x == NULL) { exit(1);} x->val = v; x->next = head; head = x; } void delete(int v) { item_t *x; if (head && head->val == v) { x = head; head = head->next; free(x); } void printlist() { item_t *x; x = head; printf(“list contents: “); while (x!=NULL) { printf(“%d “, x->val); x = x->next; } int main() { while (1) { if (fscanf(stdin, “%c %d\n”, &c, &v)!=2) { break; } if (c == ‘i’) { insert(v); }else if (c == ‘d’) { delete(v); } printlist(); } Returns a pointer to a memory block of size bytes, Returns memory block to allocator, x must come from previous calls to malloc linux%./a.out i 5 i 10 i 20 d 20 q list contents: 10 5

6 Dynamic Memory Allocation Application Dynamic Memory Allocator OS mallocfreerealloc mmap sbrk Allocate or free data of arbitrary sizes Request for or give back large chunk of page- aligned address space Dynamic memory allocator is part of user-level library. Why not implement its functionality in the kernel?

7 Types of Dynamic Memory Allocator Allocator maintains heap as collection of variable sized blocks, which are either allocated or freed Two types of allocators… Explicit allocator (used by C/C++): application allocates and frees space  malloc and free in C  new and delete in C++ Implicit allocator (used by Java,…): application allocates, but does not free space  Garbage collection in Java, Python etc. This class

8 Challenges facing a memory allocator Applications  Can issue arbitrary sequence of malloc and free requests  free request must be to a malloc ’d block Achieve good throughput  malloc/free calls should return quickly Achieve good memory utilization  Apps issue arbitrary sequence of malloc/free requests of arbitrary sizes  Utilization = sum of malloc’d data / size of heap

9 Challenges facing a memory allocator Constraints Allocators  Can’t control number or size of allocated blocks  Must respond immediately to malloc requests  i.e., can’t reorder or buffer requests  Must allocate blocks from free memory  i.e., can only place allocated blocks in free memory  Must align blocks so they satisfy all alignment requirements  8 byte alignment for GNU malloc ( libc malloc ) on Linux boxes  Can manipulate and modify only free memory  Can’t move the allocated blocks once they are malloc ’d  i.e., compaction is not allowed

10 Good Throughput Given some sequence of malloc and free requests:  R 0, R 1,..., R k,..., R n-1 Goals: maximize throughput and peak memory utilization  These goals are often conflicting Throughput:  Number of completed requests per unit time  Example:  5,000 malloc calls and 5,000 free calls in 10 seconds  Throughput is 1,000 operations/second

11 Good Utilization Given some sequence of malloc and free requests:  R 0, R 1,..., R k,..., R n-1 P k :Aggregate payload  malloc(p) results in a block with a payload of p bytes  After request R k has completed, the aggregate payload P k is the sum of currently allocated payloads H k : Current heap size - Assume H k is nondecreasing  i.e., heap only grows when allocator uses sbrk Def: Peak memory utilization after k requests  U k = ( max i<k P i ) / H k

12 Challenge #1 Design an algorithm that maximizes throughput. Design an algorithm that maximizes usage. Designing an algorithm that balances both.

13 Assumptions made in these lectures Memory is word addressed (each word can hold a pointer) Allocated block (4 words) Free block (3 words) Free word Allocated word

14 Allocation Example p1 = malloc(4) p2 = malloc(5) p3 = malloc(6) free(p2) p4 = malloc(2)

15 Fragmentation Poor memory utilization caused by fragmentation  internal fragmentation  external fragmentation

16 Internal Fragmentation Malloc allocates data from ``blocks’’ of certain sizes. Internal fragmentation occurs if payload is smaller than block size May be caused by  Limited choices of block sizes  Padding for alignment purposes  Other space overheads 100 byte Payload Internal fragmentation Block of 128-byte

17 External Fragmentation Occurs when there is enough aggregate heap memory, but no single free block is large enough Depends on the pattern of future requests  Thus, difficult to measure p1 = malloc(4) p2 = malloc(5) p3 = malloc(6) free(p2) p4 = malloc(6) Oops! (what would happen now?)

18 Challenge #2 How do we know how much memory to free given just a pointer? How do we keep track of the free blocks? What do we do with the extra space when allocating a structure that is smaller than the free block it is placed in? How do we pick a block to use for allocation -- many might fit? How do we reinsert freed block into available memory?

19 Knowing How Much to Free Standard method  Keep the length of a block in the header field preceding the block  Requires an extra word for every allocated block p0 = malloc(4) p0 free(p0) block sizedata 5

20 Keeping Track of Free Blocks Method 1: Implicit list using length—links all blocks Method 2: Explicit list among the free blocks using pointers Method 3: Segregated free list  Different free lists for different size classes Method 4: Blocks sorted by size  Can use a balanced tree (e.g. Red-Black tree) with pointers within each free block, and the length used as a key

21 Topics Basic concepts Implicit free lists

22 Method 1: Implicit List Heap is divided into variable-sized blocks Each block has a header containing size and allocation status  some low-order address bits are always 0  Instead of storing an always-0 bit, use it as a allocated/free flag  When reading size word, must mask out this bit Size 1 word Format of allocated and free blocks Payload Size: block size a = 1: Allocated block a = 0: Free block Payload: application data (allocated blocks only) a Optional padding header

23 Implicit List: Allocating in Free Block Allocating in a free block: splitting  Since allocated space might be smaller than free space, we might want to split the block malloc(3) p 2 4 space allocated split free space free space being allocated p

24 Implicit List: Freeing a Block Simplest implementation:  Need only clear the “allocated” flag  Problem?  Leads to “false fragmentation” free(p) p malloc(5) Oops! There is enough free space, but the allocator won’t be able to find it

25 Implicit List: Coalescing Join (coalesce) with next/previous blocks, if they are free  Coalescing with next block free(p) p logically gone 4426 Check if next block is free How to coalesce with a previous block?

26 Implicit List: Bidirectional Coalescing Boundary tags  Replicate size/allocated word at “bottom” (end) of free blocks  Allows us to traverse the “list” backwards, but requires extra space What does this look like?  Doubly Linked List. Size Format of allocated and free blocks Payload and padding a = 1: Allocated block a = 0: Free block Size: Total block size Payload: Application data (allocated blocks only) a Sizea Boundary tag (footer) Header

27 Constant Time Coalescing Allocated Free Allocated Free Block being freed Case 1Case 2Case 3Case 4

28 m11 Constant Time Coalescing (Case 1) m11 n1 n1 m21 1 m11 1 n0 n0 m21 1

29 m11 Constant Time Coalescing (Case 2) m11 n+m20 0 m11 1 n1 n1 m20 0

30 m10 Constant Time Coalescing (Case 3) m10 n1 n1 m21 1 n+m10 0 m21 1

31 m10 Constant Time Coalescing (Case 4) m10 n1 n1 m20 0 n+m1+m20 0

32 Additional requirements: alignment Start of heap Double word aligned 8/016/1 32/00/1 Headers: labeled with size in bytes/allocated bit Allocated blocks: shaded Free blocks Unusable

33 Implicit List: Finding a Free Block First fit:  Search from beginning, choose first free block that fits:  Time taken?  Linear time for number of blocks (both allocated and free) Next fit:  Like first fit, except search starts where previous search finished  Faster than the first?  Should be faster: avoids re-scanning unhelpful blocks  Some research suggests that fragmentation is worse Best fit:  Search the list, choose the best free block: fewest wasted bits  Keeps fragments to a minimum (best utilization)  Speed?  Will usually run slower than either

34 Key Allocator Policies Placement policy:  First-fit, next-fit, best-fit, etc.  Trades off lower throughput for less fragmentation Splitting policy:  When do we go ahead and split free blocks?  How much internal fragmentation are we willing to tolerate? Coalescing policy:  Immediate coalescing: coalesce each time free is called  Deferred coalescing: try to improve performance of free by deferring coalescing until needed. Examples:  Coalesce as you scan the free list for malloc  Coalesce when the amount of external fragmentation reaches some threshold

35 Implicit Lists: Summary Implementation: very simple Allocate cost:  linear time worst case Free cost:  constant time worst case Memory usage:  will depend on placement policy  first-fit, next-fit or best-fit Not used in practice for malloc/free because of linear- time allocation  used in many special purpose applications