1. Instructor: Haryadi Gunawi
Dynamic Memory Allocation Free-Space Management e.g., malloc(), kmalloc() CSAPP Book Chapter 9.9
Instructor:
Haryadi Gunawi
REAL
EXAMPLES /
NUMBERS
NOTE: The slides that are marked with the above label are slides with real examples/numbers. Slides that don’t have the label only focus on the high-level concepts (conceptual numbers for illustrations only).

2. Q1: How malloc() decides what address to return?
int main() {
int size = 4;
int *p = malloc(size);
int *q = malloc(size);
int *r = malloc(size);
printf(”p points to %08x \n", p);
printf(”q points to %08x \n", q);
printf(”r points to %08x \n", r); }
// Run in CSIL linux.cs.uchicago.edu
Logical View
2^n
Stack
(unused)
Q1: How malloc() decides what address to return?
Q2: Why there is a gap?
0x x30
= 0x20 bytes
= 32 bytes
(but I asked for 4 bytes)
Heap
R = 0160f050
Q = 0160f030
P = 0160f010
Code
(all hex numbers)

3. Stack (unused) P = 0160f010 Heap R = 0160f050 Q = 0160f030
(Horizontal view of the heap)
Heap
P = 0160f010
R = 0160f050
Q = 0160f030
Start of heap
Code

4. Assumptions Made in This Lecture [IMPORTANT!!]
Memory is word addressed
A box = a word = integer size = 4 bytes
malloc(X)  malloc (X words)  malloc (X * 4 bytes)
But in actual C code, malloc(Y)  Y bytes
A block = a block of words allocated by malloc()
1 word
(4 bytes)
An allocated block
(4 words)
16 bytes
A free block
(3 words)
12 bytes
Free word
Allocated word

5. Alignment/padding In CSIL example (first slide): 8-word alignment
Returned pointer is always 32-byte aligned
E.g. 0x…10, … 0x….30, …. 0x50
Last five bits are always 10000
In this lecture: 2-word alignment
Returned pointer by malloc must be 8-byte aligned
E.g. the last 3 bits of the pointer must be always 000
Ex: 0x…00, …08, …10, … 18,
Why alignment/padding?
Unit of memory line (low-level DRAM)
Aligned  1 line access
No-aligned  2 lines accesses  worse performance
R = 0x 0160f050
Q = 0x 0160f030
P = 0x 0160f010
2 words
My data is here
P0x…04
| My data
Is here |

6. Another allocated block
Allocation Example
p1 = malloc(4)
An allocated block
p2 = malloc(5)
Another allocated block
p3 = malloc(6)
free(p2)
A free block
Another
free block
p4 = malloc(2)

7. How to manage free blocks?
Method 1: Implicit list using length—links all blocks
Method 2: Explicit list among the free blocks using pointers
Method 3: Segregated free list
Different free lists for different size classes
Method 4: Blocks sorted by size
Can use a balanced tree (e.g. Red-Black tree) with pointers within each free block, and the length used as a key
(For this class, we just cover the first two)

8. Outline Basic concepts Implicit free lists
(I will just present the concept)
Please read the book for pointer arithmetic and code samples

9. Managing free and allocated blocks
Need block headers
Keep the size and status (free/allocated) of a block in the word preceding the block.
This word is often called the header field or header
Requires an extra word for every block p0
p0 = malloc(4) 5
block size
payload

10. Method 1: Implicit List
For each block we need both size and allocation status
p1 = malloc(3) a
Payload
padding
Size p1
1 word
Payload
Optional
padding
Size a
An allocated block
a = 1: Allocated block
a = 0: Free block
Size: block size
Payload: application data
(in allocated blocks only)
Format of
allocated and
free blocks:

11. Detailed Implicit Free List Example
Padding is needed for this block
Unused
Start of
heap
8/0
16/1
32/0
16/1
0/1
byteSize / alloc?
P=malloc(2 words)
Q=malloc(3 words)
Double-word
aligned
REAL
EXAMPLES /
NUMBERS
Allocated blocks: shaded
Free blocks: unshaded
Headers: labeled with size in bytes/allocated bit

12. Forms an “implicit” list
Unused
Start of
heap
8/0
16/1
32/0
16/1
0/1
Word#:
“Implicit”
List of
Free blocks
Pos: w1
Sz: 8 bytes
Pos: w7
Sz: 32 bytes …

13. Outline Basic concepts Implicit free lists How to find a free block?
Which free block to return?
Policies:
Best fit
Worst fit
First fit
Next fit

14. Best fit Example: malloc (15 bytes) Pros/cons: Best fit:
Search the list, choose the fully-fit or with fewest bytes left over
Example: malloc (15 bytes)
There are 3 free blocks
Return which block?
(the “size” below does not include header size)
Pros/cons:
(+) Keeps fragmentation low—usually improves memory utilization
(-) Slow (must iterate through all free blocks)
Pos: ..
Sz: 5
“Implicit”
List of
Free blocks
Pos: ..
Sz: 10
Pos: ..
Sz: 30
Pos: ..
Sz: 20 20

15. Worst fit Example: malloc (15 bytes) Pros/cons:; Worst fit:
Search the list, choose the largest free block
Example: malloc (15 bytes)
Return which block?
Pros/cons:;
(+) Keeps large free blocks available
(-) Slow (must iterate through all free blocks)
Pos: ..
Sz: 15
“Implicit”
List of
Free blocks
Pos: ..
Sz: 10
Pos: ..
Sz: 30
Pos: ..
Sz: 20 20

16. First fit Example: Pros/cons First fit:
Search list from beginning, choose first free block that fits
Example:
Malloc(15 bytes)
Return which block?
Pros/cons
(+) Faster than best/worst fit
(-) Still slow: can take linear time in total number of blocks (allocated and free)
Most early parts of the heap have been allocated
(-) In practice it can cause “splinters” at beginning of list
Pos: ..
Sz: 15
“Implicit”
List of
Free blocks
Pos: ..
Sz: 10
Pos: ..
Sz: 30
Pos: ..
Sz: 20

17. Next fit Example: Pros/cons Next fit:
Like first fit, but search list starting where previous search finished
Example:
Malloc(15 bytes), return which one?
Malloc(10 bytes), return which one?
Pros/cons
(+) Faster than first fit: avoids re-scanning unhelpful blocks
(-) Some research suggests that fragmentation is worse
Pos: ..
Sz: 10
Pos: ..
Sz: 5
Pos: ..
Sz: 20
“Implicit”
List of
Free blocks
Pos: ..
Sz: 10
Pos: ..
Sz: 20

18. Policies Which policy should I use?
Depends …
No single best policy for all workloads
Must know your customer’s malloc()/free() patterns
In systems, best performance is based on empirical evaluation (not theoretical)

19. Outline
Basic concepts
Implicit free lists
(Cont’d)
For SIMPLICITY of presentation, in the following slides, we will *not* show the block header

20. Allocating in Free Block
Allocating in a free block: splitting
Since allocated space might be smaller than free space, we might want to split the block 4 4 6 2
malloc(4 words) p
A free block 4 2
A new
free
block
allocated

21. Freeing a Block Simplest/naïve implementation:
Need only clear the “allocated” flag
But can lead to “false fragmentation”
Malloc(5)?  error! Oops!
There is enough free space, but the allocator won’t be able to find it
(or must check the next block  more complex) 4 2 p
free(p) 4 2

22. Freeing and Coalescing
Join (coalesce) with next/previous blocks, if they are free
Coalescing with next block 4 4 4 2 2 p
logically
gone
free(p) 4 2 6

23. Coalescing But how do we coalesce with previous block?4 4 4 2 2 p
How to merge with
this free block?
free(p)
Must iterate from beginning
O(N) not O(1)/constant-time
No “back” pointer to previous block
So far, single, one-directional linked list

24. Bidirectional Coalescing
Boundary tags [Knuth73]
Add block footer: replicate block header at “bottom” (end) of free blocks
Allows us to traverse the “list” backwards 4 6 a a
Format of
allocated and
free blocks
Payload and
padding
Size
Size
REAL
EXAMPLES /
NUMBERS
Header
Boundary tag
(footer)

25. Constant Time Coalescing
Allocated
Case 1 p
Allocated
Free
Case 2
Case 3
Free
Case 4
Free
Block being
freed
free(p) p
Allocated
(aargh, you flip the orientation again?)
Yes, above is a vertical view of the heap

26. Coalescing (Case 1) Pos: p-2words p-8bytes allocated block Becomes anm2 m1 1 n 1
Becomes an
unallocated
block
free (p) n 1 m2 1
allocated
block m2 1
This is called
Pointer arithmetic
Pos:
p – 1 word + n bytes
p – 4 byte + n bytes
prevFooter = (void*) p – 8;
nextHeader = (void*) p + (n-4)
REAL
EXAMPLES /
NUMBERS

27. Coalescing (Case 2) allocated Non- allocated m1 1 m1 1 n+m2 m1 1 n 1m1 1 n 1
free (p) n 1 m2
Non-
allocated m2

28. Coalescing (Case 3) Non-allocated Allocated m1 n+m1 m2 1 m1 n 1
n+m1 m2 1 m1 n 1
free (p) n 1 m2 1
Allocated m2 1

29. Coalescing (Case 4) Non-allocated Non-allocated m1 n+m1+m2 m1 n 1
n+m1+m2 m1 n 1
free (p) n 1 m2
Non-allocated m2

30. Outline
Basic concepts
Implicit free lists
Explicit free lists

31. Disadvantages of implicit free list?
Method 1: Implicit free list using length—links all blocks
Method 2: Explicit free list among the free blocks using pointers
No need to hop to allocated blocks
Malloc(5)
Not fit,
Go next
Allocated,
Go Next
Fit! 4 4 6 2 4 2 6
Not fit,
Go next
Fit!
REAL
EXAMPLES /
NUMBERS

32. Block format Maintain list(s) of free blocks, not all blocks
Payload is unused anyway.
Let’s use it for optimization.
Allocated block (same as before)
Size
Next free addr
Prev free addr
Free
Size 1
Payload and
padding
Important note:
Not size (implicit)
It is pointer to next free block
(explicit)
Size 1
Maintain list(s) of free blocks, not all blocks
Goal of malloc() is to manage free space, so it’s about the free list
Luckily we track only free blocks, so we can use payload area
REAL
EXAMPLES /
NUMBERS

33. Explicit Free Lists Logically:
Physically: free blocks can be linked in any order
(Explicit pointers allow different orderings of free blocks)
More powerful
4 bytes
Free
(A)
4 bytes
Free
(B)
6 bytes
Free
(C) 4 6
Forward (next) links
Back (prev) links A B C

34. Allocation (and splitting)
conceptual graphic
Before
A free block
After
(with splitting)
allocated
P = malloc(…)

35. Freeing?
Insertion policy: Where in the free list do you put a newly freed block?
One possible policy: LIFO (last-in-first-out)
Insert freed block at the beginning of the free list

36. Freeing With a LIFO Policy (Case 1)
conceptual graphic
Before
free( )
Root
Insert the freed block at the root of the list
After
Root

37. Freeing With a LIFO Policy (Case 2)
Next adjacent block is also free
Before
free( )
Root
Splice out successor block, coalesce both memory blocks and insert the new block at the root of the list
Root
After

38. Freeing With a LIFO Policy (Case 3)
conceptual graphic
Before
free( )
Root
Previous adjacent
block is also free
Splice out predecessor block, coalesce both memory blocks, and insert the new block at the root of the list
After
Root

39. Freeing With a LIFO Policy (Case 4)
conceptual graphic
Before
free( )
Root
Both adjacent blocks are free
Splice out predecessor and successor blocks, coalesce all 3 memory blocks and insert the new block at the root of the list
Root
After

40. Free-block Insertion policy
LIFO
Pro: simple and constant time
Con: studies suggest fragmentation is worse than address ordered
Another policy? Address-ordered policy
Insert freed blocks so that free list blocks are always in address order: addr(prev) < addr(curr) < addr(next)
Pro: studies suggest fragmentation is lower than LIFO
Con: requires search
Solution: segregated free list (next slide, see book)

41. Other Free-space management solutions
Method 3: Segregated free list
Different free lists for different size classes
Method 4: Blocks sorted by size
Can use a balanced tree (e.g. Red-Black tree) with pointers within each free block, and the length used as a key

42. Outline Basic concepts Implicit free lists Explicit free lists
EXTRA: RECAP – What are we doing?

43. Constraints Applications Allocators
Can issue arbitrary sequence of malloc and free requests
free request must be to a malloc’d block
Allocators
Can’t control number or size of allocated blocks
Must respond immediately to malloc requests
i.e., can’t reorder or buffer requests
Must allocate blocks from free memory
i.e., can only place allocated blocks in free memory
Must align blocks so they satisfy all alignment requirements
8-byte (x86) or 16-byte (x86-64) alignment on Linux boxes
Can manipulate and modify only free memory
Can’t move the allocated blocks once they are malloc’d
i.e., compaction is not allowed
A low-level memory management
(Unlike high-level data structures)

44. Performance Goal: Throughput
Given some sequence of malloc and free requests:
R0, R1, ..., Rk, ... , Rn-1
Goals: maximize throughput and peak memory utilization
These goals are often conflicting
Throughput:
Number of completed requests per unit time
Example:
5,000 malloc calls and 5,000 free calls in 10 seconds
Throughput is 1,000 operations/second
Malloc() and free()
must be fast

45. Performance Goal: Peak Memory Utilization
Given some sequence of malloc and free requests:
R0, R1, ..., Rk, ... , Rn-1
Def: Aggregate payload Pk
malloc(p) results in a block with a payload of p bytes
After request Rk has completed, the aggregate payload Pk is the sum of currently allocated payloads
Def: Current heap size Hk
Assume Hk is monotonically nondecreasing
i.e., heap only grows when allocator uses sbrk
Def: Peak memory utilization after k+1 requests
Uk = ( maxi<=k Pi ) / Hk

46. Internal Fragmentation
For a given block, internal fragmentation occurs if payload is smaller than block size
Caused by
Overhead of maintaining heap data structures
Padding for alignment purposes
Explicit policy decisions (e.g., to return a big block to satisfy a small request)
Depends only on the pattern of previous requests
Thus, easy to measure
Block
Internal
fragmentation
Payload
Internal
fragmentation
Must be
space
efficient

47. External Fragmentation
Occurs when there is enough aggregate heap memory, but no single free block is large enough
Depends on the pattern of future requests
Thus, difficult to measure
p1 = malloc(4)
p2 = malloc(5)
p3 = malloc(6)
free(p2)
Error! Oops! Can’t proceed!
p4 = malloc(6)
Must have good
allocation policy

48. EXTRA

49. The malloc Package #include <stdlib.h> void *malloc(size_t size)
Successful:
Returns a pointer to a memory block of at least size bytes aligned to an 8-byte (x86) or 16-byte (x86-64) boundary
If size == 0, returns NULL
Unsuccessful: returns NULL (0) and sets errno
void free(void *p)
Returns the block pointed at by p to pool of available memory
p must come from a previous call to malloc or realloc
Other functions
calloc: Version of malloc that initializes allocated block to zero.
realloc: Changes the size of a previously allocated block.
sbrk: Used internally by allocators to grow or shrink the heap

50. Implicit Lists: Summary
Implementation: very simple
Allocate cost:
linear time worst case
Free cost:
constant time worst case
even with coalescing
Memory usage:
will depend on placement policy
First-fit, next-fit or best-fit
Not used in practice for malloc/free because of linear-time allocation
used in many special purpose applications
However, the concepts of splitting and boundary tag coalescing are general to all allocators

51. Explicit List Summary Comparison to implicit list:
Allocate is linear time in number of free blocks instead of all blocks
Much faster when most of the memory is full
Slightly more complicated allocate and free since needs to splice blocks in and out of the list
Some extra space for the links (2 extra words needed for each block)
Does this increase internal fragmentation?
Most common use of linked lists is in conjunction with segregated free lists
Keep multiple linked lists of different size classes, or possibly for different types of objects

52. Dynamic Memory Allocation
Allocator maintains heap as collection of variable sized blocks, which are either allocated or free
Types of allocators
Explicit allocator: application allocates and frees space
E.g., malloc and free in C
Implicit allocator: application allocates, but does not free space
E.g. garbage collection in Java, ML, and Lisp
Will discuss simple explicit memory allocation today

53. Constraints Applications Allocators
Can issue arbitrary sequence of malloc and free requests
free request must be to a malloc’d block
Allocators
Can’t control number or size of allocated blocks
Must respond immediately to malloc requests
i.e., can’t reorder or buffer requests
Must allocate blocks from free memory
i.e., can only place allocated blocks in free memory
Must align blocks so they satisfy all alignment requirements
8-byte (x86) or 16-byte (x86-64) alignment on Linux boxes
Can manipulate and modify only free memory
Can’t move the allocated blocks once they are malloc’d
i.e., compaction is not allowed

54. Performance Goal: Throughput
Given some sequence of malloc and free requests:
R0, R1, ..., Rk, ... , Rn-1
Goals: maximize throughput and peak memory utilization
These goals are often conflicting
Throughput:
Number of completed requests per unit time
Example:
5,000 malloc calls and 5,000 free calls in 10 seconds
Throughput is 1,000 operations/second

55. Performance Goal: Peak Memory Utilization
Given some sequence of malloc and free requests:
R0, R1, ..., Rk, ... , Rn-1
Def: Aggregate payload Pk
malloc(p) results in a block with a payload of p bytes
After request Rk has completed, the aggregate payload Pk is the sum of currently allocated payloads
Def: Current heap size Hk
Assume Hk is monotonically nondecreasing
i.e., heap only grows when allocator uses sbrk
Def: Peak memory utilization after k+1 requests
Uk = ( maxi<=k Pi ) / Hk