1. CSC 660: Advanced Operating Systems
CSC 660: Advanced OS
Process Address Space
CSC 660: Advanced Operating Systems

2. CSC 660: Advanced Operating Systems
Topics
Process Address Space
Virtual Memory Areas
Page Tables
Page Fault Handler
CSC 660: Advanced Operating Systems

3. CSC 660: Advanced Operating Systems
User vs. Kernel Memory
Process requests for memory non-urgent.
Process will not immediately use all of the requested memory.
Kernel defers process memory requests until necessary.
User processes cannot be trusted.
Kernel must catch addressing errors
CSC 660: Advanced Operating Systems

4. CSC 660: Advanced Operating Systems
Process Address Space
Code 0x
Data
Heap
Stack
Env/Argv
0xBFFFFFFF
CSC 660: Advanced Operating Systems

5. CSC 660: Advanced Operating Systems
mm_struct
Type
Name
Purpose
vm_area_struct
*mmap
List of VMAs
struct rb_root
mm_rb
Tree of VMAs
pgd_t
*pgd
Page global directory
struct list_head
mmlist
List of mm_structs
atomic_t
mm_users
number of users
mm_count
primary usage count
unsigned long
start_code
start of code region
end_code
end of code region
<linux/sched.h>
CSC 660: Advanced Operating Systems

6. CSC 660: Advanced Operating Systems
mm_struct
How to access?
current->mm
What about kernel threads?
Kernel mapping identical for all tasks.
current->mm is NULL.
Use space of prev task: current->active_mm
How to allocate?
allocate_mm() uses mm_cachep slab cache.
CSC 660: Advanced Operating Systems

7. CSC 660: Advanced Operating Systems
Virtual Memory Areas
Each VMA is a unique memory object.
Contiguous page-aligned memory space.
VMA has a single set of permissions.
Common VMAs in a process space
text (code)
data, bss
zero
shared libs
mmap, shared memory, heap
<linux/mm.h>
CSC 660: Advanced Operating Systems

8. CSC 660: Advanced Operating Systems
Virtual Memory Areas
CSC 660: Advanced Operating Systems

9. CSC 660: Advanced Operating Systems
vm_area_struct
Type
Name
Purpose
struct mm_struct
*mm
Associated addr spc.
unsigned long
vm_start
Start (inclusive)
vm_end
End (exclusive)
vm_area_struct
*vm_next
List of VMAs
pgprot_t
vm_page_prot
Access permissions
vm_flags
VMA flags
struct file
*vm_file
Mapped file (if any)
<linux/sched.h>
CSC 660: Advanced Operating Systems

10. CSC 660: Advanced Operating Systems
VMA Flags
Flag
Effect
VM_READ
Pages can be read from
VM_WRITE
Pages can be written to
VM_EXEC
Pages can be executed
VM_SHARED
Pages are shared btw processes.
VM_IO
Mapping of device I/O space.
VM_RESERVED
Area must not be paged to disk.
VM_LOCKED
Pages in region are locked.
VM_GROWSUP
VMA can grow upward.
VM_GROWSDOWN
VMA can grown downward.
CSC 660: Advanced Operating Systems

11. CSC 660: Advanced Operating Systems
Hardware and Flags
VMA flags mapped to MMU hardware.
x86 without PAE
Two bits: Read/Write, User/Supervisor
Read implies Execute.
Write implies Read.
x86 with PAE
NX flag implies no execute.
CSC 660: Advanced Operating Systems

12. CSC 660: Advanced Operating Systems
VMA Operations
void open(struct vm_area_struct *area)
Invoked with VMA added to an address space.
void close(struct vm_area_struct *area)
Invoked when removed from an address space.
struct page *nopage(struct vm_area_struct *area, unsigned long address, int unused)
Invoked by page fault handler when a page not present in physical memory is accessed.
CSC 660: Advanced Operating Systems

13. CSC 660: Advanced Operating Systems
Viewing Process VMAs
e0000 r-xp : /bin/bash
080e e6000 rw-p : /bin/bash
080e rw-p 080e :00 0
b7ceb000-b7cf3000 r-xp : /lib/tls/i686/cmov/libnss_files so
b7cf3000-b7cf4000 rw-p : /lib/tls/i686/cmov/libnss_files so
b7cf4000-b7cfc000 r-xp : /lib/tls/i686/cmov/libnss_nis so
b7cfc000-b7cfd000 rw-p : /lib/tls/i686/cmov/libnss_nis so
b7cfd000-b7d0e000 r-xp : /lib/tls/i686/cmov/libnsl so
b7d0e000-b7d0f000 rw-p : /lib/tls/i686/cmov/libnsl so
b7e6f000-b7f91000 r-xp : /lib/tls/i686/cmov/libc so
b7f91000-b7f9a000 rw-p : /lib/tls/i686/cmov/libc so
b7f9a000-b7f9c000 rw-p b7f9a000 00:00 0
b7f9c000-b7f9e000 r-xp : /lib/tls/i686/cmov/libdl so
b7f9e000-b7f9f000 rw-p : /lib/tls/i686/cmov/libdl so
b7f9f000-b7fd3000 r-xp : /lib/libncurses.so.5.4
b7fd3000-b7fdb000 rw-p : /lib/libncurses.so.5.4
b7fdb000-b7fdc000 rw-p b7fdb000 00:00 0
b7fe9000-b7feb000 rw-p b7fe :00 0
b7feb000-b r-xp : /lib/ld so
b b rw-p : /lib/ld so
bffeb000-c rw-p bffeb000 00:00 0
ffffe000-fffff p :00 0
CSC 660: Advanced Operating Systems

14. CSC 660: Advanced Operating Systems
Viewing Process VMAs
K r-x-- /bash
080e K rw--- /bash
080e K rw [ anon ]
b7ceb K r-x-- /libnss_files so
b7cf K rw--- /libnss_files so
b7cf K r-x-- /libnss_nis so
b7cfc K rw--- /libnss_nis so
b7cfd K r-x-- /libnsl so
b7d0e K rw--- /libnsl so
b7e6f K r-x-- /libc so
b7f K rw--- /libc so
b7f9a K rw [ anon ]
b7f9c K r-x-- /libdl so
b7f9e K rw--- /libdl so
b7f9f K r-x-- /libncurses.so.5.4
b7fd K rw--- /libncurses.so.5.4
b7fdb K rw [ anon ]
b7fe K rw [ anon ]
b7feb K r-x-- /ld so
b K rw--- /ld so
bffeb K rw [ stack ]
ffffe K [ anon ]
CSC 660: Advanced Operating Systems

15. CSC 660: Advanced Operating Systems
Page Tables
MMU translates virtual addresses to physical.
Uses page tables to perform translation.
Operating system must manage page tables.
Each process has its own page table.
Addresses divided into two parts
Page number (p) – index into page table.
Page offset (d) – offset within page. p d 19 31
CSC 660: Advanced Operating Systems

16. CSC 660: Advanced Operating Systems
Page Tables
CSC 660: Advanced Operating Systems

17. Multi-level Page Tables
How large is the page table?
32-bit address space, 4K pages.
220 entries, 32-bits/entry = 4M
Much worse on 64-bit architectures.
Most page table entries are unused.
Solution
Multi-level page tables.
Divide address into one part for each level of tables plus one part for the offset.
CSC 660: Advanced Operating Systems

18. Linux 4-level Page Table
Converted from 3- to 4-levels in
CSC 660: Advanced Operating Systems

19. CSC 660: Advanced Operating Systems
x86 Page Tables
x86 paging
Page directory (10), page table (10), offset (12)
x86 extended paging (4MB pages)
Page table (10), offset (22)
PAE paging
Page Directory Pointer Table allows 36-bit phys addresses.
Logical addresses 32-bits in size: 4GB process limit remains.
x86-64 paging
Global(9), Upper(9), Lower(9), Table(9), Offset(12)
48-bit addressing allows 256TB of address space.
CSC 660: Advanced Operating Systems

20. Translation Lookaside Buffers
Address translation is expensive.
4-level: each virt addr => 5 memory accesses
Solution: TLB
Caches address translations.
Flushed during a context switch.
CSC 660: Advanced Operating Systems

21. CSC 660: Advanced Operating Systems
Page Cache
Primary kernel disk cache.
Each page consists of multiple block buffers.
Page types: files, block devices, swap space
Reads and writes use cache except O_DIRECT
pdflush daemon
Write operations deferred by page cache until:
Free memory shrinks below specified threshold.
Dirty data older than a specified threshold.
Multiple threads to avoid device congestion.
CSC 660: Advanced Operating Systems

22. CSC 660: Advanced Operating Systems
Page Lifecycle
Page is read into memory from disk.
File read, mmap, read ahead, page fault.
Page is made dirty by process writing to it.
Page is removed from cache
Dirty page expires or kernel needs memory.
Non-dirty pages can just be de-allocated.
Dirty pages have to be written back to disk.
CSC 660: Advanced Operating Systems

23. CSC 660: Advanced Operating Systems
Page Fault Handler
Causes of page faults
Attempt to access a virtual address that’s not mapped to a physical page.
Attempt to access a virtual address in a way that’s forbidden (ex: write to ro page.)
CPU raises page fault exception on fault.
Kernel calls do_page_fault()
CSC 660: Advanced Operating Systems

24. CSC 660: Advanced Operating Systems
Page Fault Handler
CSC 660: Advanced Operating Systems

25. CSC 660: Advanced Operating Systems
do_page_fault()
If page fault from kernel
If address is valid, handle page fault.
If address invalid, kernel oops.
If interrupt context, kernel oops.
else page fault from user process
If address is NULL or kernel, seg fault.
If address not in a VMA, seg fault.
Except if add just below stack, grow stack.
If permissions do not permit action, seg fault.
arch/i386/mm/fault.c
CSC 660: Advanced Operating Systems

26. CSC 660: Advanced Operating Systems
References
Daniel P. Bovet and Marco Cesati, Understanding the Linux Kernel, 3rd edition, O’Reilly, 2005.
Mel Gorman, Understanding the Linux Virtual Memory Manager, Prentice Hall, 2004.
Robert Love, Linux Kernel Development, 2nd edition, Prentice-Hall, 2005.
Claudia Rodriguez et al, The Linux Kernel Primer, Prentice-Hall, 2005.
Avi Silberchatz, Operating System Concepts, 7th edition, Wiley, 2004.
Andrew S. Tanenbaum, Modern Operating Systems, 3rd edition, Prentice-Hall, 2005.
CSC 660: Advanced Operating Systems