OSE 2011– processes & address spaces (lec2) 1 Operating Systems Engineering Processes & Address Spaces By Dan Tsafrir

OSE 2011– processes & address spaces (lec2) 2 Reminder: goal of “processes”  Give each process a private memory area  For code, data, stack  Illusion of its own dedicated machine  Prevent each process from reading/writing outside its address space  But allow sharing when needed  Bring a program to life

OSE 2011– processes & address spaces (lec2) 3 HW & OS collaboration  OS manages processes  (De)allocate their physical memory (create, grow, remove)  Switch between them (multiplex HW)  Keep track of them when they’re not executing  Configure HW  HW performs address translation & protection  Translate user addresses to physical addresses  Detect & prevent accesses outside address space  Allow cross-space transfers (system calls, interrupts, …)  Also  OS needs its own address space  OS should be able to easily read/write user memory

OSE 2011– processes & address spaces (lec2) 4 HW & OS collaboration  HW support not necessarily corresponds well to what the OS wants (Linux/PPC vs. AIX/PPC)  Two main approaches  Segments & page tables  Paging has won  Most OSes are designed for paging  Most modern CPU designs support only paging  BUT  x86 provides many features only via segmentation HW (interrupts, protection)  So we must learn about x86 segments, a little bit  Until recently xv6 used segments (rev 5 changed that)

OSE 2011– processes & address spaces (lec2) 5 Case study: Unix v6  Early Unix OS for DEC PDP11  By Thompson & Ritchie, 1975  From Bell labs; the 1 st version to be widely used outside Bell  Ancestor of all Unix flavors (Linux, *BSD, Solaris,…)  (But much smaller)  Written in C  Recognizable (shell, multiuser, files, directories, …)  Today’s Unix flavors have inherited many of the conceptual ideas, even though they added lots of stuff (e.g., graphics) and improved performance

OSE 2011– processes & address spaces (lec2) 6 Case study: Unix v6  1976 Commentary by Lions: a classic….  Despite its age still considered an excellent commentary on simple but high quality code  For many years, was the only Unix kernel documentation publicly available  v6 allowed classroom use of source code; v7 & onwards didn’t  Commonly held to be one of the most copied books in computer science.  Reprinted in 1996; still sold

OSE 2011– processes & address spaces (lec2) 7 Case study: Unix v6  We could have used it for OSE  But  PDP11  Old K&R C style  Missing some key issues in modern OSes (such as paging)  Luckily…

OSE 2011– processes & address spaces (lec2) 8 Case study: xv6  xv6 is an MIT reimplementation of Unix v6  Runs on x86 (well if you insist; we’ll use QEMU)  Even smaller than v6  Preserves basic structure (processes, files, pipes, etc.)  Runs on multicores  Just this year (2011) got paging  To “get it”, you’ll need to read every line in the source code a few times; but it’s really is not that hard  Student’s goal: to explain to yourself every line; the (still rough) commentary on the webpage will greatly help

OSE 2011– processes & address spaces (lec2) 9 Case study: xv6  First half of course, each lecture we will take one part of xv6 and study its source code (homework assignments will help you prepare for these lectures)  About half-way the term, you’ll understand the source code for one well-designed OS for an Intel-based  2 nd half covers important OS concepts invented after Unix v6 We will study the more modern concepts by reading research papers & discussing them in lecture  You may also implement some of these newer concepts in your OS (JOS)

OSE 2011– processes & address spaces (lec2) 10 xv6 – why?  You may wonder why we’re studying an aging OS instead of, say, Linux, or Windows, or FreeBSD, or Solaris, …  xv6 is big enough to illustrate the basic OS design & implementation  Yet xv6 is far smaller => much easier to understand  Structure similar to many modern OSes  Once you've explored xv6 you’ll find your way inside kernels such as Linux  This is, after all, OSE…  JOS occupies a very different point in the design & implementation space from xv6  Finally, “getting” xv6 will help you build your own (J)OS

OSE 2011– processes & address spaces (lec2) 11 xv6 address space  xv6 enforces memory address space isolation  No process can write on another’s space or the kernel’s  xv6 process’s memory starts at 0 & (appears) contiguous  C, as well as the linker, expect contiguity  x86 defines three kinds of memory addresses  Virtual (used by program), which is transformed to  Linear (accounting for segments), which is transformed to  Physical (actual DRAM address)  xv6 nearly doesn’t use segments  All their bases set to 0 (and their limits to the max)  virtual address = linear address  Henceforth we’ll use only the term “virtual”

OSE 2011– processes & address spaces (lec2) 12 Paging in a nutshell  Virtual address (we assume 32bit space & 4KB pages):  Given a process to run, set cr3 = pgdir (“page directory”)  Accessing pgdir[pdx], we find this PDE (page directory entry)  Flags (bits): PTE_P (present), PTE_W (write), PTE_U (user)  20bits are enough, as we count pages of the size 2^12 (=4KB)  The “page table” pgtab = pgdir[pdx] & 0x ffff f000  An actual physical address  Accessing pgtab[ptx], we find this PTE (page table entry)  Target physical address =  (pgdir[pdx] & 0x ffff f000) | (virt_adrs & 0x fff) page offset (12bit)ptx (10bit)pdx (10bit) flags (12bit)physical address (20bit) flags (12bit)physical address (20bit)

OSE 2011– processes & address spaces (lec2) 13 xv6 virtual memory  Each process has its own unique pgdir (= address space)  When the process is about to run, the cr3 register is assigned with the corresponding pgdir  Every process has at most 640 KB memory  Not more that 160 pages  For each process, for VA higher than 640 KB, xv6 maps the VA to an identical PA  => Every process’s address space simultaneously contains translations for both all of the process’s mem & all of the kernel’s mem  PTEs corresponding to addresses higher than 640KB have the PTE_U bit off, so processes can’t access them  The benefit: the kernel can use each process’s address space to access all of its memory

OSE 2011– processes & address spaces (lec2) 14 xv6 virtual memory  Assume a process P size if 12KB (3 pages), and that it wants to sbrk another page (sbrk is the syscall that dynamically allocates memory)  Assume the free page xv6 decides to give P is in PA:  0x  Thus, to ensure contiguity, the 4 th PTE of P, which covers VAs:  0x – 0x fff,  Should be mapped to physical page  0x20100 (= the upper 20 bits of PA 0x )  Two different PTEs now refer to PA = 0x  PTE of VA 0x (kernel), and  PTE of VA 0x (process)  The kernel can use both

OSE 2011– processes & address spaces (lec2) 15 Code: memory allocator