1. Nachos Instructional OS: Part 2
Tao Yang
4/26/2017
Nachos Instructional OS: Part 2
CS 170, Tao Yang, Fall 2011
Thank you for the introduction. In this presentation, I plan to talk about my research on cluster-based network services. My work has been focusing on three specific aspects: service replication, load-balancing support, and a QoS-aware resource management framework. This study was supervised by my advisor, Tao Yang, at UC Santa Barbara.

2. Topics Program execution in Nachos
Given a C program, produce binary MIPS code.
Execute this MIPS code.
Machine object
How to load a program
How to execute a program: process-level flow.
Demo of single-process execution flow.
Programming Assignment 2.
Support multiprogramming.
System calls for process execution.
System calls for a simple file system interface.
4/26/2017

3. System Layers Nachos kernel threads Thread 1 Thread 2 Thread N
User process
User process
Nachos kernel threads
Thread 1
Thread 2
Thread N
Nachos OS modules
(Threads mgm, File System, Code execution/memory mapping,
System calls/Interrupt)
MIPS Virtual Machine
(Memory, Disk, Console)
Base Operating System (Linux for our class)

4. Machine Source code: under machine subdirectory.
Virtual Machine: Roughly approximates the MIPS architecture
Simulated clock used for event scheduling and execution (interrupts).
Can execute an arbitrary program with a sequence of MIPS instructions.
Machine
object SP Rn PC
memory
page table
process page tables
registers
Clock
4/26/2017

5. Two modes of executions
User mode (executing MIPS instructions of a user program)
Example: Halt code in test directory
User-programs can only access the memory associated with the simulated machine.
Nachos Layer
4/26/2017

6. Two modes of executions
Kernel mode
kernel executes when Nachos first starts up
or when a user-program executes an instruction that causes a ``hardware’’ trap
illegal instruction,
page fault
system call
Nachos layer
4/26/2017

7. Code execution steps in Nachos
Load instructions into the machine's memory.
Initialize registers (program counter, etc).
Tell the machine to start executing instructions.
The machine fetches the instruction, decodes it, and executes it.
Repeat until all instructions are executed.
Handle interrupt/page fault when needed
User binary
code
Nachos
Nachos execution layer
MIPS Machine
4/26/2017

8. Machine Object: Implement a MIPS machine
an instance created when Nachos starts up.
Supported public variables:
Registers: 40 registers.
Memory: Byte-addressable.
Virtual memory: use a single linear page table or a software-managed TLB.
Machine
object SP Rn PC
memory
page table
process page tables
4/26/2017

9. Code/machine/machine.h
class Machine {
char *mainMemory; // physical memory to store user program,
// code and data, while executing
int registers[NumTotalRegs]; // CPU registers, for executing user programs
TranslationEntry *pageTable;
unsigned int pageTableSize;
4/26/2017

10. Machine Object: Supported operations
Machine(bool debug);//allocate memory of 32 pages (128bytes per page). Initialize memory/register/ page table
Translate(int virtAddr, int* physAddr, int size, bool writing);
OneInstruction();
// Run one instruction of a user program.
Run(); // Run a user program
ReadRegister(int num);
WriteRegister(int num, int value);
ReadMem(int addr, int size, int* value);
WriteMem(int addr, int size, int value);
4/26/2017

11. Code/machine/ipssim.cc
void Machine::Run(){
Instruction *instr = new Instruction; // storage for decoded instruction
interrupt->setStatus(UserMode);
for (;;) {
OneInstruction(instr); //fetch and execute one instruction
interrupt->OneTick(); //advance clock }
4/26/2017

12. Code/machine/machine.h
class Instruction {
public:
void Decode(); // decode binary representation of instruction
unsigned int value; // binary representation of the instruction
unsigned char opCode; // Type of instruction
unsigned char rs, rt, rd; // Three registers from instruction.
int extra; // offset or other purpose. Treat as 0 };

13. Code/machine/ipssim.cc
Void Machine::OneInstruction(Instruction *instr) {
//Fetch an instruction and then decode
if (!machine->ReadMem(registers[PCReg], 4, &raw)) return; //if error, return
instr->value = raw; instr->Decode();
// Execute the instruction
switch (instr->opCode) { …
case OP_ADDU:
registers[instr->rd] = registers[instr->rs] + registers[instr->rt];break;
case OP_SW: machine->WriteMem(registers[instr->rs], 4, registers[instr->rt]); break;
case OP_SYSCALL: RaiseException(SyscallException, 0); return; }
// Advance program counters.
registers[PCReg] = registers[NextPCReg];
registers[NextPCReg] = pcAfter; //which is registers[NextPCReg] + 4;

14. Code/machine/translate.cc
bool Machine::WriteMem(int addr, int size, int value)
{ ExceptionType exception; int physicalAddress;
exception = Translate(addr, &physicalAddress, size, TRUE);//address translation
if (exception != NoException) { machine->RaiseException(exception, addr); return FALSE; }
switch (size) { //Copy value to the target physical memory address properly
case 1: machine->mainMemory[physicalAddress] = (unsigned char) (value & 0xff); break;
case 2: *(unsigned short *) &machine->mainMemory[physicalAddress] = ShortToMachine((unsigned short) (value & 0xffff)); break;
case 4: …. }
bool Machine::ReadMem(int addr, int size, int *value) { …

15. A Simple Page Table process page table
PFN 0
PFN 1
Each process has its own page table. Virtual addresses are translated relative to the current page table.
PFN i
PFN i +
offset
page #i
offset
user virtual address
physical memory
page frames

16. Code/machine/translate.cc Physical page dirty bit
ExceptionType Machine::Translate(int virtAddr, * physAddr, size, writing) {
unsigned int vpn, offset;
TranslationEntry *entry;
unsigned int pageFrame;
// calculate virtual page number, and offset within the page
vpn = (unsigned) virtAddr / PageSize;
offset = (unsigned) virtAddr % PageSize;
entry = &pageTable[vpn];
pageFrame = entry->physicalPage;
entry->use = TRUE; // set the use, dirty bits
if (writing) entry->dirty = TRUE;
*physAddr = pageFrame * PageSize + offset; // compute physical address
return NoException; }
Physical page dirty bit

17. Interrupt Object
Maintain an event queue together with a simulated clock.
Supported operations:
Schedule(VoidFunctionPtr handler, int arg, int when, IntType type) Schedule a future event to take place at time ``when''.
Usage: schedule a yield at random interval.
SetLevel(IntStatus level). Used to temporarily disable and re-enable interrupts. Two levels are supported: IntOn and IntOff.
OneTick()—advance 1 clock tick
CheckIfDue(bool advanceClock). Examines if some event should be serviced.
Idle(). ``advances'' to the clock to the time of the next scheduled event
4/26/2017

18. Interrupt::OneTick()
Software managed clock.
The clock advances 1 tick for user mode, 10 for system mode
after every restored interrupt (disable/enable Interrupt)
or after the MIPS simulator executes one instruction.
When the ready list is empty, fast-advance ticks until the next scheduled event happens.
4/26/2017

19. Timer object Generate interrupts at regular or random intervals
Then Nachos invokes the predefined clock event handling procedure.
Supported operation:
Timer(VoidFunctionPtr timerHandler, int callArg, bool doRandom).
Create a real-time clock that interrupts
every TimerTicks (100) time units
Or set this a random number for random mode
4/26/2017

20. Console Object
Simulates the behavior of a character-oriented CRT device
Data can be written to the device one character at a time through the PutChar() routine.
Input characters arrive one-at-a-time. They can be retrieved by GetChar().
Supported operations:
Console(char *readFile, char *writeFile, VoidFunctionPtr readAvail,VoidFunctionPtr writeDone, int callArg).
Create a console instance.``readFile'' is the Unix file of where the data is to be read from; if NULL, standard input is assumed.
PutChar(char ch)
GetChar()
4/26/2017

21. Disk Object Simulates the behavior of a real disk.
The disk has only a single platter, with multiple tracks (32).
Each track contains the same number of sectors (32).
Allow only one pending operation at a time.
Contain a ``track buffer'' cache. Immediately after seeking to a new track, the disk starts reading sectors, placing them in the track buffer.
Supported operations:
Disk(char *name, VoidFunctionPtr callWhenDone, int callArg)
ReadRequest(int sectorNumber, char *data)
WriteRequest(int sectorNumber, char *data)
ComputeLatency(int newSector, bool writing)
4/26/2017

22. Executing a user program
halt
shell
user space
MIPS instructions executed
by the emulator
data
ExceptionHandler()
Nachos kernel
MIPS emulator
Machine::Run()
fetch/execute
examine/deposit
SaveState/RestoreState
examine/deposit Rn
page table
process page tables
Machine
object SP PC
registers
memory

23. From C program to MIPS binary
myprogram.c
myprogram.o
object file
int j;
char* s = “hello\n”;
int p() {
j = write(1, s, 6);
return(j); }
data
assembler
data
data
data
libraries and other objects
….. p:
store this
store that
push
jsr _write
ret
etc.
linker
Executable images are also built from separately developed components (modules)...
separate compilation
symbol tables
...linked together by system utilities.
cross-module procedure calls and data references
relocation records and linkage sections
static link-and-load in “traditional” Unix
DLLs and shared libraries
importance of calling conventions
gcc
compiler
data
program
myprogram.s
myprogram
(executable file)

24. Binary code format (Noff)
Source code: under userprog subdirectory.
The current Nachos can run a single MIPS binary (Noff format) e.g. type ``nachos -x ../test/halt''.
A user program must be compiled using a cross-platform gcc compiler that generates MIPS code.
A Noff-format file contains (bin/noff.h)
the Noff header, describes the contents of the rest of the file
executable code segment (TEXT)
initialized data segment (DATA)
uninitialized data segment (BSS).
4/26/2017

25. Space usage during execution of a C program
STACK
HEAP
BSS
DATA
TEXT
Stack grows from top-down.
Heap grows bottom-up
Uninitialized data
Initialized data
Code

26. TEXT, DATA, BSS, HEAP, STACK in C
Tao Yang
4/26/2017
TEXT, DATA, BSS, HEAP, STACK in C
Int f3=3; /* DATA segment */
Int f1; /*BSS segment*/
def[] = "1"; /* DATA segment */
int main(void) { static char abc[12], /* BSS segment */ static float pi = ; /* DATA segment */ int i = 3; /* Stack*/
char *cp; /*stack*/
cp= malloc(10); /*malloc allocates space from HEAP*/
f1= i+f3; /* code is in TEXT*/
strcpy(abc , "Test" ); /* “Test” is located in DATA segment */ }
4/26/2017

27. Noff format virtualAddr: virtual address that segment begins at.
. Each segment has the following information:
virtualAddr: virtual address that segment begins at.
inFileAddr: Pointer within the Noff file where that section actually begins.
The size (in bytes) of that segment.
4/26/2017

28. User process for executing a program
A Nachos thread is extended as a process
Each process has its own address space containing
Executable code (Code segment)
Initialized data (Data segment)
Uninitialized data (BSS)
Stack space for function calls/local variables
how big is address space?
A process owns some other objects, such as open file descriptors.
4/26/2017

29. Steps in User process creation
Tao Yang
4/26/2017
Steps in User process creation
Currently only execute a single user program.
Create an address space.
Zero out all of physical memory (machine->mainMemory)
Read the binary into physical memory and initialize data segment.
Initialize the translation tables to do a one-to-one mapping between virtual and physical addresses.
Zero all registers, setting PCReg and NextPCReg to 0 and 4 respectively.
Set the stackpointer to the largest virtual address of the process (stack grows downward).
4/26/2017

30. Key Calling graph when Nachos executes under userprog directory
Executable file
ReadAt()
Space=
New AddrSpace()
in addrspace.cc
Initialize()
in system.cc
Machine->
WriteRegister()
Space->
InitRegisters()
main() in
main.cc
StartProcess ()
in progtest.cc
Space
->RestoreState()
Machine->
OneInstruction()
Machine->Run ()
in mipssim.cc
Interupt->
OneTick()
In Interupt.cc
4/26/2017

31. Creating a Nachos Process (code/userprog/progtest.cc)
void StartProcess(char *filename) {
OpenFile *executable;
AddrSpace *space;
executable = fileSystem->Open(filename);
if (executable == NULL) {
printf("Unable to open file %s\n", filename);
return; }
space = new AddrSpace(executable);
currentThread->space = space;
delete executable; // close file
space->InitRegisters();
space->RestoreState();
machine->Run();
ASSERT(FALSE);
Create a handle for reading text and initial data out of the executable file.
Create an AddrSpace object, allocating physical
memory and setting up the process page table.
Set address space of current thread/process.
Initialize registers, load pagetable, and begin execution in user
mode.

32. Creating a Nachos Address Space (code/userprog/addrspace.cc)
AddrSpace::AddrSpace(OpenFile *executable) {
NoffHeader noffH;
unsigned int i, size;
executable->ReadAt((char *)&noffH, sizeof(noffH), 0);
// how big is address space?
size = noffH.code.size + noffH.initData.size + noffH.uninitData.size
+ UserStackSize; // we need to increase the size to leave room for the stack
numPages = divRoundUp(size, PageSize);
size = numPages * PageSize;
pageTable = new TranslationEntry[numPages];
for (i = 0; i < numPages; i++) {
pageTable[i].virtualPage = i; // for now, virtual page # = phys page #
pageTable[i].physicalPage = i;
pageTable[i].valid = TRUE; }
....
Read the header of binary file
Compute address space need
Setup a page table for address translation

33. Initializing a Nachos Address Space
Zero out memory allocated
bzero(machine->mainMemory, size);
// copy in the code and data segments into memory
if (noffH.code.size > 0) {
executable->ReadAt(&(machine->mainMemory[noffH.code.virtualAddr]),
noffH.code.size, noffH.code.inFileAddr); }
if (noffH.initData.size > 0) {
executable->ReadAt(&(machine->mainMemory[noffH.initData.virtualAddr]),
noffH.initData.size, noffH.initData.inFileAddr);
Copy code segment to memory
Copy initialized data segment to memory

34. Compilation of halt.c Test/halt.c: halt.s: ssembly code of halt.c
#include "syscall.h"
main() {
Halt();
/* not reached */ }
halt.s: ssembly code of halt.c
gcc -S
main: ….
jal Halt …
Test/start.s …
Halt:
addiu $2,$0,SC_Halt
syscall
j $31
.end Halt
start.s has system call entries.
System call number is always in register 2

35. Nachos –x halt using halt.c in test directory
Machine->
RaiseException(SyscallException)
Machine->Run ()
in mipssim.cc
ExceptionHandler(SyscallException)
OneInstruction()
Interrupt->
Halt()
In Interupt.cc
4/26/2017

36. Code/userprog/exception.cc
ExceptionType Machine::Translate(int virtAddr, * physAddr, size, writing) {
void ExceptionHandler(ExceptionType which) {
int type = machine->ReadRegister(2);
if ((which == SyscallException) && (type == SC_Halt)) {
DEBUG('a', "Shutdown, initiated by user program.\n");
interrupt->Halt(); } }
// Code Convention:
// system call code -- r2
// arg1 -- r4, arg2 -- r5, arg3 -- r6, arg4 -- r7
4/26/2017

37. When typing “nachos –x halt”
The main thread starts by running function StartProcess() in file progtest.cc. This thread is used to run halt binary.
StartProcess() allocates a new address space and loads the halt binary. It also initializes registers and sets up the page table.
Call Machine::Run() to execute the halt binary using the MPIS emulator.
The halt binary invokes the system call Halt(), which causes a trap back to the Nachos kernel via functions RaiseException() and ExceptionHandler().
The exception handler determines that a Halt() system call was requested from user mode, and it halts Nachos.
4/26/2017

38. Assignment 2: Multiprogramming&System Calls
Modify source code under userprog subdirectory.
~1200 lines of code.
The crossplatform compiler is under ~cs170/gcc/.
This compiler on x86 machines produces a.out with the coff format.
Use utility coff2noff (under nachos’ bin directory) to convert it as Noff.
Check the makefile under test subdirectory on how to use gcc and coff2noff.
System calls to be implemented:
Multiprogramming: Fork(), Yield(), Exit(), Exec() and Join().
File and console I/O: Creat(), Open(), Read(), Write(), and Close().
Test program: ~cs170/nachos-projtest/proj2
4/26/2017

39. Multi-Processes and the Kernel
Fork or Exec
text
data
BSS
user stack
text
data
BSS
user stack
text
data
BSS
user stack
2n-1
Nachos kernel
2n-1

40. To run multiple processes
Tao Yang
4/26/2017
To run multiple processes
Nachos should
Provide the physical memory management;
Fill memory with proper data, instruction.
Set up an address translation table with linear page tables;
Save/restore address-space related state during process switching (AddrSpace::SaveUserState() and AddrSpace:RestoreUserState() are called).
4/26/2017

41. Assignment 2: Files involved
Key files.
progtest.cc -- test routines to run user code.
addrspace.h addrspace.cc -- create an address space and load the program from disk.
syscall.h -- the system call interface.
exception.cc -- the handler for system calls and other user-level exceptions such as page faults.
filesys.h, openfile.h console.h -- interface to the Nachos file system and console.
Other related files:
bitmap.h bitmap.cc -- manipulate bitmsps (useful for keeping track of physical page frames).
translate.h, translate.cc -- translation tables.
machine.h, machine.cc -- emulates main memory, processor, etc.
mipsim.cc -- emulates MPIS R2/3000 instructions.
console.cc -- emulates a terminal using UNIX files.
4/26/2017

42. Assignment 2: Implementation Notes
Tao Yang

43. Part I: Multiprogramming
Fork(func) creates a new user-level (child) process, whose address space starts out as an exact copy of that of the caller (the parent),
Yield(): temporarily relinquish the CPU to another process.
Exit(int) call takes a single argument, which is an integer status value as in Unix. The currently executing process is terminated..
Exec(filename) spawns a new user-level thread (process), but creates a new address space. It should return to the parent a SpaceId.
Join(ID) call waits and returns only after a process with the specified ID has finished.

44. Getting Started
Review start.s (under test directory) which includes all system call stubs, following the style of Halt.
Modify ExceptionHandler() in exception.cc to include all system call entries.
After each system call, increment PC registers so that ExceptionHandler() execution flow returns back to next instruction after user’s system call place.
counter = machine->ReadRegister(PCReg);
machine->WriteRegister(PrevPCReg,counter);
counter = counter + 4;
machine->WriteRegister(PCReg,counter);
machine->WriteRegister(NextPCReg,counter);
Arguments of a system call are in Registers 4, 5, 6 etc.
how to verify? You may review MPIS assembly code produced for a test C program using gcc with -S.
If needed, return result is register 2
machine->WriteRegister(2,result);

45. PCB (Process Control Block)
Write the PCB and a process manager. Create a PCB class that will store the necessary information about a process. Don't worry about putting everything in it right now, you can always add more as you go along. To start, it should have a PID, parent PID, and Thread*.
The process manager- it can have getPID and clearPID methods, which return an unused process id and clear a process id respectively.

46. Memory Manager
Write a Memory Manager that will be used to facilitate memory allocation:
Track memory page usage.
Allocate a page
Free a page
Modify AddrSpace:AddrSpace (addrspace.cc) to use the memory manager.
Modify the page table constructors to use pages allocated by your memory manager
Create a PCB (process control block) also for each process to include key control information.

47. AddSpace.cc
Write a function (e.g. AddrSpace::Translate), which converts a virtual address to a physical address. It does so by breaking the virtual address into a page table index and an offset.
Write a function( e.g. AddrSpace::ReadFile), which loads the code and data segments into the translated memory, instead of at position 0.
Read data into a system buffer (diskBuffer).
Copy buffered data into proper memory locations (e.g. at machine->mainMemory[physAddr].)

48. How to Run User Processes Concurrently?
Forked process
Forked process
Main program
text
data
BSS
user stack
2n-1
2n-1 …
Machine->run()
Nachos Kernel Threads

49. Compilation of a fork program
#include "syscall.h“
FunEntry(){}
main() {
Fork(FuncEntry); }
Assembly code
gcc -S
FuncEntry: …
main:
...
la $4, FuncEntry
jal Fork
jal Fork
Test/start.s …
Fork:
addiu $2,$0,SC_Fork
syscall
j $31
.end Fork
System call number is in register 2.
Argument 1 is in register 4

50. Questions on Forking a Process
When to spawn a new kernel thread?
Do we directly use thread fork function to execute the binary of a child?
Thread->Fork(FuncEntry, NULL)?
Thread->Fork(Machine->Run, NULL)?
Who needs to set the program counter for the new process? Parent thread vs child thread?
Machine->
RaiseException(SyscallException)
Machine->Run ()
in mipssim.cc
ExceptionHandler(SyscallException)
OneInstruction()
4/26/2017

51. User Process and Nachos Kernel Threads
Forked process
Forked process
Main program
text
data
BSS
user stack
2n-1 …
Machine->run()
ForkBridge { …
Machine->run() }
ForkBridge { …
Machine->run() }
Thread fork
Nachos Kernel Threads
2n-1

52. Implement Fork() in ExceptionHandler()
FuncEntry = Value of register 4 ( sys call argu)
Target function to be executed in the new space.
Create a new kernel thread.
Create a new AddrSpace to be a duplicate of the CurrentThread's space and get a new PCB.
The current thread calls Yield() so the new thread can run.
The new thread runs a dummy function that creates a bridge for execution of the user function).
Call NewThread->Fork(ForkBridge, FuncEntry)

53. ForkBridge() : Key parts
Set counter = FuncEntry
Initialize and restore the registers. For example,
currentThread->RestoreUserState();
currentThread->space->RestoreState();
machine->WriteRegister(PCReg, counter);
machine->WriteRegister(PrevPCReg,counter-4);
machine->WriteRegister(NextPCReg,counter+4);
Call machine->Run() which executes the forked user process in the desired FuncEntry address.
ForkBridge() is called helpFork() in the sample code.

54. Implement Exec()
Exec is creating a new process with new code and data segments from a file.
Allocate a new address space which fits this file.
Load data/code from an OpenFile object constructed from the filename passed in by the user.
In order to get that file name you will have to write a function that copies over the string from user space.
Allocate a new kernel thread and a PCB to execute with the above space.
Fork the new thread to run a dummy bridge function that sets the machine registers straight and runs the code
Call NewThread->Fork(ExecBridge ,NULL);
The calling thread should yield to give control to the newly spawned thread.
Return the process ID that executes this binary.

55. ExecBridge(): Key parts
Initialize registers/restore state.
(currentThread->space)->InitRegisters();
(currentThread->space)->RestoreState();
Machine->run();

56. System Calls for File System
For the entire system, maintain a set of objects (SysOpenFile class) representing system-wide opened files.
Each file may be opened by many user processes.
Call it fileOpenTable[];
For each process, maintain a set of objects in PCB (UserOpenFile class) representing files opened by this process.
Each file has filename, index to the system-wide open table entry, and offset for the current read/write pointer

57. Implement create() Nachos already has a simple file system implemented
filesys.cc and openfile.cc under filesys directory.
Use Nachos fileSystem to add a new file to its directory.

58. Implement open()
Use fileSystem->Open() (nachos’open) to locate the file object.
Add the opened file object to the system-wide fileOpenTable.
If it is already opened by some other user process, just share the entry.
Else insert to the local file open table in PCB.

59. Implement read() Allocate an internal buffer.
If Inputfile ID is ConsoleInput
Then use getchar() and save data in the internal buffer.
Read until EOF or \n.
Else, find the current read offset.
Use openFile:ReadAt (defined in openfile.cc) to read data from the nachos file.
Copy data from the internal buffer to the destination memory location.
Need to copy one by one since the address needs to be translated one by one.

60. Implement write() Allocate an internal buffer.
Copy data from the source memory location to the internal buffer.
Need to copy one by one since the address needs to be translated one by one.
If Outputfile ID is ConsoleOutput
Then use printf (assuming string type).
Else, find the current write offset.
Use openFile:WriteAt (defined in openfile.cc) to write data to the nachos file.