Nachos Instructional OS: Part 3B CS 170, Tao Yang

10/2/20162 Topics File system implementation. Assignment 3B for File system extension

10/2/20163 Nachos File System Two versions: A ``stub'' version is simply a front-end to the Linux. A Nachos version implemented on top of a raw disk. Function layers (from top to down): FileSystem object Directory support. File control block (I-node) support Free block map OpenFile objects (many) Created by every call to open/access a file in a directory. SynchDisk object Created by synchDisk = new SynchDisk("DISK") in system.cc The Disk object. Created by disk = new Disk(name,..) in SynchDisk::SynchDisk() Nachos Kernel Read/write openFile objects SynchDisk Disk Linux openFile objects

10/2/20164 Disk Size Machine/disk.h #define SectorSize 128 // number of bytes per disk sector #define SectorsPerTrack 32 // number of sectors per disk track #define NumTracks 32 // number of tracks per disk #define NumSectors (SectorsPerTrack * NumTracks) // total # of sectors per disk Nachos Kernel Read/write openFile objects SynchDisk Disk

File System Layers in Nachos Base Operating System (Linux for our class) FileSystem : Directory management file create/open/delete Disk: low-level interface that initiates asynchronous IO to physical disk sectors OpenFile: access individual file with read/write/seek. Operate on bytes. SynchDisk: synchronous access to a disk with concurrent threads. Operate on sectors. Nachos kernel thead Directory: Manage a directory of file names FreeMap Bitmap for 32x32 disk sectors FileHeader: i-node (File control block)

10/2/20166 Operations of File System Object FileSystem(bool format). called once at ``boot'' time. For the stub version, just use Unix file system Otherwise create a directory/control info. Sector 0 is fileheader (i-node) for FreeMap. Sector 1 is i-node for directory table. Create(char *name, int initialSize). Create a Nachos disk file. ``initialSize'' is ignored in the stub version; Unix doesn't require it. OpenFile *Open(char *name). Assume both read and write access. in the stub version, Open simply opens the Unix file. Remove(char *name) List()

10/2/20167 Directory Object contains 10 DirectoryEntry objects (directory.h) Directory content has 10 table entries.  fit into a 128B sector. Stored in Sector 1 Directory space is allocated in FileSystem() during initialization … Bool inUse --- is this entry used? Int sector --- location of file header (i-node) Filename -- 9 characters+ 1 for ‘\n’ … 128 bytes

memory 10/2/20168 directory.cc Limit FileNameMaxLen 9 // 9 characters at most for a file name NumDirEntries 10 // at most 10 files Directory Object attributes tableSize -- # of directory entries which is 10 now. table = in-memory directory entries, each of which is for one Nachos file. Content is copied from and copies back to a disk DirectoryFile. Operations Directory (int size) -- Create an in-memory directory of the desired size. FetchFrom (OpenFile *file) -- Fetch disk DirectoryFile that contains directory entries, and copy to in-memory directory table. WriteBack(OpenFile *file) -- Write back in-memory directory content to disk DirectoryFile called every time when a file is added or removed to directory. FindIndex (Char *name) --Search a file string name in a directory Add (Char *name, int sector) – Add file name into a directory with disk sector no that contains the file header (i-node). Remove(char *name) – remove a file. Directory Disk Directory Block

10/2/20169 filehdr.h: File header (File control block, i-node) … int numBytes; // file size Data sector pointer 0 … int numSectors; //sectors used Data sector pointer 29 Data sector pointer 1 Data sector Data sector Data sector int dataSectors[NumDirect] Fit in one sector (128 bytes) Operations Allocate(BitMap *bitMap, int fileSize); FetchFrom (int SectorNo) -- Fetch Fileheader sector from disk. WriteBack(int SectorNo) -- Write back in-memory file header content to disk ByteToSector(int offset) –Convert a file offset in bytes to a sector no memory File control block Disk File header

10/2/ OpenFile Object OpenFile(int sector) sector – disk location of file control block. Load FCB from disk create an in-memory OpenFile object Seek(int position) Read/write in the current position Read(char *buffer, int numBytes) Write(char *buffer, int numBytes) Read/write in a specific position ReadAt(char *buffer, int numBytes, int FilePosition). Identify specific sectors in the file and fetch them, copy to buffer. WriteAt(char *buffer, int numBytes, int FilePosition) May read partially written sectors first, and then write out updated sectors. Length(). Return the file size. Nachos Kernel Read/write openFile objects SynchDisk Memory File control block Disk OpenFile Obj openFile objects

10/2/ SynchDisk Object: Synchronously access a disk Filesys/synchDisk.cc SynchDisk () Called only once to create a physical disk object. Set the disk interrput hander as RequstDone(). Create lock/semaphore for synchronization. Notice that physical disk is an asynchronous device (disk requests return immediately, and an interrupt happens later on). ReadSector(int sectorNumber, char* buffer) Send a read request for a sector Wait IO complete (wait for interrupt using a semaphone) WriteSector(int sectorNumber, char* data) Send a sector write request. Wait for interrupt. RequestDone() Disk interrupt handler. Wake up any thread waiting for the disk request to finish. Nachos Kernel Read/write openFile objects SynchDisk

10/2/ Disk Object Machine/disk.h and disk.cc Simulates the behavior of a disk I/O device. a single platter, with multiple tracks (32). Each track contains 32 sectors Each sector is 128 bytes Disk size = 32x32*128 =128K Asynchronous read/write with an option of disk cache Supported operations: Disk(char *name, VoidFunctionPtr callWhenDone, int callArg) Create a disk: just create a Unix file to simulate. Setup a handling function (RequestDone in SynchDisk.cc) when a read/write request is done. Nachos Kernel Read/write openFile objects SynchDisk Disk 128K

10/2/ Operations in Disk device ReadRequest(int sectorNumber, char *buffer) Read from the specific location of the “physical disk”. ticks = ComputeLatency(sectorNumber, FALSE); interrupt->Schedule(DiskDone, (int) this, ticks, DiskInt); WriteRequest(int sectorNumber, char *data) Write data to specific location of the “physical disk”. ticks = ComputeLatency(sectorNumber, TRUE); interrupt->Schedule(DiskDone, (int) this, ticks, DiskInt); ComputeLatency(int newSector, bool writing) Latency= seek time + rotation time Nachos Kernel Read/write openFile objects SynchDisk Disk

10/2/ Assignment 3B Workload Test C programs ~cs170/nachos- projtest/proj3 Part B. ~200 in filesys (few in userprog) Part A and B can be done in parallel Part B may use Project 2 code without VM. Work on filesys subdirectoy mainly. And change system call handling code in userprog if needed (to use this extended file system). Extend the size of each file from 4K to 100K single indirect pointers in i-node (file control block).

10/2/ Emphasis of Project 3B Play tradeoffs Understand the design options Deadline-driven approach Choose something simpler and easier to do first. Still not enough time? Complete with minimal efforts to pass the tests. Hidden tests are very similar to public test cases. No complex test cases. You are good as long as your code can run virtual memory>physical, can support files up to 100K. Interpret the requirement towards your advantage if vague. If not specified, do something easy.

Part B: Steps for Nachos file system extension 1. Understand how the Nachos file system operates and how Nachos implements files. 2. Modify the file system calls from Project 2 to use Nachos file system 1. Donot have to support VM (Part A/Part B can be done in parallel) 3. Modify Nachos file system so that the file size can be upto 100K

Files to be modified Directory filesys filehdr.cc/.h Support 30 direct data pointers New: Support single indirect pointers in i-node and can expand the file length with more sectors allocated. Write method ExtendFile() that allows a file to grow by certain number of sectors. openfile.cc Expand file length/allocate sectors when writing data at the end or beyond. IndirectPointerBlock.cc (new file to add) This class provides a way for a file header to contain not just direct sectors, but indirect blocks that each have their individual collection of sectors. This allows the file size to exceed the usual 4K. Directory userprog exception.cc Make sure file system calls use extended Nachos file system.

Useful methods in IndirectPointerBlock.cc // Constructor IndirectBlock::IndirectBlock() //Writes the modified object back to the specified sector on disk. void IndirectBlock::WriteBack(int sectorNum) // Retrieves the serialized object from disk. void IndirectBlock::FetchFrom(int sectorNum) // Adds a newly allocated data sector from the file header to this indirect pointer block's list of allocated sectors. void IndirectBlock::PutSector(int sectorNum) // Removes the allocated sectors from the global map of allocated sectors. Essentially removes the data held by this indirect pointer block. void IndirectBlock::Deallocate(BitMap* allocSectorsMap) // Returns the disk sector that is corresponding the specified byte at offset from this indirect pointer block int IndirectBlock::ByteToSector(int offsetFromStartOfIB)

Comparison: Unix file i-node (4K bytes per block)

Step 3: Extend maximum file size Modify the FileHeader i-node (file control block) to support 100K size. Option 1: Include 5 data sector pointers, and 25 sectors for single indirect mapping. At most map *32=805 blocks. Maximum file length = 805*128=103040bytes. Option 2: single indirect mapping at every entry Suggest to add a class called IndirectPointerBlock which takes 1 sector (32 entries) FetchFrom (int sector). WriteBack(int sector). AddSector (int sector) – add a new sector. Deallocate(BitMap *freeMap) – deallocate all sectors. ByteToSector (int offset). – convert an offset in bytes to a sector.

Step 4: File size can grow Modify the current implementation so that a write beyond the end of the file extends the size of the file. Gracefully handle the case of the disk being full or maximum size is reached - the user process should not crash. Instead an error should be returned.

Step 4 Modify WriteAt() method in openfile.cc : Case 1: Write request is within the bound of the file. Handle like the existing code Case 2: Write request overwrites some portion of existing file and goes beyond by N bytes. Case 3: No overwrite to the current data, but goes beyond the end of the file. Here. To extend a file dynamically, Extend i-node/data dynamically: include a routine in FileHeader to extend a file by N bytes (by a sector or by a set of sectors)

Testing issues Need to load user binaries/data into Nachos file system during execution Example: $ pwd cs170/nachos/code/filesys $./nachos -f -cp../test/Prog1 Prog1 -x Prog1 Or nachos –f nachos -cp Prog1 Prog1 nachos -x Prog1 Linux file system Nachos file system Nachos –cp Prog1 Prog1 Nachos –x Prog1 Nachos –f