cs4411 – Operating Systems Practicum December 2, 2011 Zhiyuan Teo Supplementary lecture 5

Administrative information General implementation hints Our last lecture… A review of things we have done Where do we go from here? Offline Q&A session

Deadline for Project 6 has been extended till 11.59pm Sunday, 11 December. Administrative Information The zipfile for Project 6 has been updated on CMS. Please use our new disk.c file. CS4410 MP4 is optional for 4411 students.

Administrative Information We have been busy catching up on research and final reports. One final round of office hours on Monday, 5 December from 3pm – late. - We promise to honor all regrade requests. - We want to help you!

These slides generally reveal implementation hints. Before we begin… The only concurrency we will test for is simultaneous writes. Focus on correctness first, then performance later. - You do not have to follow the implementation we describe here! - Consider following the hints only if you are stuck. - But ideally, we want you to design the file system to be correctly concurrent under any kind of workload. inode = “eye node”, not “ee node”. - Don’t embarrass yourself and your instructors!

mkfs (and fsck) should be minithread programs. Before we begin… - You need to compile them as separate programs. - Don’t make mkfs or fsck function calls in your minifile implementation.

How do the global disk variables work? Getting started - They are set inside main(), before minithread_system_initialize() is called. int main(int argc, char** argv) { use_existing_disk=0; disk_name = “disk0”; disk_flags = DISK_READWRITE; disk_size = 1000; minithread_system_initialize(entrypoint, NULL); } void minithread_system_initialize(proc_t mainproc, arg_t arg) { disk_initialize(&disk); install_disk_handler(disk_handler); }

On-disk data structures Keep things simple: 1 disk block per item. - One disk block for superblock. - One disk block for each inode. - Don’t share disk blocks. Concurrency-related structures should not be on disk. - Reference counters, locks etc. Pointers on the disk point refer to disk block number.

The big picture superblock magic no. size of disk root inode first free inode first free data block type size direct ptr indirect ptr dir inode nameblock no. dir data nameblock no. nameblock no. nameblock no. direct ptr indirect ptr direct ptr indirect ptr next free block file inode type size direct ptr indirect ptr data nameblock no. nameblock no. file data free block

The big picture superblock (blk 0) magic no.: 4411 size: 1000 blks root inode: 1 1 st free inode: 3 first free data block DIR_INODE size: 3 ents. dir ptr 1: dir inode (blk 1)..at inode 1 dir data (blk 100).at inode 1 abc.txtat inode 2 0 nx free inode: 4nx free inode: 5 file inode (blk 2) FILE_INODE size: 12 bytes dir ptr 1: Hello world! 0 0 file data (blk 101) free block (blk 3) 0 … last free inode block (blk 99)

Superblock magic number size of disk root inode first free inode first free data block superblock Use disk block 0 for the superblock. Root inode field should contain the value 1. - Since that inode is located at disk block 1.

Inodes inode type size direct ptr 1 direct ptr 2 direct ptr n inode You can use the same structure for file and directory inodes. Size: number of directory entries if inode is a directory inode. indirect ptr Size: number of bytes of a file if inode is a file inode.

Directory data blocks directory data block This is just a table with 2 columns. You can’t tell from this table if a certain entry is a file or a directory. No indirect pointers in this block. Directory data blocks are stored in the region of disk reserved for data blocks. nameinode ptr nameinode ptr nameinode ptr nameinode ptr nameinode ptr nameinode ptr nameinode ptr nameinode ptr nameinode ptr nameinode ptr - Implement it as two arrays. - But you can easily access that entry’s inode and look up its type.

Free blocks ptr to next free block free block Just store an integer that points to the next free block. If the next free block says 0, there are no more free blocks after this. Use the same data structure for free inodes and data blocks. Returning blocks to the free list: prepend and modify superblock.

Data structures for blocks You can apply a trick we’ve seen before. struct superblock { union { struct { char magic_number[4]; char disk_size[4]; char root_inode[4]; char first_free_inode[4]; char first_free_data_block[4]; } data; char padding[DISK_BLOCK_SIZE]; } }

Data structures for blocks You can apply a trick we’ve seen before. struct inode { union { struct { char inode_type; char size[4]; char direct_ptrs[TABLE_SIZE][4]; char indirect_ptr[4]; } data; char padding[DISK_BLOCK_SIZE]; } }

Data structures for blocks You can apply a trick we’ve seen before. struct dir_data_block { union { struct { char dir_entres[TABLE_SIZE][256]; char inode_ptrs[TABLE_SIZE[4]; } data; char padding[DISK_BLOCK_SIZE]; } }

Data structures for blocks You can apply a trick we’ve seen before. struct free_data_block { union { char next_free_block[4]; char padding[DISK_BLOCK_SIZE]; } }

Benefits You can cast the struct into a char* and directly use it in disk read and write operations. - The struct is of size DISK_BLOCK_SIZE, so you will read/write exactly one block. - Use pack and unpack functions to fill/read the fields. No need to worry about padding. Endianness-independent disk implementation.

You can use single/double/triple indirect pointers, similar to Linux. Variations You can use a bitmap instead of a free list. You may want to use different structures for blocks. - As long as your file system is reasonable and concurrent. - Describe your implementation in the README file. Remember: you don’t have to follow our suggestions.

Storing names in inodes. What are not acceptable variations Storing directory data or indirect blocks in the inode-reserved section of the disk. Constricting free expansion for the number of directory entries or a file’s size. - But you can assume there will not be more than 2 32 directory entries. - File sizes will not exceed 2 32 bytes (4Gb).

Our suggestion: one ‘big lock’ for metadata accesses that can potentially span multiple inodes. Concurrency One lock per inode for file updates. Create some in-memory protection structures. - They have to be dynamically allocated since disk_size is a variable. - Lock this inode when performing reading/writing, but release it as soon as you can.

Contains lots of low-level, step-by-step instructions. Need more implementation hints? Credits to Ashik, Christopher and Sean for contributing this gold nugget. The Design of the UNIX Operating System, Maurice J. Bach

You built an OS! We’ve come to the end. Very few students in the world have an opportunity to practice extensive design of real OS components. This was not an easy class. Hindsight is always perfect.

My parting quote “Good design comes from experience. Experience comes from bad design.” -Theodore von Karman It is our great pleasure to have accompanied you on a part of your academic journey.