1. OSes: 11. FS Impl. 1 Operating Systems v Objectives –discuss file storage and access on secondary storage (a hard disk) Certificate Program in Software Development CSE-TC and CSIM, AIT September--November, 2003 11. File System Implementation (S&G, Ch. 11)

2. OSes: 11. FS Impl. 2 Contents 1.A Hard Disk 2.File System Organization 3.Allocation Methods 4.File System Management 5.Directory Implementation 6.Improving Performance

3. OSes: 11. FS Impl. 3 1. A Hard Disk Fig. 2.5, p.33 : rotation arm platter cylinder c track t read-write head spindle actuator sector s

4. OSes: 11. FS Impl. 4 Notes v To read/write disk sector –move head to cylinder (seek) –wait for sector to rotate to head (latency) –read or write (transfer) v Controller –one controller for several disks –intelligent controllers –caching continued

5. OSes: 11. FS Impl. 5 v I/O transfer is carried out in blocks –one or more sectors v Sector size: 32 - 4096 bytes –usually 512 bytes

6. OSes: 11. FS Impl. 6 2. File System Organization Fig. 11.1, p.370 application program logical file system file-organization module basic file system I/O control hardware devices

7. OSes: 11. FS Impl. 7 Features v Basic File System –identifies a hardware block using drive/cylinder/track/sector numbers v I/O Control –consists of device drivers and interrupt handlers –translates hardware block commands (e.g. “retrieve block 123”) into hardware specific instructions continued

8. OSes: 11. FS Impl. 8 v File Organization Module –translates logical block addresses into physical blocks and then to drive / cylinder / track / sector numbers

9. OSes: 11. FS Impl. 9 2.1. Open File Table  An open() searches for a file and then records its details in an open file table –returns an index to the table entry u called a file descriptor in UNIX –the index is used by subsequent file operations

10. OSes: 11. FS Impl. 10 Open File Table Diagram Fig. 11.2, p.372 test.c mail.txt : : rw- rw- rw- rw- --- ---... 0 1 : : n file namepermissionsaccess dates pointer to disk block index

11. OSes: 11. FS Impl. 11 3. Allocation Methods v How is space allocated to files on a disk? v Aims: –utilize disk space effectively –allow fast file access v Three main methods: –contiguous allocation –linked allocation –indexed allocation

12. OSes: 11. FS Impl. 12 3.1. Contiguous Allocation v Each file occupies a set of contiguous (continuous) blocks on the disk. v Supports fast sequential and direct (random) access –e.g. the 5th block of a file starting at block 19 is at block 24 (19+5)

13. OSes: 11. FS Impl. 13 Diagram Fig. 11.3, p.374 0123 4567 891011 15141312 19181716 23222120 27262524 31302928 filestartlength count 0 2 tr 14 3 mail 19 6 list 28 4 f 6 2 directory 'physical block' view

14. OSes: 11. FS Impl. 14 Problems v Finding space for a new file or a resized file. v Determining size requirements. v External fragmentation of the hard disk.

15. OSes: 11. FS Impl. 15 3.2. Linked Allocation v Each file is stored as a linked list of disk blocks –the disk blocks may be anywhere on the disk

16. OSes: 11. FS Impl. 16 Diagram Fig. 11.4, p.377 0 10 123 4567 89 11 15141312 19181716 23222120 27262524 31302928 filestartend jeep directory 1625 1 925

17. OSes: 11. FS Impl. 17 Advantages v Solves the problems with contiguous allocation –the space allocation problem –no external fragmentation v Fast for sequential access.

18. OSes: 11. FS Impl. 18 Disadvantages v Direct access is not supported (efficiently). v Space overhead of pointers –solution: cluster disk blocks v Reliability if a pointer ‘fails’ –solution: use richer (bigger) links

19. OSes: 11. FS Impl. 19 3.2.1. File-allocation Table (FAT) Fig. 11.5, p.378 618 eof 339 217...test directory entry name start block 0 217 339 618 no. of disk blocks FAT

20. OSes: 11. FS Impl. 20 Notes v The table has one entry for each disk block on the disk, and is indexed by block number. v Records the sequence of blocks used by each file. v No need for pointers in the actual disk blocks. continued

21. OSes: 11. FS Impl. 21 v Used by MS-DOS and OS/2 v Drawback: the disk head needs to do many seeks: –move to the start of the FAT –then move to the required FAT entry –then move to the actual disk block

22. OSes: 11. FS Impl. 22 3.3. Indexed Allocation v A modification of linked allocation where the disk block pointers for a file are all placed in an index block.

23. OSes: 11. FS Impl. 23 Diagram Fig. 11.6, p.379 0123 4567 891011 15141312 19181716 23222120 27262524 31302928 fileindex block jeep directory 19 9 16 1 10 25 -1 -1 -1 19

24. OSes: 11. FS Impl. 24 Notes v Supports direct access. v Wasteful if only a few pointers are stored in the index block.

25. OSes: 11. FS Impl. 25 3.3.1. Variations v Linked Scheme –link together the (smaller) index blocks used for a file v Multilevel Index –use a first-level index block to point to second-level index blocks which point to the actual disk blocks –supports much bigger files continued

26. OSes: 11. FS Impl. 26 v Combined Schemes –e.g. the UNIX file index block (inode) mode owners (2) timestamps (3) size block count single indirect double indirect triple indirect direct blocks data........................ 12 :::::: : : : Fig. 11.7, p.381

27. OSes: 11. FS Impl. 27 3.4. Performance v Allocation methods vary in their: –storage efficiency –disk block access times v Measurements depend on the file mix in the OS –e.g. no. of sequential access files vs. no. of direct/random access files continued

28. OSes: 11. FS Impl. 28 v Contigous allocation –constant time access for sequental or direct access files v Linked allocation –should not be used for direct access files continued

29. OSes: 11. FS Impl. 29 v Indexed allocation –faster if the index block for the file is already in memory –large files require more index block searching, so are slower to access

30. OSes: 11. FS Impl. 30 Performance Optimisations v Since CPU speeds are often 10,000 times faster than hard disk speeds, then very complex disk accessing optimisations, coded in software/hardware are worthwhile.

31. OSes: 11. FS Impl. 31 4. File Space Management v Use a free-space list to record which blocks are free (i.e. not allocated to a file or directory). v Implementation approaches: –bit vector –linked list –grouping –counting

32. OSes: 11. FS Impl. 32 4.1. Bit Vector v Represent each free block by a ‘1’ bit, each allocated block as a ‘0’ bit in a bit vector. v Example: free blocks are 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26, 27,… v Bit vector: 001111001111110001100000011100000... continued

33. OSes: 11. FS Impl. 33 v OSes contain fast bit manipulation instructions to find a free block or blocks quickly. v Problem: size of the bit vector –1 bit required for each disk block

34. OSes: 11. FS Impl. 34 4.2. Linked List v Link together all the free disk blocks. v Problems: –traversal speed –pointer failure Fig. 11.8, p.384 0123 4567 891011 15141312 19181716 23222120 27262524 31302928 free-space list head

35. OSes: 11. FS Impl. 35 4.3. Grouping  Store the addresses of the n free blocks in the first free disk block. v Use pointers to link together multiple disk blocks of this information.

36. OSes: 11. FS Impl. 36 4.4. Counting v In contiguous allocation schemes, free blocks are usually contiguous.  Instead of making a list of n free disk blocks, store the address of the first free block and a counter (containing n ).

37. OSes: 11. FS Impl. 37 5. Directory Implementations v Linear list of files –simple to code –time-consuming to search u solution: cache recently used directory info. v Hash table –compute the index into a list of files based on a hash function (applied to the file name) u collisions u cost of directory reorganization

38. OSes: 11. FS Impl. 38 6. Improving Performance v Most disk controllers include local memory to store previously accessed tracks –repeated accesses are faster –called a track cache (or track buffer) v Some systems store recently accessed disk blocks in memory –called a disk cache (or block buffer) continued

39. OSes: 11. FS Impl. 39 v Some systems support RAM disks (virtual disks) in memory –users can carry out standard disk operations, but they are much faster since the data never leaves memory

40. OSes: 11. FS Impl. 40 Diagram Fig. 11.9, p.389 hard disk track cache ram disk disk cache CPU main memory disk controller