1. Case Study – The Linux OS Part II Mostly taken from: Silberschatz – Chapter 20 Nutt – Chapter 20

2. Linux Operating System - Part -II - Slide 2 Fakhar Lodhi Virtual Memory  The VM system maintains the address space visible to each process: It creates pages of virtual memory on demand, and manages the loading of those pages from disk or their swapping back out to disk as required  The VM manager maintains two separate views of a process’s address space: A logical view describing instructions concerning the layout of the address space  The address space consists of a set of nonoverlapping regions, each representing a continuous, page-aligned subset of the address space A physical view of each address space which is stored in the hardware page tables for the process

3. Linux Operating System - Part -II - Slide 3 Fakhar Lodhi Virtual Memory (Cont.)  On executing a new program, the process is given a new, completely empty virtual-address space The program-loading routines populate the address space  Creating a new process with fork involves creating a complete copy of the existing process’s virtual address space The kernel copies the parent process’s VMA descriptors, then creates a new set of page tables for the child The parent’s page tables are copied directly into the child’s, with the reference count of each page covered being incremented After the fork, the parent and child share the same physical pages of memory in their address spaces Copy-on-write mechanism

4. Linux Operating System - Part -II - Slide 4 Fakhar Lodhi Virtual Memory (Cont.)  The VM paging system relocates pages of memory from physical memory out to disk when the memory is needed for something else  The VM paging system can be divided into two sections: The pageout-policy algorithm decides which pages to write out to disk, and when The paging mechanism actually carries out the transfer, and pages data back into physical memory as needed

5. Linux Operating System - Part -II - Slide 5 Fakhar Lodhi Virtual Memory (Cont.)  The Linux kernel reserves a constant, architecture- dependent region of the virtual address space of every process for its own internal use  This kernel virtual-memory area contains two regions: A static area that contains page table references to every available physical page of memory in the system, so that there is a simple translation from physical to virtual addresses when running kernel code The reminder of the reserved section is not reserved for any specific purpose; its page-table entries can be modified to point to any other areas of memory

6. Linux Operating System - Part -II - Slide 6 Fakhar Lodhi Executing and Loading User Programs  Linux maintains a table of functions for loading programs; it gives each function the opportunity to try loading the given file when an exec system call is made  The registration of multiple loader routines allows Linux to support both the Executable and Linking Format (ELF) and a.out binary formats  An ELF-format binary file consists of a header followed by several page-aligned sections The ELF loader works by reading the header and mapping the sections of the file into separate regions of virtual memory  Initially, binary-file pages are mapped into virtual memory Only when a program tries to access a given page will a page fault result in that page being loaded into physical memory

7. Linux Operating System - Part -II - Slide 7 Fakhar Lodhi Mapping of Programs into Memory Points to the current extent Top of user mode Write protected Variable size region Fixed size region

8. Linux Operating System - Part -II - Slide 8 Fakhar Lodhi Static and Dynamic Linking  A program whose necessary library functions are embedded directly in the program’s executable binary file is statically linked to its libraries  The main disadvantage of static linkage is that every program generated must contain copies of exactly the same common system library functions  Dynamic linking is more efficient in terms of both physical memory and disk-space usage because it loads the system libraries into memory only once

9. Linux Operating System - Part -II - Slide 9 Fakhar Lodhi Linux File Manager  Defines a single internal view of files that can be used to read/write files onto a storage device using a variety of file managers  A file is viewed as a named byte stream that can be saved in the secondary storage  Supports directory and file operations  Implements asynchronous read/write operations I/O Buffers Device Driver Storage Device File Manager Write Read

10. Linux Operating System - Part -II - Slide 10 Fakhar Lodhi Linux File Manager – VFS  Virtual File System (VFS)  Implements FS independent operations  VFS switch – provides the API to user space programs and establishes an internal interface used by different file system translators  The VFS inode contains details about the file. Linux File Manager System Call Interface VFS Switch MS-DOSMINIXExt2/proc

11. Linux Operating System - Part -II - Slide 11 Fakhar Lodhi File Systems  To the user, Linux’s file system appears as a hierarchical directory tree obeying UNIX semantics  Internally, the kernel hides implementation details and manages the multiple different file systems via an abstraction layer, that is, the virtual file system (VFS)  The Linux VFS is designed around object-oriented principles and is composed of two components: A set of definitions that define what a file object is allowed to look like  The inode-object and the file-object structures represent individual files  the file system object represents an entire file system A layer of software to manipulate those objects

12. Linux Operating System - Part -II - Slide 12 Fakhar Lodhi The Linux Ext2 File System  Ext2 performs its allocations in smaller units The default block size on ext2fs is 1Kb, although 2Kb and 4Kb blocks are also supported  Ext2 uses allocation policies designed to place logically adjacent blocks of a file into physically adjacent blocks on disk, so that it can submit an I/O request for several disk blocks as a single operation

13. Linux Operating System - Part -II - Slide 13 Fakhar Lodhi  Two parts Partition into multiple block groups Each group belongs to a single cylinder of the physical disk May be Different sizes of groups  ext2 selects the block group for the file  tries to keep the related information in the same group block  for data block it tries to choose the same block group in which the file’s inode has been allocated  for inodes it selects the same group as the file’s parent directory  directory files are not kept together to spread out disk load  within the block groups, it tries to keep the allocation physically contiguous  maintains a bitmap of all free blocks in a block group The Linux Ext2 File System

14. Linux Operating System - Part -II - Slide 14 Fakhar Lodhi  when allocating the first block for a new file, it starts searching for a free block  for extending the file it continues the search from the block most recently used  the search is performed in two stages first it searches for an entire free byte in the bitmap – aims to allocate disk blocks in chunks (at least 8 where possible). if it fails then it searches for any free bit  once a free block has been identified, the search is extended backwards until an allocated block is encountered  when a free byte is found in the bitmap. this backward extension prevents leaving a hole between the most recently allocated block in the previous non-zero byte and the zero byte found.  extends the allocation forward for up to eight blocks and preallocates these extra blocks to the file to reduce fragmentation and save disk allocation time.  preallocated blocks are returned to free space when the file is closed. The Linux Ext2 File System

15. Linux Operating System - Part -II - Slide 15 Fakhar Lodhi Ext2 Block-Allocation Policies if we can find any free blocks sufficiently near the start of the search then we allocate them no matter how fragmented

16. Linux Operating System - Part -II - Slide 16 Fakhar Lodhi  Ext2 internals divided into group Super block, Group descriptor, Block descriptor, inode bitmap, inode table, data blocks  Different block size length 1KB, 2KB, 4KB, 8KB  Root has 5% of space reserved Enable filesystem recovery The Linux Ext2 File System (cont.)

17. Linux Operating System - Part -II - Slide 17 Fakhar Lodhi  The inode contains important information regarding the file  Kernel keeps the inodes of open files in memory  A structure that contains file’s description: Type Access rights Owners Timestamps Size 15 Pointers to data blocks  First twelve points to data block.  Thirteenth is called indirect pointer  Fourteenth is called doubly indirect pointer  Fifteenth is called triple indirect pointer The Linux Ext2 File System - inode

18. Linux Operating System - Part -II - Slide 18 Fakhar Lodhi inode structure File Info IND DIND TIND pointers to data blocks indirect pointer double indirect pointer triple indirect pointer

19. Linux Operating System - Part -II - Slide 19 Fakhar Lodhi Directories  A directory is just a special type of file  These are structured in a tree hierarchy  Each can contain both files and directories  Special user-functions for directory access  Each dentry contains filename + inode-no  Kernel searches the direrctory tree  translates a pathname to an inode-number

20. Linux Operating System - Part -II - Slide 20 Fakhar Lodhi Directory diagram Inode Table Directory i1 i2 i3 i4 name1 name2 name3 name4

21. Linux Operating System - Part -II - Slide 21 Fakhar Lodhi Links  Multiple names can point to same inode  The inode keeps track of how many links  If a file gets deleted, the inode’s link-count gets decremented by the kernel  File is deallocated if link-count reaches 0  This type of linkage is called a ‘hard’ link  Hard links may exist only within a single FS  Hard links cannot point to directories (cycles)

22. Linux Operating System - Part -II - Slide 22 Fakhar Lodhi The Linux /proc File System  The /proc file system does not store data, rather, its contents are computed on demand according to user file I/O requests  proc must implement a directory structure, and the file contents within; it must then define a unique and persistent inode number for each directory and files it contains It uses this inode number to identify just what operation is required when a user tries to read from a particular file inode or perform a lookup in a particular directory inode When data is read from one of these files, proc collects the appropriate information, formats it into text form and places it into the requesting process’s read buffer

23. Linux Operating System - Part -II - Slide 23 Fakhar Lodhi  The /proc file system is a direct reflection of the system kept in memory and represented in a hierarchal manner.  The effort of the /proc file system is to provide an easy way to view kernel and information about currently running processes.  As a result, some commands (ps for example) read /proc directly to get information about the state of the system.  The premise behind /proc is to provide such information in a readable manner instead of having to invoke difficult to understand system calls. The Linux /proc File System

24. Linux Operating System - Part -II - Slide 24 Fakhar Lodhi Input and Output  Linux splits all devices into three classes: block devices allow random access to completely independent, fixed size blocks of data character devices include most other devices; they don’t need to support the functionality of regular files network devices are interfaced via the kernel’s networking subsystem  User cannot directly transfer data to network devices  All device drivers appear as normal files  The Linux device-oriented file system accesses disk storage through two caches: Data is cached in the page cache, which is unified with the virtual memory system Metadata is cached in the buffer cache, a separate cache indexed by the physical disk block

25. Linux Operating System - Part -II - Slide 25 Fakhar Lodhi Device-Driver Block Structure  Provide the main interface to all disk devices in a system  The block buffer cache serves two main purposes:  it acts as a pool of buffers for active I/O  it serves as a cache for completed I/O  The request manager manages the reading and writing of buffer contents to and from a block device driver Block Devices

26. Linux Operating System - Part -II - Slide 26 Fakhar Lodhi Device-Driver Block Structure  A device driver which does not offer random access to fixed blocks of data  A character device driver must register a set of functions which implement the driver’s various file I/O operations  The kernel performs almost no preprocessing of a file read or write request to a character device, but simply passes on the request to the device  The main exception to this rule is the special subset of character device drivers which implement terminal devices, for which the kernel maintains a standard interface Character Devices

27. Linux Operating System - Part -II - Slide 27 Fakhar Lodhi Device-Driver Block Structure  Networking is a key area of functionality for Linux. It supports the standard Internet protocols for UNIX to UNIX communications It also implements protocols native to nonUNIX operating systems, in particular, protocols used on PC networks, such as Appletalk and IPX Network Devices

28. Linux Operating System - Part -II - Slide 28 Fakhar Lodhi Device-Driver Block Structure  Internally, networking in the Linux kernel is implemented by three layers of software: The socket interface Protocol drivers Network device drivers Network Devices

29. Linux Operating System - Part -II - Slide 29 Fakhar Lodhi Device-Driver Block Structure  The most important set of protocols in the Linux networking system is the internet protocol suite It implements routing between different hosts anywhere on the network On top of the routing protocol are built the UDP, TCP and ICMP protocols Network Devices

30. Linux Operating System - Part -II - Slide 30 Fakhar Lodhi Interprocess Communication  Linux informs processes that an event has occurred via signals  There is a limited number of signals, and they cannot carry information: Only the fact that a signal occurred is available to a process

31. Linux Operating System - Part -II - Slide 31 Fakhar Lodhi Passing Data Between Processes  The pipe mechanism allows a child process to inherit a communication channel to its parent, data written to one end of the pipe can be read a the other  Each pipe has a pair of wait queues to synchronize readers and writers

32. Linux Operating System - Part -II - Slide 32 Fakhar Lodhi Shared Memory Object  Shared memory offers an extremely fast way of communicating; any data written by one process to a shared memory region can be read immediately by any other process that has mapped that region into its address space

33. Linux Operating System - Part -II - Slide 33 Fakhar Lodhi Security – Pluggable Authentication Module (PAM)  PAM provides a way to develop programs that are independent of authentication scheme.  By using PAM, multiple authentication technologies, such as RSA, DCE, Kerberos, smart card and S/Key, can be added without changing any of the login services, thereby preserving existing system environments.  Which authentication module is to be attached is dependent upon the local system setup and is at the discretion of the local system administrator.  PAM is based on a shared library that can be used by any system component that needs to authenticate users

34. Linux Operating System - Part -II - Slide 34 Fakhar Lodhi Security – Pluggable Authentication Module (PAM)  Access control under UNIX systems, including Linux, is performed through the use of unique numeric identifiers (uid and gid)  Access control is performed by assigning objects a protections mask, which specifies which access modes—read, write, or execute—are to be granted to processes with owner, group, or world access

35. End of Linux Overview