Windows 2000 I/O System, Cache Manager and File Systems Computing Department, Lancaster University, UK

Overview Goals… Goals… –Overview of I/O System –Overview of Cache Manager –Overview of File Systems

I/O System Design goals Design goals –Make I/O processing fast on both single and multiprocessor systems –Protect shareable resources by using standard Windows 2000 security mechanisms –Meet requirements for the various sub-systems available –Provide services to make device driver development as easy as possible –Allow device drivers to be added or removed dynamically –Provide support for multiple installable file systems

I/O System Components (1) The I/O manager The I/O manager –Connects applications and system components to virtual, logical and physical devices A device driver A device driver –Typically provides an I/O interface for a particular type of device The PnP manager The PnP manager –Responsible for loading a device driver when a device is detected The power manager The power manager –Guides the system through power-state transitions

I/O System Components (2) The registry The registry –Serves as a database that stores a description of basic hardware devices attached to the system INF files INF files –Driver installation files –The link between a particular device, and the driver that assumes its primary control Hardware Abstraction Layer (HAL) Hardware Abstraction Layer (HAL) –Insulates the drivers from the specifics of the processor and interrupt controller –Provides APIs that hide the differences between platforms

I/O Operations Most I/O operations dont involve all components… Most I/O operations dont involve all components… Typical request starts with an app executing an I/O related function, that is processed by the I/O manager, one or more device drivers and the HAL Typical request starts with an app executing an I/O related function, that is processed by the I/O manager, one or more device drivers and the HAL The OS abstracts all I/O requests as operations on a virtual file: The OS abstracts all I/O requests as operations on a virtual file: –Hiding the fact that the target might not be a file-structured device –Abstraction generalises an applications interface to devices –All data that is read or written is regarded as a simple stream of bytes directed to these files –The I/O manager dynamically directs these virtual file requests to the appropriate device driver

Flow of a Typical I/O Request I/O Subsystem API I/O Manager Kernel mode device drivers HAL I/O access routines User-mode API Driver support routines

Types of Device Drivers File system drivers File system drivers –Accept I/O requests to files Windows 2000 drivers Windows 2000 drivers –Include drivers for mass storage devices, net. adaptors Legacy drivers Legacy drivers –Written for NT Display drivers Display drivers –Translate device-independent graphics requests to device-specific requests WDM drivers (Windows Driver Model) WDM drivers (Windows Driver Model) –Includes support for power management, PnP, etc…

User-mode Drivers Virtual device drivers (VDDs) Virtual device drivers (VDDs) –Used to emulate 16-bit MS-DOS applications Printer drivers Printer drivers –Translate device-independent graphics requests to printer-specific commands

Win32 Input/Output APIs A few device-independent APIs available A few device-independent APIs available –CreateFile –ReadFile / WriteFile –CloseHandle Many (over 100) device-type-dependent APIs Many (over 100) device-type-dependent APIs –disk –tape –console –pipes –sound cards –etc…

Device and File Naming File names File names –\\.\disk\directory\…\file –\\server\disk\directory\…\file Local device names Local device names –\\.\name-of-device Remote device names; Universal Name Convention Remote device names; Universal Name Convention –\\server\name-of-shared-resource DOS-style built-in device names for compatibility DOS-style built-in device names for compatibility –A:, B:, D: etc… Windows 2000 internal name differ from externally visible names… Windows 2000 internal name differ from externally visible names…

Overview of Cache Manager What is the cache manager? What is the cache manager? –A set of kernel-mode functions and system threads that cooperate with the memory manager to provide data caching for all Windows 2000 file system drivers Most OSs rely on each individual file system to cache data Most OSs rely on each individual file system to cache data –Duplicate caching / memory management code –Limitations on the kinds of data that can be cached Windows 2000 facilitates the caching of all externally stored data: Windows 2000 facilitates the caching of all externally stored data: –Whether on local hard disks, floppy disks, network file servers, CD-ROMs Any kind of data can be cached: Any kind of data can be cached: –User data streams (the contents of a file, + on-going r/w) –File system metadata (e.g. directory and file headers)

Cache Manager Key Features Supports all file system types Supports all file system types –Removes the need for each file system to implement its own cache management code Uses the memory manager to control what parts of the files are in physical memory (see next slide) Uses the memory manager to control what parts of the files are in physical memory (see next slide) –Allowing a trade-off for physical memory between user processes and the OS Caches data on a virtual block basis Caches data on a virtual block basis –In contrast to most caching systems, which cache on a logical block basis provides fast I/O intelligent read-ahead and high- speed access to the cache without involving file-system drivers provides fast I/O intelligent read-ahead and high- speed access to the cache without involving file-system drivers

Interaction Between the Cache and the Memory Manager Unusual aspect: Cache manager never actually knows how much cached data is in physical memory! Unusual aspect: Cache manager never actually knows how much cached data is in physical memory! This is because it maps views of files into system virtual address spaces This is because it maps views of files into system virtual address spaces –The memory manager then pages in blocks that arent in physical memory, and pages data out too… The approach avoids the cache manager generating read or write I/O request packets to access the data for files its caching The approach avoids the cache manager generating read or write I/O request packets to access the data for files its caching –Instead, it copies data from the virtual address of where the portion of the file is, and relies on the memory manager to fault in the data, if needed…

Overview of File Systems NTFS – native file system format for Win2000 NTFS – native file system format for Win2000 Also includes support for: Also includes support for: –CDFS –UDF –FAT12, FAT16 and FAT32

CDFS File System CD-ROM file system – simple format defined in 1998 as the read-only formatting standard for CD CD-ROM file system – simple format defined in 1998 as the read-only formatting standard for CD Format has a number of restrictions Format has a number of restrictions –Directory and file names must be fewer than 32 characters long –Directory trees can be no more than eight levels deep Now considered legacy…. because of… Now considered legacy…. because of…

UDF File System Format replacing CDFS for optical storage media Format replacing CDFS for optical storage media –DVD-ROM in mind… Designed with rewritable media in mind, Windows 2000 UDF driver only provides read- only support Designed with rewritable media in mind, Windows 2000 UDF driver only provides read- only support File systems have the following features: File systems have the following features: –Filenames can be 255 characters long –Maximum path length is 1023 characters –Filenames can be upper and lower case

FAT12, FAT16 and FAT32 Supported to enable upgrades from other versions of Windows, and for compatibility with other OSs, and for use as a floppy disk format Supported to enable upgrades from other versions of Windows, and for compatibility with other OSs, and for use as a floppy disk format 12, 16 and 32 represent the number of bits the format uses to identify clusters on a disk 12, 16 and 32 represent the number of bits the format uses to identify clusters on a disk Windows 2000 uses cluster sizes of 512 bytes to 8 KB in size, limiting a FAT12 volume to 32MB Windows 2000 uses cluster sizes of 512 bytes to 8 KB in size, limiting a FAT12 volume to 32MB –Windows 2000 uses FAT12 as the format for floppy disks FAT32 (most recently defined) FAT32 (most recently defined) –Used in Win95 (OSR2), Win98 and Win Millennium

NTFS Uses 64-bit cluster indexes Uses 64-bit cluster indexes NTFS has the ability to address volumes up to 16 exabytes (16 million GB) NTFS has the ability to address volumes up to 16 exabytes (16 million GB) Windows 2000 limits the size of an NTFS volume to that addressable using 32-bit clusters Windows 2000 limits the size of an NTFS volume to that addressable using 32-bit clusters –128 TB (using 64KB clusters)

NTFS Advanced Features (1) Recoverability Recoverability –File system structures can be repaired to a consistent state (no loss of file or directory structure info – file data can be lost) File and directory security File and directory security –Files and directories are protected from unauthorised access using a combination of a security ID (stored as part of the file) and the password entered… File compression File compression –Transparent to application –Directory compression also possible

NTFS Advanced Features (2) Disk quotas Disk quotas –Per-user volume quotas –NTFS tags files and directories with the security ID of the user who created them… Encryption Encryption –Supports EFS – Encrypting File System –Transparent to application –Files are accessed using using the private key of an accounts EFS private / public key pair etc… etc…