1. Yet Even More on Drives
CSC 370 (Blum)

2. FAT Review
FAT (16) uses up to 16 bits to address data on the hard drive (or partition thereof)
216 = 65,536
If you address 65,536 sectors, each having 512 bytes then you would have
65,536  = 33,554,432 bytes
= 32,768 kilobytes
= 32 megabytes (MB)
CSC 370 (Blum)

3. Partition Capacity (MB)
Cluster
Clusters are groups of sectors addressed in the FAT system. Within FAT(16)
Sectors/Cluster
Cluster Size (KB)
Partition Capacity (MB) 1
0.5 32 2 64 4
128 8
256 16
512
1024 (= 1 GB)
2048 (= 2 GB)
CSC 370 (Blum)

4. The bigger the cluster, the more the slack
The cluster size is the minimal space that can be used to store a file.
With 32 sectors per cluster, a cluster was 16KB, much larger than many of the files that need to be stored on a typical partition.
The unused portion of all of these clusters is called slack.
While large clusters allowed for larger partitions, they resulted in unacceptable amounts of slack.
CSC 370 (Blum)

5. More addresses needed
To have large capacity partitions without sacrificing much of that capacity to slack, a larger address space is needed
FAT 32 can devote up to 28 bits to addressing (the other four bits are reserved for other purposes)
Allows one to address 228 = 268,435,456 things
CSC 370 (Blum)

6. 228 is a lot
Even if one addressed sectors then theoretically one could a capacity of
268,435,456  512 bytes
137,438,953,472 bytes
134,217,728 kilobytes
131,072 megabytes
128 gigabytes
CSC 370 (Blum)

7. A very fat FAT
The price one pays for having small clusters (which save on slack) is to have a large FAT table.
The FAT table does not take up much room as far as disk space is concerned but it is something one probably wants in memory (disk cache). But a large FAT table will take up too much space in memory.
So partition size, cluster size and FAT table size is a balancing act.
CSC 370 (Blum)

8. FAT32 Table
The table shows the FAT32 Table size for various choices of the partition size and cluster size. The size should be compared to the amount of memory since the FAT table is often in memory (disk cache).
CSC 370 (Blum)

9. NTFS
New Technology File System (NTFS) was built to provide features like:
Reliability: introduced ideas like “transactions” (grouping certain updates together to maintain integrity)
Security and Access Control: built-in features to manage who can access files and what type of access they have
CSC 370 (Blum)

10. NTFS Features (Cont.)
Large-capacity partitions: allows large partitions and even RAID (Redundant Array of Inexpensive Disks, treating multiple disk as one large disk)
Slack reduction: allocates space differently from FAT
Allows for long file names (not limited to 8-character names with 3 character extensions)
Networking: built with networking in mind
CSC 370 (Blum)

11. A more structured file system
In NTFS files are more than just pools of data, they have structure
The difference between FAT and NTFS is analogous to the difference between a flat file and a database.
Just as in databases where one has data and metadata (the data about the data), NTFS has metadata files (files that contain data about other files)
CSC 370 (Blum)

12. Partition/Volume Boot Sector/Record
One of the first things made when an NTFS partition is created is the volume boot sector, which contains:
BIOS parameter block: identifies the partition, how big it is, etc.
Volume boot code: code that starts to load the operating system
CSC 370 (Blum)

13. All else is files
After the volume boot sector, just about everything else is a file. There are
metadata files: files about files
Created automatically when the partition is formatted
Placed at the beginning
(Actual or real) Data files
CSC 370 (Blum)

14. MFT
Think of the Master File Table (MFT) as a database containing records about all of the files (both data and metadata, including itself).
Each file’s record holds the values of its attributes.
The actual data in a data file is simply one of its attributes.
CSC 370 (Blum)

15. The first several records
The first several records in the MFT are about other important metadata files, including
MFT itself
MFT Mirror (1st 16 records)
Log file (keeps account of transactions)
Attribution Definition Table (names file properties and says what they are
Root Directory Folder
Bad cluster file
Etc.
CSC 370 (Blum)

16. MFT Zone
There will be a record in the MFT for every file on the partition.
Thus the MFT needs room to grow.
Some space in the partition, called the MFT Zone, is reserved for this purpose.
If one needs part of the MFT zone for storage, it will eventually be used.
On the other hand. The MFT can grow to be larger than the MFT zone. It is then fragmented which could affect performance.
CSC 370 (Blum)

17. Resident vs. Non-Resident Attributes
The MFT’s record size is fixed (between 1KB and 4KB), but the attributes may be of any size (especially since a data file’s data is an attribute).
Attributes that are contained in the MFT are called resident.
A small file may be entirely resident.
Attributes that are linked to but not actually contained in the MFT are called non-resident.
CSC 370 (Blum)

18. Extents Small files are contained within the MFT
For larger files, the MFT contains a collection of pointers to the data runs of extents which actually hold the data.
If the collection of pointers grows too large, then it is placed in a separate file and the MFT points to this file, which in turns points to the data runs.
CSC 370 (Blum)

19. Some File Attributes
File name: (can be up to 255 characters, allows a file to have aliases)
Standard Information: read-only, hidden, archived, time stamps, etc.
Security Descriptor: Access Control Lists (ACLs) who owns the file, who has what privilege, etc.
Data: the actual data
CSC 370 (Blum)

20. Security
NTFS was designed with the idea of multiple users and security in mind.
The features necessary to implement a security policy are built directly into the file system.
In FAT32 a file may be hidden or read-only, nut in NTFS a file can be hidden from user1, read-only to user2 and fully accessible to user3.
CSC 370 (Blum)

21. Security Concepts
Ownership: some user owns a file/folder and he or she grants permissions to other users.
Permissions: what a user can do with a file/folder (read, read-write, delete, etc.)
Users are placed in groups (possibly more than one) and permissions are assigned to groups
Permissions can be inherited, e.g. new files gets permissions of folder it was created in
Auditing: tracking information about users’ access to and modification of files
CSC 370 (Blum)

22. ACLs
An important security attribute of a file is its Access Control List (ACL).
The ACL specifies which users can access the file and in what way they can access the file
There are two types of ACL:
System ACL: used for auditing purposes
Discretionary ACL: explicit assigning of permissions to users or groups
CSC 370 (Blum)

23. Permissions
CSC 370 (Blum)

24. Versions
NTFS version 1.1 (a.k.a. version 4.0) is used with Windows NT 4.0.
Usually Windows 2000 uses a revised version known as NTFS version 5.0
CSC 370 (Blum)

25. Improvements (NTFS 5 over 4)
Reparse Points: One can associate an action or actions with a file. So that if the file is accessed, the action is performed.
Analogous to a trigger in a database
Reparse points is very flexible, one example is redirection sending one to another file or directory, it may be on another drive or even have been archived.
CSC 370 (Blum)

26. Improvements (NTFS 5 over 4)
Improved Security and Permissions: one change is from static to dynamic permission inheritance
Change Journals: improved auditing (journaling) of file/folder access activity.
Encryption: Automatic encryption/decryption of files (when accessed by users with the appropriate permissions).
CSC 370 (Blum)

27. Improvements (NTFS 5 over 4)
Disk Quotas: Users or groups of users can be limited in the amount of disk space they can use.
Sparse File Support: A sparse file is one that may be big but hold very little data (relative to its size). NTFS has utilities to help store sparse files more efficiently.
Disk Defragmenter: Strictly speaking part of the operating system, it affects the file system.
CSC 370 (Blum)

28. Transactions Don’t forget NTFS is pretty much a database.
Almost any activity involving the drive in anyway is going to affect a number of files.
NTFS introduces the notion of a transaction – the grouping together of various operations to form an “atomic” unit.
In other words these operations should be viewed as “all or nothing” in order to maintain the file system’s integrity.
Recall the “ACID test” from databases?
CSC 370 (Blum)

29. Logging and Committing
There is a special metafile for logging all activity.
When all of the components of a transaction are complete, this completion is indicated in the log file and the transaction is said to be committed.
If something goes wrong (e.g. power failure) before a transaction is completed, the file system can undo the partially enacted transaction to return the file system to a consistent state. Doing so is said to be rolling back the transaction.
It is also called transaction recovery.
CSC 370 (Blum)

30. Effect on Performance
Logging each activity which is great for security and integrity of the file system but does have some negative effects on performance.
Each file access now requires an another file access (writing to the log file).
One way to save on performance but risk somewhat integrity is to cache the activity log changes rather than write to disk every time.
The cached log results are written to disk periodically but not continuously.
CSC 370 (Blum)

31. Recovery Recovery then involves three passes over the log file:
Analysis pass: determine the part of the disk affected
Redo pass: perform any transaction that was completed since the last “checkpoint”
Undo pass: roll back any incomplete transactions
CSC 370 (Blum)

32. Change Journal
NTFS can record changes to files, these are kept in the Change Journal.
Each change is assigned an ID, an Update Sequence Number (USN).
It will record that a file was written to but not what was written. Otherwise it would be gargantuan.
CSC 370 (Blum)

33. Fault Tolerance
NTFS has a fault tolerance disk driver known as FTDISK.
That’s where one can find the transaction recovery features.
Also where one finds support for RAID (redundant array of inexpensive (or is that independent) disks).
And where you’ll find dynamic bad cluster remapping.
Basically the drive reads immediately after writing to ensure that the cluster written to was OK. If it was not, it writes it somewhere else and marks the cluster as bad.
CSC 370 (Blum)

34. Compression NTFS has build-in utilities for file compression
File compression takes advantage of patterns in data to reduce the amount of space required to store it
E.g. instead of ASCII code for text (each character 8 bits) one might use a variable length code with short codes for common letters like e and longer codes for uncommon letters like q or j. On average the files are much smaller.
In NTFS one can compress any part of the partition.
CSC 370 (Blum)

35. POSIX support NTFS offers POSIX support.
POSIX stands for Portable Operating System Interface for UNIX
It allows software developments to make sure that their code can be ported to a POSIX-compliant operating system, which includes most versions of UNIX.
CSC 370 (Blum)

36. Supports Encryption NTFS supports Encrypting File System (EFS).
EFS is really part of the operating system (Windows 2000). But the operating system works with the file system to make this feature easy to use.
CSC 370 (Blum)

37. Disk Quota support
As a genuinely multi-user file system, NTFS support disk quotas
A quota can be set for a particular user or on a particular partition or the combination.
Allows for limits and warnings. The user is warned when he or she exceeds the warning amount. The user is blocked (from writing?) when he or she exceeds the limit amount.
Monitor and log events that cause a user to go over the "limit" or "warning" levels.
CSC 370 (Blum)

38. References http://www.pcguide.com
PC Hardware in a Nutshell, Thompson and Thompson
CSC 370 (Blum)