File Systems and Partitioning Systems © Dr. D. Kall Loper, all rights reserved Digital Forensics File Systems and Partitioning Systems

File Systems The Windows Family of File Systems A file system is part of the computer’s operating system. Several different systems have been developed to meet the growing needs of users for larger drives, greater efficiency, more speed, and security. © Dr. D. Kall Loper, all rights reserved

File Systems When a disk is formatted, it is imprinted with the structures used to control files. Low Level Formats (done at the factory) imprint cylinders, tracks, & sectors. High Level Format (user-level) define file system, clusters, partition system, & data structure (MFT, FAT, etc.) © Dr. D. Kall Loper, all rights reserved

File Systems Definitions Storage Volume A storage volume is defined by the operating system and is the logical unit used by the operating system. Storage volume is often used interchangeably with partition, but this is not always accurate. Storage volumes may exist without a physical partition. Definitions

File Systems Floppy Disk Illustration Hard Disk – Bootable Partition

File Systems Storage Volumes The floppy disk has no partition information. The boxes define where partition information would be. The text is also a clue. © Dr. D. Kall Loper, all rights reserved

File Systems Definitions Partition A partition is a division of space on a storage device. A storage volume must be created within a partition to allow the operating system to interact with the disk space. Partitions allow a single storage medium to act like multiple “virtual” storage media. For example, it allows multiple file systems (FAT & NTFS) on a single disk. Definitions

File Systems Partition Structures File Allocation Table (FAT)/MBR Master File Table (MFT)/MBR Master File Table (MFT)/GPT MBR, http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/prork/prcb_dis_stfl.mspx © Dr. D. Kall Loper, all rights reserved

FAT File Allocation Table

File Systems DOS, Windows 3.x, Windows 95, Windows 98, & Windows ME It is not necessary to know all of the arcane details of file systems to know where evidence comes from. However, a few details of the FAT file system can make recovering hidden data much easier. © Dr. D. Kall Loper, all rights reserved

File Systems DOS, Windows 3.x, Windows 95, Windows 98, & Windows ME There has been a historic progression of FAT systems from early DOS versions using FAT12 through FAT 16 of the mature DOS and early Windows systems to the final version of FAT, FAT32. © Dr. D. Kall Loper, all rights reserved

File Systems FAT-12 FAT-16 FAT-32 File Size Limit 32 MB 2 GB 4 GB Files Limit 4,077 65,517 268,435,437 Volume Size Limit 2 GB / 4GB 2 TB © Dr. D. Kall Loper, all rights reserved

File System Illustration

File System Illustration FAT 16 Structure Sector 0 Boot Sector Root Directory Sector 33 Beginning of Data Area Illustration © Dr. D. Kall Loper, all rights reserved

File System Illustration Partition relative offset and disk absolute offset Illustration Determining absolute offset from relative offset

File Systems Interpreting the FAT16 Table entries by hand…because you’re a huge nerd. 0x00 8 8 character DOS file name 0x08 3 3 character DOS .3 extension 0x0b 1 File Attributes (bit flags) 0x0c Reserved (NT use) 0x0d Create Time, high precision (x10ms 0-199) 0x0e 2 Create Time 0x10 Create Date 0x12 Last Access Date 0x14 Reserved (NT use) also used in FAT32 0x16 Last Modified Time 0x18 Last Modified Date 0x1a First Cluster (FAT16) also used in FAT32 0x1c 4 File Size in bytes (volumes and subdirs =0) Columns = Hex offset, length in bytes, description Attribute bit flags 00x01 Read Only 10x02 Hidden 20x04 System 30x08 Volume Label 40x10 Subdirectory 50x20 Archive 60x40 Device (not an expected value) 70x80 Unused 15-11Hours (0-23)10-5Minutes (0-59)4-0Seconds/2 (0-29) 15-9Year (0 = 1980, 127 = 2107)8-5Month (1 = January, 12 = December)4-0Day (1 - 31) Long File Names (LFN) are stored on a FAT file system using a trick—adding (possibly multiple) additional entries into the directory before the normal file entry. The additional entries are marked with the Volume Label, System, Hidden, and Read Only attributes (yielding 0x0F), which is a combination that is not expected in the MS-DOS environment, and therefore ignored by MS-DOS programs and third-party utilities. Notably, a directory containing only volume labels is considered as empty and is allowed to be deleted; such a situation appears if files created with long names are deleted from plain DOS. (Wikipedia)

File System Illustration

File Systems FAT-based Folders Directories have a set of 32-byte Folder Entries for each file and subfolder contained in the folder. © Dr. D. Kall Loper, all rights reserved

File System FAT32 file entry © Dr. D. Kall Loper, all rights reserved 0003EE80 4D 59 46 49 4C 45 20 20 54 58 54 20 00 C3 D6 93 MYFILE TXT .AO“ 0003EE90 56 2B 56 2B 00 00 EE 93 56 2B 03 00 33 B7 01 00 V+V+..i“V+..3•.. Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F © Dr. D. Kall Loper, all rights reserved

File System FAT32 file entry 00 01 b7 33xh = 112,435xd 0003EE80 4D 59 46 49 4C 45 20 20 54 58 54 20 00 C3 D6 93 MYFILE TXT .AO“ 0003EE90 56 2B 56 2B 00 00 EE 93 56 2B 03 00 33 B7 01 00 V+V+..i“V+..3•.. Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F 00 01 b7 33xh = 112,435xd File Size = 112,435 bytes © Dr. D. Kall Loper, all rights reserved

File System FAT32 file entry Starting Cluster = 3 0003EE80 4D 59 46 49 4C 45 20 20 54 58 54 20 00 C3 D6 93 MYFILE TXT .AO“ 0003EE90 56 2B 56 2B 00 00 EE 93 56 2B 03 00 33 B7 01 00 V+V+..i“V+..3•.. Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F Starting Cluster = 3 File length / Cluster size = clusters allocated © Dr. D. Kall Loper, all rights reserved

File Systems FAT-based Folders Name (8.3 format) Attribute byte (8 bits) Create time (24 bits) Create date (16 bits) Last access date (16 bits) Last modified time (16 bits) Last modified date (16 bits.) Starting cluster number in the file allocation table (16 bits) File size (32 bits) © Dr. D. Kall Loper, all rights reserved

File Systems FAT-based File Systems One of the most useful characteristics of the FAT systems to forensic analysts is that deleted files aren’t wiped from the drive. Most people know that somehow deleted files can be recovered. © Dr. D. Kall Loper, all rights reserved

File Systems FAT-based File Systems The reason that works is the need for speed. To save time on unneeded operations, when a file is deleted, the file is not altered, except for a marker at the beginning. © Dr. D. Kall Loper, all rights reserved

File Systems FAT-based File Systems The only real change is that the file allocation table marks the clusters used by the file as open or ‘unallocated’. If no new data is stored to those clusters, the data can sit there intact for years. © Dr. D. Kall Loper, all rights reserved

File Systems FAT-based File Systems Even if new data is saved, it may not completely fill the unallocated space. This leaves fragments of the previous file in places that cannot normally be accessed. We call this slack space. © Dr. D. Kall Loper, all rights reserved

MFT

File Systems NTFS: New Technology File System Some utilities have been created that allow DOS to read NTFS, for example NTFS-DOS is freeware that allows DOS operations on an NTFS volume. Although NTFS implements a different method for dealing with files, it still has slack space and unallocated clusters that may contain old data. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: New Technology File System Like FAT, NTFS does not manage individual 512-byte sectors independently. It groups them into clusters, or ‘allocation units.’ © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: New Technology File System If individual sectors were used for data storage on a very large disk, many resources would be required to keep track of what was in each one, and fragmentation of the disk would be much more of a problem. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS allows system administrators to set cluster size; however, the default values are the most commonly used and efficient values. Partition Size Default Cluster Size 0 MB – 512 MB 512 Bytes 512 MB – 1 GB 1 KB (1024 Bytes) 1 GB – 2 GB 2 KB (2048 Bytes) 2 GB + 4 KB (4096 Bytes) © Dr. D. Kall Loper, all rights reserved

File Systems MFT The MFT is an index of data about the files, directories, and metafiles in the system According to the logic of the MFT, all the data in a file is simply a ‘property’ or extent of the file object. © Dr. D. Kall Loper, all rights reserved

File System Illustration

File Systems MFT: Resident Files The MFT stores all data about a file. If the file is small, upto 800 bytes may be stored in the MFT itself. Long file names and Alternate Streams can force the file out of residence due to space used. EFS encrypted files, sparse files, and compressed files cannot be resident. Through this mechanism, there CAN be more files than clusters. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: File Properties Standard Information Includes information such as timestamp and link count. Attribute List Lists the location of all attribute records that do not fit in the MFT record. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: File Properties File Name A repeatable attribute for both long and short file names. The long name of the file can be up to 255 Unicode characters. The short name is the 8.3, case-insensitive name for the file. Additional names, or hard links, required by POSIX can be included as additional file name attributes. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: File Properties Security Descriptor Describes who owns the file and who can access it. Data Contains file data. NTFS allows multiple data attributes per file. Each file typically has one unnamed data attribute. A file can also have one or more named data attributes, each using a particular syntax. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: File Properties Object ID A volume-unique file identifier. Used by the distributed link tracking service. Not all files have object identifiers. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: Special Files $Mft Contains one base file record for each file and folder on an NTFS volume. If the allocation information for a file or folder is too large to fit within a single record, other file records are allocated as well. $MftMirr A duplicate image of the first four records of the MFT. This file guarantees access to the MFT in case of a single-sector failure. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: Special Files $LogFile Contains a list of transaction steps used for NTFS recoverability. Log file size depends on the volume size and can be as large as 4 MB. It is used by Windows NT/2000 to restore consistency to NTFS after a system failure. $Volume Contains information about the volume, such as the volume label and the volume version. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: Special Files $AttrDef A table of attribute names, numbers, and descriptions. $ The root folder. $Bitmap A representation of the volume showing which clusters are in use. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: Special Files $Boot Includes the BPB used to mount the volume and additional bootstrap loader code used if the volume is bootable. $BadClus Contains bad clusters for the volume. $Secure Contains unique security descriptors for all files within a volume. © Dr. D. Kall Loper, all rights reserved

File Systems NTFS: Special Files $Upcase Converts lowercase characters to matching Unicode uppercase characters. $Extend Used for various optional extensions such as quotas, reparse point data, and object identifiers. http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/prork/prcb_dis_stfl.mspx? © Dr. D. Kall Loper, all rights reserved

MBR Master Boot Record

Partition Systems Boot Code “Intel-based computers rely on the system BIOS to load and execute bootstrap code.” “The first physical sector is called the master boot record (MBR) and contains the primary bootstrap loader code.” Microsoft Knowledgebase article 149877 http://support.microsoft.com/kb/149877 © Dr. D. Kall Loper, all rights reserved

Partition Systems Boot Code “The BIOS bootstrap routine generates an int 0x19 which loads the first sector of the floppy or hard disk (0:0:1 in CHS) in memory at segment address 0000:7C00H.” Microsoft Knowledgebase article 149877 http://support.microsoft.com/kb/149877 © Dr. D. Kall Loper, all rights reserved

Partition Systems Boot Code “After loading sector zero, the BIOS checks that the last two bytes of that sector are 55AA” in hexadecimal. “This… is called a boot record signature and is kind of like an EOF when the sector is read. This is something that [older] BIOS requires when booting.” Microsoft Knowledgebase article 149877 http://support.microsoft.com/kb/149877 © Dr. D. Kall Loper, all rights reserved

Partition Systems Hard Disk – Bootable Partition Illustration

GUID Partition Table (Globally Unique Identifier) GPT GUID Partition Table (Globally Unique Identifier)

Partition Systems GPT The GUID Partition Table (GPT) was introduced as part of the Extensible Firmware Interface (EFI) initiative. In theory, a GPT disk can be up to 264 logical blocks in length. Logical blocks are commonly 512 bytes in size. In other words, over 8 Zettabytes (8x 10247) http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx © Dr. D. Kall Loper, all rights reserved

Partition Systems GPT Sector Zero The GPT includes a “protective MBR” that prevents MBR-based systems from overwriting the GPT. The entire GPT is identified as a single partition with the type identifier 0xEE. If the disk exceeds two terabytes (32bit LBA max size), 2 TB is used and the rest of the disk is ignored. http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx © Dr. D. Kall Loper, all rights reserved

Partition Systems How big can GPT systems get? The maximum partition size is a function of the operating system version. Windows XP and Windows Server 2003 have a limit of 2TB per physical disk. Windows Server 2003 SP1 Windows XP x64 edition, and later versions, the maximum raw partition of 18 exabytes can be supported. Windows file systems currently are limited to 256 terabytes each.

Partition Systems How big can GPT systems get? Most calculations assume 512 byte sectors. However, other devices may use different sector sizes. The International Diskdrive Equipment and Materials Association (IDEMA) has announced production of 4,096 byte sectors which firmware can present to the OS as 512 byte sectors for legacy compatibility.

Partition Systems GPT Partition Structure Windows defined GPTs have a complex of partitions. Some are variable, some are optional, but every Windows-based GPT disk must contain an MSR (Microsoft Reserved Partition). http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx © Dr. D. Kall Loper, all rights reserved

Partition Systems GPT Partition Structure The order of partitions on the disk should be: ESP (optional), OEM (optional and variable), and MSR followed by primary data partition(s). http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx © Dr. D. Kall Loper, all rights reserved

Partition Systems ESP, EFI System Partition This system partition contains boot files and drivers. On Windows systems, this includes HAL, NTLDR, Boot.txt, and others. Size: ≈100MB GUID for the ESP: {C12A7328L-F81F-11D2-BA4B00A0C93EC93B} http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx © Dr. D. Kall Loper, all rights reserved

Partition Systems OEM Reserved Partition Manufacturers place value-add contents in an OEM-specific partition. The contents of GPT OEM partitions are not exposed (given drive letters or returned in volume lists). Users are warned that deleting the partition can cause the system to fail to operate. Size: Defined by OEM GUID for the OEM: Defined by OEM http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx Microsoft encourages OEMs to use an OEM partition. © Dr. D. Kall Loper, all rights reserved

Partition Systems MSR, Microsoft Reserved Partition GPT disks do not allow hidden sectors. Software components that formerly used hidden sectors now allocate portions of the MSR for component-specific partitions. Size: 32-128 MB GUID for the MSR {E3C9E316-0B5C-4DB8-817DF92DF00215AE} http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx © Dr. D. Kall Loper, all rights reserved

Partition Systems Basic Data Partition User and system data. Size: User defined at creation GUID for the Basic Data Type: {EBD0A0A2-B9E5-4433-87C068B6B72699C7} http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx Dynamic disks use two different GPT partitions: •A data container partition corresponding to the MBR partition 0x42, with the following GUID: DEFINE_GUID (PARTITION_LDM_DATA_GUID, 0xAF9B60A0L, 0x1431, 0x4F62, 0xBC, 0x68, 0x33, 0x11, 0x71, 0x4A, 0x69, 0xAD);•A partition to contain the dynamic configuration database, with the following GUID: DEFINE_GUID(PARTITION_LDM_METADATA_GUID, 0x5808C8AAL, 0x7E8F, 0x42E0, 0x85, 0xD2, 0xE1, 0xE9, 0x04, 0x34, 0xCF, 0xB3);Volumes are created in the data container and mounted by default. Again, this is exactly the same as the contents of 0x42 MBR partitions. © Dr. D. Kall Loper, all rights reserved

Partition System Basic Data Partition Type GUID Image Source: http//:www.digitalforensics.ch

Partition Systems GUID Collision Although GUID’s are so large that it is extremely unlikely that two will collide randomly, algorithms have been developed to ensure uniqueness. V1 GUIDs use the MAC address for the fifth group in the GUID. Example: {FFFFFFFF-FFFF-1FFF-FFFF-FFFFFFFFFFFF} V3 GUIDs use 6 bytes from the name V4 GUIDs use 6 bytes pseudo-randomly generated Paul J. Leach, Microsoft and Rich Salz, Certco 2/4/1998 © Dr. D. Kall Loper, all rights reserved

Partition Systems WTF-MAC address V1 GUIDs use the MAC address for the fifth group in the GUID. Example: {FFFFFFFF-FFFF-1FFF-FFFF-FFFFFFFFFFFF} If you noticed that my example’s MAC address was the broadcast address, you are a huge nerd.

Partition Systems GUID Time The timestamp is a 60 bit value. For UUID version 1, this is represented by Coordinated Universal Time (UTC) as a count of 100- nanosecond intervals since 00:00:00.00, 15 October 1582 (the date of Gregorian reform to the Christian calendar). Paul J. Leach, Microsoft and Rich Salz, Certco 2/4/1998

Partition System Illustration

Partition Systems © Dr. D. Kall Loper, all rights reserved Offset Length Contents 8 bytes Signature “EFI PART” 8 4 bytes Revision number 12 Header size 16 CRC32 of header with first 20 bytes= 00 20 00 00 00 00 24 LBA location of Primary GPT Header 32 LBA location of Secondary GPT Header 40 First usable LBA for partitions 48 Last usable LBA for partitions 56 16 bytes Disk GUID 72 Partition entries starting LBA 80 Number of partition entries 84 Size of partition entries (128) 88 CRC32 of partition array 92 420 bytes* 00 00 00 00 00 00 00… Assumes 512 byte LBA’s See also http://www.microsoft.com/whdc/device/storage/gpt_faq.mspx GPT Header: 45 46 49 20 50 41 52 54 = EFI PART © Dr. D. Kall Loper, all rights reserved

Partition Systems Offset Length Contents 16 bytes Partition type GUID 16 Partition GUID 32 8 bytes First LBA 40 Last LBA 48 Attribute Flags 56 72 bytes Partition name (UTF-16LE, 36 units) 128 * Assumes 512 byte LBA’s All partition entries share this format in GPT system. The OS determines how many entries are reserved, but 128 is the minimum. © Dr. D. Kall Loper, all rights reserved

Partition Systems Offset Length Contents 16 bytes Partition type GUID 16 Partition GUID 32 8 bytes First LBA 40 Last LBA 48 Attribute Flags 56 72 bytes Partition name (UTF-16LE, 36 units) 128 * Assumes 512 byte LBA’s The OS determines how many entries are reserved, but 128 is the minimum.* © Dr. D. Kall Loper, all rights reserved

Partition Systems EFI Specification The EFI specification requires 16,384 bytes be reserved for the partition table. The most common implementation of this is 128 partitions with 128 bytes per record. However, OS vendors are warning programmers not to hard code these values. Paul J. Leach, Microsoft and Rich Salz, Certco 2/4/1998

Partition Systems Entry 1: EFI System Partition (ESP) Entry 2: OEM Partition Entry 3: MSR Partition Entry 4: Basic Data Partition 1 (FAT) Entry 5: Basic Data Partition 2 (NTFS) * Assumes 512 byte LBA’s © Dr. D. Kall Loper, all rights reserved

Posix Speak Unix or Go Home

File Systems Posix: IEEE 1003 Posix is not a file system, but it provides a layer of abstraction setting standards for storage across Unix systems. Unix uses iNodes to organize and retrieve stored data; these are broadly defined under Posix. © Dr. D. Kall Loper, all rights reserved

File Systems Definitions Posix http://standards.ieee.org/develop/wg/POSIX.html © Dr. D. Kall Loper, all rights reserved

Posix Standard for inodes The length of the file (bytes). (Storage) Device ID UID of the file's owner. GID of the file's group. The file mode (permissions & access rights)

Posix Standard for inodes Timestamps telling when the: inode was last changed (ctime, changing time), the file content was last modified (mtime, modification time), and Inode was last accessed (atime, access time). Link count telling how many hard links point to the inode. Pointers to the disk blocks that store the file's contents.