EXT in Detail High-Performance Database Research Center School of Computer and Information Sciences Francisco R. Ortega, Ph.D.

The Sleuth Kit File System Forensic Analysis The Sleuth Kit Chapter 14,15 Slides from those chapters. The Sleuth Kit http://www.sleuthkit.org From site: “is a collection of command line tools and a C library that allows you to analyze disk images and recover files from them. It is used behind the scenes in Autopsy and many other open source and commercial forensics tools.”

The Sleuth Kit

Important Links http://wiki.sleuthkit.org/ http://wiki.sleuthkit.org/index.php?title=NTFS_Implementation_Notes Tools overview http://wiki.sleuthkit.org/index.php?title=TSK_Tool_Overview

Let’s review EXT in more detail Derives from Unix File Systems (UFS) This book (and ppt)describes EXT3 which is an extension of EXT2. However, most systems uses EXT4, which is similar We already went over EXT4 FS Starts with an optional reserved space and then block groups of equal size (except for last one)

Relationship

File System Layout Sample Group Layout

File System Category File System Category of Data is where the general data of a file is located We have two Superblock Group Descriptor

Super Block The ExtX superblock is located 1,024 bytes from the start of the file system and is 1,024 bytes in size although most of the bytes are not used. This data structure contains only configuration values and no boot code. Backup copies of the superblock are typically stored in the first block of each block group.

Super Block The superblock contains block size the total number of blocks the number of blocks per block group the number of reserved blocks before the first block group. the total number of inodes the number of inodes per block group. the volume name the last write time the last mount time

Super Block The superblock contains the path where the file system was last mounted. There are also values that identify if the file system is clean or if a consistency check needs to be run on it. The superblock also keeps some bookkeeping data about the total number of free inodes and blocks. These are used when new inodes and blocks are allocated.

How to determine file system layout? use the block size and number of blocks to calculate the file system size. If this value is less than the volume size, there could be hidden data following the file system, which is called volume slack. The first block group is located in the block following the reserved area.

Superblock The superblock also defines what features in the file system are enabled. For example, if sparse super group is used, only some numbers of blocks (e.g., 1,3,5,7,9,25,27) will contain backup copies. This is enabled by default in Linux

Block Group Descriptors Group after super block, there is a group descriptor table Contains every block group in the file system. Backup copies of the table in every block group Unless sparse option is enabled

Block Group Descriptors in addition to file content, block groups contain administrative data superblocks group descriptor tables inode tables inode bitmaps block bitmaps. The group descriptor describes where these data can be found

Block Descriptor The block bitmap manages the allocation status of the blocks in the group its starting block address is given in the group descriptor. Its size in bytes can be calculated by dividing the number of blocks in the group by eight. When Linux creates a file system, it defines the number of blocks per group to be equal to the number of bits in a block. Therefore, the block bitmap will require exactly one block.

Block Descriptor The inode bitmap manages the allocation status of the inodes in the group, and its starting block address is also given in the group descriptor. Its size in bytes can be calculated by dividing the number of inodes per group by eight. In general, there are fewer inodes than blocks per group but a user can choose these values when they create the file system.

Block Descriptor The starting block address of the inode table is given in the group descriptor its size is calculated by multiplying the number of inodes per group by the size of each inode, which is 128 bytes. The group descriptor also contains the number of free blocks and inodes in the block group. The superblock contains the total number of free blocks and inodes in all groups.

Boot Code Block It is optional. Non-boot drive will not have this clock. If it does exist, it will be found 1024 bytes before the SuperBlock. Most linux do not use the boot code. It uses MBR (master boot loder) to know which block the kernel is found.

Content Category The content category of data includes file and directory contents

Content Categories: Blocks An ExtX block can be 1,024; 2,048; or 4,096 bytes, and the size is given in the superblock. EXT Does not support Fragments (UFS DOES) Therefore, it is assume that fragments are same as block It is a obsolete feature for what EXT is concerned.

Content Categories: Blocks All blocks are given an address, starting with 0, and block 0 is located in the first sector of the file system. All blocks belong to a block group, except in cases where the superblock has defined a reserved area at the beginning of the file system. In that case, the reserved blocks do not belong to a group, and group 0 starts immediately following the reserved blocks.

Content Categories: Blocks How do we know which block does a group belong? (using the number of blocks per group, which is given in the superblock): group = (block – FIRST_DATA_BLOCK) / BLOCKS_PER_GROUP

Content Category: Allocation Status The allocation status of a block is determined by the group's block bitmap, the location of which is given in the group descriptor. The block bitmap will have a full block allocated to it, and each bit corresponds to a block in the group. To determine which bit corresponds to a given block, we need to first determine the block's address relative to the start of the group.

Content Category: Allocation Status This can be thought of as the block's logical group address. The calculation for determining the first block in a group is a variation of the previous one. first_block = group * BLOCKS_PER_GROUP + FIRST_DATA_BLOCK Then OFFSET = BLOCK_ADDRESS – FIRST_BLOCK

Content Category: Allocation Status For example BASE FOR GROUP 1 = 32,768 BLOCK = 60,000 OFFSET = 27232 Not every allocated file block is allocated to a file Administrative data Superblocks Group descriptors Bitmaps for blocks Different than NTFS

Metadata Catagory Metadata is stored in inode and extended attributes

Inodes Ext inodes are of equal size defined in the superblock. One inode is allocated to every file and directory, and each inode has an address, starting with 1. A set of inodes is assigned to each block group. The inodes in each group are stored in a table, whose location is given in the group descriptor.

Inode Address Ext inodes are of equal size defined in the superblock. One inode is allocated to every file and directory, and each inode has an address, starting with 1. A set of inodes is assigned to each block group. The inodes in each group are stored in a table, whose location is given in the group descriptor.

INodes How to find its group? Inodes 1 to 10 are usually reserved Group = (inode – 1 ) / INODES_PER_GROUP Inodes 1 to 10 are usually reserved And allocated Inode 1 keeps track of black blocks Inode 2 is root Inode 8 is ussually reserved for journaling. First user file is in Inode 11 lost+found directory Use in file system check consistency (and files placed there) Any Inode allocated there and that does not have a file pointing there is stored here.

INode Inodes have static number of fields Additional information may be stored in extended attributes or indirect block pointers An inode contains the file's size, ownership, and temporal information.

INode The size value in newer versions of ExtX is 64 bits, but older versions had only 32 bits and therefore could not handle files over 4GB. Newer versions utilize an unused field for the upper 32 bits of the size value and set a read-only compatible feature flag when a large file exists.

INode The type of the file is stored in the mode field, which also contains basic permission values. Everything is a file in Unix, and therefore there are many file types. A normal file that a user creates is called a regular file, and a directory is, intuitively, called a directory. The files are simply there as a name for programs to use when referring to hardware devices or other communication services. Hardware devices are assigned one or more file names, and each will have a file type of a block or a character device. A block device is used for devices that operate on only block- sized chunks of data, such as hard disks, for example. As we saw in Chapter 2, "Computer Foundations," to read any data from a hard disk, you need to read at least 512 bytes. If an application reads less than a sector from a block device, then the OS will read the needed sectors and return only what the application requested. On the other hand, a character device, also called a raw device, is used for devices that do not need to operate in blocks, such as keyboards. A block device typically also has a character device created for it, but an error is generated if it is used to read and write data using non-block sized chunks. The inode space that usually stores information about what blocks has been allocated to a file are used to store device identifier information.

INode Hardware devices are assigned one or more file names, and each will have a file type of a block or a character device. A block device is used for devices that operate on only block- sized chunks of data, such as hard disks. a character device, also called a raw device, is used for devices that do not need to operate in blocks, such as keyboards

INode Inode contains additional information including temporal, permissions, type of file, and so on. See pages 413-415 for additional information

Block Pointers If a file has more blocks than can fit in the 12 direct pointers and the indirect block, a double indirect block is used. A double indirect block is when the inode points to a block that contains a list of single indirect block pointers, each of which point to blocks that contain a list of direct pointers. Lastly, if a file needs still more space, it can use a triple indirect block pointer. A triple indirect block

Block Pointers

Block Pointers

Block Pointers ExtX allows a file to have sparse blocks. Sparse blocks exist when the original block was either undefined or all zeros. Undefined blocks exist because the program that created the file forced the file to be a specific size but never wrote data to some parts. Instead of allocating a block of all 0s, the OS will place the address 0 in the block pointer.

Attributes We will skip the attributes but refer to pages, 417

Directories Entries An ExtX directory is just like a regular file except that it has a special type value in its inode. Directories allocate blocks that will contain a list of directory entry data structures. A directory entry is a simple data structure that contains the file name and the inode address where the file's metadata can be found. The size of the directory corresponds to the number of blocks that it has allocated and is irrelevant to how many files actually exist.

Directories Entries

Directory Entries A directory entry has a dynamic length because the file name can be anywhere from 1 to 255 characters long. Therefore, the data structure has a field that identifies how long the name is and where the next directory entry can be found. The length of the entry is rounded up to a multiple of four

Directory Entries

Directories Entries

Links ExtX provides both hard and soft links so that users can define multiple names for a file or directory. A hard link is an additional name for a file or directory in the same file system. After a hard link is created, you will not be able to tell if it is the original name or a link. To make a hard link, the OS allocates a new directory entry and points it to the original inode. The link count in the inode is incremented by one to account for the new name. A file will not be deleted until all its hard links are deleted.

Links Note that the '.' and '..' entries in each directory are hard links to the current and parent directory. Therefore, the link count for a directory is equal to at least two plus the number of subdirectories it has.

Links Soft links are also a second name for a file or directory, but they can span different file systems. The OS creates a soft link using a symbolic link, which is a special type of file. The full path of the destination file or directory is stored in either blocks allocated to the file or in the inode if the path is less than 60 characters long

Links and Mount Points

Mount Points In Unix, directories can be used for both storing files and volume mount points. Consider a directory dir1 that is in a file system named FS1. If file system FS2 is mounted on the dir1 directory, when a user changes into that directory and lists the contents, the files from FS2 are shown. Even if the dir1 directory has its own files in FS1, they will not be shown when FS2 is mounted on it.

Mount Points

Mount Points Many current post-mortem investigation tools do not show volumes at their mount point, and therefore you will need to determine which volume should be there. On the plus side, because the tools do not show volumes at their mount points, you can see the directory contents of the mount points. One hiding technique is to create files in a directory and then mount a volume on the directory so that a casual observer would not notice them.

Mount Points It is important to know where the files were mount because different directories could have been on different volumes. Many current post-mortem investigation tools do not show volumes at their mount point, and therefore you will need to determine which volume should be there. On the plus side, because the tools do not show volumes at their mount points, you can see the directory contents of the mount points. One hiding technique is to create files in a directory and then mount a volume on the directory so that a casual observer would not notice them.

Hash-trees The user may choose a hash-tree as opposed to a unordered list.

Hash Trees If a directory is using a hash tree, it will have multiple blocks and each will be a node in the tree. Each node contains the files whose hash value is in a given range. The first block of the directory is the root node, and it contains the '.' and '..' directory entries. The rest of the first block contains node descriptors, which contain a hash value and a block address. The OS uses the node descriptors to determine to which block it should jump for a given hash value

Hash Trees

More Look at the rest of Chapter 14 Read Chapter 15.