File Systems Forensics
1
File Systems Forensics A FS can be viewed as a method to store or retrieve data for long term, from a storage array, using a computer. It has 2 basic functions: ● Map Physical spaces on the drive to logical addresses that comprise files. ● Read/Write capability to open/change/delete files. FS does not permanently erase files on the hard drive, just creates a flag that indicates sector can be reused. 5 Layer FS Reference Model: ● File system category ● Content category ● Metadata category ● Filename category ● Application category
Physical Data Hiding Techniques ● Deleted files and files contained in the swap space in virtual memory created on hard drive by the OS Linux ● Used when the system needs more memory resources and the RAM is full. Inactive pages in memory are moved to the swap space. Windows ● Hidden system file used for virtual memory when there is not enough physical memory to run programs. ● Space on the hard drive is temporarily swapped with the RAM as programs are running. Slack Space ● Happens when a FS creates some space to store the data (clusterin Windows or block in Linux), but do not necessarily use the entire fixed length space that was allocated Unallocated Space ● When a user deletes a file, it is flagged as no longer needed, but it remains on the system until it is overwritten. A forensic software tool can identify and restore these files.
Data Hiding Techniques (Hiding data on Hard drive) ●
Hard drive structure to identify those hidden spaces
2
● ● ● ● ● ●
●
Host Protected Area (HPA) – reserved area in hard drive where data can be hidden. Device Configuration Overlay (DCO) – using this, some features provided by hard drive can be modified. Master Boot Record (MBR) – space reserved at the beginning of the HDD. Sector – minimum addressable unit of a disk or cluster. Partition – set of consecutive blocks on a hard disk that appear to the OS as a separate logical volume Partition slack – when the total number of sectors in a partition is not a multiple of the block size, some sectors at the end of the partition may end up not being accessed by the OS (this is referred to as partition slack) Volume stack – when a hard disk is partitioned and not all the spaces are utilized, the OS can take
3
● ● ● ● ● ●
advantage of these spaces by using them (this is volume slack) Virtual file system (VFS) – a hard drive can be partitioned into VFSs before being used Boot sectors in nonbootable partitions can be used to hide data Unallocated space – those spaces in a partition that are not allocated to a particular file cannot be accessed by the OS. They can be used to hide data. Disk slack – when a file is not an exact multiple of the sector size, the OS must pad the last sector with some data. Block groups are used to store file names, metadata, and file content. Superblocks – Information about block group size and configuration is stored in a superblock, located at the beginning of the FS.
Volatility of Data Hiding Areas
FAT File System Forensics Simple FS found in common OS’s. 2 main data structures: ● File Allocation Table ● Directory Entries Versions of FAT: FAT12, FAT16, FAT32 Eliminate the disadvantages of the linked list allocation by taking the link out of each disk block and gathering them together in a table (on disk and cached in memory). The layout of the FAT file system consists of 3 physical sections: ● Reserved area – for file system category ● FAT area – primary and backup FAT structures ● Data area – clusters used for storing file and directory content
4
FAT Cluster Allocation Status Clusters: ● Group of consecutive sectors; ● Number of sectors: power of 2. ● Located in the data area region of the FS. ● Difficult to find the location of the 1st cluster (Cluster 2) because it is not at the beginning of the FS. Some algorithms can be used for such purpose. ● The allocation status of a cluster is determined by an Allocation Algorithm for Clusters (chosen by the OS), based on the FAT structure. Allocation Algorithms for Clusters ● Basic – to find an unallocated cluster, the OS scans the FAT for an entry with a 0 in it. ● First available – always start the scan at the beginning of the file system (not good for sequential writing)(content at the end of the FS may persist) ● Next available – maintain a pointer to the last cluster allocated and start the scan there (good for sequential writing of a file in a scatter storage environment) ● Best fit – Search for consecutive clusters that fit the size of file. Only works for files that do not grow Directory Entry Structure ● 32 Bytes in size ● Stores information on: ○ Location where contents are stored ○ File timestamps ○ Permissions ○ File Metadata (name, size, etc.) ● The allocation status of a directory entry is determined by using the first byte File Timestamps Each directory entry has 3 times in it: ● Time Created – accurate to a tenth of a second. ● Last Accessed – accurate to the day. ● Last Written – accurate to 2 seconds. Created ● Set when OS allocates a new directory entry for a new file
5 ●
If file has been renamed or moved to another directory, original creation time kept. If file is copied, a new file will be created and and a new creation time will be written to the new entry. Written ● Set when OS writes new file content ● If file has been renamed or moved to another directory, original written and creation time. ● If files are moved or copied, the new directory entry will have the written time from the original file. ● If you move a file, the resulting file will have the original written times and creation time. ● If file is copied, the resulting file will have the original written time and a new creation time. Accessed: Updated the most frequently when ● File is opened ● File is moved to a new volume. Exception is when moving within the same volume. ● Copying and Moving files
Creating Files (dir1/file1.txt, cluster size=4096 bytes, filesize= 6000 bytes) 1. Read the boot sector from sector 0 of the partition and locate the FAT structures, data area and root directory. 2. Process each directory entry to find dir1 and its starting cluster (90). 3. Read the contents of dir1 and find an unallocated directory entry. 4. Set its allocation status by writing the filename file1.txt, update the written time. 5. To find clusters for content, search the FAT structure. Allocate cluster (200) and set its entry to the EOF value 6. Write the address of cluster (200) in the starting cluster field of the directory entry. Write the 4096 bytes to that cluster; 1094 bytes remain so a second cluster is needed 7. Search FAT for another cluster and allocate cluster (201) 8. The FAT entry for the first cluster (200) is changed to contain (201). The remaining bytes are written and cluster (201) is set to EOF in the FAT.
6
Delete the dir1/file1.txt file 1. Read the boot sector from sector 0 of the partition and locate the relevant structures. 2. Locate the dir1 directory as before. 3. Process the contents of dir1’s starting cluster (90) to find a directory entry for file1.txt. We find cluster 200. 4. Use the FAT structure to determine the cluster chain for the file. Here, it has clusters 200 and 201. 5. Set the FAT entries for clusters 200 and 201 to 0 6. Unallocate the directory entry for file1.txt by changing the first byte to .
NTFS File System No specific layout like for other FS, (i.e) ● Every byte of data in NTFS is allocated to a file. ● Everything is a file, including the file that contain the administrative data. ● The entire FS is considered a data area and any sector can be allocated to a file. ● Only consistent layout is that the first sectors contain the boot sector and boot code. Master File Table (MTF) ● Is a file ● Has an entry for itself, $MFT ● Boot sector has starting location of the MFT ● First normal entry in the table determines the layout and size of the MFT. ● Contains the information about all files and directories. ● Uses Clusters ● Every file and directory has at least one entry in the table.
7 ● ● ● ●
Entries are 1 KB in size. Only the first 42 bytes of an entry have a defined purpose The remaining bytes are used to store file attributes File attributes: are small data structures with specific purpose, (e.g.) filename $FILE_NAME or file contents $DATA
Master File Table Entries ● The first 42 bytes of an MFT entry contain 12 fields and the remaining 982 bytes are unstructured and can be filled with attributes. ● First field is signature (magic number): FILE or BAAD. ● A flag field indicates if the MFT entry is being used and is for a directory. ● When a file cannot fit its attributes into one MFT entry, more MFT entries (hence clusters/disk space) can be allocated for that file. Attribute referred to as nonresident attribute. Master File Table Attribute ● NTFS exist to read/write file attributes ● Every file has a $DATA attribute, which stores the file contents ● 2 Parts: Header and Content ○ Header identifies the type of attribute, its size, name, and flags (compressed, encrypted, etc.). ○ Stored in 2 locations: ■ In the MFT entry with the attribute header ■ In an external cluster or cluster run Alternative Data Stream (ADS) ● If a file has > 1 $DATA attribute, the additional ones are called Alternate Data Streams (ADS), with names ● Command line can access ADSs using redirection operators filename:stream name ● Creating an ADS: writing a stream into an ADS named hidden: ○ (ex) C:\> echo Hidden text > test.txt:hidden ● The metadata can be viewed by redirecting from it to more ○ C:\>more
File Systems Forensics A FS can be viewed as a method to store or retrieve data for long term, from a storage array, using a computer. It has 2 basic functions: ● Map Physical spaces on the drive to logical addresses that comprise files. ● Read/Write capability to open/change/delete files. FS does not permanently erase files on the hard drive, just creates a flag that indicates sector can be reused. 5 Layer FS Reference Model: ● File system category ● Content category ● Metadata category ● Filename category ● Application category
Physical Data Hiding Techniques ● Deleted files and files contained in the swap space in virtual memory created on hard drive by the OS Linux ● Used when the system needs more memory resources and the RAM is full. Inactive pages in memory are moved to the swap space. Windows ● Hidden system file used for virtual memory when there is not enough physical memory to run programs. ● Space on the hard drive is temporarily swapped with the RAM as programs are running. Slack Space ● Happens when a FS creates some space to store the data (clusterin Windows or block in Linux), but do not necessarily use the entire fixed length space that was allocated Unallocated Space ● When a user deletes a file, it is flagged as no longer needed, but it remains on the system until it is overwritten. A forensic software tool can identify and restore these files.
Data Hiding Techniques (Hiding data on Hard drive) ●
Hard drive structure to identify those hidden spaces
2
● ● ● ● ● ●
●
Host Protected Area (HPA) – reserved area in hard drive where data can be hidden. Device Configuration Overlay (DCO) – using this, some features provided by hard drive can be modified. Master Boot Record (MBR) – space reserved at the beginning of the HDD. Sector – minimum addressable unit of a disk or cluster. Partition – set of consecutive blocks on a hard disk that appear to the OS as a separate logical volume Partition slack – when the total number of sectors in a partition is not a multiple of the block size, some sectors at the end of the partition may end up not being accessed by the OS (this is referred to as partition slack) Volume stack – when a hard disk is partitioned and not all the spaces are utilized, the OS can take
3
● ● ● ● ● ●
advantage of these spaces by using them (this is volume slack) Virtual file system (VFS) – a hard drive can be partitioned into VFSs before being used Boot sectors in nonbootable partitions can be used to hide data Unallocated space – those spaces in a partition that are not allocated to a particular file cannot be accessed by the OS. They can be used to hide data. Disk slack – when a file is not an exact multiple of the sector size, the OS must pad the last sector with some data. Block groups are used to store file names, metadata, and file content. Superblocks – Information about block group size and configuration is stored in a superblock, located at the beginning of the FS.
Volatility of Data Hiding Areas
FAT File System Forensics Simple FS found in common OS’s. 2 main data structures: ● File Allocation Table ● Directory Entries Versions of FAT: FAT12, FAT16, FAT32 Eliminate the disadvantages of the linked list allocation by taking the link out of each disk block and gathering them together in a table (on disk and cached in memory). The layout of the FAT file system consists of 3 physical sections: ● Reserved area – for file system category ● FAT area – primary and backup FAT structures ● Data area – clusters used for storing file and directory content
4
FAT Cluster Allocation Status Clusters: ● Group of consecutive sectors; ● Number of sectors: power of 2. ● Located in the data area region of the FS. ● Difficult to find the location of the 1st cluster (Cluster 2) because it is not at the beginning of the FS. Some algorithms can be used for such purpose. ● The allocation status of a cluster is determined by an Allocation Algorithm for Clusters (chosen by the OS), based on the FAT structure. Allocation Algorithms for Clusters ● Basic – to find an unallocated cluster, the OS scans the FAT for an entry with a 0 in it. ● First available – always start the scan at the beginning of the file system (not good for sequential writing)(content at the end of the FS may persist) ● Next available – maintain a pointer to the last cluster allocated and start the scan there (good for sequential writing of a file in a scatter storage environment) ● Best fit – Search for consecutive clusters that fit the size of file. Only works for files that do not grow Directory Entry Structure ● 32 Bytes in size ● Stores information on: ○ Location where contents are stored ○ File timestamps ○ Permissions ○ File Metadata (name, size, etc.) ● The allocation status of a directory entry is determined by using the first byte File Timestamps Each directory entry has 3 times in it: ● Time Created – accurate to a tenth of a second. ● Last Accessed – accurate to the day. ● Last Written – accurate to 2 seconds. Created ● Set when OS allocates a new directory entry for a new file
5 ●
If file has been renamed or moved to another directory, original creation time kept. If file is copied, a new file will be created and and a new creation time will be written to the new entry. Written ● Set when OS writes new file content ● If file has been renamed or moved to another directory, original written and creation time. ● If files are moved or copied, the new directory entry will have the written time from the original file. ● If you move a file, the resulting file will have the original written times and creation time. ● If file is copied, the resulting file will have the original written time and a new creation time. Accessed: Updated the most frequently when ● File is opened ● File is moved to a new volume. Exception is when moving within the same volume. ● Copying and Moving files
Creating Files (dir1/file1.txt, cluster size=4096 bytes, filesize= 6000 bytes) 1. Read the boot sector from sector 0 of the partition and locate the FAT structures, data area and root directory. 2. Process each directory entry to find dir1 and its starting cluster (90). 3. Read the contents of dir1 and find an unallocated directory entry. 4. Set its allocation status by writing the filename file1.txt, update the written time. 5. To find clusters for content, search the FAT structure. Allocate cluster (200) and set its entry to the EOF value 6. Write the address of cluster (200) in the starting cluster field of the directory entry. Write the 4096 bytes to that cluster; 1094 bytes remain so a second cluster is needed 7. Search FAT for another cluster and allocate cluster (201) 8. The FAT entry for the first cluster (200) is changed to contain (201). The remaining bytes are written and cluster (201) is set to EOF in the FAT.
6
Delete the dir1/file1.txt file 1. Read the boot sector from sector 0 of the partition and locate the relevant structures. 2. Locate the dir1 directory as before. 3. Process the contents of dir1’s starting cluster (90) to find a directory entry for file1.txt. We find cluster 200. 4. Use the FAT structure to determine the cluster chain for the file. Here, it has clusters 200 and 201. 5. Set the FAT entries for clusters 200 and 201 to 0 6. Unallocate the directory entry for file1.txt by changing the first byte to .
NTFS File System No specific layout like for other FS, (i.e) ● Every byte of data in NTFS is allocated to a file. ● Everything is a file, including the file that contain the administrative data. ● The entire FS is considered a data area and any sector can be allocated to a file. ● Only consistent layout is that the first sectors contain the boot sector and boot code. Master File Table (MTF) ● Is a file ● Has an entry for itself, $MFT ● Boot sector has starting location of the MFT ● First normal entry in the table determines the layout and size of the MFT. ● Contains the information about all files and directories. ● Uses Clusters ● Every file and directory has at least one entry in the table.
7 ● ● ● ●
Entries are 1 KB in size. Only the first 42 bytes of an entry have a defined purpose The remaining bytes are used to store file attributes File attributes: are small data structures with specific purpose, (e.g.) filename $FILE_NAME or file contents $DATA
Master File Table Entries ● The first 42 bytes of an MFT entry contain 12 fields and the remaining 982 bytes are unstructured and can be filled with attributes. ● First field is signature (magic number): FILE or BAAD. ● A flag field indicates if the MFT entry is being used and is for a directory. ● When a file cannot fit its attributes into one MFT entry, more MFT entries (hence clusters/disk space) can be allocated for that file. Attribute referred to as nonresident attribute. Master File Table Attribute ● NTFS exist to read/write file attributes ● Every file has a $DATA attribute, which stores the file contents ● 2 Parts: Header and Content ○ Header identifies the type of attribute, its size, name, and flags (compressed, encrypted, etc.). ○ Stored in 2 locations: ■ In the MFT entry with the attribute header ■ In an external cluster or cluster run Alternative Data Stream (ADS) ● If a file has > 1 $DATA attribute, the additional ones are called Alternate Data Streams (ADS), with names ● Command line can access ADSs using redirection operators filename:stream name ● Creating an ADS: writing a stream into an ADS named hidden: ○ (ex) C:\> echo Hidden text > test.txt:hidden ● The metadata can be viewed by redirecting from it to more ○ C:\>more
