File Systems in Operating Systems: Structure & Design

The file system is the operating system’s structured interface to persistent storage. It organizes raw block devices into the familiar hierarchy of files and directories, managing allocation, naming, permissions, and metadata. The choice of file system affects every aspect of storage performance — throughput, latency, fragmentation resistance, crash recovery, and maximum file and volume sizes. This guide examines file system architecture, allocation strategies, journaling mechanisms, and the practical trade-offs between popular file systems including ext4, NTFS, APFS, and the FAT family.

File System Fundamentals

A file system manages three categories of data: the user’s file content, the directory hierarchy that organizes files, and metadata describing file attributes (size, timestamps, ownership, permissions). The file system also tracks free space — which blocks on the storage device are available for allocation — and allocates new blocks as files grow.

The abstraction layer that enables multiple file system types to coexist is the Virtual File System (VFS). The VFS defines a common interface — inodes, directory entries, file objects, and superblocks — that every file system must implement. Application code calls open(), read(), write(), and close() through the VFS, which dispatches to the appropriate file system’s implementation. Linux supports over 50 file systems through this mechanism.

Inode-Based File Systems

In Unix-derived systems, every file is represented by an inode (index node). The inode contains all metadata about the file except its name: file type (regular, directory, symlink, socket), permission bits, owner and group IDs, three timestamps (access, modify, change), file size, and block pointers that locate the file’s data on disk.

The block pointer structure in ext4 uses a combination of direct blocks (pointing directly to data blocks), indirect blocks (pointing to a block of block pointers), double indirect, and triple indirect blocks. For small files, only direct blocks are needed. For large files, the indirect chain extends reach. Ext4 also uses extents — a contiguous range of blocks — which reduce the number of block pointers needed for large files and improve sequential read performance.

Directory entries (dentries) map file names to inode numbers. A directory is simply a file whose data is a list of name-inode pairs. When a file is opened by name, the VFS traverses the directory tree, looking up each path component in the dentry cache (dcache) to avoid repeated disk reads.

File Allocation Strategies

Contiguous Allocation

Each file occupies a contiguous range of blocks on disk. This layout provides excellent sequential read performance (the disk head moves linearly) and simple implementation. The fatal flaw is external fragmentation: as files are created and deleted, free space becomes divided into small gaps that may not accommodate new files. The system must either compact storage (expensive) or reject file creations. Contiguous allocation is used in ISO 9660 (optical discs) and raw disk images.

Linked Allocation

Each block contains a pointer to the next block in the file. There is no external fragmentation — any free block can be used. The price is poor random access: reading block N requires reading blocks 0 through N-1 sequentially. The pointer overhead (4–8 bytes per block) also consumes space. The File Allocation Table (FAT) file system improves on this by storing the linked list in a dedicated table (the FAT), enabling faster random access without reading the data blocks.

Indexed Allocation

Each file has an index block (or multiple levels of index blocks) listing all its data block addresses. Random access is efficient — the OS reads the index block and jumps directly to the target data block. The overhead is the index block itself, which for tiny files wastes space. Unix inodes with direct/indirect pointers and ext4 extents are practical implementations of indexed allocation. Silberschatz, Galvin, and Gagne (Operating System Concepts) note that indexed allocation combines the random-access advantage of contiguous allocation with the space efficiency of linked allocation.

Journaling and Crash Recovery

File system operations — creating a file, extending it, deleting it — involve multiple disk writes. If the system crashes midway through (power failure, kernel panic), the file system can be left in an inconsistent state: blocks marked allocated but not belonging to any file, or blocks simultaneously allocated to two files.

Journaling solves this by writing a record of the intended operation to a circular journal (log) before applying it to the main file system. If a crash occurs, the journal is replayed on the next mount, completing or undoing incomplete operations. Three journaling modes exist:

• Writeback: Only metadata is journaled. Data may be written before or after the journal. Fastest but risks data corruption — a file can contain garbage after recovery if data wasn’t written before the crash.
• Ordered (default in ext4 and most Linux file systems): Metadata is journaled, and data blocks are written to disk before the journal entry marks the metadata as committed. The file’s content is always consistent after recovery, though recently created files in directories may be lost.
• Data: Both metadata and data are journaled. Slowest but safest: every write is logged, and after recovery every file contains its pre-crash data.

NTFS uses a journal (the $LogFile) that journals metadata changes in redo/undo format, enabling rollback of incomplete transactions. APFS uses a copy-on-write (CoW) approach rather than traditional journaling — metadata changes are written to new locations, and the file system switches atomically to the new metadata on success. CoW eliminates the journal write entirely but requires more free space.

Popular File Systems in Detail

ext4 (Fourth Extended File System)

Ext4, the default file system for most Linux distributions, supports volumes up to 1 exabyte (EB) and files up to 16 terabytes (TB). Key features include extents (replacing the block mapping of ext3 with range-based allocation), delayed allocation (buffering writes and allocating blocks only when data is flushed), multi-block allocator (mballoc), and fast file system checking (uninit_bg). Ext4’s delayed allocation improves performance by coalescing small writes into larger sequential writes but risks data loss on power failure if the system hasn’t flushed the write-back cache — a trade-off addressed by barriers (cache flushes) in the block layer.

NTFS (New Technology File System)

NTFS, introduced with Windows NT 3.1 in 1993, is the primary file system for Windows. Its master file table (MFT) is a relational database of file records — each file is an entry in the MFT containing attributes (name, data, security descriptor, timestamps). NTFS supports file-level encryption (EFS), compression, disk quotas, sparse files, hard links, and reparse points (the foundation for directory junctions and symbolic links). Its journal ($LogFile) and self-healing capabilities make NTFS robust against corruption, though fsck-equivalent tools (chkdsk) run when dirty bits are detected.

APFS (Apple File System)

APFS, introduced with macOS High Sierra and iOS 10.3, is optimized for flash storage. Its copy-on-write metadata design provides crash-safe snapshots (instant, space-efficient point-in-time copies), cloning (instant file copies that share data blocks until one is modified), and space sharing (multiple volumes can share the same free space pool, growing and shrinking dynamically). APFS uses 64-bit inode numbers, supports up to 8 exabyte volumes, and includes native encryption (single-key, multi-key, and hardware-backed). Its primary limitation is the absence of native checksumming for user data (only metadata is checksummed).

FAT32 and exFAT

The File Allocation Table family dates to 1977 and remains relevant for cross-platform compatibility. FAT32’s 4 GB file size limit makes it unsuitable for modern media files (videos, disk images). exFAT, introduced with Windows Embedded CE 6.0, removes the file size and partition size limits while maintaining the simplicity and broad compatibility of FAT. exFAT is the default file system for SDXC cards and is widely supported across Windows, macOS, and Linux (via kernel support since 5.4).

Choosing the Right File System

The optimal file system depends on the workload and platform. For Linux system drives, ext4 provides an outstanding balance of performance, reliability, and tooling. For large-scale storage servers, XFS offers superior performance with parallel allocation (allocation groups) and defragmentation — it handles concurrent writes better than ext4 on multi-threaded workloads. ZFS and Btrfs offer advanced features (checksumming, snapshots, RAID-like volume management, compression) at the cost of higher memory usage and management complexity. For removable media requiring cross-platform access, exFAT is the practical choice.

FAQ

What is the difference between a file system and a volume manager?

A file system organizes data within a single logical volume (partition). A volume manager (LVM on Linux, Logical Disk Manager on Windows) aggregates multiple physical disks into a single logical volume and provides features like snapshots, striping, and resizing. The file system sits on top of the volume manager, unaware of the underlying physical layout.

How does defragmentation work, and does it help SSDs?

Defragmentation rearranges files so that their blocks are contiguous on disk, reducing seek time on HDDs. For SSDs, defragmentation is unnecessary (random access has no seek penalty) and harmful (it creates extra write cycles that wear the NAND flash). Modern operating systems (Windows 8+, macOS, Linux) automatically disable defragmentation on SSDs and use TRIM/discard to maintain write performance.

What is the maximum file size for ext4, NTFS, and APFS?

Ext4 supports individual files up to 16 TB (with 4 KB blocks) or larger with 64 KB block sizes (implementation-dependent). NTFS supports files up to 16 EB theoretically (Windows implementation limits to 256 TB due to volume size constraints). APFS supports files up to 8 EB.

What happens when I delete a file on a journaling file system?

The file system removes the directory entry and marks the inode and data blocks as free in the block bitmap. The file’s data remains on disk until those blocks are overwritten by new writes. This is why forensic tools can recover deleted files unless the data is explicitly erased (shred, secure erase).

How do file systems handle bad blocks on hard drives?

Modern drives have a spare pool of sectors (typically hundreds to thousands). The drive’s firmware remaps bad sectors transparently — anytime the drive detects a write or read failure, it marks the sector as bad and redirects subsequent accesses to a spare sector. The file system never sees the bad block. Some file systems (Btrfs, ZFS) maintain their own checksums of data blocks and can detect and repair silent corruption that the drive firmware missed.

The Memory Management Guide explains the page cache that bridges file systems and physical memory. The Virtual Memory Guide covers memory-mapped files, where the file system and virtual memory subsystem cooperate. Comprehensive coverage of file system internals appears in Silberschatz, Galvin, and Gagne, Operating System Concepts (10th ed.), Chapter 10, and in Tanenbaum and Bos, Modern Operating Systems (4th ed.), Chapter 4.