Linux Storage Management Complete Guide

Linux Storage Management Overview

Linux storage management spans physical disk identification, partition management, filesystem creation, logical volume management, and ongoing monitoring. The Linux kernel abstracts storage hardware through the block device layer, presenting disks as device files under /dev/. Storage management tools range from low-level partition editors (fdisk, parted) to sophisticated logical volume managers (LVM) and filesystem-specific utilities (mkfs.ext4, xfs_admin). Understanding these layers is essential for system administrators managing production servers: disk full errors, filesystem corruption, and capacity planning are among the most common operational challenges. According to the Linux kernel documentation, the virtual filesystem (VFS) layer provides a single interface for all filesystem operations, meaning tools like df and mount work uniformly across ext4, XFS, Btrfs, and network filesystems (Love, “Linux Kernel Development,” 3rd Edition, Addison-Wesley). Modern Linux systems use udev for dynamic device management, creating /dev/ entries as hardware is discovered and removed. The sysfs filesystem at /sys/block/ exposes low-level device parameters including queue settings and I/O schedulers.

Identifying Block Devices

The lsblk command lists block devices in a tree format showing partition relationships:

lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 238.5G  0 disk
├─sda1   8:1    0   512M  0 part /boot/efi
├─sda2   8:2    0 100.5G  0 part /
└─sda3   8:3    0 137.5G  0 part /home

lshw -class disk provides hardware-level details including vendor, model, and firmware version. smartctl -a /dev/sda (from smartmontools) retrieves S.M.A.R.T. data for predictive failure analysis. NVMe devices appear as /dev/nvme0n1, with namespaces for multi-tenant storage configurations. The iostat command monitors I/O performance metrics including throughput, latency, and queue depth.

Partitioning

fdisk for Traditional Partitioning

fdisk is the most widely used partition editor for MBR and GPT partition tables. Interactive mode (fdisk /dev/sdb) provides menu-driven operations. Common commands: m (help), p (print table), n (new partition), d (delete), t (change type), w (write), g (create GPT table). For non-interactive scripting:

printf 'g\nn\n\n\n+100G\nt\n\n8E00\nw\n' | fdisk /dev/sdb

The 8E00 code marks a partition as Linux LVM. GPT is the modern standard, supporting disks larger than 2 TB and up to 128 primary partitions. MBR is limited to 2 TB and 4 primary partitions.

parted for Advanced Partitioning

GNU parted supports both interactive and scripted partitioning:

parted /dev/sdc mklabel gpt
parted /dev/sdc mkpart primary ext4 1MiB 100GiB
parted /dev/sdc set 1 lvm on

Alignment to optimal I/O boundaries (usually 1 MiB) is critical for SSD performance — parted handles this automatically.

Filesystem Creation and Tuning

ext4

ext4 is the default filesystem for most Linux distributions, offering journaling, extents, delayed allocation, and online defragmentation:

mkfs.ext4 -O ^has_journal /dev/sdb1  # Disable journaling (performance, no recovery)
tune2fs -m 1 /dev/sdb1  # Reserve 1% instead of default 5% for root

XFS

XFS excels with large files and parallel I/O workloads, suitable for high-performance computing:

mkfs.xfs -m reflink=1 -m crc=1 /dev/sdb1
xfs_fsr /dev/sdb1  # Defragment (online)

Btrfs

Btrfs provides copy-on-write, subvolumes, snapshots, compression, and built-in RAID:

mkfs.btrfs -L mydata /dev/sdb1
btrfs subvolume create /mnt/mydata/@snapshots
btrfs subvolume snapshot -r /mnt/mydata /mnt/mydata/@snapshots/backup-2025-01-01
mount -o compress=zstd /dev/sdb1 /mnt/mydata

LVM — Logical Volume Manager

LVM adds a flexible abstraction layer between physical disks and filesystems. It enables resizing, snapshots, and striping without downtime:

pvcreate /dev/sdb /dev/sdc
vgcreate vg_data /dev/sdb /dev/sdc
lvcreate -L 500G -n projects vg_data
mkfs.ext4 /dev/vg_data/projects
vgextend vg_data /dev/sdd
lvextend -L +100G /dev/vg_data/projects
resize2fs /dev/vg_data/projects

LVM thin provisioning allows overcommitting storage, creating large logical volumes backed by smaller physical storage pools.

Monitoring and Maintenance

Use df -h for disk usage, du -sh for directory size, iostat -x 1 for I/O performance, and smartctl -a for drive health. Schedule periodic smartctl --test=long tests and monitor SMART attributes for early failure detection.

Advanced LVM Features

LVM provides capabilities beyond basic volume management. Thin provisioning creates sparse logical volumes that report full capacity but only consume physical storage on writes: lvcreate -L 500G --thin vg_data/thin_pool; lvcreate -V 100G --thin -n app_data vg_data/thin_pool. Thin snapshots are instantaneous and space-efficient, unlike thick snapshots that reserve full capacity. LVM caching accelerates slow HDD volumes by adding an SSD cache layer: lvcreate -L 50G --cachepool vg_data/cache_pool; lvconvert --type cache --cachepool vg_data/cache_pool vg_data/slow_volume. RAID integration within LVM provides striping (RAID 0), mirroring (RAID 1), and striped mirrors (RAID 10) across physical volumes. LVM’s pvmove migrates data between physical devices online, enabling disk replacement without downtime. Integration with systemd through lvm2-activation-generator ensures LVM volumes are activated early in the boot process. For large-scale deployments, LVM’s tagging system organizes volumes by application, environment, or backup schedule.

Filesystem Performance Tuning

Filesystem performance tuning depends on workload characteristics and hardware capabilities. For ext4 on SSDs, the noatime mount option eliminates access time updates, reducing write amplification. The discard mount option enables automatic TRIM for SSDs, but periodic fstrim is preferred for many workloads. The nobarrier option disables write barriers for battery-backed RAID controllers, improving throughput at the cost of crash-safety guarantees. For XFS, the allocation group count (-d agcount=N) should match the number of CPU cores for concurrent allocation workloads. The largeio and nolargeio mount options control read-ahead behavior for large sequential I/O. Btrfs compression with compress-force=zstd reduces storage usage 2-3x for compressible data with minimal CPU overhead. The /sys/block/sdX/queue/scheduler file selects the I/O scheduler: none for NVMe, mq-deadline for spinning disks, bfq for desktop workloads. The ionice command sets I/O priority class for processes independently of CPU scheduling. Filesystem benchmarks using fio with realistic workload profiles should guide tuning decisions rather than default assumptions.

LVM and Volume Management

The Logical Volume Manager (LVM) provides flexible storage allocation beyond physical disk boundaries. LVM combines physical volumes (PVs) into volume groups (VGs) from which logical volumes (LVs) are allocated. Snapshots create point-in-time copies of logical volumes using copy-on-write, enabling consistent database backups without downtime. Thin provisioning allows over-allocating storage with lvcreate --type thin-pool and lvcreate --type thin, with automatic data blocks allocated on write. LVM striping across multiple PVs improves sequential I/O performance similar to RAID 0.

Resizing is straightforward: lvextend -L +10G /dev/vg/lv extends the LV, then filesystem-specific resize commands (resize2fs for ext4, xfs_growfs for XFS) expand the filesystem to fill the LV. Shrinking requires filesystem shrinking first, then LV reduction — and is only supported for ext4, not XFS. The lvchange -an /dev/vg/lv deactivates a logical volume for offline maintenance. LVM caching uses fast SSDs as cache for spinning disks: lvcreate --type cache --cachevol fast_lv slow_lv. For monitoring, pvs, vgs, and lvs display concise summaries, while pvdisplay, vgdisplay, and lvdisplay show detailed metadata. The lvm.conf file controls global LVM behavior including device filtering and locking.

RAID Configuration with mdadm

Software RAID with mdadm provides redundancy and performance improvements for Linux storage. RAID levels serve different purposes: RAID 0 stripes data across disks for maximum throughput but provides no redundancy; RAID 1 mirrors data for fault tolerance with 50% capacity efficiency; RAID 5 distributes parity across 3+ disks for single-disk fault tolerance; RAID 6 tolerates two disk failures with dual parity; RAID 10 combines mirroring and striping for performance and redundancy. Creating a RAID array: mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1.

The /etc/mdadm/mdadm.conf configuration file defines arrays for automatic assembly at boot. The mdadm --detail /dev/md0 command shows array health, and mdadm --monitor provides event notification for disk failures. Hot spare disks automatically replace failed drives: mdadm --add /dev/md0 /dev/sde1. Monitoring /proc/mdstat shows sync progress and health status. The mdadm --fail /dev/md0 /dev/sdb1 simulates a disk failure for testing. For production deployments, combine RAID with LVM: create RAID arrays as physical volumes, pool them into volume groups, and allocate logical volumes. This hybrid approach separates redundancy management from volume allocation, simplifying capacity planning and replacement.

Filesystem Repair and Recovery

Filesystem corruption requires immediate attention to prevent data loss. The fsck utility checks and repairs filesystem inconsistencies. For ext4, fsck.ext4 -f /dev/sda1 forces a check even if the filesystem appears clean. Run fsck on unmounted filesystems or in single-user mode to prevent further damage. The e2fsck -c option checks for bad blocks and marks them. The debugfs tool provides low-level filesystem inspection and repair for ext2/3/4 filesystems.

For XFS, xfs_repair is the repair tool, run with -n for a dry run first. XFS has better recovery capabilities for metadata corruption compared to ext4. The xfs_metadump tool creates metadata snapshots for debugging without exposing file contents. Btrfs scrub (btrfs scrub start /mountpoint) reads all data and metadata, verifying checksums and repairing from mirrors or parity when available. The btrfs check command repairs Btrfs filesystems offline. Regular integrity checking via smartctl (SMART monitoring) identifies failing hardware before filesystem corruption occurs. Filesystem-independent recovery tools like testdisk and photorec recover deleted files from damaged filesystems.

Frequently Asked Questions

What is the difference between ext4, XFS, and Btrfs? ext4 is the standard general-purpose filesystem. XFS excels with large files and parallel I/O but cannot be shrunk. Btrfs provides snapshots, compression, and checksums.

How do I recover a deleted file on Linux? Stop writing immediately. Use extundelete (ext4) or restore from backup. LVM snapshots enable instant rollback.

What is the difference between soft and hard RAID? Software RAID (mdadm) uses the CPU — flexible and hardware-independent. Hardware RAID uses a dedicated controller.

How do I resize a filesystem without data loss? Most online resizing is safe on modern filesystems. Always backup first.

What is the 5% reserved space on ext4? Reserved space ensures root can always log in and run recovery tools. Reduce with tune2fs -m 1 on data partitions.

Related: Linux Namespaces | Linux Systemd Guide | Linux Text Processing