Systemd: A Complete Guide to System and Service Management

Linux Linux 7 min read 1460 words Beginner ExcellentWiki Editorial Team

Systemd is the init system and service manager used by most modern Linux distributions. It replaces SysV init and provides parallel service startup, on-demand daemon activation, dependency management, and centralized logging through journald.

Core Concepts

Systemd organizes the system into units — configuration files that describe services, sockets, devices, mount points, timers, and more. Units are defined in .service, .timer, .socket, .mount, .target, and other files.

Unit Locations

/etc/systemd/system/       # System administrator units (override /lib)
/run/systemd/system/       # Runtime units (generated)
/lib/systemd/system/       # Distribution-packaged units
/usr/lib/systemd/system/   # Alternative packaged location

Units in /etc/systemd/system/ override those in /lib/systemd/system/. Administrator-created units should go in /etc/systemd/system/.

Service Units

Anatomy of a Service Unit

# /etc/systemd/system/myapp.service
[Unit]
Description=My Application Service
Documentation=https://example.com/docs
After=network.target postgresql.service
Wants=redis.service
Requires=postgresql.service

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
EnvironmentFile=/opt/myapp/.env
ExecStart=/usr/local/bin/myapp --config /etc/myapp/config.yaml
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target

Service Types

simple — main process is the service (default)
exec — similar to simple but waits until the binary is executed
forking — service daemonizes (forks); specify PIDFile
oneshot — runs once and exits (for startup tasks)
dbus — service registers on D-Bus
notify — service sends notification when ready (sd_notify)

Common Service Directives

[Service]
Type=simple
ExecStart=/usr/bin/myapp
ExecStop=/bin/kill -TERM $MAINPID
ExecReload=/bin/kill -HUP $MAINPID
Restart=always             # on-failure, always, no
RestartSec=5               # wait before restart
TimeoutStartSec=30         # max time for startup
TimeoutStopSec=30          # max time for shutdown
KillMode=process           # process, control-group, mixed, none
Environment=NODE_ENV=production
EnvironmentFile=/etc/default/myapp
LimitNOFILE=65536          # file descriptor limit
LimitNPROC=4096            # process limit
WorkingDirectory=/opt/myapp
User=myapp
Group=myapp

Managing Services with systemctl

# Service lifecycle
sudo systemctl start myapp.service    # start service
sudo systemctl stop myapp.service     # stop service
sudo systemctl restart myapp.service  # restart service
sudo systemctl reload myapp.service   # reload config (SIGHUP)
sudo systemctl reload-or-restart myapp.service  # reload if supports, else restart

# Enable/disable (start at boot)
sudo systemctl enable myapp.service   # enable at boot
sudo systemctl disable myapp.service  # disable at boot
sudo systemctl enable --now myapp.service  # enable and start immediately
sudo systemctl disable --now myapp.service # disable and stop immediately

# Status and information
systemctl status myapp.service        # current status, recent logs
systemctl is-active myapp.service     # active/inactive
systemctl is-enabled myapp.service    # enabled/disabled
systemctl show myapp.service          # all properties
systemctl list-units                  # all active units
systemctl list-units --type=service   # all service units
systemctl list-unit-files             # all installed unit files

# Check unit dependencies
systemctl list-dependencies myapp.service
systemctl list-dependencies --reverse myapp.service  # what depends on this

# Daemon reload (after modifying unit files)
sudo systemctl daemon-reload

Journalctl: Viewing Logs

Systemd collects logs through journald, indexed by service, priority, time, and other metadata.

# Service logs
journalctl -u myapp.service           # all logs for service
journalctl -u myapp.service -f        # follow (tail -f)
journalctl -u myapp.service --since "1 hour ago"
journalctl -u myapp.service --since today
journalctl -u myapp.service --until "2024-01-15 12:00"

# System-wide
journalctl -f                         # follow all logs
journalctl -p err                     # only errors
journalctl --since "2024-01-01"
journalctl -k                         # kernel messages
journalctl --list-boots               # list boot sessions

# Output control
journalctl -n 50                      # last 50 lines
journalctl --no-pager                 # no pager
journalctl -o json                    # JSON output
journalctl -o verbose                 # all fields

# Cleanup
sudo journalctl --vacuum-size=500M    # keep only 500MB
sudo journalctl --vacuum-time=30d     # keep only 30 days

Timer Units (Replacing Cron)

Systemd timers replace cron with better integration (logging, dependencies, failure handling):

Timer File

# /etc/systemd/system/backup.timer
[Unit]
Description=Daily backup timer
Requires=backup.service

[Timer]
OnCalendar=daily
Persistent=true
RandomizedDelaySec=1800

[Install]
WantedBy=timers.target

Timer Options

OnCalendar=daily — daily at midnight (see systemd.time(7) for format)
OnCalendar=Mon..Fri 09:00:00 — weekdays at 9 AM
OnCalendar=*-*-1 03:00:00 — first of every month at 3 AM
OnBootSec=5min — 5 minutes after boot
OnUnitActiveSec=1h — 1 hour after last activation
Persistent=true — catch up if system was off
RandomizedDelaySec=1800 — random delay (avoids thundering herd)

Corresponding Service

# /etc/systemd/system/backup.service
[Unit]
Description=Backup script

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh
User=root

Managing Timers

sudo systemctl start backup.timer
sudo systemctl enable backup.timer
sudo systemctl enable --now backup.timer

systemctl list-timers                 # show all active timers
systemctl list-timers --all           # including inactive

# View next timer run
systemctl status backup.timer

Targets (Runlevels)

Targets group units and represent system states:

# Common targets
poweroff.target    # system shutdown
rescue.target      # single-user rescue mode
multi-user.target  # multi-user, non-graphical
graphical.target   # multi-user with GUI
reboot.target      # reboot

# View current target
systemctl get-default

# Set default target
sudo systemctl set-default multi-user.target

# Change to different target
sudo systemctl isolate multi-user.target    # switch to text mode

# List all targets
systemctl list-units --type=target

Custom Snippets (Drop-Ins)

Override specific directives without modifying the original unit file:

# Create override directory
sudo mkdir -p /etc/systemd/system/myapp.service.d/

# Create override file
sudo cat > /etc/systemd/system/myapp.service.d/override.conf << EOF
[Service]
Restart=always
RestartSec=10
Environment=DEBUG=1
EOF

sudo systemctl daemon-reload
sudo systemctl restart myapp.service

# View effective configuration
systemctl cat myapp.service

# Edit conveniently (single command)
sudo systemctl edit myapp.service

Real-World Implementation Tips

Production Considerations

When moving from development to production, several factors become critical. Error handling should be comprehensive — every external call (database, API, file system) should have proper error checking, logging, and retry logic where appropriate. Performance monitoring through metrics and structured logging helps identify bottlenecks before they affect users.

Testing Strategy

A thorough testing approach combines multiple levels:

Unit tests verify individual functions and methods in isolation
Integration tests validate that components work together correctly
Edge case tests cover boundary conditions, empty inputs, and error states
Performance tests ensure the system meets latency and throughput requirements

Test data should be realistic but controlled. Mock external dependencies to make tests fast and deterministic. Aim for tests that are independent, repeatable, and fast enough to run on every commit.

Documentation

Good documentation is essential for maintainable code. Follow these principles:

Document the “why” not just the “what” — explain design decisions
Keep examples up to date with the code
Include usage examples for public APIs
Document configuration options and their defaults
Explain error conditions and recovery strategies

Security Best Practices

Security should be considered throughout development:

Validate all inputs at system boundaries
Use parameterized queries for database access
Store secrets in environment variables or secret managers
Keep dependencies updated to patch vulnerabilities
Apply the principle of least privilege

Performance Optimization

Optimize based on measured data, not assumptions:

Profile before optimizing — identify actual bottlenecks
Measure the impact of each change
Consider the trade-off between speed and readability
Cache expensive operations with appropriate invalidation
Use connection pooling for database and network resources

Monitoring and Observability

Production systems need visibility:

Structured logging with correlation IDs for request tracking
Metrics for latency, throughput, error rates, and resource usage
Health check endpoints for load balancers and orchestration
Distributed tracing for request flows across services
Alerts for anomaly detection based on baselines

These patterns apply across all programming languages and frameworks. The specific implementation varies, but the principles remain consistent.

FAQ

Q: How do I check if systemd is the init system? A: Check for PID 1: ps -p 1 -o comm=. If it shows systemd, systemd is active. Alternatively, check for /run/systemd/system/.

Q: What is the difference between enable and start? A: enable creates symlinks so the service starts at boot. start starts the service immediately. Use enable --now for both.

Q: How do I run a script at boot with systemd? A: Create a oneshot service and enable it. Add [Install] WantedBy=multi-user.target to the service unit.

Q: How do I view logs for a specific time window? A: Use journalctl --since "10 minutes ago" or journalctl --since "2024-01-01 10:00" --until "2024-01-01 12:00".

Q: Why did my service fail to start? A: Check systemctl status myapp.service for error messages. Use journalctl -xe for detailed logs. Verify the ExecStart path, permissions, and all dependencies.

Q: How do I pass environment variables to a systemd service? A: Use Environment=KEY=value in the service unit or EnvironmentFile=/path/to/env/file. The file should contain KEY=value pairs, one per line.

Q: How do I make a service restart automatically? A: Add Restart=on-failure (or always) to the [Service] section, and set RestartSec=5 for the delay between restarts.

systemd Advanced Usage

Service Dependencies

[Unit]
Description=My App
After=network.target postgresql.service
Requires=postgresql.service
Wants=redis.service
BindsTo=postgresql.service

[Service]
Type=notify
ExecStart=/usr/local/bin/myapp
Restart=always
RestartSec=5
LimitNOFILE=65536

Journalctl Usage

# Follow logs in real time
journalctl -u myapp.service -f

# Show logs since last boot
journalctl -u myapp.service -b

# Filter by time range
journalctl -u myapp.service --since "1 hour ago" --until "30 minutes ago"

# Show logs with priority
journalctl -u myapp.service -p err

# Export logs to file
journalctl -u myapp.service --output=json > /tmp/myapp-logs.json

# Show disk usage
journalctl --disk-usage

# Vacuum old logs
journalctl --vacuum-time=7d
journalctl --vacuum-size=500M

Timers (Cron Replacement)

# /etc/systemd/system/backup.timer
[Unit]
Description=Daily backup

[Timer]
OnCalendar=daily
Persistent=true
RandomizedDelaySec=1800

[Install]
WantedBy=timers.target

systemctl enable --now backup.timer
systemctl list-timers

Resource Control

[Service]
# CPU and memory limits
CPUQuota=50%
MemoryMax=512M
IOWeight=100
TasksMax=200

For a comprehensive overview, read our article on Bash Scripting Basics.

For a comprehensive overview, read our article on Cron Jobs Guide.

Share this article

X LinkedIn Facebook Email

Section: Linux 1460 words 7 min read Beginner 756 articles in section Report inaccuracy Back to top

Continue Learning in This Category

1 Intermediate

Bash Scripting Basics: A Beginner's Guide 8 min read

2 Intermediate

Linux Command Cheat Sheet 9 min read

3 Beginner

Linux File Permissions: Understanding chmod, chown, and umask 7 min read

4 Beginner

Linux Storage Management Complete Guide 8 min read

5 Beginner

Linux Environment Variables: Complete Guide 7 min read

6 Beginner

Systemd Guide: Services, Timers & Logging 8 min read

Next SSH Guide: Secure Shell for Remote Access