Redis: A Beginner's Guide to Caching and NoSQL

Redis is an in-memory data structure store that is used as a database, cache, message broker, and queue. It stores data in RAM for lightning-fast access while optionally persisting to disk for durability. Redis is often called a “NoSQL” database, but it is more accurate to describe it as a data structure server — it provides native support for strings, hashes, lists, sets, sorted sets, streams, and more.

Installation

# macOS
brew install redis
brew services start redis

# Ubuntu/Debian
sudo apt install redis-server
sudo systemctl start redis

# Docker
docker run -d -p 6379:6379 redis:7-alpine

# Test connection
redis-cli ping
# PONG

Core Data Types

Strings

Strings are the most basic Redis type. They can hold text, numbers, binary data, or serialized JSON:

SET user:1:name "Alice"
SET user:1:visits 0
GET user:1:name
# "Alice"

INCR user:1:visits
# (integer) 1
INCRBY user:1:visits 5
# (integer) 6

# Set with expiry (seconds)
SET session:abc123 "user_data" EX 3600
TTL session:abc123
# (integer) 3598

Lists

Lists are sequences of strings ordered by insertion. Use them for queues, timelines, and message buffers:

LPUSH notifications:user:1 "New message"
LPUSH notifications:user:1 "Friend request"
RPUSH notifications:user:1 "System update"

LRANGE notifications:user:1 0 -1
# 1) "Friend request"
# 2) "New message"
# 3) "System update"

LPOP notifications:user:1
# "Friend request"

LLEN notifications:user:1
# (integer) 2

Sets

Sets are unordered collections of unique strings. Use them for tags, followers, or any deduplicated data:

SADD post:42:tags "redis" "database" "caching"
SADD post:42:tags "redis" "nosql"

SMEMBERS post:42:tags
# 1) "caching"
# 2) "database"
# 3) "nosql"
# 4) "redis"

# Set operations
SADD post:43:tags "redis" "python"
SINTER post:42:tags post:43:tags
# 1) "redis"

SUNION post:42:tags post:43:tags
# 1) "caching" 2) "database" 3) "nosql" 4) "python" 5) "redis"

Sorted Sets

Like sets, but every member has a score. Members are ordered by score:

ZADD leaderboard 100 "Alice"
ZADD leaderboard 85 "Bob"
ZADD leaderboard 95 "Charlie"

# Top scores
ZREVRANGE leaderboard 0 2 WITHSCORES
# 1) "Alice"   2) 100
# 3) "Charlie" 4) 95
# 5) "Bob"     6) 85

# Increment score
ZINCRBY leaderboard 10 "Bob"

Hashes

Hashes store field-value pairs. Use them for objects:

HSET user:1 name "Alice" email "alice@example.com" age 30
HGET user:1 name
# "Alice"
HGETALL user:1
# 1) "name"    2) "Alice"
# 3) "email"   4) "alice@example.com"
# 5) "age"     6) "30"

HINCRBY user:1 age 1
# (integer) 31

Streams

Streams are append-only logs that support consumer groups. Introduced in Redis 5.0, they are ideal for event sourcing, activity feeds, and message queuing with reliable delivery:

# Add to stream
XADD mystream * sensor_id 1234 temperature 22.5
# "1712345678000-0"

# Read from stream
XRANGE mystream - + COUNT 10

# Create consumer group
XGROUP CREATE mystream mygroup $
XREADGROUP GROUP mygroup consumer1 COUNT 1 BLOCK 5000 STREAMS mystream >

Caching Strategies

Cache-Aside (Lazy Loading)

The application checks the cache before querying the database:

import redis

r = redis.Redis()

def get_user(user_id):
    # Check cache
    user_data = r.hgetall(f"user:{user_id}")
    if user_data:
        return user_data

    # Cache miss — query database
    user = db.query("SELECT * FROM users WHERE id = ?", user_id)

    # Store in cache with expiry
    r.hset(f"user:{user_id}", mapping=user)
    r.expire(f"user:{user_id}", 3600)
    return user

Write-Through

The cache is updated synchronously with every write:

def update_user(user_id, data):
    # Update database
    db.execute("UPDATE users SET ... WHERE id = ?", user_id, data)

    # Update cache
    r.hset(f"user:{user_id}", mapping=data)
    r.expire(f"user:{user_id}", 3600)

Write-Behind (Lazy Write)

Writes go to Redis first and are asynchronously flushed to the database. This provides the fastest write performance but risks data loss if Redis crashes before the flush completes.

Cache Eviction Policies

Redis provides several eviction policies when memory is full:

maxmemory 512mb
maxmemory-policy allkeys-lru   # evict least recently used keys

For most caching use cases, allkeys-lru is the right choice. For production traffic with predictable access patterns, allkeys-lfu often yields better hit rates.

Persistence

RDB (Snapshot)

# Manual save
redis-cli SAVE     # synchronous, blocks
redis-cli BGSAVE   # background, non-blocking

# Configuration
save 900 1     # save if 1 key changed in 900 seconds
save 300 10    # save if 10 keys changed in 300 seconds
save 60 10000  # save if 10000 keys changed in 60 seconds

AOF (Append-Only File)

appendonly yes
appendfsync everysec   # sync every second (good balance)
# appendfsync always   # sync every write (slow, safe)
# appendfsync no       # let OS decide (fast, less safe)

AOF logs every write operation. It is more durable than RDB but uses more disk space and can be slower.

Best Practice: Both

Use both RDB and AOF for production:

save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync everysec

RDB provides fast restarts. AOF provides durability.

Pub/Sub

Redis supports publish/subscribe messaging:

# Publisher
import redis
r = redis.Redis()
r.publish("notifications", "Hello, subscribers!")

# Subscriber (separate process)
import redis
r = redis.Redis()
pubsub = r.pubsub()
pubsub.subscribe("notifications")

for message in pubsub.listen():
    if message["type"] == "message":
        print(f"Received: {message['data']}")

Pub/Sub is fire-and-forget — messages are lost if no subscriber is listening. For reliable messaging, use Redis Streams with consumer groups.

Redis Cluster and Sentinel

Redis Sentinel

Sentinel provides high availability through automatic failover. It monitors the master, detects failures, and promotes a replica if the master goes down:

# sentinel.conf
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000

Sentinel also acts as a service discovery layer — clients query Sentinel for the current master address.

Redis Cluster

Cluster provides automatic sharding across multiple nodes. Data is distributed across 16384 hash slots, and each node handles a subset of slots:

redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 \
  127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
  --cluster-replicas 1

Cluster supports automatic failover, resharding, and linear scalability. Use it when a single Redis instance’s memory or throughput capacity is insufficient.

Common Use Cases

Rate Limiting

import time

def is_rate_limited(user_id, max_requests=10, window=60):
    key = f"ratelimit:{user_id}:{int(time.time() / window)}"
    count = r.incr(key)
    if count == 1:
        r.expire(key, window)
    return count > max_requests

Session Storage

# Store session
r.hset(f"session:{session_id}", mapping={
    "user_id": user.id,
    "ip": request.remote_addr,
    "expires": time.time() + 3600
---)
r.expire(f"session:{session_id}", 3600)

# Retrieve session
session = r.hgetall(f"session:{session_id}")

Distributed Locks

lock = r.setnx(f"lock:resource:{resource_id}", "locked")
if lock:
    r.expire(f"lock:resource:{resource_id}", 10)
    try:
        process_resource(resource_id)
    finally:
        r.delete(f"lock:resource:{resource_id}")

For production use, implement the Redlock algorithm or use Redisson to handle edge cases around clock drift and lock expiration.

Redis Transactions

Redis supports a form of transactions using MULTI, EXEC, DISCARD, and WATCH. Unlike SQL transactions, Redis transactions are optimistic — commands are queued and executed atomically but rollback is not supported:

MULTI
SET user:1:name "Alice"
INCR user:1:visits
EXEC

WATCH implements optimistic locking — if the watched key changes before EXEC, the transaction aborts:

WATCH user:1:balance
val = GET user:1:balance
if val >= 50:
    MULTI
    DECRBY user:1:balance 50
    EXEC
else:
    UNWATCH

Common Pitfalls

Blocking commands in production — KEYS * scans every key and blocks the event loop. Use SCAN for production iteration. Large key values — storing multi-megabyte values in Redis wastes memory and slows replication. Keep values under 1MB or use a separate blob store. Missing expiry — keys without TTLs accumulate indefinitely, eventually exhausting memory. Always set expirations on cache data. Incorrect eviction policy — using noeviction (the default) causes writes to fail when memory is full. Switch to allkeys-lru for caching workloads.

FAQ

What is the difference between Redis and Memcached? Both are in-memory caches, but Redis supports rich data structures (lists, sets, sorted sets, hashes, streams), persistence (RDB/AOF), pub/sub, Lua scripting, and replication. Memcached is simpler — key-value only, no persistence, multithreaded by default. Redis is better for feature-rich caching and data structure use cases.

How much memory does Redis need? Estimate your working set — the data accessed frequently. Redis works best when all data fits in RAM. Monitor used_memory and maxmemory settings. Use INFO memory to see breakdowns by data type. Enable compression or switch to a memory-optimized serialization format for large string values.

Is Redis suitable for persistent storage? Yes, with caveats. Use AOF with appendfsync everysec for durability. RDB snapshots provide periodic persistence. For data that absolutely cannot be lost, pair Redis with a traditional database (PostgreSQL, MySQL) and treat Redis as a cache layer.

How do I handle cache invalidation? Set appropriate TTLs on cached data. Use write-through or write-behind strategies for data that changes frequently. For complex invalidation patterns, publish invalidation events via Redis Pub/Sub to notify all application instances.

What is Redis Lua scripting? Redis supports running Lua scripts server-side with EVAL. Scripts execute atomically, making them useful for multi-step operations that require atomicity (e.g., check-then-set patterns). The SCRIPT LOAD command preloads scripts, and EVALSHA runs them by hash to reduce bandwidth overhead.

For a comprehensive overview, read our article on Acid Transactions Guide.

For a comprehensive overview, read our article on Database Backup Recovery.