Async API Patterns: WebSockets, Webhooks, and SSE

API Development API Development 9 min read 1740 words Intermediate ExcellentWiki Editorial Team

Traditional request-response APIs work well for many scenarios, but they fail when clients need real-time updates. Polling wastes resources and introduces latency — a client checking for updates every 5 seconds creates unnecessary load when nothing has changed, yet may still introduce seconds of delay when an event does occur. Asynchronous API patterns solve this problem by enabling servers to push data to clients when events occur, eliminating the trade-off between resource waste and latency.

WebSockets, webhooks, and server-sent events each address different real-time requirements. Choosing the right pattern depends on whether communication needs to be bidirectional, whether the client can maintain a persistent connection, and what latency requirements apply. The AsyncAPI specification, modeled after OpenAPI for event-driven APIs, provides a standardized way to document these asynchronous interfaces.

WebSockets

WebSockets (standardized as RFC 6455) provide full-duplex communication channels over a single TCP connection. Both client and server can send messages at any time, making WebSockets ideal for bidirectional, low-latency communication where either party may initiate data transfer.

How WebSockets Work

A WebSocket connection starts with an HTTP upgrade request:

GET /chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

The server responds with a 101 Switching Protocols status, and the connection upgrades from HTTP to WebSocket. After upgrade, both sides can send and receive frames until either side closes the connection. The handshake uses HTTP for compatibility — HTTP proxies and load balancers see an initial HTTP request and can route it appropriately, while the persistent connection that follows bypasses HTTP overhead for subsequent messages.

Server Implementation

WebSocket servers can be built with most modern frameworks. Python’s websockets library provides an clean async implementation:

import asyncio
import websockets

async def handler(websocket):
    async for message in websocket:
        # Process incoming message
        response = await process_message(message)
        await websocket.send(response)

async def main():
    async with websockets.serve(handler, "localhost", 8765):
        await asyncio.Future()

asyncio.run(main())

Client Implementation

const ws = new WebSocket('wss://example.com/chat');

ws.onopen = () => {
    ws.send(JSON.stringify({ type: 'join', room: 'general' }));
---;

ws.onmessage = (event) => {
    const message = JSON.parse(event.data);
    console.log('Received:', message);
---;

ws.onclose = () => {
    console.log('Disconnected');
---;

Use Cases

WebSockets excel in applications requiring real-time bidirectional communication. Chat applications (Slack, Discord) use WebSockets for instant message delivery and typing indicators. Collaborative editing tools (Google Docs, Notion) use WebSockets to broadcast changes to all connected clients. Live gaming, financial trading platforms (with sub-millisecond latency requirements), and real-time dashboards all benefit from WebSocket connections.

The trade-off is that WebSockets require persistent connections, consuming server memory and file descriptors for each connected client. Scaling WebSocket connections across multiple servers requires sticky sessions (session affinity) or a pub/sub layer like Redis Pub/Sub, Redis Streams, or Apache Kafka for broadcasting messages across server instances. Libraries like Socket.IO handle scaling concerns with built-in adapter support for Redis and other backends.

Webhooks

Webhooks provide a different model. Instead of maintaining a persistent connection, the server sends an HTTP request to a pre-registered URL when an event occurs. This makes webhooks fundamentally simpler than WebSockets because they use standard HTTP and do not require persistent connections.

How Webhooks Work

A client registers a callback URL with the server. When the server has new data, it sends an HTTP POST to the callback URL with a payload describing the event:

POST /webhooks/order-update HTTP/1.1
Host: client.example.com
Content-Type: application/json
X-Webhook-Signature: sha256=abc123def456

{
  "event": "order.shipped",
  "order_id": "12345",
  "status": "shipped",
  "tracking_number": "1Z999AA10123456784"
---

Implementing Webhooks

The webhook sender must support registration, authentication, retry logic, and payload delivery:

import requests
import hmac
import hashlib

def send_webhook(url: str, payload: dict, secret: str):
    body = json.dumps(payload)
    signature = hmac.new(
        secret.encode(), body.encode(), hashlib.sha256
    ).hexdigest()

    headers = {
        'Content-Type': 'application/json',
        'X-Webhook-Signature': f'sha256={signature}'
    }

    try:
        response = requests.post(url, data=body, headers=headers, timeout=10)
        if response.status_code >= 500:
            # Queue for retry
            queue_retry(url, payload)
    except requests.RequestException:
        queue_retry(url, payload)

Critical webhook implementation features include: signing payloads with HMAC-SHA256 so receivers can verify authenticity, implementing retry with exponential backoff (immediate, then 1 minute, 5 minutes, 30 minutes, 2 hours), supporting idempotency keys in event payloads to handle duplicate deliveries, and providing a dashboard for monitoring delivery status and replaying failed deliveries.

Use Cases

Webhooks are ideal for event notifications where the client does not need immediate updates (latency of seconds to minutes is acceptable). Payment processing notifications (Stripe sends webhooks for payment success, failure, disputes), CI/CD pipeline events (GitHub sends webhooks for push, pull request, and release events), and order fulfillment updates all use webhooks.

Webhooks are simpler than WebSockets because they use standard HTTP and do not require persistent connections. The trade-off is higher latency (the event must be serialized, the HTTP request created, and the response received before the client knows about the event) and the need for the receiver to be reachable via HTTP. Receivers must be online to receive webhooks — if the receiver is down, the delivery fails and must be retried later. Webhook delivery typically offers at-least-once guarantees with retries, but clients must handle duplicate deliveries.

Server-Sent Events

Server-sent events (SSE, standardized as part of HTML5) provide a unidirectional channel from server to client over a standard HTTP connection. The client opens a connection, and the server sends events as they occur. SSE is significantly simpler than WebSockets because it uses standard HTTP and does not require a protocol upgrade.

How SSE Works

The client creates an EventSource:

const source = new EventSource('https://example.com/events');

source.addEventListener('order', (event) => {
    const data = JSON.parse(event.data);
    console.log('New order:', data);
---);

source.onerror = () => {
    console.log('Connection error, reconnecting...');
---;

The server sends events in a specific text format with event: and data: fields:

event: order
data: {"order_id": "12345", "status": "new"}

event: notification
data: {"message": "System maintenance at 2 AM"}

:heartbeat

Server Implementation

from flask import Response, stream_with_context

def event_stream():
    while True:
        # Wait for new events from a queue
        event = get_next_event()
        yield f"event: {event.type}\ndata: {json.dumps(event.data)}\n\n"

@app.route('/events')
def stream():
    return Response(
        stream_with_context(event_stream()),
        mimetype='text/event-stream'
    )

Use Cases

SSE is ideal for one-way real-time updates. Live sports scores, stock tickers, social media feeds, system monitoring dashboards, and notification streams all benefit from SSE’s simplicity. SSE automatically reconnects when connections drop (with built-in last-event-ID tracking for resuming), providing reliable delivery without custom reconnection logic.

SSE has significantly simpler infrastructure requirements than WebSockets. It works through standard HTTP proxies and load balancers, including CDNs like Cloudflare. The limitation is unidirectional communication — clients cannot send data to the server over the same connection. If the client needs to send data, it must use separate HTTP requests in addition to the SSE connection.

Message Queues and Event Streaming

For server-to-server async patterns, message queues and event streaming platforms provide reliable, persistent async communication. Apache Kafka provides durable, ordered event logs with replay capability, making it suitable for event sourcing and stream processing. RabbitMQ offers flexible routing with exchanges and queues for traditional message broker patterns. Redis Pub/Sub provides lightweight publish-subscribe for real-time messaging without persistence.

The AsyncAPI specification documents these patterns with the same rigor that OpenAPI brings to REST APIs. AsyncAPI supports MQTT, AMQP, Kafka, WebSocket, and HTTP streaming protocols, making it valuable for documenting event-driven architectures. The specification describes channels (topics/queues), messages (payload schemas), and bindings (protocol-specific configuration).

Choosing the Right Pattern

Choose WebSockets for bidirectional, low-latency communication where clients send and receive data frequently — chat, collaborative editing, live gaming. Choose webhooks for event notifications where the client does not need immediate delivery and cannot maintain persistent connections — payment notifications, CI/CD events, email delivery status. Choose SSE for unidirectional real-time updates from server to client — live feeds, dashboards, monitoring, notifications where the client only receives data.

Many applications combine patterns. A real-time dashboard might use SSE for live data streaming and WebSockets for user interactions. An e-commerce platform might use WebSockets for the checkout process (bidirectional communication between browser and server) and webhooks for order fulfillment notifications (asynchronous, server-to-server). A chat application might use WebSockets for message delivery and webhooks for push notification delivery to offline users.

Async APIs are essential for modern applications. Users expect real-time updates — data that appears without refreshing, notifications that arrive instantly, and collaborative features that reflect other users’ actions immediately. Choosing the right async pattern ensures your application meets those expectations efficiently and scales predictably as your user base grows.

Frequently Asked Questions

When should I use WebSockets vs SSE?

Use WebSockets when the client needs to send data to the server over the same connection (bidirectional communication) or when you need the lowest possible latency. Use SSE when the server only pushes data to the client (unidirectional) and you want simpler infrastructure — SSE works through standard HTTP proxies and CDNs, while WebSockets require protocol-aware infrastructure. SSE also has automatic reconnection built in, while WebSocket reconnection requires custom logic.

How do I secure WebSocket connections?

Use wss:// (WebSocket over TLS) for all production connections. Authenticate during the initial HTTP upgrade handshake using cookies, tokens in the URL (for the initial request only), or the Sec-WebSocket-Protocol header for custom subprotocol negotiation. Validate the Origin header in WebSocket upgrade requests to prevent cross-site WebSocket hijacking. Implement per-message authentication for long-lived connections.

How do I handle webhook retries?

Implement exponential backoff: retry after 1 minute, then 5 minutes, 30 minutes, 2 hours, 6 hours, and 24 hours. Stop retrying after 24-48 hours and log the failure for manual review. Use idempotency keys in webhook payloads so receivers can safely process duplicate deliveries. Provide a webhook delivery dashboard where users can see delivery status, payload content, and manually trigger retries.

What is the AsyncAPI specification?

AsyncAPI is an open-source specification for describing event-driven APIs, similar to what OpenAPI does for REST APIs. It defines channels (topics, queues, or streams), messages with schemas, and protocol-specific bindings. AsyncAPI supports Kafka, MQTT, AMQP, WebSocket, HTTP streaming, and other protocols. It enables documentation generation, code generation, and validation for async APIs.

Can I use WebSockets and HTTP together?

Yes, this is common. Use HTTP for RESTful CRUD operations and WebSockets for real-time updates. Many applications use HTTP for initial data loading and WebSockets for subsequent real-time synchronization. A single server can handle both HTTP and WebSocket connections on the same port using protocol detection. Socket.IO and similar libraries handle this transparently.

For more on API design patterns, see the REST API Design Guide. For API gateway management, read Microservices API Gateway. For GraphQL subscriptions (real-time GraphQL), see GraphQL Beginner’s Guide.

For a comprehensive overview, read our article on Api Authentication Guide.

For a comprehensive overview, read our article on Api Caching Strategies.

Share this article

X LinkedIn Facebook Email