Python Concurrency: Threading, Asyncio, Multiprocessing
Python offers three distinct approaches to concurrent execution: threading for I/O-bound tasks with shared state, asyncio for high-concurrency I/O with cooperative multitasking, and multiprocessing for CPU-bound computation that bypasses the Global Interpreter Lock. Choosing the right approach — or combining them — depends on understanding your workload’s characteristics and the tradeoffs each model imposes. This guide covers all three models with practical examples, performance characteristics, and decision criteria.
Concurrency vs Parallelism
Concurrency is the composition of independently executing tasks — managing multiple tasks at once. Parallelism is simultaneous execution — running multiple tasks at the literal same time on multiple CPU cores. Concurrency enables structure; parallelism enables speed.
Python’s threading provides concurrency without parallelism for CPU-bound work due to the Global Interpreter Lock (GIL). Multiprocessing provides both concurrency and parallelism by spawning separate processes. Asyncio provides concurrency within a single thread through cooperative multitasking. Understanding this distinction is critical because choosing the wrong model can lead to code that appears correct during testing but performs poorly under load.
The decision framework depends on the bottleneck. I/O-bound tasks — network requests, file operations, database queries — spend most of their time waiting. Threading and asyncio both handle I/O-bound workloads effectively, with asyncio scaling to higher concurrency levels. CPU-bound tasks — numerical computation, image processing, data transformation — require multiprocessing or alternative approaches like NumPy/C extensions that release the GIL.
Threading
The threading module provides threads — lightweight execution contexts sharing the same memory space. Threads are managed by the operating system, which preemptively switches between them. Threading is appropriate for I/O-bound tasks where the GIL is released during blocking operations. File I/O, network I/O, and database operations release the GIL while waiting, allowing other threads to run.
import threading
import time
results = []
def fetch_url(url):
time.sleep(0.1) # Simulate network I/O
results.append(f"Data from {url}")
urls = [f"https://api.example.com/{i}" for i in range(20)]
threads = [threading.Thread(target=fetch_url, args=(url,)) for url in urls]
for t in threads:
t.start()
for t in threads:
t.join()
print(f"Fetched {len(results)} URLs")Thread safety is the primary challenge with threading. Shared data accessed from multiple threads requires synchronization. Python’s threading.Lock provides mutual exclusion:
import threading
lock = threading.Lock()
counter = 0
def increment():
global counter
for _ in range(100000):
with lock:
counter += 1
t1 = threading.Thread(target=increment)
t2 = threading.Thread(target=increment)
t1.start()
t2.start()
t1.join()
t2.join()
print(counter) # 200000Without the lock, the increment operation (which is three bytecode instructions: load, add, store) can interleave between threads, producing incorrect results. The with statement acquires the lock before accessing the shared counter and releases it afterward.
threading.ThreadPoolExecutor provides a higher-level interface for managing thread pools. It is preferred over manual thread management for most applications:
from concurrent.futures import ThreadPoolExecutor
import time
def fetch_url(url):
time.sleep(0.1)
return f"Data from {url}"
with ThreadPoolExecutor(max_workers=8) as executor:
urls = [f"https://api.example.com/{i}" for i in range(20)]
results = list(executor.map(fetch_url, urls))The ThreadPoolExecutor manages thread lifecycle, handles exceptions, and provides clean error propagation through the Future API. According to the Python documentation on concurrent.futures, the max_workers parameter should be tuned based on the nature of the I/O workload and the system’s thread capacity.
Asyncio
Asyncio provides single-threaded cooperative concurrency. Tasks voluntarily yield control at await points, allowing the event loop to resume other tasks. This model scales to tens of thousands of concurrent tasks because the overhead per task is much lower than threads — approximately 10-50 KB per task compared to 1-8 MB per OS thread.
import asyncio
async def fetch_url(url):
await asyncio.sleep(0.1) # Simulate async I/O
return f"Data from {url}"
async def main():
tasks = [fetch_url(f"https://api.example.com/{i}") for i in range(20)]
results = await asyncio.gather(*tasks)
print(f"Fetched {len(results)} URLs")
asyncio.run(main())Asyncio is most effective when all I/O operations have async libraries. The aiohttp library replaces requests for HTTP. aiomysql and aiosqlite provide async database access. aiofiles provides async file operations. The async context manager pattern is common for resource management:
import aiohttp
async def fetch_json(url):
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.json()Python 3.11 introduced asyncio.TaskGroup for structured concurrency. TaskGroup ensures all tasks complete before the group scope exits, and collects exceptions into an ExceptionGroup as specified in PEP 654:
async def fetch_all(urls):
async with asyncio.TaskGroup() as tg:
tasks = [tg.create_task(fetch_url(url)) for url in urls]
return [t.result() for t in tasks]Asyncio’s limitation is that CPU-bound operations block the event loop. If a task performs computation without awaiting, no other task can run until the computation completes. For mixed workloads, offload CPU-bound work to a thread pool executor. The Real Python asyncio tutorial explores this pattern in depth.
Multiprocessing
The multiprocessing module spawns separate Python processes, each with its own GIL. This enables true parallelism for CPU-bound computation.
from multiprocessing import Pool
def is_prime(n):
if n < 2:
return False
for i in range(2, int(n ** 0.5) + 1):
if n % i == 0:
return False
return True
with Pool(4) as pool:
numbers = range(1, 100000)
results = pool.map(is_prime, numbers)
prime_count = sum(results)The Pool distributes work across processes, each running on a separate CPU core. The default Pool size is the number of CPU cores, providing optimal throughput for CPU-bound tasks. Multiprocessing communication between processes uses pickle serialization, which adds overhead for large data transfers.
Shared memory (multiprocessing.shared_memory in Python 3.8+) provides zero-copy data sharing for numerical arrays:
from multiprocessing import shared_memory
import numpy as np
# Create shared array
shm = shared_memory.SharedMemory(create=True, size=1000000 * 8)
arr = np.ndarray((1000000,), dtype=np.float64, buffer=shm.buf)ProcessPoolExecutor mirrors the ThreadPoolExecutor API for process-based parallelism:
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor(max_workers=4) as executor:
results = list(executor.map(is_prime, range(100000)))The Global Interpreter Lock
The GIL is a mutex that protects CPython’s internal state, preventing multiple threads from executing Python bytecode simultaneously. It exists because CPython’s memory management is not thread-safe. The GIL simplifies C extension development and makes single-threaded code faster. The GIL affects threading: CPU-bound threads cannot run in parallel. For I/O-bound threads, the GIL is released during blocking operations, enabling effective concurrency.
The GIL does not affect multiprocessing — each process has its own GIL. Asyncio avoids the GIL entirely by running in a single thread. Workarounds for GIL-limited CPU-bound work include using multiprocessing, using C extensions (NumPy, Cython) that release the GIL, using alternative Python implementations without a GIL, or accepting the GIL and using single-threaded optimization.
Python 3.12 introduced per-interpreter GILs (PEP 684), enabling sub-interpreters with independent GILs. This is an early feature with limited library compatibility but represents the first step toward removing the GIL entirely, a goal pursued by PEP 703 (nogil) which is targeting Python 3.13+.
Thread Safety
Thread safety requires synchronization when multiple threads access mutable shared state. Python provides several synchronization primitives. Lock provides mutual exclusion. RLock (reentrant lock) allows the same thread to acquire the lock multiple times without deadlock. Semaphore limits concurrent access to a resource to a fixed count. Condition allows threads to wait for a condition and be notified when it changes.
The queue.Queue class is thread-safe and is the preferred mechanism for producer-consumer patterns. Multiple producer threads can put items, and multiple consumer threads can get items, without explicit synchronization:
from queue import Queue
from threading import Thread
def producer(q, items):
for item in items:
q.put(item)
q.put(None) # Sentinel
def consumer(q):
while True:
item = q.get()
if item is None:
break
process(item)Local storage (threading.local()) provides thread-specific data that does not require synchronization because each thread has its own copy. This is ideal for per-thread database connections or request context.
Combining Approaches for Complex Workloads
Combine approaches for complex workloads. Use asyncio for the I/O coordination layer and offload CPU-intensive subtasks to a ProcessPoolExecutor. This hybrid approach provides the concurrency of asyncio with the parallelism of multiprocessing:
import asyncio
from concurrent.futures import ProcessPoolExecutor
async def compute_on_cpu(data):
loop = asyncio.get_running_loop()
with ProcessPoolExecutor() as pool:
result = await loop.run_in_executor(pool, heavy_computation, data)
return resultThis pattern is used extensively in modern Python web frameworks. FastAPI, for example, uses this pattern to offload CPU-bound operations from the async request handler to a process pool, keeping the event loop responsive for handling other requests.
FAQ
Does Python’s GIL make threading useless?
No. For I/O-bound tasks, threads are effective because the GIL is released during blocking operations. Only CPU-bound tasks are limited by the GIL.
Can asyncio and threading be used together?
Yes. Use loop.run_in_executor() to run blocking code in a thread pool from asyncio. This is the standard pattern for integrating blocking I/O or CPU-bound work into an async application.
How many concurrent tasks can asyncio handle?
Asyncio can handle tens of thousands of concurrent tasks on a typical system. Each task consumes approximately 10-50 KB of memory, compared to approximately 1-8 MB per OS thread.
What is the difference between asyncio.gather() and TaskGroup?
asyncio.gather() returns results and exceptions separately, requiring manual handling. TaskGroup provides structured concurrency with automatic exception collection into ExceptionGroup and cancellation of sibling tasks on failure.
When should I use ProcessPoolExecutor vs multiprocessing.Pool?
Both provide process-based parallelism. ProcessPoolExecutor has a cleaner API aligned with concurrent.futures and integrates with asyncio. multiprocessing.Pool provides more options for worker initialization and result handling.
Related: Explore Python 3.11 features for TaskGroup and ExceptionGroup patterns. Study Python design patterns for thread-safe implementation patterns. Learn Python error handling for robust concurrent exception management.