Containers and Kubernetes: Orchestration and Deployment
Introduction
A developer builds an application on their laptop. It runs perfectly. The same code is deployed to a staging server and crashes with mysterious errors. The operating system is slightly different. A library version has changed. A configuration file lives in a different directory. This scenario — “it works on my machine” — has frustrated developers for decades.
Containers solve this problem by packaging an application with everything it needs to run: code, runtime, system tools, libraries, and settings. A containerized application runs identically on a laptop, a test server, a production cluster, or a cloud instance. Containers transformed how modern applications are built, shipped, and deployed. Combined with Kubernetes for orchestration, they form the foundation of cloud-native infrastructure.
Understanding Containers
Containers are lightweight, standalone executable packages that include application code and all dependencies. Unlike virtual machines that virtualize hardware and run full operating systems, containers virtualize at the operating system level, sharing the host kernel while isolating processes through kernel namespaces and control groups.
Containers vs Virtual Machines
Virtual machines include a full guest operating system, consuming gigabytes of disk space and significant memory and CPU overhead. A VM boots in minutes and runs a complete OS stack. Containers share the host kernel, consuming megabytes of disk space and adding minimal overhead. A container starts in milliseconds.
This efficiency enables density. A physical server might run ten virtual machines. The same server can run hundreds of containers. Containers also provide consistent environments — developers, testers, and production systems run the same container image with identical dependencies and configurations.
Docker and Container Runtimes
Docker popularized containerization and remains the most widely used container platform. Docker provides tooling for building container images, running containers, managing container lifecycles, and sharing images through registries. The Dockerfile format defines how to build an image — starting from a base image, installing dependencies, copying application code, and configuring the runtime environment.
Container images are built in layers. Each instruction in a Dockerfile creates a new layer. Layers are cached and reused across builds, so rebuilding after a code change only recreates layers affected by the change. This layer caching dramatically speeds up development workflows.
Alternative container runtimes include containerd, which Docker itself uses under the hood, and CRI-O, designed specifically for Kubernetes integration. Podman provides a daemonless alternative to Docker that runs containers without a central daemon.
Container Orchestration
Running a single container is straightforward. Running hundreds or thousands of containers across a cluster of servers — managing placement, networking, scaling, health monitoring, and updates — requires orchestration.
What Orchestration Provides
Orchestration platforms automate container lifecycle management. They schedule containers onto cluster nodes based on resource requirements and constraints. They monitor container health and restart failed containers automatically. They manage networking so containers can communicate across nodes. They handle scaling — increasing or decreasing container replicas based on load. They coordinate rolling updates and rollbacks so applications update without downtime.
Kubernetes Architecture
Kubernetes has become the standard container orchestration platform. Originally developed by Google and now maintained by the Cloud Native Computing Foundation, Kubernetes runs in production across organizations of every size and in every industry.
A Kubernetes cluster consists of control plane nodes and worker nodes. The control plane manages the cluster — scheduling workloads, maintaining desired state, and responding to events. Worker nodes run containerized applications. The control plane includes several components:
The API server exposes the Kubernetes API, which all internal and external interactions use. The scheduler assigns new workloads to worker nodes based on resource availability and scheduling policies. The controller manager runs controllers that maintain desired state — ensuring the correct number of replicas, responding to node failures, and managing endpoints. etcd stores cluster state as a distributed key-value store.
Worker nodes run the kubelet, which communicates with the control plane and manages containers on the node. kube-proxy handles network routing and load balancing for services.
Pods: The Atomic Unit
The pod is the smallest deployable unit in Kubernetes. A pod wraps one or more containers that share the same network namespace, storage volumes, and lifecycle. Containers within a pod communicate over localhost and share access to mounted volumes.
Single-container pods are most common, but multi-container pods serve specific patterns. Sidecar containers extend the main container’s functionality — a sidecar might handle log forwarding, proxy network traffic, or manage configuration synchronization. Ambassador containers proxy network connections to external services. Adapter containers transform data formats between containers.
Services and Service Discovery
Pods are ephemeral — they come and go as scaling events, node failures, and updates occur. Kubernetes services provide stable network endpoints for accessing pods. A service selects pods by labels and load-balances traffic across healthy replicas.
Service types include ClusterIP for internal cluster access, NodePort for external access through a static port on each node, and LoadBalancer for integrating with cloud provider load balancers. Ingress resources provide HTTP routing, enabling multiple services to share a single external endpoint with path-based and host-based routing.
Deployments and StatefulSets
Deployments manage stateless application workloads. They define the desired number of pod replicas, the container image to run, update strategies, and health check configurations. The deployment controller ensures the actual state matches the desired state, creating or terminating pods as needed.
StatefulSets manage stateful applications — databases, message queues, and key-value stores — that require stable network identities, persistent storage per instance, and ordered deployment and scaling. Each pod in a StatefulSet has a unique identity that persists across rescheduling.
ConfigMaps and Secrets
Configuration that varies between environments — database URLs, feature flags, external service endpoints — should not be hardcoded in container images. ConfigMaps store non-sensitive configuration data as key-value pairs that pods consume as environment variables or mounted files.
Secrets store sensitive data — passwords, API keys, certificates — in base64-encoded format. Kubernetes encrypts secrets at rest and transmits them securely to pods. Secrets mount only on nodes running pods that need them.
Storage in Kubernetes
Containers are ephemeral — data written to a container’s filesystem disappears when the container restarts. Persistent volumes provide long-term storage that survives pod restarts and rescheduling. Persistent volume claims request storage resources — specifying capacity and access modes — that Kubernetes binds to available persistent volumes.
Storage classes define different storage tiers — SSD-backed fast storage, HDD-backed capacity storage, or cloud provider-managed storage. The Container Storage Interface enables any storage provider to offer persistent storage through a standard interface.
Scaling and Autoscaling
Kubernetes supports multiple scaling mechanisms. Manual scaling changes replica counts through kubectl or the API. Horizontal pod autoscaling automatically adjusts replica counts based on CPU utilization, memory usage, or custom metrics. Cluster autoscaling adds or removes worker nodes based on pending pod resource requests.
Vertical pod autoscaling adjusts CPU and memory requests for existing pods based on actual usage, eliminating the need for manual resource tuning. The combination of horizontal and vertical autoscaling enables clusters to respond automatically to changing workload demands.
Networking
Kubernetes networking addresses four requirements: containers within a pod communicate freely, pods communicate with all other pods across nodes, pods communicate with services, and external traffic reaches services. The Container Network Interface standard enables a variety of networking plugins — Calico, Flannel, Cilium, Weave — that implement these requirements.
Network policies define firewall rules at the pod level, controlling which pods can communicate with each other. By default, all pods communicate freely. Network policies restrict this to implement micro-segmentation and least-privilege networking.
Production Best Practices
Running Kubernetes in production requires attention to security, monitoring, and operational practices.
Security
Enable role-based access control to restrict what users and service accounts can do. Use pod security policies to prevent privileged containers and enforce security contexts. Scan container images for vulnerabilities as part of the CI/CD pipeline. Implement network policies to restrict pod-to-pod communication to only what applications require. Regularly update Kubernetes versions and apply security patches.
Monitoring and Observability
Kubernetes generates metrics for CPU, memory, disk, and network for every pod and node. Prometheus collects and stores metrics. Grafana visualizes dashboards for cluster and application health. Structured logging through fluentd or similar tools aggregates logs across pods. Distributed tracing with Jaeger or OpenTelemetry tracks requests across microservices.
Resource Management
Set resource requests and limits for every container. Requests guarantee resources; limits cap maximum usage. Overcommitting resources without limits causes node pressure and pod evictions. Use namespaces to isolate environments and enforce resource quotas at the namespace level.
Real-World Kubernetes Deployments
Kubernetes powers production infrastructure at organizations of every scale. Spotify runs millions of containers on Kubernetes. The New York Times migrated from a monolithic content management system to a Kubernetes-based microservices architecture. Financial institutions like JPMorgan Chase run trading applications on Kubernetes. For more on building cloud-native applications on Kubernetes, see the Cloud Architecture Patterns guide.
FAQ
Do I need Kubernetes for containers? No. Single containers or small numbers of containers run fine on a single host with Docker. Kubernetes adds value when you run many containers across multiple hosts and need automated orchestration, scaling, and self-healing.
Is Kubernetes hard to learn? Kubernetes has a steep learning curve due to its many abstractions — pods, deployments, services, ingresses, configmaps, secrets, persistent volumes. Start with simple deployments and add complexity as needed. Managed Kubernetes services from cloud providers simplify cluster setup and maintenance.
How does Kubernetes compare to Docker Swarm? Kubernetes has won the orchestration wars. Docker Swarm is simpler to set up but lacks Kubernetes’s ecosystem, extensibility, and community support. Most organizations choose Kubernetes for production workloads.
What is the difference between a pod and a container? A pod is the smallest deployable unit in Kubernetes and can contain one or more containers. Containers within a pod share networking, storage, and lifecycle. Single-container pods are the most common pattern.
Should I manage my own Kubernetes cluster? For most organizations, managed Kubernetes services — Amazon EKS, Azure Kubernetes Service, Google Kubernetes Engine — are the right choice. They provide automated control plane management, updates, and integrations with cloud services. Self-managed clusters make sense for specialized requirements or on-premises deployments.