Image Processing Basics: Filters, Convolution, Fourier Transform,...

Computer Vision Computer Vision 8 min read 1555 words Beginner ExcellentWiki Editorial Team

Digital Image Processing Fundamentals

Image processing is the manipulation of digital images through mathematical operations. Unlike computer vision, which aims to extract semantic understanding, image processing transforms images into other images — enhancing quality, extracting features, or preparing data for further analysis. Every computer vision pipeline begins with image processing steps: noise reduction, contrast enhancement, geometric transformation, and color space conversion. The quality of these preprocessing steps directly determines the performance of downstream deep learning models.

A digital image is fundamentally a 2D discrete signal. Each pixel represents the light intensity at a specific spatial location. The image can be analyzed in the spatial domain (operating directly on pixel values) or the frequency domain (operating on the decomposition of the image into sinusoidal components). Understanding both perspectives is essential for selecting the right processing technique.

The Convolution Operation

Convolution is the most fundamental operation in image processing. A kernel — a small matrix typically 3x3, 5x5, or 7x7 — slides across the image, and at each position, the sum of element-wise multiplications between the kernel and the underlying image pixels becomes the output pixel value. The choice of kernel determines the effect: smoothing, sharpening, edge detection, or feature extraction.

In OpenCV, convolution is implemented via the filter2D function:

import cv2
import numpy as np

# Sharpening kernel
kernel = np.array([[-1,-1,-1],
                   [-1, 9,-1],
                   [-1,-1,-1]], dtype=np.float32)
sharpened = cv2.filter2D(image, -1, kernel)

Common Convolution Kernels

The Gaussian blur kernel smooths images by averaging nearby pixels with weights determined by the Gaussian distribution. The standard deviation (sigma) controls the amount of smoothing — larger sigma produces more blur. Gaussian blur is used for noise reduction, creating image pyramids, and as a preprocessing step for edge detection.

The Sobel operator computes the gradient magnitude in the horizontal and vertical directions. The horizontal Sobel kernel detects vertical edges; the vertical kernel detects horizontal edges. The gradient magnitude combines both directions for full edge detection. The Laplacian kernel computes the second derivative, highlighting regions of rapid intensity change. It is sensitive to noise and often applied to a Gaussian-blurred image (Laplacian of Gaussian, LoG).

Image Filtering Techniques

Linear filtering encompasses any operation where each output pixel is a weighted sum of input pixels — convolution is the canonical example. Linear filters are shift-invariant: the same operation is applied at every position. This property enables efficient implementation through the convolution theorem, which states that convolution in the spatial domain is equivalent to element-wise multiplication in the frequency domain.

Non-linear filters provide capabilities that linear filters cannot achieve. The median filter replaces each pixel with the median value in its neighborhood, excellently removing salt-and-pepper noise while preserving edges (which linear blurring filters smear). The bilateral filter combines a spatial Gaussian kernel with a range Gaussian kernel (based on intensity difference), enabling edge-preserving smoothing — smooth flat regions while keeping edges sharp. Non-local means filtering searches the entire image for similar patches, providing state-of-the-art denoising at higher computational cost.

The Fourier Transform

The Fourier Transform decomposes an image into its frequency components. Low frequencies correspond to smooth regions and gradual intensity changes. High frequencies correspond to edges, textures, and noise. The 2D Fourier Transform produces a complex-valued frequency representation where magnitude indicates the strength of each frequency component and phase indicates its position.

The OpenCV documentation on discrete Fourier transform explains the implementation:

import numpy as np

f = np.fft.fft2(image)
fshift = np.fft.fftshift(f)  # Move zero frequency to center
magnitude_spectrum = np.log(np.abs(fshift) + 1)

Applications of frequency-domain processing include: low-pass filtering (Gaussian blur in frequency domain removes high-frequency noise), high-pass filtering (removes low-frequency illumination variations), periodic noise removal (filtering out specific frequency peaks from sensor interference), and convolution acceleration (multiplying Fourier transforms instead of convolving spatial kernels). The JPEG compression standard uses the Discrete Cosine Transform (DCT), a real-valued variant of the Fourier Transform, to compress image blocks.

Morphological Operations

Morphological operations process images based on shape, operating on binary or grayscale images. Erosion shrinks foreground regions: a pixel is set to foreground only if all pixels under the structuring element are foreground. This removes small noise blobs and separates touching objects. Dilation expands foreground regions: a pixel is set to foreground if any pixel under the structuring element is foreground. This fills small holes and connects nearby objects.

Opening (erosion followed by dilation) removes small foreground objects while preserving the shape and size of larger objects. Closing (dilation followed by erosion) fills small holes inside foreground objects while preserving overall shape. The morphological gradient (dilation minus erosion) extracts object boundaries. Top-hat and black-hat transforms extract bright and dark features respectively.

kernel = np.ones((5,5), np.uint8)
erosion = cv2.erode(binary_img, kernel, iterations=1)
dilation = cv2.dilate(binary_img, kernel, iterations=1)
opening = cv2.morphologyEx(binary_img, cv2.MORPH_OPEN, kernel)
closing = cv2.morphologyEx(binary_img, cv2.MORPH_CLOSE, kernel)

Color Spaces

The RGB color space represents images as red, green, and blue channels, matching how displays work. However, RGB is not well-suited for image analysis because the three channels are highly correlated and perceptual differences do not correspond to Euclidean distances in RGB space.

The HSV (Hue, Saturation, Value) color space separates color information (hue) from intensity (value), making it ideal for color-based segmentation. The CIELAB (LAB) color space is perceptually uniform — Euclidean distance in LAB space corresponds to perceived color difference. LAB is preferred for color comparison and image analysis. YCbCr separates luminance from chrominance, used in video compression standards.

hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
lab = cv2.cvtColor(image, cv2.COLOR_BGR2LAB)

Histogram Equalization

Histogram equalization improves image contrast by spreading pixel intensity values across the full available range. The transformation function is the cumulative distribution function (CDF) of the pixel intensities. In a low-contrast image where pixel values cluster in a narrow range, the CDF spreads them across the full 0-255 range, revealing detail in both shadows and highlights.

Adaptive Histogram Equalization (AHE) operates on local image regions rather than the global histogram, providing better contrast enhancement in images with varying lighting conditions. Contrast Limited AHE (CLAHE) prevents noise amplification by clipping the histogram at a predefined threshold before computing the CDF, making it suitable for medical imaging and X-ray enhancement.

clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
equalized = clahe.apply(gray_image)

Edge Detection

The Canny edge detector, developed by John Canny in 1986, remains the gold standard. The multi-stage algorithm applies Gaussian smoothing, computes the gradient magnitude and direction using Sobel operators, performs non-maximum suppression to thin edges to single-pixel width, applies double thresholding to identify strong and weak edges, and tracks edges by hysteresis — weak edges are included only if connected to strong edges. The low and high thresholds are the critical parameters, typically set in a 1:2 or 1:3 ratio.

edges = cv2.Canny(blurred_image, threshold1=100, threshold2=200)

The Hough Transform detects lines and circles in edge images. Each pixel in the edge image votes for all possible lines passing through it in parameter space. Accumulator peaks indicate detected lines. The probabilistic Hough Transform is faster and more suitable for real-time applications.

Image Pyramids and Multi-Scale Processing

Image pyramids represent images at multiple resolutions, enabling algorithms to process information at different scales. The Gaussian pyramid is built by repeatedly applying Gaussian blur and downsampling by a factor of 2. Each level has half the width and height of the previous level. Applications include template matching across scales, image blending for panorama stitching, and feature detection invariant to object size.

The Laplacian pyramid stores the difference between adjacent Gaussian pyramid levels, encoding the band-pass information. Laplacian pyramids are used for image compression (the JPEG-2000-like encoding) and multi-resolution blending. In panorama stitching, Laplacian pyramid blending creates seamless transitions between overlapping images by blending low frequencies over a wide region and high frequencies over a narrow region, avoiding the double-edge artifacts of simple linear blending.

Frequently Asked Questions

What is the difference between image processing and computer vision? Image processing transforms images into other images (enhance, filter, compress). Computer vision extracts semantic information from images (detect objects, recognize faces, estimate depth). Image processing is typically a preprocessing step in computer vision pipelines.

Which color space should I use for segmentation? HSV is generally best for color-based segmentation because hue is independent of lighting intensity. For example, segmenting a red object: threshold on hue (0-10 and 170-180 for red) and saturation (above some minimum) while ignoring value. LAB is better when perceptual uniformity matters, such as measuring color similarity.

Why does Canny edge detection work better than simple gradient thresholding? Canny combines multiple refinements: non-maximum suppression produces single-pixel edges rather than thick bands, and hysteresis tracking connects fragmented edges while suppressing noise. Simple thresholding on gradient magnitude produces either noisy edges (low threshold) or fragmented edges (high threshold).

How do I remove noise without blurring edges? Use the bilateral filter, which averages pixels with similar intensity values, preserving sharp edges while smoothing flat regions. Anisotropic diffusion is another edge-preserving denoising method that reduces diffusion across edges. Both are more computationally expensive than Gaussian blur but produce better results.

What is the difference between erosion and dilation? Erosion shrinks foreground objects, removing small protrusions and separating connected objects. Dilation grows foreground objects, filling small holes and connecting nearby objects. Opening (erosion then dilation) removes small objects while preserving shape. Closing (dilation then erosion) fills holes while preserving shape.

OpenCV Beginners Guide — practical implementation of image processing operations
Convolutional Neural Networks — how learned filters extend classical convolution
Image Segmentation — pixel-level classification using processing techniques

Share this article

X LinkedIn Facebook Email