What Is Image Filtering in the Spatial Domain?

Image filtering is a technique for modifying or enhancing an image. For example, you can filter an image to emphasize certain features or remove other features. Image processing operations implemented with filtering include smoothing, sharpening, and edge enhancement.

In image processing applications, filtering is a commonly a neighborhood operation. A neighborhood operation determines the value of any given pixel in the output image by applying some algorithm to the values of the pixels in the neighborhood of the corresponding input pixel. The neighborhood of a pixel neighborhood is a set of pixels that are defined by their locations relative to that pixel. Different filters change how the neighborhood of a pixel contributes to the output value of the pixel.

In linear filtering, the value of an output pixel is a weighted sum of the values of the pixels in the neighborhood of the input pixel. The matrix of weights is called the kernel or the filter. Functions that filter an image perform either a correlation operation using a correlation kernel or a convolution operation using a convolution kernel. These two operations are closely related, and you can express each in terms of the other. The kernels are both matrices of weights. The difference is that correlation kernels are rotated by 180 degrees around the center pixel with respect to convolution kernels, and likewise convolution kernels are rotated by 180 degrees around the center pixel with respect to correlation kernels. For more information, see Compare Image Filtering Using Correlation and Convolution.

MATLAB^® and Image Processing Toolbox™ provide many functions that enable you to design and apply image filters using either correlation or convolution operations. The table summarizes common filtering functions. The table also indicates whether each function expects a correlation kernel or a convolution kernel. If you have a kernel of the opposite type, then you can convert the kernel to the expected type by rotating the kernel by 180 degrees about the center pixel.

Function	Kernel	Description
`conv2`	Convolution kernel	Perform convolution of a 2-D grayscale or binary image.
`convn`	Convolution kernel	Perform convolution of an N-D grayscale or binary image.
`filter2`	Correlation kernel	Filter a 2-D grayscale image using correlation.
`imfilter`	Correlation kernel, by default This function can also perform convolution when you specify a convolution kernel and the value `"conv"` as an optional input argument.	Filter an image using either correlation (by default) or convolution. This function supports grayscale, RGB, multispectral, and binary images of any dimensionality, and supports image-specific padding options for the boundary of the image.
`fspecial`	Correlation kernel	Create a predefined 2-D filter for use with `imfilter`.
`fspecial3`	Correlation kernel	Create a predefined 3-D filter for use with `imfilter`.

In nonlinear filtering, the value of an output pixel depends on the neighborhood of the input pixel, but the output value is not a linear combination of the pixel values in the neighborhood. An example of a nonlinear filter is a median filter, in which the output value of a pixel is equal to the median value of the neighborhood.

Linear Filtering Using Correlation

Correlation is a neighborhood operation in which each output pixel is the weighted sum of neighboring input pixels. The matrix of weights is called the correlation kernel, also known as the filter.

These steps show how the correlation operation calculates the value of an output pixel. Suppose you start with a grayscale image. This image shows an enlarged 5-by-5 pixel region of the image, with pixel values superimposed over each pixel. Because filtering can result in negative pixel values, assume that this image is of data type double, for illustration purposes. (Note that many Image Processing Toolbox functions expect images of data type double to have pixel values in the range [0, 1].)

Also, suppose you define the following correlation kernel. This particular kernel is based on the Sobel operator and can be used to detect horizontal edges.

You can follow these steps to calculate the output pixel at position (2, 4) using the correlation operation.

Slide the center element of the correlation kernel so that the center of the kernel lies on top of the (2, 4) element of the input image.
Multiply each weight in the correlation kernel by the value of the pixel underneath it.
Sum the individual products from step 3.
169+244+51-97-88-57 = 222
Therefore, the value of the (2, 4) output pixel is 222.

This figure displays the result of filtering the entire region with the correlation kernel. The (2, 4) pixel with an intensity value of 222 is circled in yellow.

Linear Filtering Using Convolution

The convolution operation is closely related to correlation. In convolution, you still calculate the value of an output pixel as a weighted sum of its neighboring pixels. The matrix of weights is called the convolution kernel, also known as the filter. The difference is that, before multiplying the neighborhood pixel values by the convolution kernel, the convolution operation rotates the convolution kernel by 180 degrees about the center element.

These steps show how the convolution operation calculates the value of an output pixel. Suppose you start with the same grayscale image as before. This image shows the same enlarged 5-by-5 pixel region of the image, with pixel values superimposed over each pixel.

Also, suppose you define the following convolution kernel. This particular kernel is based on a Roberts cross operator and emphasizes diagonal edges oriented from the bottom-left to the top-right.

You can follow these steps to calculate the output pixel at position (2, 4) using the convolution operation.

Rotate the convolution kernel by 180 degrees about its center element.
Slide the center element of the rotated convolution kernel so that the center of the kernel lies on top of the (2, 4) element of the input image.
Multiply each weight in the rotated convolution kernel by the value of the pixel underneath it.
Sum the individual products from step 3.
122+148-48-44 = 178
Therefore, the value of the (2, 4) output pixel is 178.

This figure displays the result of filtering the entire region with the convolution kernel. The (2, 4) pixel with an intensity value of 178 is circled in yellow.

Nonlinear Filtering

In nonlinear filtering, you specify a mathematical operation to apply to the neighborhood of each pixel. Using the nlfilter function, you can specify any mathematical operation that accepts a 2-D matrix as input and returns a numeric scalar. Image Processing Toolbox also offers many functions that perform common nonlinear filtering operations. For example, the medfilt2 function applies a median filter to the neighborhood of each input pixel.

These steps show how a nonlinear filter calculates the value of an output pixel. Suppose you start with the same grayscale image as before, with added "salt and pepper" noise that has turned one pixel of the image white. This image shows the same enlarged 5-by-5 pixel region of the image, with pixel values superimposed over each pixel.

Also, suppose you specify that the filter return the median value of a 3-by-3 pixel neighborhood. Median filtering is often used in image processing to reduce salt and pepper noise. A median filter is more effective than correlation when the goal is to simultaneously reduce noise and preserve edges.

You can follow these steps to calculate the output pixel at position (2, 4) using the median filtering operation.

Identify the 3-by-3 pixel neighborhood that is centered on the (2, 4) element of the input image.
Apply the mathematical operation to the pixels in the neighborhood.
median([169 122 51 148 255 48 97 44 57]) = 97
Therefore, the value of the (2, 4) output pixel is 97.

This figure displays the result of filtering the entire region with the median filter. The (2, 4) pixel with an intensity value of 97 is circled in yellow.