Convolution

Convolution is a mathematical operation that combines two functions into a third by integrating or summing their products over relative shifts. Unlike pointwise multiplication, it combines values at different arguments. It provides a common framework for describing the response of linear systems, calculating distributions of sums of independent random variables, and implementing spatial filters. Its continuous and discrete forms share the same underlying structure. (dlmf.nist.gov)

Definition and interpretation

For functions $f$ and $g$ on the real line, the usual continuous convolution is

whenever this integral exists. The functions may take real or complex values. In Euclidean space $\mathbb R^d$, the same definition applies with integration over all $d$ coordinates. Some references incorporate a normalization factor into convolution, so formulas involving integral transforms must be read with their conventions in mind. (math.mit.edu)

Geometrically, $g(x-t)$ is obtained by reflecting $g(t)$ about the origin and then shifting it by $x$. Multiplying this reflected, shifted function by $f(t)$ and integrating measures their weighted overlap. Repeating the calculation for different shifts produces the output function. Equivalently, convolution adds shifted copies of one function, weighted by values of the other. (ocw.mit.edu)

For sequences indexed by integers, discrete convolution replaces the integral with a sum:

Finite sequences are commonly extended by zeros outside their recorded ranges. If their lengths are $N$ and $M$, the full output contains $N+M-1$ positions. For example, the sequences $(1,2)$ and $(3,4)$, both starting at index zero, produce $(3,10,8)$: the middle value is $1\cdot4+2\cdot3$. (numpy.org)

Existence and algebraic properties

Existence requires appropriate assumptions; arbitrary functions need not have a well-defined convolution. A standard setting is the $L^1$ space of absolutely integrable functions. If $f,g\in L^1(\mathbb R^d)$, their convolution exists almost everywhere, belongs to $L^1$, and satisfies

This estimate is a case of Young’s convolution inequality. (math.mit.edu)

Under suitable convergence assumptions, convolution is commutative, associative, and distributive:

It is linear in each argument separately. Consequently, fixing $g$ makes $f\mapsto f*g$ a linear map, whereas treating both arguments as variable gives a bilinear operation. These properties allow convolution expressions to be regrouped and decomposed without changing their values. (ocw.mit.edu)

Convolution also interacts with differentiation. For instance, if $f$ is continuous with compact support and $g$ is smooth with compact support, derivatives can be transferred to $g$:

Convolution with a suitable smooth kernel therefore produces a smooth function. Such kernels, called mollifiers, are used in mathematical analysis to approximate less regular functions by smooth ones. For continuous, compactly supported functions, appropriately rescaled mollifiers yield uniform convergence to the original function. (math.mit.edu)

Fourier transforms and computation

The convolution theorem connects convolution with the Fourier transform. Using the convention

the theorem states

Thus an operation that combines many shifted products becomes pointwise multiplication in the frequency domain. Other Fourier conventions introduce different constant factors. (dlmf.nist.gov)

This identity supports efficient numerical algorithms using the fast Fourier transform: transform the inputs, multiply corresponding transform values, and apply an inverse transform. Discrete Fourier multiplication naturally corresponds to circular convolution, in which indices wrap periodically. To recover ordinary finite linear convolution, sufficient zero-padding must prevent this wraparound from mixing output values. (numpy.org)

Software commonly distinguishes three output modes. “Full” retains the entire convolution; “same” selects a centered portion matching a specified input size; “valid” retains only positions that do not depend on zero-padding. Boundary conventions therefore affect both output dimensions and edge values. Direct summation and Fourier-based methods are alternative implementations of the same operation. (scipy.github.io)

Probability

In probability theory, convolution describes the distribution of a sum of independent random variables. If $X$ and $Y$ have probability density functions $p_X$ and $p_Y$, then $Z=X+Y$ has density

For integer-valued variables, the corresponding probability mass functions are combined by discrete convolution. Independence is essential: it allows the joint probabilities or densities to factor into products of marginal quantities. (statproofbook.github.io)

The operation adds the random variables, not their probabilities pointwise. For example, convolving two independent Poisson distributions with parameters $\lambda_1$ and $\lambda_2$ gives a Poisson distribution with parameter $\lambda_1+\lambda_2$. (web.stanford.edu)

Signals, images, and neural networks

In signal processing, the zero-state output of a linear time-invariant system is the convolution of its input with its impulse response. Linearity permits an input to be decomposed into weighted impulses, while time invariance makes each shifted impulse produce a correspondingly shifted response. Their superposition yields the convolution formula. In image processing, the same framework describes spatially invariant blur. (ocw.mit.edu)

Convolutional neural networks use learned kernels that combine local input values, often across multiple channels. Terminology requires care: PyTorch’s two-dimensional convolution layer computes cross-correlation rather than mathematical convolution, omitting the kernel reversal. Its stride controls sampling intervals, padding controls boundary extension, and dilation controls spacing between kernel positions. These choices determine how local neighborhoods contribute to the output. (docs.pytorch.org)