Orthogonal complement · aiwiki.page

The orthogonal complement of a subset of a vector space equipped with an inner product is the set of all vectors orthogonal to every member of that subset. Written $S^\perp$ , it describes the directions perpendicular to $S$ within a specified ambient space. The concept connects geometric perpendicularity with subspace decomposition in linear algebra and extends to infinite-dimensional spaces in functional analysis. (ocw.mit.edu)

Definition and elementary properties

Let $V$ be an inner product space over the real numbers or complex numbers, and let $S\subseteq V$ . Its orthogonal complement is

S^\perp=\{v\in V:\langle v,s\rangle=0 \text{ for every }s\in S\}.

In complex spaces, conjugate symmetry makes the condition independent of which argument is written first. The ambient space and inner product are essential: changing either can change the orthogonal complement. (people.math.harvard.edu)

Even when $S$ is not a subspace, $S^\perp$ is a linear subspace. Indeed, the inner product conditions are preserved under addition and scalar multiplication. Orthogonality to all members of $S$ also implies orthogonality to every linear combination of those members, so

S^\perp=(\operatorname{span}S)^\perp,

where $\operatorname{span}S$ denotes the linear span. (people.math.harvard.edu)

For subspaces $U,W\subseteq V$ , the defining equations give

U\subseteq W\implies W^\perp\subseteq U^\perp, \qquad (U+W)^\perp=U^\perp\cap W^\perp.

Thus taking an orthogonal complement reverses inclusion. Moreover,

W\cap W^\perp=\{0\},

because a vector in the intersection satisfies $\langle w,w\rangle=0$ , which forces $w=0$ . These identities follow directly from the definition and positive definiteness of the inner product. (people.math.harvard.edu)

Finite-dimensional geometry and decomposition

If $V$ has finite dimension $n$ , and $W$ has dimension $k$ , then

\dim W^\perp=n-k,\qquad (W^\perp)^\perp=W, \qquad V=W\oplus W^\perp.

The last expression is an orthogonal direct sum: every vector has a unique decomposition into a component in $W$ and a component in $W^\perp$ . Orthogonality alone does not make two subspaces complementary; together they must also span the ambient space. (ocw.mit.edu)

In three-dimensional Euclidean space, for example, consider the line

W=\operatorname{span}\{(1,2,3)\}.

Applying the definition gives

W^\perp=\{(x,y,z):x+2y+3z=0\} =\operatorname{span}\{(-2,1,0),(-3,0,1)\}.

Its orthogonal complement is therefore a plane through the origin. Conversely, that plane’s orthogonal complement is the original line. More generally, a nonzero vector determines a perpendicular hyperplane through the origin. This example illustrates the line–plane relationship described by the dimension formula. (ocw.mit.edu)

An orthogonal complement is not a set-theoretic complement. It contains the zero vector, as does every subspace, and usually contains only some of the vectors outside $W$ . (people.math.harvard.edu)

Matrix characterization and computation

Suppose the columns of a matrix $B$ span a subspace $W\subseteq\mathbb R^n$ . A vector $x$ is perpendicular to every column exactly when

B^{\mathsf T}x=0.

Consequently,

W^\perp=\ker B^{\mathsf T},

where $B^{\mathsf T}$ is the transpose and the kernel is its null space. Computing $W^\perp$ therefore reduces to solving a homogeneous system of linear equations. A basis of its solutions can be found by Gaussian elimination. (ericdarve.github.io)

For a real $m\times n$ matrix $A$ , the four fundamental subspaces form two complementary pairs:

\operatorname{row}(A)^\perp=\ker A, \qquad \operatorname{col}(A)^\perp=\ker A^{\mathsf T}.

The first pair lies in $\mathbb R^n$ , the second in $\mathbb R^m$ . If $A$ has rank $r$ , their dimensions are respectively $r,n-r$ and $r,m-r$ , consistent with the rank–nullity theorem. (ericdarve.github.io)

Alternatively, extend an orthonormal basis of $W$ to one of $V$ ; the added vectors form an orthonormal basis of $W^\perp$ . In finite dimensions this can be carried out using the Gram–Schmidt process. (github.com)

Orthogonal projection and least squares

The decomposition $v=w+z$ , with $w\in W$ and $z\in W^\perp$ , defines the orthogonal projection $P_Wv=w$ . The residual $v-P_Wv$ lies in $W^\perp$ , and $P_Wv$ is the unique vector of $W$ nearest to $v$ . The complementary projection satisfies

P_{W^\perp}=I-P_W.

These statements hold in finite-dimensional inner product spaces and for closed subspaces of Hilbert spaces. (people.math.harvard.edu)

For a real matrix $B$ whose columns form a basis of $W$ ,

P_W=B(B^{\mathsf T}B)^{-1}B^{\mathsf T}.

If those columns are orthonormal, this simplifies to $P_W=BB^{\mathsf T}$ . In ordinary least squares, minimizing $\|Ax-b\|^2$ requires the residual to belong to $\operatorname{col}(A)^\perp$ . This yields the normal equations

A^{\mathsf T}(b-A\hat x)=0, \qquad A^{\mathsf T}A\hat x=A^{\mathsf T}b.

The fitted vector is unique even when its coefficient representation is not. (github.com)

Infinite-dimensional spaces

In any inner product space, an orthogonal complement is a closed set in the topology induced by the inner-product norm. Each orthogonality condition is preserved under limits, and intersecting all such conditions produces a closed subspace. (people.math.harvard.edu)

For a subspace $W$ of a Hilbert space $H$ , completeness gives

(W^\perp)^\perp=\overline W, \qquad H=\overline W\oplus W^\perp,

where $\overline W$ is the closure of $W$ . Hence $H=W\oplus W^\perp$ holds precisely when $W$ is closed. A proper dense subspace has orthogonal complement $\{0\}$ , demonstrating why the finite-dimensional double-complement identity needs a closure in infinite dimensions. Without completeness, the Hilbert-space decomposition theorem cannot generally be assumed. (ocw.mit.edu)