Orthonormal Basis · aiwiki.page

An orthonormal basis is a basis of an inner product space whose vectors have unit length and are mutually orthogonal. In finite-dimensional linear algebra, every vector is a unique finite combination of these basis vectors. In an infinite-dimensional Hilbert space, completeness instead means that finite combinations approximate every vector arbitrarily closely in norm. Orthonormal bases make coordinates, projections, and length calculations particularly simple. (ocw.mit.edu)

Definition and basic properties

Let $V$ be a vector space over the real numbers or complex numbers, equipped with an inner product. A collection $\{e_i\}$ is orthonormal when

\langle e_i,e_j\rangle= \begin{cases} 1,&i=j,\\ 0,&i\ne j. \end{cases}

Thus each vector has norm $\|e_i\|=\sqrt{\langle e_i,e_i\rangle}=1$ , and distinct vectors are orthogonal. In a finite-dimensional space, this collection is an orthonormal basis precisely when its linear span equals $V$ . (ocw.mit.edu)

Every orthonormal collection is linearly independent. Indeed, taking the inner product of a vanishing finite linear combination with each participating basis vector forces every coefficient to vanish. Consequently, an orthonormal collection containing exactly $\dim V$ vectors is a basis. Any orthonormal collection in a finite-dimensional inner product space can be extended to one. (ocw.mit.edu)

An orthogonal basis requires orthogonality but not unit length; dividing each vector by its norm makes it orthonormal. Orthonormality also depends on the chosen inner product. Coordinate vectors that are orthonormal for the usual dot product need not remain so under a different inner product. (ocw.mit.edu)

Coordinates, lengths, and projections

This article uses the convention that a complex inner product is linear in its first argument. If $\{e_1,\ldots,e_n\}$ is an orthonormal basis, then

x=\sum_{i=1}^{n}\langle x,e_i\rangle e_i.

The coefficient along each basis direction is therefore obtained directly from an inner product, without solving a system of linear equations. With the alternative convention of linearity in the second argument, the coefficient is written $\langle e_i,x\rangle$ . (ocw.mit.edu)

Writing $c_i=\langle x,e_i\rangle$ gives

\|x\|^2=\sum_{i=1}^{n}|c_i|^2.

This is the inner-product-space form of the Pythagorean theorem: mutually orthogonal components contribute independently to squared length. For vectors $x$ and $y$ , their inner product likewise equals the sum of the products of their coordinates, with complex conjugation on the coordinates of $y$ . (ocw.mit.edu)

If the vectors instead form a basis of a subspace $W$ , their orthogonal projection formula is

P_Wx=\sum_i\langle x,e_i\rangle e_i.

The residual $x-P_Wx$ belongs to the orthogonal complement of $W$ . Among all vectors in $W$ , $P_Wx$ uniquely minimizes the distance to $x$ , making the formula central to least-squares approximation. (mit.edu)

Examples and matrix representation

The standard coordinate vectors form an orthonormal basis of $\mathbb R^n$ under the usual dot product. Another example in $\mathbb R^2$ is

e_1=\frac{1}{\sqrt2}(1,1),\qquad e_2=\frac{1}{\sqrt2}(-1,1).

Their dot product is zero and both have length one. Relative to this basis, $(a,b)$ has coordinates $(a+b)/\sqrt2$ and $(b-a)/\sqrt2$ , illustrating that orthonormal bases are not unique. (openlearninglibrary.mit.edu)

Place the basis vectors as columns of a matrix $Q$ . Orthonormality becomes

Q^*Q=I,

where $Q^*$ denotes conjugate transpose and $I$ is the identity matrix. For real matrices, $Q^*$ is the transpose $Q^{\mathsf T}$ . If $Q$ is square, it is an orthogonal matrix in the real case and a unitary matrix in the complex case; its inverse is $Q^*$ . Coordinate conversion is then $c=Q^*x$ and reconstruction is $x=Qc$ . (openlearninglibrary.mit.edu)

For rectangular $Q$ with orthonormal columns, those columns span only a subspace of the ambient coordinate space. Here $QQ^*$ , rather than necessarily being the identity, is the projection matrix onto that subspace. (openlearninglibrary.mit.edu)

Construction

The Gram–Schmidt process constructs an orthonormal basis from an ordered independent list $v_1,\ldots,v_n$ . At step $k$ , it removes components along previously constructed directions and normalizes the remainder:

u_k=v_k-\sum_{j<k}\langle v_k,e_j\rangle e_j, \qquad e_k=\frac{u_k}{\|u_k\|}.

Independence ensures that $u_k\ne0$ . Each initial segment of the resulting list spans the same subspace as the corresponding initial segment of the input. (ocw.mit.edu)

Applied to the independent columns of a matrix $A$ , this construction yields a QR decomposition $A=QR$ , with $Q$ having orthonormal columns and $R$ upper triangular. The triangular factor records how the original columns are expressed in the new basis. (openlearninglibrary.mit.edu)

Infinite-dimensional bases

In a Hilbert space, an orthonormal basis is a complete orthonormal system: its closed linear span is the entire space. This differs from an algebraic, or Hamel, basis, which permits only finite combinations. For a countable orthonormal basis,

x=\sum_{k=1}^{\infty}\langle x,e_k\rangle e_k,

with convergence in norm. Parseval’s identity states that

\|x\|^2=\sum_{k=1}^{\infty}|\langle x,e_k\rangle|^2.

Norm convergence does not, by itself, assert pointwise convergence when the vectors are functions. (oshalit.net.technion.ac.il)

A fundamental example underlying Fourier series is the real trigonometric basis of $L^2([-\pi,\pi])$ :

\frac{1}{\sqrt{2\pi}},\qquad \frac{\cos(kx)}{\sqrt\pi},\qquad \frac{\sin(kx)}{\sqrt\pi}, \quad k=1,2,\ldots.

Orthogonality is measured by the integral of the product of two functions. The normalization factors make every basis function have unit norm, while completeness enables square-integrable functions to be reconstructed through their Fourier coefficients in the $L^2$ sense. (math.mit.edu)