Eigenvalues and eigenvectors

In linear algebra, an eigenvector of a linear transformation is a nonzero vector that the transformation maps to a scalar multiple of itself; the corresponding scalar is its eigenvalue. For a square matrix $A$, this relationship is expressed as $Av=\lambda v$. Eigenvalues and eigenvectors identify invariant directions and help describe a transformation’s structure, its repeated application, and its representation in different coordinates. (math.mit.edu)

Definition and geometric meaning

Let $T:V\to V$ be a linear transformation on a vector space over the real or complex numbers. A vector $v\ne0$ is an eigenvector with eigenvalue $\lambda$ when

The zero vector is excluded because it satisfies this equation for every scalar and therefore cannot identify a particular eigenvalue. Multiplying an eigenvector by any nonzero scalar produces another eigenvector with the same eigenvalue. (math.mit.edu)

For real eigenvectors and eigenvalues, the geometric interpretation is scaling along a line through the origin. A positive eigenvalue preserves orientation, a negative eigenvalue reverses it, and zero maps the vector to zero. The absolute value measures the length-scaling factor. For example,

has eigenvectors $(1,0)^T$ and $(0,1)^T$, with eigenvalues $3$ and $-2$: the first coordinate direction is stretched threefold, while the second is reversed and stretched twofold. These conclusions follow directly from matrix multiplication. (math.mit.edu)

The scalar field matters. A quarter-turn rotation in a real plane has no real eigenvectors, although its matrix has complex eigenvalues $i$ and $-i$ when considered over the complex numbers. (ocw.mit.edu)

Characteristic polynomial and eigenspaces

Rearranging the eigenvalue equation gives

where $I$ is the identity matrix. A nonzero solution exists precisely when $A-\lambda I$ is singular, or equivalently when its determinant vanishes. Thus eigenvalues are roots of the characteristic polynomial

For an $n\times n$ matrix, this is a degree-$n$ polynomial. The fundamental theorem of algebra guarantees $n$ complex roots counted with multiplicity. Nonreal eigenvalues of a real matrix occur in complex-conjugate pairs. (ocw.mit.edu)

For a fixed eigenvalue, its eigenspace is

It contains all corresponding eigenvectors together with the zero vector. Once an eigenvalue is known, finding this space amounts to solving a homogeneous system of linear equations. (ocw.mit.edu)

An eigenvalue’s algebraic multiplicity is its multiplicity as a characteristic-polynomial root; its geometric multiplicity is the dimension of its eigenspace. The latter is at least one and no greater than the former. Counting algebraic multiplicities, the sum of the eigenvalues equals the matrix’s trace, and their product equals its determinant. Consequently, a square matrix is invertible exactly when zero is not an eigenvalue. (esp.mit.edu)

Diagonalization and the spectral theorem

Eigenvectors associated with distinct eigenvalues are linearly independent. An $n\times n$ matrix is diagonalizable over a chosen field exactly when it has a basis of $n$ eigenvectors over that field. Placing these vectors in the columns of an invertible matrix $P$ yields

where $D$ is diagonal and lists the corresponding eigenvalues in matching order. This is an eigenvalue decomposition: in the eigenvector basis, the transformation acts independently on each coordinate. (esp.mit.edu)

Distinct eigenvalues are sufficient but not necessary for diagonalizability. Repeated eigenvalues are compatible with it when every eigenspace has the required dimension. By contrast,

has eigenvalue $1$ twice but only a one-dimensional eigenspace, so it is not diagonalizable. This follows by solving its eigenvector equation. (esp.mit.edu)

The spectral theorem gives stronger results for real symmetric matrices: their eigenvalues are real, and they possess an orthonormal eigenvector basis. Consequently,

with $Q^{-1}=Q^T$, where $Q^T$ denotes the matrix transpose. Complex Hermitian matrices have the analogous decomposition using conjugate transpose. (ocw.mit.edu)

Computation

Characteristic polynomials provide a theoretical description and are useful for small exact examples. Once their roots are found, eigenvectors can be obtained through Gaussian elimination. Large numerical problems instead use methods from numerical linear algebra that work directly with the matrix. LAPACK’s nonsymmetric routines, for example, compute eigenvalues through Schur factorization; symmetric routines reduce matrices to tridiagonal form before solving the reduced problem. (ocw.mit.edu)

The QR algorithm is an important eigenvalue method. LAPACK also supplies divide-and-conquer methods for symmetric problems. The appropriate computation depends on matrix structure and on whether all eigenvalues, selected eigenvalues, or eigenvectors are required. (netlib.org)

Applications

Eigenpairs describe elementary modes of linear dynamical systems. If $Av=\lambda v$, then $A^kv=\lambda^kv$. For the constant-coefficient differential equation $x'(t)=Ax(t)$, the vector-valued function $x(t)=e^{\lambda t}v$ is a solution. Eigenvalue magnitudes therefore govern repeated discrete scaling, while real parts govern exponential growth or decay in continuous time. (ocw.mit.edu)

In principal component analysis, eigenvectors of the covariance matrix identify principal directions of variation. Their eigenvalues measure the corresponding variances. Retaining directions with the largest eigenvalues provides dimensionality reduction. This analysis is closely related to singular value decomposition: for centered data $X$, the right singular vectors are eigenvectors of $X^TX$, and squared singular values determine covariance eigenvalues through the chosen normalization. (web.stanford.edu)