Tensor Contraction · aiwiki.page

Tensor contraction is an operation on a tensor that pairs a vector-space factor with a corresponding dual-space factor, removing both through evaluation. In coordinates, it identifies two compatible indices and sums over their common range. It generalizes familiar operations in linear algebra, including matrix trace and matrix multiplication. In numerical computing, the term also denotes multiplying multidimensional arrays and summing over specified matching axes. (elliptic.space)

Coordinate-free definition

Let $V$ be a finite-dimensional vector space over a field $\mathbb F$ . Its dual space $V^*$ consists of linear functionals $\alpha:V\to\mathbb F$ . The canonical evaluation pairing induces a linear map

C:V^*\otimes V\longrightarrow\mathbb F, \qquad C(\alpha\otimes v)=\alpha(v).

Its existence follows from the universal property of the tensor product: evaluation is bilinear, so it extends uniquely to a linear map on the product space. No metric or choice of coordinates is required. (elliptic.space)

For a mixed tensor

T\in V^{\otimes p}\otimes(V^*)^{\otimes q},

contracting one $V$ factor with one $V^*$ factor produces a tensor of type $(p-1,q-1)$ . On an elementary tensor, the selected vector and functional are replaced by the scalar obtained by evaluation; the remaining factors retain their order. Extending linearly defines the operation for arbitrary tensors. Each such contraction therefore reduces total tensor order by two. Here “order,” sometimes called tensor rank, counts factors and must not be confused with matrix rank. (maths.ed.ac.uk)

Index notation and invariance

Choose a basis $e_1,\ldots,e_n$ and its dual basis $e^1,\ldots,e^n$ , satisfying $e^i(e_j)=\delta^i_j$ . A type- $(2,2)$ tensor has components $T^{ab}{}_{cd}$ . Contracting its second upper index with its first lower index gives

S^a{}_d=\sum_{i=1}^{n}T^{ai}{}_{id}.

The indices $a,d$ are free indices, describing the output; $i$ is a dummy index used only for summation. Renaming a dummy index does not alter the result. (maths.ed.ac.uk)

Under the Einstein summation convention, the summation sign is omitted when an index occurs once upstairs and once downstairs. The opposite transformation laws of these two indices cancel when they are paired, ensuring that contraction is independent of the chosen basis. An expression with free indices remains a tensor, while a fully contracted expression is a scalar. Coordinate independence does not imply that the components of a partially contracted tensor remain numerically unchanged under a basis transformation. (maths.ed.ac.uk)

Familiar examples

Evaluation of a covector on a vector is the simplest example:

\alpha(v)=\alpha_i v^i.

For an endomorphism $A:V\to V$ , contracting its two indices gives the matrix trace,

\operatorname{tr}A=A^i{}_i.

Thus trace is intrinsically an operation on a linear operator, not merely a rule for adding diagonal entries of a displayed matrix. (elliptic.space)

Matrix multiplication is a contraction of a tensor product:

(AB)^i{}_k=\sum_j A^i{}_jB^j{}_k.

The shared index $j$ disappears, while $i,k$ remain. Similarly, multiplying a matrix by a vector contracts the matrix’s input index with the vector index. These examples distinguish contraction from an outer product, which retains all input indices, and from componentwise multiplication, which alone performs no summation. (elliptic.space)

Contraction with a metric

Canonical contraction pairs opposite index types. Pairing two upper indices, or two lower indices, requires additional structure. A nondegenerate metric tensor $g$ identifies vectors with covectors, permitting indices to be lowered or raised. For vectors $u,v$ ,

g(u,v)=g_{ij}u^iv^j.

For a covariant two-tensor $A_{ij}$ , its metric trace is

g^{ij}A_{ij},

where $g^{ij}$ represents the inverse metric. These expressions combine metric insertion with canonical contractions. (damtp.cam.ac.uk)

In a real inner-product space with an orthonormal basis, $g_{ij}=\delta_{ij}$ , making metric contraction look like ordinary matching-index summation. This simplification is basis-dependent. In differential geometry, contraction acts pointwise on tensor fields. The Ricci tensor is obtained by contracting the Riemann curvature tensor, and contracting the Ricci tensor with the inverse metric gives scalar curvature. Both constructions enter general relativity. (damtp.cam.ac.uk)

Arrays and computational notation

Array libraries generally encode axis lengths rather than geometric distinctions between covariant and contravariant indices. For arrays $A_{ijk}$ and $B_{jkl}$ , a two-axis contraction is

C_{il}=\sum_{j,k}A_{ijk}B_{jkl}.

Each paired axis must have matching length. If the operands have orders $r,s$ and $m$ axis pairs are contracted, the output has order $r+s-2m$ . These array operations implement coordinate formulas; their geometric meaning depends on how the arrays represent tensors. (numpy.org)

NumPy’s einsum expresses the example as einsum('ijk,jkl->il', A, B). Its notation also distinguishes a trace, 'ii->', from diagonal extraction, 'ii->i': repeated labels do not necessarily imply summation when explicitly retained in the output. (numpy.org)

Tensor networks and contraction order

A tensor network represents tensors as nodes and contracted indices as connecting edges. Unconnected edges correspond to free output indices; a network without free edges evaluates to a scalar. Such representations support calculations of wave-function norms and observable expectation values in quantum mechanics. Classical statistical mechanics also uses network contractions to evaluate partition functions. (arxiv.org)

Although exact arithmetic gives the same result for equivalent contraction orders, intermediate tensor sizes and operation counts can differ substantially. Choosing which tensors to combine first is therefore an important algorithmic problem. Large networks may require approximate contraction, replacing growing intermediate tensors with compressed representations. Such approximations introduce truncation error, distinct from the exact contraction operation itself. (arxiv.org)

References

Linear Algebra and Quantum Information, Section 9.4: Contractions and Traceelliptic.space
Riemannian Manifolds: An Introduction to Curvaturemaths.ed.ac.uk
Lecture 3: Tensors Continuedocw.mit.edu
numpy.einsum — NumPy v2.1 Manualnumpy.org
numpy.tensordot — NumPy Manualnumpy.org
A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair Statesarxiv.org
Lecture Notes of Tensor Network Contractionsarxiv.org