Tensor

A tensor is an object in linear algebra that expresses multilinear relationships between vectors, covectors, and scalars independently of a chosen coordinate system. Scalars, vectors, and certain objects represented by matrices are elementary examples. Tensor components change when the basis changes, while the underlying object remains the same. In numerical computing, “tensor” also commonly means a multidimensional array, without necessarily specifying a geometric transformation law. These related meanings distinguish an abstract mathematical object from its representation or storage format. (damtp.cam.ac.uk)

Mathematical definition

Let $V$ be a finite-dimensional vector space over a field $F$, typically the real numbers or complex numbers. Its dual space $V^*$ consists of linear maps from $V$ to $F$; its elements are called covectors. A tensor of type $(p,q)$ is an element of the tensor product space

Equivalently, it is a multilinear map

“Multilinear” means linear in each argument separately, with the remaining arguments fixed. The equivalence uses the natural identification of a finite-dimensional vector space with its double dual. (damtp.cam.ac.uk)

The integers $p$ and $q$ count contravariant and covariant factors; their sum is the tensor’s order. A scalar has type $(0,0)$, a vector type $(1,0)$, and a covector type $(0,1)$. A bilinear form has type $(0,2)$, whereas a linear operator $V\to V$ corresponds to type $(1,1)$. Both can be represented by a matrix, but their transformation laws differ. (damtp.cam.ac.uk)

Components and changes of basis

Choosing a basis $e_i$ and its dual basis $\varepsilon^j$ expresses a tensor through components

Upper indices conventionally denote contravariant components, and lower indices covariant components. A general order-$r$ tensor on an $n$-dimensional space has $n^r$ component positions, although symmetry can reduce the number of independent values. (arxiv.org)

For an explicit basis change $e'_i=e_jA^j{}_i$, vector components transform by $A^{-1}$, while covector components transform by $A$. Each index of a higher-order tensor receives the corresponding factor. For a linear operator with component matrix $L$,

For a bilinear form with component matrix $B$,

Thus, an array’s dimensions alone do not determine its tensor type. A component array together with the appropriate transformation rule represents the same mathematical object in different bases. (damtp.cam.ac.uk)

Tensor operations and symmetry

Tensors of the same type can be added and multiplied by scalars. Taking their tensor product combines their factors: the product of types $(p,q)$ and $(r,s)$ has type $(p+r,q+s)$. For vectors $u$ and $v$, the components of $u\otimes v$ are $u^iv^j$. (damtp.cam.ac.uk)

Contraction pairs a vector factor with a covector factor and sums over their matching indices, reducing the order by two. For a type-$(1,1)$ tensor, contraction gives the trace $T^i{}_i$. The Einstein summation convention suppresses the summation sign when an index occurs twice, once above and once below. (damtp.cam.ac.uk)

A tensor is symmetric in selected slots if exchanging them leaves it unchanged, and antisymmetric if exchanging them reverses its sign. Totally antisymmetric covariant tensor fields are differential forms. These symmetry properties are intrinsic rather than consequences of a particular coordinate choice. (damtp.cam.ac.uk)

Tensor fields and geometry

In differential geometry, a tensor field assigns a tensor smoothly to every point of a manifold, using the tangent space and its dual at that point. The distinction matters: an individual tensor describes an object at one point, whereas a tensor field can vary across a space. (damtp.cam.ac.uk)

A metric tensor is a symmetric, nondegenerate type-$(0,2)$ tensor field. A positive-definite metric determines lengths and angles; a Lorentzian metric supplies the corresponding geometric structure for spacetime. A metric and its inverse also permit indices to be lowered or raised, for example $v_i=g_{ij}v^j$. This identification of vectors with covectors requires additional structure, rather than following from the vector space alone. (damtp.cam.ac.uk)

In general relativity, the metric describes spacetime geometry, and the Riemann curvature tensor characterizes its curvature. Tensor equations express relationships without making their validity depend on a particular coordinate chart. Ordinary partial derivatives of tensor components need not transform tensorially; a covariant derivative incorporates a connection to obtain a tensorial derivative. (damtp.cam.ac.uk)

Arrays, order, and tensor rank

Computing libraries commonly use “tensor” for an array with a specified shape and element type. An array of shape $(2,3,4)$ has three axes and 24 entries. In this vocabulary, rank usually means the number of axes. Reshaping reorganizes entries into another compatible shape; transposition permutes axes. Neither operation should automatically be interpreted as a geometric change of basis. (tensorflow.org)

In tensor decomposition, tensor rank has a different meaning: the minimum number of simple tensors required to express an object as a sum. A nonzero simple tensor is a product $u_1\otimes\cdots\otimes u_r$. For order two, this rank agrees with matrix rank; for higher orders, decomposition and approximation have substantially different properties. (cs.cornell.edu)

The CP decomposition expresses a tensor as a sum of rank-one terms. Tucker decomposition combines a smaller core tensor with factor matrices along its modes and generalizes aspects of principal component analysis. Such methods are used in signal processing, computer vision, and multiway data analysis, where separate axes preserve distinct kinds of variation instead of flattening all observations into a single matrix. (epubs.siam.org)