Stationary Process · aiwiki.page

A stationary process is a stochastic process whose statistical description is unchanged when its time index is shifted. Stationarity concerns the probabilistic laws governing observations, not whether an individual trajectory remains constant. Two principal definitions are used: strict stationarity, which requires invariance of all finite-dimensional distributions, and weak stationarity, which requires invariance of first and second moments. The distinction is fundamental in time series analysis and signal processing, where observations at different times are interpreted through a common statistical structure. (sia.mit.edu)

Strict stationarity

Let $X=\{X_t:t\in T\}$ be a collection of random variables on a common probability space, usually indexed by $T=\mathbb Z$ or $T=\mathbb R$ . The process is strictly stationary if, for every positive integer $n$ , every selection of times $t_1,\ldots,t_n$ , and every admissible shift $h$ ,

(X_{t_1},\ldots,X_{t_n}) \overset{d}{=} (X_{t_1+h},\ldots,X_{t_n+h}).

Here $\overset{d}{=}$ denotes equality of joint probability distributions. For a restricted index set, both sets of times must lie in that set. (sia.mit.edu)

Taking $n=1$ shows that each observation has the same marginal distribution. Identical marginals alone are insufficient, however: relationships among observations must also be invariant. Strict stationarity imposes no requirement that a mean or a finite variance exist; it is a distributional property rather than a moment condition. (bookdown.org)

Weak stationarity

A real-valued process is weakly stationary, also called wide-sense stationary or covariance stationary, if it has finite second moments and satisfies

\mathbb E[X_t]=\mu,\qquad \operatorname{Cov}(X_{t+h},X_t)=\gamma(h).

Thus its expected value is constant and its covariance depends only on the lag $h$ , rather than on absolute time. Its variance is consequently constant, since $\operatorname{Var}(X_t)=\gamma(0)$ . When $\gamma(0)>0$ , the autocorrelation function is

\rho(h)=\frac{\gamma(h)}{\gamma(0)}.

This describes correlation at each time separation. (online.stat.psu.edu)

Strict stationarity implies weak stationarity when finite second moments exist. The converse generally fails because first and second moments do not determine arbitrary distributions. For a Gaussian process, however, the two definitions are equivalent: every finite-dimensional multivariate normal distribution is determined by its mean vector and covariance matrix. Normal marginal distributions alone do not establish this Gaussian-process property. (sia.mit.edu)

Examples and nonexamples

An independent, identically distributed sequence is strictly stationary; if its variance is finite, it is also weakly stationary. Such a sequence illustrates that independence is compatible with stationarity, although stationarity does not require independence between times. (stat153.berkeley.edu)

A standard dependent example is a causal first-order autoregressive process,

X_t=c+\phi X_{t-1}+\varepsilon_t,

with zero-mean independent, identically distributed innovations of variance $\sigma^2$ . For $|\phi|<1$ , its stationary causal solution has

\mu=\frac{c}{1-\phi},\qquad \gamma(h)=\frac{\sigma^2}{1-\phi^2}\phi^{|h|}.

These formulas describe the stationary solution, not an arbitrary initialization: starting from a fixed value generally introduces a transient period. (online.stat.psu.edu)

By contrast, a random walk starting at zero, with independent zero-mean increments of positive variance $\sigma^2$ , has variance $t\sigma^2$ at integer time $t$ , so it is not weakly stationary. Standard Brownian motion similarly has variance $t$ . Both illustrate the difference between stationarity of levels and stationary increments: increment distributions can depend only on interval length even when level distributions change with time. (sia.mit.edu)

Stationarity and ergodicity

Ergodicity addresses whether averages along a single sufficiently long trajectory reproduce ensemble averages. For an integrable, strictly stationary and ergodic sequence, the ergodic theorem gives

\frac1n\sum_{t=1}^{n}X_t \longrightarrow \mathbb E[X_0] \quad\text{almost surely}.

This extends the averaging interpretation associated with the law of large numbers beyond independent observations. (ocw.mit.edu)

Stationarity alone does not guarantee this convergence to the ensemble mean. Let $X_t=Z$ at every time, where $Z$ is a nondegenerate integrable random variable. The process is strictly stationary, but its sample mean always equals $Z$ , rather than generally approaching $\mathbb E[Z]$ . A time-homogeneous Markov chain provides another distinction: initializing it in a stationary distribution makes the resulting process strictly stationary, while an arbitrary initial distribution need not do so. (mit.edu)

Spectral description

Weak stationarity supports a frequency-domain description of temporal dependence. If a discrete-time autocovariance sequence is absolutely summable, its spectral density can be written, using frequency $\nu$ in cycles per observation, as

f(\nu)=\sum_{h=-\infty}^{\infty} \gamma(h)e^{-2\pi i\nu h}, \qquad -\tfrac12\leq\nu\leq\tfrac12.

The inverse relation is

\gamma(h)=\int_{-1/2}^{1/2} e^{2\pi i\nu h}f(\nu)\,d\nu.

The autocovariance and density therefore form a Fourier transform pair. Integrating the density gives the variance. More generally, a stationary covariance sequence has a spectral measure, which need not possess a density. (online.stat.psu.edu)

Use in statistical modeling

In statistics, stationarity allows observations from different times to inform common quantities such as a mean and lag-dependent covariance. Dependence still affects estimation, and reliable temporal averaging requires additional assumptions. Trends, changing variances, and changing dependence structures violate weak stationarity. Differencing can remove some forms of nonstationarity—for example, it converts a random walk into its increment sequence—but does not universally produce a stationary process. (stat153.berkeley.edu)