Conditional Expectation

Conditional expectation is a fundamental concept in probability and statistics that describes the average value of a random variable given specified information. It generalizes the ordinary expected value: conditioning on an event produces a number, whereas conditioning on another random variable generally produces a random variable whose value depends on the observed information. It provides a framework for averaging within subpopulations, predicting unobserved quantities, and describing uncertainty as information accumulates. (ocw.mit.edu)

Elementary definitions

For an integrable random variable $X$ and an event $A$ with $P(A)>0$ , the conditional expectation is

E[X\mid A]=\frac{E[X\mathbf 1_A]}{P(A)},

where $\mathbf 1_A$ equals one on $A$ and zero elsewhere. For discrete $X$ , this becomes

E[X\mid A]=\sum_x xP(X=x\mid A).

Thus the average uses probabilities adjusted to the specified event rather than the original probabilities. (ocw.mit.edu)

If $Y$ is discrete, define

m(y)=E[X\mid Y=y] =\sum_x xP(X=x\mid Y=y)

for values with $P(Y=y)>0$ . The expression $E[X\mid Y]=m(Y)$ denotes a random variable, while $E[X\mid Y=y]$ denotes its numerical value at a specified observation. This distinction is essential: the conditional mean changes with $Y$ , even though the unconditional mean is constant. (ocw.mit.edu)

For real-valued $X,Y$ having a joint distribution with density $f_{X,Y}$ , the corresponding formula uses a conditional density:

m(y)=\int_{\mathbb R}x\,f_{X\mid Y}(x\mid y)\,dx, \qquad f_{X\mid Y}(x\mid y)=\frac{f_{X,Y}(x,y)}{f_Y(y)},

where $f_Y(y)>0$ and the integral exists. This is not ordinary division by $P(Y=y)$ , which is zero for continuously distributed $Y$ . (live.ocw.mit.edu)

Measure-theoretic definition

The general definition belongs to measure theory. Let $(\Omega,\mathcal F,P)$ be a probability space, let $E|X|<\infty$ , and let $\mathcal G\subseteq\mathcal F$ be a sigma-algebra representing the available information. A conditional expectation $Z=E[X\mid\mathcal G]$ is an integrable, $\mathcal G$ -measurable random variable satisfying

\int_A Z\,dP=\int_A X\,dP \quad\text{for every }A\in\mathcal G.

These integral identities preserve the averages detectable using that information. Measurability ensures that $Z$ does not depend on information outside $\mathcal G$ . (math.ucdavis.edu)

Existence follows from the Radon–Nikodym theorem: for nonnegative integrable $X$ , the measure $A\mapsto E[X\mathbf 1_A]$ on $\mathcal G$ has a Radon–Nikodym derivative relative to $P$ . General integrable $X$ is handled through its positive and negative parts. Conditional expectations are unique almost surely, meaning that versions may differ on sets of probability zero. (math.ucdavis.edu)

Conditioning on $Y$ means conditioning on its generated sigma-algebra $\sigma(Y)$ . For real-valued $Y$ , a version can be written $m(Y)$ , with $m$ measurable. Its values at individual observations of probability zero need not be uniquely determined; additional regularity may select a convenient version. (math.ucdavis.edu)

Principal properties

For integrable variables, conditional expectation is linear and order-preserving. It leaves $\mathcal G$ -measurable variables unchanged. If $X$ is independent of $\mathcal G$ , then

E[X\mid\mathcal G]=E[X] \quad\text{almost surely}.

Independence is sufficient for this identity, but the identity alone does not establish independence. (math.ucdavis.edu)

The “taking out what is known” rule states that, for bounded $\mathcal G$ -measurable $W$ ,

E[WX\mid\mathcal G]=W E[X\mid\mathcal G].

The tower property states that if $\mathcal H\subseteq\mathcal G$ , then

E[E[X\mid\mathcal G]\mid\mathcal H] =E[X\mid\mathcal H].

Averaging a finer-information estimate using coarser information therefore produces the coarser-information estimate directly. Its special case, the law of total expectation, is

E[E[X\mid\mathcal G]]=E[X].

These identities hold almost surely where appropriate. (people.math.wisc.edu)

Conditional Jensen’s inequality gives

\varphi(E[X\mid\mathcal G]) \leq E[\varphi(X)\mid\mathcal G]

for a convex function $\varphi$ , assuming the relevant expectations exist. In particular, conditioning cannot increase the expected absolute magnitude:

E|E[X\mid\mathcal G]|\leq E|X|.

(samuel-drapeau.info)

Prediction and geometric interpretation

When $E[X^2]<\infty$ , conditional expectation is the orthogonal projection of $X$ onto the closed subspace of $\mathcal G$ -measurable variables in the Hilbert space $L^2$ . Its residual satisfies

E[(X-E[X\mid\mathcal G])W]=0

for every square-integrable, $\mathcal G$ -measurable $W$ . Orthogonality here uses the inner product $E[UV]$ . (samuel-drapeau.info)

Consequently, $E[X\mid\mathcal G]$ minimizes mean squared error among square-integrable predictions based on $\mathcal G$ . For observed predictors $Y$ , the minimizing prediction is $m(Y)=E[X\mid Y]$ . Unlike linear regression, this optimization does not restrict predictions to linear functions; the optimal conditional mean may be nonlinear. (ocw.mit.edu)

Variance decomposition and stochastic processes

For square-integrable $X$ , define conditional variance by

\operatorname{Var}(X\mid\mathcal G) =E[(X-E[X\mid\mathcal G])^2\mid\mathcal G].

The law of total variance then gives

\operatorname{Var}(X) =E[\operatorname{Var}(X\mid\mathcal G)] +\operatorname{Var}(E[X\mid\mathcal G]).

It separates average variability remaining within information-defined groups from variability between their conditional means. (ocw.mit.edu)

In a stochastic process, an increasing filtration $(\mathcal F_t)$ represents information available over time. An integrable adapted process $M_t$ is a martingale when

E[M_t\mid\mathcal F_s]=M_s,\qquad s\leq t.

For an integrable terminal quantity $Z$ , the process $M_t=E[Z\mid\mathcal F_t]$ satisfies this identity by the tower property, expressing successive estimates of the same quantity as information grows. (people.math.wisc.edu)