Normal Distribution · aiwiki.page

The normal distribution, also called the Gaussian distribution, is a continuous probability distribution whose density forms a symmetric, bell-shaped curve. It describes a random variable through two parameters: its mean, which determines the center, and its variance, which determines the spread. Normal distributions are fundamental to statistics, both as models for observations and as approximations to the distributions of sample averages and other statistics. (itl.nist.gov)

Definition and parameters

A random variable $X$ has a normal distribution with mean $\mu$ and variance $\sigma^2>0$ , written $X\sim N(\mu,\sigma^2)$ , if its probability density function is

f(x)=\frac{1}{\sigma\sqrt{2\pi}} \exp\left[-\frac{(x-\mu)^2}{2\sigma^2}\right], \qquad -\infty<x<\infty.

The parameter $\mu$ can be any real number, while $\sigma>0$ is the standard deviation. Some sources use $N(\mu,\sigma)$ instead, so the meaning of the second parameter must be checked. (itl.nist.gov)

The expected value is $\mu$ , and the variance is $\sigma^2$ . The mean, median, and mode coincide. Increasing $\mu$ shifts the curve without changing its shape; increasing $\sigma$ makes it wider and lowers its peak. Its skewness is zero and its kurtosis is three, giving zero excess kurtosis. (itl.nist.gov)

The total area beneath the density is one. The probability of an interval is the integral of the density over that interval, rather than the density at an individual point. Consequently, a particular exact value has probability zero, although every interval of positive length has positive probability. (itl.nist.gov)

Standardization and probabilities

The standard normal distribution has mean zero and variance one. If $X\sim N(\mu,\sigma^2)$ , then

Z=\frac{X-\mu}{\sigma}\sim N(0,1).

An observed standardized value is commonly called a z-score: it expresses distance from the mean in standard-deviation units. Standardization allows probabilities for every normal distribution to be calculated using the same reference distribution. (itl.nist.gov)

The standard normal cumulative distribution function is

\Phi(z)=\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{z}e^{-t^2/2}\,dt.

Thus,

P(a\le X\le b)= \Phi\left(\frac{b-\mu}{\sigma}\right) -\Phi\left(\frac{a-\mu}{\sigma}\right).

This integral has no elementary antiderivative; numerical methods and probability tables provide its values. Symmetry gives $\Phi(-z)=1-\Phi(z)$ . (itl.nist.gov)

Approximately 68.27% of probability lies within one standard deviation of the mean, 95.45% within two, and 99.73% within three. These proportions are often rounded to the 68–95–99.7 rule. They describe the theoretical distribution, not guaranteed percentages in every finite sample. (itl.nist.gov)

Sums and the central limit theorem

Normal distributions are closed under sums of independent normal variables. If $X\sim N(\mu_X,\sigma_X^2)$ and $Y\sim N(\mu_Y,\sigma_Y^2)$ are independent, then

X+Y\sim N(\mu_X+\mu_Y,\sigma_X^2+\sigma_Y^2).

More generally, a linear combination of independent normal variables is normal, provided its variance is positive; a zero-variance combination is constant. (new.statlect.com)

The central limit theorem explains why normal approximations arise even from nonnormal populations. For independent, identically distributed variables with finite mean $\mu$ and finite positive variance $\sigma^2$ , the standardized sample mean converges in distribution to $N(0,1)$ :

\frac{\sqrt n(\overline X_n-\mu)}{\sigma} \xrightarrow{d}N(0,1).

Accordingly, sufficiently large sample means can often be approximated by $N(\mu,\sigma^2/n)$ . The theorem concerns averages, not a transformation of the original observations into normally distributed data. The required sample size depends on the underlying distribution; there is no universal threshold guaranteeing a satisfactory approximation. (mathworks.com)

Estimation and statistical inference

For independent observations $x_1,\ldots,x_n$ from a normal population, maximum likelihood estimation gives

\widehat\mu=\overline x,\qquad \widehat{\sigma}^{\,2}= \frac1n\sum_{i=1}^{n}(x_i-\overline x)^2.

The variance estimate uses denominator $n$ and is biased downward. Replacing $n$ with $n-1$ gives the usual unbiased sample variance. The distinction separates likelihood maximization from unbiased estimation. (mail.statlect.com)

Normal distributions underpin many confidence intervals and procedures in statistical hypothesis testing. When independent observations are normal, their sample mean is exactly normal at every sample size. If population variance is unknown, inference about the mean commonly uses Student’s t-distribution, accounting for uncertainty in estimating the standard deviation. (itl.nist.gov)

In linear regression, normality is commonly an assumption about errors conditional on predictors, not about every predictor or the response considered separately. Normal probability plots of fitted residuals provide a graphical diagnostic: approximate linearity supports a normal error model, while systematic departures indicate a mismatch. (itl.nist.gov)

Multivariate extension and historical development

The multivariate normal distribution extends the model to random vectors. A mean vector specifies the center, and a covariance matrix specifies variances and dependence. Every linear combination of a jointly normal vector’s components is normal or constant. However, individually normal components do not necessarily form a jointly normal vector. (itl.nist.gov)

Historically, Abraham de Moivre derived a normal approximation to the binomial distribution in 1733. Pierre-Simon Laplace developed its role in probability and error theory. The name Gaussian reflects the influential treatment of observational errors by Carl Friedrich Gauss, notably in his 1809 work on celestial motion. (mathshistory.st-andrews.ac.uk)