Analysis of Variance · aiwiki.page

Analysis of variance (ANOVA) is a family of methods in statistics that partitions variation in a quantitative response into components associated with explanatory factors and unexplained error. Its best-known application is hypothesis testing for equality of two or more population means. Despite its name, this application compares means by examining estimates of variance, rather than primarily testing whether population variances are equal. ANOVA is closely connected to experimental design and linear statistical models. (itl.nist.gov)

Historical development

Ronald Fisher and his colleagues developed ANOVA into a central method for analyzing agricultural experiments at Rothamsted Experimental Station. Fisher’s Statistical Methods for Research Workers, published in 1925, helped disseminate the approach. Its development linked the statistical analysis of experimental results with the deliberate organization of treatments, replication, and sources of experimental variation. (rothamsted.ac.uk)

The one-way model

A one-way ANOVA examines one categorical factor, whose categories are called levels. For example, measurements from products made using three manufacturing processes constitute three groups. A fixed-effects model is

Y_{ij}=\mu_i+\varepsilon_{ij},

where $Y_{ij}$ is observation $j$ in group $i$ , $\mu_i$ is that group’s population mean, and $\varepsilon_{ij}$ is a random error. Equivalently, $\mu_i=\mu+\alpha_i$ , with an identifying constraint on the group effects $\alpha_i$ . The null hypothesis is

H_0:\mu_1=\mu_2=\cdots=\mu_k.

The alternative states that not all means are equal; it does not require every pair to differ. (itl.nist.gov)

Partitioning variation

Let group $i$ contain $n_i$ observations, let $N=\sum_i n_i$ , and denote the group and overall sample means by $\bar y_i$ and $\bar y$ . The corrected total sum of squares separates exactly into between-group and within-group components:

SS_{\mathrm{total}}= \underbrace{\sum_i n_i(\bar y_i-\bar y)^2}_{SS_{\mathrm{between}}} + \underbrace{\sum_i\sum_j(y_{ij}-\bar y_i)^2}_{SS_{\mathrm{within}}}.

The between-group component measures separation of sample means, weighted by group size. The within-group component measures dispersion around each group’s own mean. Their degrees of freedom are $k-1$ and $N-k$ , respectively; total degrees of freedom are $N-1$ . Dividing each component by its degrees of freedom produces a mean square. These quantities form the conventional ANOVA table. (itl.nist.gov)

The F test

The classical test statistic is

F= \frac{MS_{\mathrm{between}}}{MS_{\mathrm{within}}} = \frac{SS_{\mathrm{between}}/(k-1)} {SS_{\mathrm{within}}/(N-k)}.

Under the null hypothesis, independent normal errors, and equal error variances, its sampling distribution is an F distribution with $k-1$ numerator and $N-k$ denominator degrees of freedom. Both mean squares then estimate the same error variance; unequal means tend to increase the numerator. The p-value is the upper-tail probability of an F statistic at least as large as the observed value under this null model. (itl.nist.gov)

A significant omnibus result does not identify which groups differ. Conversely, failure to reject the null does not establish equality of the population means. (itl.nist.gov)

Assumptions and diagnostics

The classical independent-groups F test assumes independent errors, a normal distribution of errors within groups, and equal error variances. Normality concerns deviations from group means, not necessarily the pooled observations: pooling groups with different means can produce a non-normal overall distribution. (itl.nist.gov)

Diagnostics examine residuals, including their distribution, spread, and relationship to observation order. Balanced designs have some robustness to unequal variances, but this does not remove the assumption. Welch’s ANOVA provides an alternative test of means without assuming equal group variances, using an approximate reference distribution and adjusted degrees of freedom. (itl.nist.gov)

Multifactor and related designs

Two-way ANOVA examines two factors. In a factorial design, all combinations of their levels occur. A model can contain separate main effects and a statistical interaction: an interaction means that the effect of one factor depends on the level of the other. For balanced replicated designs, total variation separates into components for both main effects, their interaction, and error. (itl.nist.gov)

Fixed-effects models concern specifically selected factor levels. In random-effects models, levels represent a random sample from a wider population, and interest includes variance components. Repeated-measures ANOVA addresses observations collected repeatedly from the same units. Its conventional univariate tests can require sphericity, meaning equal variances of pairwise differences among repeated conditions; violations can affect false-positive rates. (itl.nist.gov)

Comparisons, effect sizes, and regression

Specific comparisons can be expressed as contrasts among means. Testing many comparisons creates a multiple-testing problem. Tukey’s procedure addresses all pairwise mean comparisons, while the Bonferroni correction can control familywise error for a specified collection. Simultaneous confidence intervals describe both the direction and uncertainty of differences. (itl.nist.gov)

Statistical significance is distinct from effect size. In one-way ANOVA, $\eta^2=SS_{\mathrm{between}}/SS_{\mathrm{total}}$ describes the fraction of observed variation associated with group membership. In multifactor models, eta squared and partial eta squared use different denominators and are not interchangeable. (graphpad.com)

ANOVA can also be formulated as linear regression using indicator variables for group membership and fitting by ordinary least squares. The equal-variance one-way F test is therefore obtainable from the corresponding regression model, connecting ANOVA with the broader framework of linear-model comparison. (stat.ethz.ch)