Analysis of variance (ANOVA)

In short

Analysis of variance (ANOVA) tests whether the means of three or more populations are equal by splitting the total variability in the data into a part explained by the groups (treatments) and a part left as random error, then comparing the two with an F ratio. It avoids the inflated Type I error that repeated t-tests would create. ANOVA is a core unit of the Inference paper in every B.Sc Statistics CBCS syllabus and appears in GATE Statistics (ST), ISS, CSIR-UGC NET, ICAR-ASRB and university practicals. In industry it is the standard tool for A/B and multi-arm testing, Six Sigma quality projects, agricultural and pharmaceutical trials, and analytics interviews at Indian firms.

Explained

Idea, model and assumptions

• Core logic: if all group means are equal, variation between group means should be about the same size as variation within groups; a between/within variance ratio much larger than 1 contradicts equality.
• One-way fixed-effects model: y_ij = μ + α_i + ε_ij, i = 1…k treatments, j = 1…n_i observations, with Σ n_i α_i = 0 (or Σ α_i = 0 for equal n) and ε_ij ~ N(0, σ²) independent.
• Hypotheses: H₀: μ₁ = μ₂ = … = μ_k (equivalently all α_i = 0) against H₁: at least one mean differs. The alternative is never "all means differ".
• Four assumptions: (1) independent random samples, (2) normality of errors, (3) homoscedasticity — equal variances across groups, (4) additive effects (treatment effects add to the overall mean).
• Checks: normality by Shapiro–Wilk or a normal Q–Q plot of residuals; equality of variances by Bartlett's test (needs normality), Levene's or Brown–Forsythe test (robust); independence by design (randomisation) and a residuals-versus-order plot.
• Robustness: ANOVA is fairly robust to mild non-normality (central limit effect) and to unequal variances when sample sizes are equal and the largest-to-smallest variance ratio is under about 3; it is not robust to dependence.
• Remedies when assumptions fail: variance-stabilising transformations — √y for counts (Poisson), arcsin√p for proportions (binomial), log y when standard deviation rises with the mean; or use Welch's ANOVA / Kruskal–Wallis.

Partition of variability (the algebra)

• Identity: (y_ij − ȳ) = (ȳ_i − ȳ) + (y_ij − ȳ_i). Squaring and summing kills the cross-product (because Σ_j (y_ij − ȳ_i) = 0), giving SST = SSTr + SSE.
• Degrees of freedom add too: (N − 1) = (k − 1) + (N − k), where N = Σ n_i.
• Mean squares: MSTr = SSTr/(k−1), MSE = SSE/(N−k). MSE is always an unbiased estimator of σ²; MSTr is unbiased for σ² only under H₀.
• Expectations: E(MSE) = σ², E(MSTr) = σ² + Σ n_i α_i²/(k−1). So a large F signals non-zero treatment effects.
• Distribution theory: under H₀ and normality, SSTr/σ² ~ χ²_(k−1) and SSE/σ² ~ χ²_(N−k), independent (Cochran's theorem), so F = MSTr/MSE ~ F(k−1, N−k).
• Test: right-tailed only. Reject H₀ if F_cal > F_α(k−1, N−k), usually at α = 0.05. Equivalently if p-value < α.
• k = 2 special case: F = t², and the one-way ANOVA is exactly the two-sample pooled t-test (two-sided).

One-way ANOVA: step-by-step

1. Compute the grand total G = ΣΣ y_ij, N, and the correction factor CF = G²/N.
2. Raw sum of squares = ΣΣ y_ij². Then SST = ΣΣ y_ij² − CF.
3. SSTr = Σ (T_i²/n_i) − CF, where T_i is the total of treatment i.
4. SSE = SST − SSTr (obtained by subtraction, never computed first).
5. Fill the ANOVA table, compute F = MSTr/MSE, compare with the table value, and state the conclusion in words.
6. If H₀ is rejected, run a post-hoc test to find which pairs differ.

Two-way ANOVA and higher layouts

• Two-way without replication (randomised block design): y_ij = μ + α_i + β_j + ε_ij. SST = SSTr + SSBl + SSE, with df (k−1), (b−1) and (k−1)(b−1). Two F ratios are computed, one for treatments and one for blocks.
• Two-way with replication: y_ijk = μ + α_i + β_j + (αβ)_ij + ε_ijk. SST = SSA + SSB + SSAB + SSE, df (a−1), (b−1), (a−1)(b−1), ab(r−1). Interaction must be tested first — if interaction is significant, main effects are interpreted only within levels of the other factor.
• Interaction meaning: the effect of one factor depends on the level of the other; non-parallel lines in an interaction plot. With one observation per cell, interaction is inestimable and is confounded with error (Tukey's one-degree-of-freedom test is the partial remedy).
• Fixed, random and mixed models: fixed (Model I) — levels chosen deliberately, inference about those levels; random (Model II) — levels a random sample, inference about variance components σ²_α; mixed (Model III). SS computations are identical; expected mean squares and hence the denominator of F differ, and only random models estimate variance components.
• ANCOVA: combines ANOVA with regression — adjusts treatment means for a continuous covariate (e.g. initial weight), reducing error variance.
• MANOVA: several response variables simultaneously, using Wilks' lambda or Pillai's trace.
• Unequal sample sizes (unbalanced data): one-way formulae handle it directly; in two-way layouts orthogonality is lost and SS depend on the order of fitting (Type I vs Type III SS) — a standard interview trap.

Post-hoc and multiple comparisons

• Why needed: a significant F says "not all equal" but not which; m pairwise t-tests at α each give a family-wise error of about 1 − (1−α)^m (for k = 5, m = 10, that is ~40%).
• Fisher's LSD: LSD = t_(α/2, N−k) × √(MSE(1/n_i + 1/n_j)); most powerful, least protective — use only after a significant F.
• Tukey's HSD: uses the studentised range q_(α, k, N−k); controls family-wise error for all pairwise comparisons; the standard choice for equal n.
• Bonferroni: test each pair at α/m; simple, conservative, good for few planned comparisons.
• Scheffé: valid for all linear contrasts, most conservative; Dunnett for comparing every treatment against one control only.
• Contrasts: a linear combination Σ c_i μ_i with Σ c_i = 0; orthogonal contrasts (Σ c_i c_i' = 0) split SSTr into independent single-df components.
• Effect size: η² = SSTr/SST (proportion of variance explained); ω² and ε² are less biased estimators, while partial η² is used in multifactor designs (it is not a bias correction); always report alongside p.

Alternatives, limitations and Indian practice

• Welch's ANOVA: unequal variances; adjusts df, preferred over classical F when Levene's test is significant.
• Kruskal–Wallis H test: the non-parametric rank-based alternative for k independent samples; Friedman test for a randomised block layout.
• Repeated-measures ANOVA: the same subjects measured under all conditions; needs sphericity (Mauchly's test, Greenhouse–Geisser correction).
• Limitations: detects a difference but not direction or magnitude without follow-up; sensitive to outliers; needs balanced, randomised data for clean interpretation; cannot establish causation from observational groups.
• Software: R (aov, anova, TukeyHSD), Python (statsmodels, scipy.stats.f_oneway), SPSS, Minitab, Excel's Data Analysis ToolPak, and SAS. ICAR's web portal and IASRI's SPAD/OPSTAT tools are used routinely for Indian agricultural trials.
• Indian standards context: ANOVA underpins gauge repeatability and reproducibility (Gauge R&R) and measurement-system analysis in quality work, and inter-laboratory precision studies following the ISO 5725 series (adopted as the corresponding IS/ISO standards); BIS sampling-inspection and precision standards are revised periodically, so cite the current edition.

Key points

• ANOVA — partitions total variation into between-treatment and within-treatment parts to test equality of k ≥ 3 means.
• Why not many t-tests — family-wise Type I error inflates rapidly with the number of pairs.
• SST = SSTr + SSE — sums of squares and degrees of freedom both partition additively.
• MSE — always unbiased for σ²; the yardstick against which treatment variation is judged.
• F ratio — MSTr/MSE, right-tailed test, F(k−1, N−k) under H₀.
• E(MSTr) = σ² + Σn_iα_i²/(k−1) — the reason F exceeds 1 when treatments differ.
• Correction factor — CF = G²/N, subtracted from every raw sum of squares.
• Assumptions — independence, normality of errors, equal variances, additivity.
• Bartlett / Levene — tests of homoscedasticity; Levene is robust to non-normality.
• Transformations — √y for counts, arcsin√p for proportions, log y for multiplicative data.
• Two-way ANOVA — removes block (nuisance) variation, raising precision; gives separate F tests per factor.
• Interaction — needs replication; test it before interpreting main effects.
• Fixed vs random model — same SS, different expected mean squares and inference scope.
• Post-hoc tests — LSD (liberal) < Tukey HSD (standard) < Bonferroni < Scheffé (most conservative); Dunnett versus a control.
• η² = SSTr/SST — effect size; statistical significance is not practical importance.
• k = 2 — ANOVA reduces to the pooled two-sample t-test, with F = t².
• Alternatives — Kruskal–Wallis (one-way) and Friedman (blocks) are non-parametric; Welch's ANOVA is parametric, for unequal variances.
• Unbalanced two-way data — non-orthogonal; Type I and Type III sums of squares differ.

Formulas and examples

Example 1 — One-way ANOVA, three machines (output in units/shift). Machine A: 20, 22, 24, 26 (T₁ = 92); B: 28, 30, 32, 30 (T₂ = 120); C: 18, 20, 22, 24 (T₃ = 84). n = 4 each, k = 3, N = 12.

• G = 92 + 120 + 84 = 296; CF = 296²/12 = 87,616/12 = 7,301.333.
• Σy² = (400+484+576+676) + (784+900+1,024+900) + (324+400+484+576) = 2,136 + 3,608 + 1,784 = 7,528.
• SST = 7,528 − 7,301.333 = 226.667.
• SSTr = (92² + 120² + 84²)/4 − CF = (8,464 + 14,400 + 7,056)/4 − 7,301.333 = 29,920/4 − 7,301.333 = 7,480 − 7,301.333 = 178.667.
• SSE = 226.667 − 178.667 = 48.000.

• F_cal = 89.333/5.333 = 16.75; F_0.05(2, 9) = 4.26. Since 16.75 > 4.26, reject H₀ — machine mean outputs differ significantly.
• η² = 178.667/226.667 = 0.788, i.e. machines explain about 79% of the variation.
• Post-hoc (Tukey, q_0.05(3,9) = 3.95): HSD = 3.95 √(5.333/4) = 3.95 × 1.1547 = 4.56. Means are 23, 30, 21. |23−30| = 7 > 4.56 (significant); |30−21| = 9 > 4.56 (significant); |23−21| = 2 < 4.56 (not significant). So B differs from both A and C; A and C do not differ.

Example 2 — Two-way ANOVA without replication (RBD): 3 fertilisers × 4 plots, yield in quintals.

• N = 12, CF = 120²/12 = 1,200. Σy² = 64+144+100+100+196+144+36+100+64+64+144+100 = 1,256. SST = 56.
• SSTr = (32² + 48² + 40²)/4 − 1,200 = (1,024 + 2,304 + 1,600)/4 − 1,200 = 4,928/4 − 1,200 = 1,232 − 1,200 = 32.
• SSBl = (30² + 36² + 24² + 30²)/3 − 1,200 = (900 + 1,296 + 576 + 900)/3 − 1,200 = 3,672/3 − 1,200 = 1,224 − 1,200 = 24.
• SSE = 56 − 32 − 24 = 0 (this constructed table is perfectly additive).

• With SSE = 0 the data are exactly additive, so F is undefined (division by zero) and no valid F test exists — this contrived table cannot be used to declare significance; in real data SSE > 0, and one would compare F against F_0.05(2,6) = 5.14 for fertilisers and F_0.05(3,6) = 4.76 for blocks.

Example 3 — Reading an ANOVA table backwards (common exam format). Given: k = 4 treatments, n = 6 per treatment, SSTr = 150, SST = 450.

• N = 24; SSE = 450 − 150 = 300; df: treatments 3, error 20, total 23.
• MSTr = 150/3 = 50; MSE = 300/20 = 15; F = 50/15 = 3.33.
• F_0.05(3, 20) = 3.10 → reject H₀ at 5%. F_0.01(3, 20) = 4.94 → do not reject at 1%. So the result is significant at 5% but not at 1%.
• LSD at 5% = t_0.025,20 √(15(1/6 + 1/6)) = 2.086 × √5 = 2.086 × 2.2361 = 4.66 quintals.

Example 4 — F = t² check. Two groups of 5: A = 4, 6, 8, 10, 12 (mean 8); B = 9, 11, 13, 15, 17 (mean 13). Each group's Σ(y−ȳ)² = 40, so SSE = 80, MSE = 80/8 = 10. SSTr = 5(8−10.5)² + 5(13−10.5)² = 31.25 + 31.25 = 62.5. F = 62.5/10 = 6.25. Pooled t = (13−8)/√(10(1/5+1/5)) = 5/2 = 2.5, and 2.5² = 6.25 — confirming F = t².

# One-way ANOVA with post-hoc test in R
out  <- c(20,22,24,26, 28,30,32,30, 18,20,22,24)
mach <- factor(rep(c("A","B","C"), each = 4))
fit  <- aov(out ~ mach)
summary(fit)            # F = 16.75 on 2 and 9 df
bartlett.test(out ~ mach)   # equality of variances
TukeyHSD(fit)               # which pairs differ

Where it is used

• Agriculture and food research: ICAR institutes, state agricultural universities and IASRI analyse variety, fertiliser and irrigation trials almost entirely by one-way, RBD and split-plot ANOVA.
• Pharmaceuticals and clinical research: Sun Pharma, Cipla, Dr Reddy's and CROs use ANOVA and ANCOVA for dissolution profiles, stability batches, bioequivalence and multi-arm trials under CDSCO and ICH guidelines.
• Manufacturing quality and Six Sigma: Tata Steel, Maruti Suzuki, Bosch and Ashok Leyland use ANOVA in Measure/Analyse phases, Gauge R&R, and Taguchi designs to compare machines, shifts, suppliers and operators.
• Analytics, product and marketing roles: data scientists at Flipkart, Swiggy, Zomato, PhonePe and banks use ANOVA for multi-variant A/B tests on pricing, UI and campaigns; the F test and post-hoc correction are standard interview questions.
• Social science and survey research: NSSO/MoSPI analyses, NCAER and IIM research comparing mean income, expenditure or scores across states, strata or education groups.
• Psychometrics and education: comparing mean marks across teaching methods, centres or coaching batches; repeated-measures ANOVA in learning studies.

Interview questions

• Q: Why use ANOVA instead of several t-tests? A: Multiple pairwise t-tests inflate the family-wise Type I error (about 40% for five groups at α = 0.05); ANOVA gives one test at the stated α and pools all data for a better error estimate.
• Q: What exactly does a significant F tell you? A: Only that the group means are not all equal — not which ones differ, by how much, or in which direction; a post-hoc test is needed for that.
• Q: Why is the ANOVA F test one-tailed? A: Treatment effects can only inflate MSTr relative to MSE, so evidence against H₀ lies in the upper tail; an F well below 1 indicates no treatment effect, not the reverse.
• Q: What are the assumptions, and which matters most? A: Independence, normality of errors, equal variances and additivity; independence matters most, since ANOVA is reasonably robust to mild departures from the other two with balanced data.
• Q: How do you handle unequal variances? A: Use Welch's ANOVA, a variance-stabilising transformation, or the non-parametric Kruskal–Wallis test.
• Q: What is the difference between a fixed-effects and a random-effects model? A: Fixed effects test specific chosen levels; random effects treat levels as a sample from a population and estimate the variance component σ²_α — the sums of squares are the same but the expected mean squares and F denominators can differ.
• Q: What does interaction mean, and can you test it with one observation per cell? A: It means a factor's effect depends on the other factor's level; with one observation per cell there are no degrees of freedom left, so interaction is confounded with error and cannot be tested without replication.
• Q: Why is SSE computed by subtraction? A: Because the partition SST = SSTr + SSE is exact, subtraction is faster and avoids rounding and arithmetic errors in computing deviations within every group.
• Q: When would you choose Tukey over Bonferroni? A: Tukey for all pairwise comparisons with equal group sizes (tighter and exact for that family); Bonferroni for a small number of pre-planned comparisons.
• Q: How is ANOVA related to regression? A: ANOVA is a linear model with dummy-coded categorical predictors; its F test is the overall significance test of that regression, and η² corresponds to R².
• Q: Your F is significant but all pairwise tests are not — what happened? A: The F test is sensitive to overall spread and to contrasts other than simple pairs; with borderline significance, a conservative post-hoc test can fail to flag any single pair, so examine contrasts and effect sizes.
• Q: How does blocking improve an experiment? A: It removes an identified nuisance source of variation from the error term, shrinking MSE and raising the power of the treatment F test — at the cost of error degrees of freedom.

Common mistakes

• Writing H₁ as "all means are different" instead of "at least one mean differs".
• Using a two-tailed critical value or the wrong order of degrees of freedom — F_α(k−1, N−k), numerator df first.
• Forgetting the correction factor G²/N, or applying it to only one sum of squares.
• Jumping straight to pairwise t-tests after a significant F without any multiple-comparison control.
• Computing MS as SS divided by the number of groups or observations rather than by the correct degrees of freedom.
• Treating a significant F as proof of a large or practically important difference, and never reporting an effect size.
• Interpreting main effects in a two-way design while ignoring a significant interaction.
• Running ANOVA on paired or repeated measurements as if the samples were independent, instead of using repeated-measures ANOVA or the Friedman test.

Related topics

Hypothesis testing · Design of experiments · Non-parametric tests · Estimation theory · Regression analysis · Sampling methods and survey design