Population and quantitative genetics

In short

Population genetics describes how allele and genotype frequencies change in populations under mutation, migration, selection, genetic drift and non-random mating; quantitative genetics extends this to continuously varying traits controlled by many loci plus environment. Together they supply the theory behind GWAS, polygenic risk scores, plant and animal breeding, conservation of endangered species and forensic DNA statistics. The topic is examined in UGC-CBCS B.Sc. Genetics/Biotechnology papers, CSIR-UGC NET Life Sciences (Unit on evolution and population genetics), GATE BT/XL, ICAR-NET and JGEEBILS/TIFR entrances. In India it is applied at NIPGR, ICAR institutes (IARI, NBPGR, NDRI), CSIR-CCMB and IGIB (Indian Genome Variation, GenomeIndia), CDFD (DNA fingerprinting), seed companies, and bioinformatics service firms building variant-analysis pipelines.

Explained

Basic concepts and allele frequency estimation

• Population: a group of interbreeding individuals sharing a gene pool. Deme = a local interbreeding unit; Mendelian population = members sharing a common gene pool.
• Allele (gene) frequency p = proportion of a given allele among all alleles at a locus; genotype frequency = proportion of individuals of a genotype. Both sum to 1.
• For a diallelic autosomal locus with counts of AA, Aa, aa: p = (2·AA + Aa)/(2N), q = (2·aa + Aa)/(2N) = 1 − p.
• Codominant/SNP markers allow direct counting; dominance forces estimation by assuming Hardy-Weinberg: q = √(freq of recessives).
• X-linked loci: males carry one allele, females two, so p = (A alleles counted in females + A alleles counted in males)/(2N_f + N_m).
• Polymorphism: a locus is polymorphic if the commonest allele is below 0.99 (or MAF ≥ 1%). Heterozygosity (H) and PIC measure marker informativeness.

Hardy-Weinberg equilibrium (HWE)

• Statement: in a large, randomly mating population without selection, mutation, migration or drift, allele frequencies stay constant and genotype frequencies become p² : 2pq : q² after one generation of random mating.
• Derivation outline: random union of gametes → gamete array (pA + qa) combined with itself gives (p + q)² = p² + 2pq + q².
• Key properties: equilibrium is reached in one generation for autosomal loci; it is reached only asymptotically for X-linked loci (female and male frequencies oscillate, converging as the difference halves each generation) and for two-locus gamete frequencies (LD decays).
• Maximum heterozygote frequency is 0.5 at p = q = 0.5. Rare recessive alleles are overwhelmingly hidden in heterozygotes: ratio 2pq/q² = 2p/q.
• Uses: estimating carrier frequency of recessive disease alleles, checking genotyping quality (HWE deviation in controls signals genotyping error), forensic match probability.
• Testing: chi-square goodness-of-fit with 1 degree of freedom for a diallelic locus (k alleles → df = k(k−1)/2), or an exact test for small samples. Deviations arise from inbreeding, population structure (Wahlund effect), selection, non-random mating or technical error.

Forces that change allele frequencies

• Mutation: the ultimate source of variation; rate per locus per generation ~10⁻⁵–10⁻⁶ (human per-base ~1.25 × 10⁻⁸). Alone it changes frequencies extremely slowly; recurrent forward/back mutation gives equilibrium p̂ = ν/(µ + ν).
• Selection: differential survival and reproduction. Fitness (w) = relative contribution of a genotype; selection coefficient s = 1 − w. Types: directional, stabilising (favours intermediates, reduces variance), disruptive/diversifying, balancing.
• Modes by dominance: selection against a recessive is slow when the allele is rare (because it hides in heterozygotes); selection against a dominant is fast; heterozygote advantage (overdominance) gives a stable polymorphism — classic Indian example: HbS sickle-cell and HbE alleles and β-thalassaemia trait persisting in malarial belts of central and eastern India; G6PD deficiency likewise.
• Migration (gene flow): one-way island model p₁' = (1 − m)p₁ + m·p_m; homogenises populations and opposes differentiation. Hybrid zones and admixture result.
• Genetic drift: random change in frequencies due to finite sampling of gametes; variance of change = pq/2N per generation. Effects: loss of heterozygosity at rate 1/(2N_e) per generation, random fixation or loss, founder effect (new population from few individuals) and bottleneck (drastic size reduction). Indian examples of founder/endogamy effects: high frequency of specific founder mutations within endogamous groups documented by the Indian Genome Variation Consortium and CCMB studies.
• Non-random mating: assortative mating (like with like — raises homozygosity for the trait loci), disassortative, and inbreeding (mating of relatives).
• Mutation-selection balance: for a deleterious recessive q̂ = √(µ/s); for a dominant q̂ ≈ µ/s. Explains persistence of rare disease alleles.

Inbreeding, effective population size and structure

• Inbreeding coefficient F = probability that the two alleles at a locus in an individual are identical by descent (IBD). Genotype frequencies become p² + Fpq, 2pq(1 − F), q² + Fpq — heterozygosity falls by a factor (1 − F).
• Computed from pedigrees by the path method: F = Σ (½)^n₁+n₂+1(1 + F_A). First-cousin marriage gives F = 1/16 = 0.0625; uncle-niece 1/8; second cousins 1/64.
• Consanguinity in India is high in several southern and some Muslim communities (commonly reported around 20–30% of marriages in parts of Tamil Nadu, Andhra Pradesh, Telangana and Karnataka, far lower in the north) — the basis of elevated recessive disease and the reason genetic counselling units compute F.
• Inbreeding depression: loss of fitness from exposure of deleterious recessives and loss of heterozygote advantage; the converse in crops/livestock is heterosis (hybrid vigour), exploited in hybrid maize, rice, cotton, bajra and sunflower.
• Effective population size N_e: size of an ideal population showing the same drift as the real one. Unequal sex ratio: N_e = 4N_mN_f/(N_m + N_f); variable size across generations: harmonic mean; variance in family size and overlapping generations reduce N_e. N_e is usually far below census size — critical in conservation (Project Tiger, Asiatic lion at Gir, great Indian bustard captive breeding under the National Wildlife Action Plan).
• Wahlund effect: pooling subpopulations with different frequencies shows a heterozygote deficit even if each subpopulation is in HWE.
• Wright's F-statistics: F_IT = total inbreeding, F_IS = within-subpopulation (relative to its own HWE), F_ST = among-subpopulation differentiation. Relation (1 − F_IT) = (1 − F_IS)(1 − F_ST). F_ST = (H_T − H_S)/H_T. Guideline: <0.05 little, 0.05–0.15 moderate, 0.15–0.25 great differentiation. Island-model expectation F_ST ≈ 1/(1 + 4N_em). Indian human F_ST between most caste/linguistic groups is low (roughly 0.01–0.02) but higher for isolated tribal groups.
• Linkage disequilibrium (LD): non-random association of alleles at two loci. D = p_AB − p_A p_B; D' = D/D_max; r² = D²/(p_A q_A p_B q_B). Decays as D_t = D₀(1 − c)^t. Created by admixture, drift, selection and new mutation; the basis of association mapping and haplotype blocks.
• Coalescent and neutral theory: Kimura's neutral theory — most molecular variants are neutral and fixed by drift; expected heterozygosity under mutation-drift balance H = 4N_eµ/(1 + 4N_eµ), θ = 4N_eµ. Tests of neutrality: Tajima's D, Fu and Li, Fay-Wu H, McDonald-Kreitman, extended haplotype homozygosity (EHH/iHS) for recent selective sweeps.

Quantitative genetics: variance and heritability

• Quantitative (metric) traits show continuous variation: height, yield, milk output, blood pressure, polygenic disease liability. Explained by the multiple-factor hypothesis (Nilsson-Ehle's wheat kernel colour): many loci of small additive effect + environment → normal distribution.
• Threshold traits: discontinuous expression over an underlying continuous liability (cleft lip, type 2 diabetes).
• Partition: V_P = V_G + V_E + V_GE; and V_G = V_A (additive) + V_D (dominance) + V_I (epistatic/interaction).
• Broad-sense heritability H² = V_G/V_P; narrow-sense heritability h² = V_A/V_P — only h² predicts response to selection because only additive effects transmit reliably.
• Heritability is a property of a population in an environment, not of an individual, and says nothing about the cause of differences between groups.
• Estimation: parent-offspring regression (single-parent regression slope b = ½h²; mid-parent regression slope = h²), sib correlations, twin studies (H² ≈ 2(r_MZ − r_DZ)), ANOVA of half-sib families, realised heritability from a selection experiment, and modern GREML/LDSC SNP-heritability from genome-wide data.
• Repeatability = fraction of variance due to permanent differences between individuals (upper bound for heritability). Genetic correlation r_A between traits drives correlated response and trade-offs (yield vs protein content).
• Breeding value (A) = sum of average additive effects of an individual's alleles; BLUP and genomic selection (GBLUP, Bayesian methods) predict it from pedigree and markers — used in NDRI/ICAR cattle and buffalo progeny-testing programmes.
• Response to selection R = h²S (breeder's equation); with selection intensity, R = i·h·σ_A per generation and per year R/L. Genetic gain methods: mass selection, progeny testing, index selection (Smith-Hazel), reciprocal recurrent selection.
• Selection limits/plateaus, loss of V_A, drift in small breeding populations and genotype × environment interaction (multilocation trials, AMMI/GGE analysis) limit long-term gain.
• QTL mapping: cross two divergent lines → F₂/RIL/BC/NIL/DH populations → genotype markers → single-marker ANOVA, interval mapping (Lander-Botstein), composite interval mapping; LOD ≥ 3 conventional threshold, permutation thresholds preferred. MAS (marker-assisted selection) moved Sub1 (submergence tolerance) into Swarna-Sub1 and bacterial blight Xa genes into Improved Samba Mahsuri — Indian flagship examples.
• GWAS: case-control or quantitative association over millions of SNPs, genome-wide significance p < 5 × 10⁻⁸ (Bonferroni for ~1 million independent tests), correction for structure by principal components or mixed models (EMMAX, GEMMA, BOLT-LMM, SAIGE); Manhattan and QQ plots; polygenic risk scores sum effect-weighted alleles but transfer poorly across ancestries — the rationale for GenomeIndia and IndiGen reference panels.

Tools, data and Indian practice

• Software: PLINK (HWE, LD, association), VCFtools/bcftools, GCTA, ADMIXTURE/STRUCTURE, EIGENSOFT, Arlequin, GenAlEx, PopART, SNeP; R packages adegenet, hierfstat, qtl/qtl2, rrBLUP, sommer, pegas; MEGA for phylogenetics; TASSEL for plants.
• Public data: 1000 Genomes, gnomAD, HapMap legacy, dbSNP, GWAS Catalog; Indian resources IndiGen (CSIR), Indian Genome Variation database, GenomeIndia (DBT), ICMR MedGenome-type clinical cohorts, NBPGR germplasm and ICAR crop databases.
• Regulation and ethics: ICMR National Ethical Guidelines for Biomedical and Health Research Involving Human Participants (2017) and the ICMR-DBT National Guidelines for Gene Therapy Product Development; the DNA-based technology (use and application) regulation Bill has not been enacted, so DNA profiling in India is governed by case law and laboratory accreditation (NABL ISO/IEC 17025) rather than a dedicated statute; Biological Diversity Act, 2002 as amended by the Biological Diversity (Amendment) Act, 2023 governs access to Indian biological resources and benefit sharing; PPV&FR Act, 2001 governs plant variety registration and farmers' rights; the PCPNDT Act, 1994 restricts sex-selective use of genetic/prenatal diagnostic tests. Data protection: until the DPDP Rules, 2025 duties commence on 13 May 2027, genomic personal data is handled under IT Act s.43A and the SPDI Rules, 2011.
• NGS pipelines in Indian service companies compute allele frequencies, HWE filters, relatedness (KING/IBD), ancestry PCs and PRS as routine QC steps — the direct employable skill from this topic.

Key points

• Gene pool - all alleles in a population; population genetics tracks its allele frequencies over time.
• Hardy-Weinberg - p² + 2pq + q² = 1 under large size, random mating, no selection/mutation/migration/drift; reached in one generation for autosomal loci.
• HWE uses - carrier-frequency estimation, genotyping QC, forensic match probability; deviation signals structure, inbreeding or error.
• Mutation - creates alleles but changes frequencies very slowly; equilibrium with selection gives q̂ = √(µ/s) for recessives.
• Selection coefficient - s = 1 − w; selection against a rare recessive is slow because the allele hides in heterozygotes.
• Heterozygote advantage - maintains polymorphism; equilibrium p̂ = s₂/(s₁ + s₂); sickle-cell and thalassaemia in malarial India.
• Genetic drift - random frequency change ∝ 1/N_e; causes fixation, loss of heterozygosity, founder effects and bottlenecks.
• Effective population size - N_e = 4N_mN_f/(N_m + N_f); harmonic mean over generations; usually far below census size.
• Inbreeding coefficient F - probability of IBD; genotypes become p² + Fpq, 2pq(1 − F), q² + Fpq; first cousins F = 1/16.
• Wahlund effect - pooling structured subpopulations creates apparent heterozygote deficiency.
• F-statistics - F_ST = (H_T − H_S)/H_T measures differentiation; ≈ 1/(1 + 4N_em) under the island model.
• Linkage disequilibrium - D, D′, r²; decays by (1 − c) per generation; underlies GWAS and haplotype blocks.
• Neutral theory - θ = 4N_eµ; Tajima's D tests departure from neutral mutation-drift equilibrium.
• Variance partition - V_P = V_A + V_D + V_I + V_E (+ V_GE).
• Heritability - H² = V_G/V_P (broad), h² = V_A/V_P (narrow); population- and environment-specific, never individual.
• Breeder's equation - R = h²S; only additive variance converts selection differential into genetic gain.
• QTL vs GWAS - QTL mapping uses designed crosses and few recombinations (low resolution, high power); GWAS uses natural populations and historical LD (high resolution, needs large N).
• Genomic selection - marker-based BLUP predicts breeding values without phenotyping every candidate; standard in dairy cattle improvement.

Formulas and examples

Example 1 - allele frequency and HWE test (MN blood group, a sample of 1,000). Observed: MM = 360, MN = 480, NN = 160.

• p(M) = (2×360 + 480)/2000 = 1200/2000 = 0.60; q(N) = 0.40.
• Expected: p²N = 0.36 × 1000 = 360; 2pqN = 0.48 × 1000 = 480; q²N = 0.16 × 1000 = 160.
• χ² = 0 → the sample is in perfect Hardy-Weinberg equilibrium (df = 1, p = 1).

Example 2 - carrier frequency of a recessive disease. Incidence of an autosomal recessive disorder = 1 in 10,000 births.

• q² = 1/10,000 → q = 0.01; p = 0.99.
• Carrier frequency 2pq = 2 × 0.99 × 0.01 = 0.0198 ≈ 1 in 50.
• Carriers : affected = 0.0198/0.0001 = 198 : 1.
• If a carrier (known heterozygote) marries at random, risk of an affected child = 1 × 0.0198 × ¼ = 0.00495 ≈ 1 in 202.
• If the same person marries a first cousin, the probability the spouse carries the same allele is roughly r + (1 − r)·2pq, where r = 1/8 is the coefficient of relationship for first cousins: 0.125 + 0.875×0.0198 ≈ 0.142, so risk ≈ 0.142 × ¼ ≈ 1 in 28 — a ~7-fold rise, the standard counselling point.

Example 3 - heterozygote advantage (sickle-cell model). Fitnesses: AA = 0.89, AS = 1.00, SS = 0.20. So s_AA = 0.11, s_SS = 0.80.

• Equilibrium frequency of S: q̂ = s_AA/(s_AA + s_SS) = 0.11/0.91 = 0.1209.
• p̂(A) = 0.8791. Mean fitness w̄ = p̂²(0.89) + 2p̂q̂(1.00) + q̂²(0.20) = 0.7728×0.89 + 2×0.8791×0.1209 + 0.01462×0.20 = 0.6878 + 0.2126 + 0.0029 = 0.9033.
• Expected SS births at equilibrium = q̂² = 1.46% — of the order reported in high-prevalence tribal pockets of Odisha, Chhattisgarh and Madhya Pradesh before screening programmes.

Example 4 - drift, N_e and loss of heterozygosity. A captive breeding group has 10 breeding males and 40 breeding females.

• N_e = 4 × 10 × 40/(10 + 40) = 1600/50 = 32 (against a census of 50).
• Heterozygosity after 20 generations: H₂₀ = H₀(1 − 1/64)²⁰ = H₀ × (0.984375)²⁰ = H₀ × 0.7295 → 27% of variation lost.
• Per-generation inbreeding ΔF = 1/(2N_e) = 1.56%, above the 1% ceiling usually recommended for conservation herds — so the sex ratio must be equalised.
• If sizes over three generations were 100, 10, 100: N_e = 3/(1/100 + 1/10 + 1/100) = 3/0.12 = 25 — the bottleneck dominates.

Example 5 - heritability and response to selection (milk yield). Herd mean lactation yield = 3,000 kg, σ_P = 400 kg, h² = 0.25. Selected cows average 3,400 kg.

• S = 3,400 − 3,000 = 400 kg; R = h²S = 0.25 × 400 = 100 kg per generation.
• Generation interval L = 5 years → genetic gain = 100/5 = 20 kg per year.
• Selection intensity i = S/σ_P = 1.0 (corresponding to selecting roughly the top 38%). Using R = i·h²·σ_P = 1.0 × 0.25 × 400 = 100 kg — consistent.
• If realised offspring gain were only 60 kg, realised h² = 60/400 = 0.15, indicating the original estimate included non-additive or common-environment variance.

# PLINK: basic population-genetics QC and association
plink --vcf cohort.vcf.gz --make-bed --out cohort
plink --bfile cohort --hardy --out hwe          # per-SNP HWE exact test
plink --bfile cohort --freq --out af            # allele frequencies / MAF
plink --bfile cohort --geno 0.05 --maf 0.01 --hwe 1e-6 --mind 0.1 --make-bed --out clean
plink --bfile clean --indep-pairwise 50 5 0.2 --out prune   # LD pruning (r2 < 0.2)
plink --bfile clean --extract prune.prune.in --pca 10 --out pcs
plink --bfile clean --logistic --covar pcs.eigenvec --ci 0.95 --out gwas
plink --bfile clean --score weights.txt --out prs           # polygenic score

Where it is used

• Plant breeding (ICAR-IARI, NBPGR, IRRI-India, seed companies such as Nuziveedu, Rasi, Kaveri, Mahyco): heritability and GCA/SCA estimates, QTL mapping and marker-assisted selection for drought, submergence and disease resistance; genomic selection in rice, wheat, maize and chickpea.
• Animal breeding (NDRI Karnal, NBAGR, ICAR-NIANP, BAIF, Amul/NDDB): BLUP sire evaluation, progeny testing in Sahiwal, Gir and Murrah buffalo; genomic selection pilots and inbreeding monitoring in closed herds.
• Human and clinical genomics (CSIR-IGIB IndiGen, DBT GenomeIndia, CCMB, ICMR institutes, diagnostic firms such as MedGenome, Strand, Neuberg): variant allele-frequency reference panels, carrier screening for thalassaemia and sickle-cell under the National Sickle Cell Anaemia Elimination Mission, GWAS and polygenic risk research; roles as variant analyst and bioinformatics scientist.
• Forensic DNA (CDFD Hyderabad, CFSLs, state FSLs): allele frequency databases for Indian populations, random match probability and likelihood ratios computed under HWE with a θ correction for population substructure.
• Conservation genetics (WII Dehradun, NCBS, ZSI, Project Tiger/Project Elephant): effective population size, F_ST among tiger landscapes, inbreeding in the Gir lion and genetic monitoring of the African cheetahs reintroduced at Kuno, under the Biological Diversity Act, 2002 (as amended 2023) and Wild Life (Protection) Act permits.
• Pharmacogenomics and insect/pathogen resistance monitoring: allele frequency tracking of CYP variants, insecticide-resistance alleles in Helicoverpa and Anopheles, and antimicrobial resistance alleles.

Interview questions

• Q: State the Hardy-Weinberg law and its assumptions. A: In a large, randomly mating population free of selection, mutation, migration and drift, allele frequencies remain constant and genotype frequencies are p², 2pq, q²; it is reached in one generation of random mating for an autosomal locus.
• Q: If a disease occurs in 1 in 2,500, what is the carrier frequency? A: q = √(1/2500) = 0.02, so 2pq = 2 × 0.98 × 0.02 = 0.0392, about 1 in 26 people is a carrier.
• Q: Why does selection fail to eliminate a rare recessive lethal quickly? A: Nearly all copies are in phenotypically normal heterozygotes that selection cannot see, so Δq ∝ q², which becomes vanishingly small as q falls.
• Q: Distinguish broad-sense and narrow-sense heritability, and say which one a breeder uses. A: H² = V_G/V_P includes dominance and epistasis; h² = V_A/V_P includes only additive variance; the breeder uses h² because only additive effects are transmitted predictably and enter R = h²S.
• Q: Can heritability of 0.8 mean a trait cannot be changed by environment? A: No — heritability describes variance partitioning in one population and environment; changing the environment (nutrition, management) can shift the mean substantially without altering h².
• Q: What is effective population size and why is it usually smaller than the census size? A: It is the size of an idealised population with the same rate of drift; unequal sex ratio, variance in family size, fluctuating numbers and overlapping generations all reduce it.
• Q: What does F_ST = 0.02 tell you about two populations? A: Only 2% of total genetic variation lies between them — little differentiation, typical of most human population comparisons including those within India.
• Q: Why does the Wahlund effect matter in forensic DNA calculations? A: Hidden substructure inflates homozygote frequencies, so match probabilities computed naïvely under HWE can be too small; a θ (co-ancestry) correction of about 0.01–0.03 is applied.
• Q: Why does a GWAS need principal components or a mixed model? A: Population structure correlates allele frequencies with ancestry and with trait prevalence, producing spurious associations; PCs or a kinship matrix absorb that confounding.
• Q: Why is the genome-wide significance threshold 5 × 10⁻⁸? A: It is roughly 0.05 divided by one million effectively independent common variants in the genome after accounting for LD.
• Q: How do QTL mapping and GWAS differ in resolution and why? A: QTL mapping relies on recombinations in a few generations of a cross, giving broad intervals but high power; GWAS uses many generations of historical recombination, giving fine resolution but needing very large samples.
• Q: Why do polygenic risk scores perform worse in Indian populations? A: Most discovery cohorts are European; differences in allele frequencies, LD patterns and effect sizes reduce transferability, which is why IndiGen and GenomeIndia reference data are being built.
• Q: What maintains the sickle-cell allele at a few per cent in parts of central India? A: Heterozygote advantage — AS carriers resist severe falciparum malaria — giving a balanced polymorphism despite strong selection against SS homozygotes.

Common mistakes

• Using the observed heterozygote count as 2pq when the population may not be in HWE, instead of counting alleles directly.
• Taking the frequency of recessive phenotypes as the allele frequency q rather than q².
• Using the wrong degrees of freedom in the HWE chi-square test (it is 1 for a diallelic locus, not 2, because p is estimated from the data).
• Applying R = h²S with broad-sense heritability, or treating heritability as a fixed property of a trait or of an individual.
• Assuming equilibrium is reached in one generation for X-linked loci or for two-locus LD — both approach it gradually.
• Confusing effective with census population size, and ignoring that the harmonic mean makes a single bottleneck dominate N_e.
• Interpreting an HWE deviation as biology without first excluding genotyping error, null alleles or sample duplication.
• Reading a GWAS peak as the causal gene; the lead SNP is usually only in LD with the functional variant.

Related topics

Human genetics and genetic counselling · Evolution · Molecular markers and DNA fingerprinting · Plant and animal breeding · Statistical genomics and GWAS analysis · Biostatistics and experimental design