Random variables and expectation

In short

A random variable is a real-valued function on the sample space that converts an experimental outcome into a number, so that probability can be handled with algebra and calculus instead of event lists. Expectation is the probability-weighted mean of that number, and variance, moments, skewness and kurtosis are all expectations of simple functions of it. The topic is the structural core of B.Sc Statistics Probability papers, is examined heavily in GATE Statistics (ST), ISI/CMI entrances, IIT JAM MS, UGC-NET Statistics and IAI (Institute of Actuaries of India) CS1, and appears in analytics placement rounds at Mu Sigma, ZS Associates, Fractal, Swiss Re, EXL, Tiger Analytics and bank risk teams. It is the machinery behind insurance premium loading under IRDAI rules, expected credit loss under RBI norms, and expected-value decisions in pricing and inventory. Questions mix proofs (linearity, variance identity, Chebyshev) with numerical evaluation of means, variances and moment generating functions.

Explained

Random variables: definition and classification

• Definition: given a probability space (Ω, F, P), a random variable X is a function X : Ω → ℝ such that {ω : X(ω) ≤ x} ∈ F for every real x. The measurability condition is what makes P(X ≤ x) meaningful.
• Induced probability: X transfers P on Ω to a distribution on ℝ; the sample space itself then disappears from most calculations.
• Discrete: X takes finitely or countably many values x₁, x₂, … Described by a probability mass function p(xᵢ) = P(X = xᵢ) with p(xᵢ) ≥ 0 and Σ p(xᵢ) = 1. Examples: number of defectives in a sample, number of claims in a month.
• Continuous: X has a probability density function f(x) ≥ 0 with ∫f(x)dx = 1 over ℝ, and P(a ≤ X ≤ b) = ∫ₐᵇ f(x)dx. Here P(X = c) = 0 for every single point c, so ≤ and < may be interchanged.
• Mixed type: a distribution function with jumps and continuous stretches — e.g. insured loss that is 0 with probability 0.7 and continuous above 0. Common in actuarial papers.
• Indicator random variable: I_A(ω) = 1 if ω ∈ A, else 0. Then E(I_A) = P(A) — the bridge between probability and expectation, and the standard trick for counting problems.
• Degenerate random variable: P(X = c) = 1; its variance is 0. Conversely Var(X) = 0 implies X is degenerate (almost surely constant).

Distribution function and its properties

• CDF: F(x) = P(X ≤ x), defined for every type of random variable — this universality is why it is the primary object in theory questions.
• Properties examined as a short proof: (i) F is non-decreasing; (ii) F(−∞) = 0, F(+∞) = 1; (iii) F is right-continuous; (iv) P(a < X ≤ b) = F(b) − F(a); (v) P(X = a) = F(a) − F(a⁻), the jump at a.
• Discrete case: F is a step function; the step height at xᵢ equals p(xᵢ). Continuous case: F is continuous and f(x) = dF/dx wherever the derivative exists.
• Survival function S(x) = 1 − F(x) = P(X > x) — the reliability and life-insurance form.
• Quantiles: the p-th quantile is the smallest x with F(x) ≥ p; p = 0.5 gives the median. Mode is the x maximising p(x) or f(x).
• Inverse transform: if U ~ Uniform(0,1) then X = F⁻¹(U) has CDF F — the basis of Monte Carlo simulation in risk and operations work.

Expectation: definition, existence and properties

• Definition. Discrete: E(X) = Σ xᵢ p(xᵢ). Continuous: E(X) = ∫ x f(x) dx. It is a weighted average, interpreted as the long-run mean per the Law of Large Numbers, and as the centre of gravity of the distribution.
• Existence. E(X) exists only if the sum/integral converges absolutely: Σ|xᵢ|p(xᵢ) < ∞ or ∫|x|f(x)dx < ∞. The Cauchy distribution f(x) = 1/[π(1+x²)] has no mean — a favourite one-mark trap. The St. Petersburg game (pay 2ⁿ with probability 2⁻ⁿ) has infinite expectation.
• Properties (standard proof questions):
  1. E(c) = c for a constant.
  2. Linearity: E(aX + b) = aE(X) + b, and E(ΣaᵢXᵢ) = ΣaᵢE(Xᵢ) — valid with *no* independence assumption.
  3. If X ≥ 0 then E(X) ≥ 0; if X ≤ Y then E(X) ≤ E(Y) (monotonicity).
  4. |E(X)| ≤ E|X|.
  5. Product rule: E(XY) = E(X)E(Y) if X and Y are independent (converse false).
• Function of a random variable: E[g(X)] = Σ g(xᵢ)p(xᵢ) or ∫ g(x)f(x)dx — the "law of the unconscious statistician". Note E[g(X)] ≠ g(E[X]) in general; Jensen's inequality gives E[g(X)] ≥ g(E X) for convex g (so E(X²) ≥ (E X)², E(1/X) ≥ 1/E(X) for positive X).
• Tail formula: for X ≥ 0, E(X) = ∫₀^∞ [1 − F(x)]dx (continuous) or Σ_{k≥0} P(X > k) (non-negative integer case) — quick route to the mean of geometric and exponential variables.

Variance, moments and shape measures

• Variance: Var(X) = E[(X − μ)²] = E(X²) − μ², where μ = E(X). Units are the square of X's units; standard deviation σ = √Var(X) carries X's own units (₹, hours, kg).
• Scaling: Var(aX + b) = a²Var(X) — adding a constant shifts the mean, not the spread. Coefficient of variation = σ/μ (unit-free, used to compare volatility of two portfolios or two processes).
• Raw moments μ′ᵣ = E(Xʳ); central moments μᵣ = E[(X − μ)ʳ]. So μ₁ = 0, μ₂ = σ².
• Conversions: μ₂ = μ′₂ − μ′₁², μ₃ = μ′₃ − 3μ′₂μ′₁ + 2μ′₁³, μ₄ = μ′₄ − 4μ′₃μ′₁ + 6μ′₂μ′₁² − 3μ′₁⁴.
• Skewness β₁ = μ₃²/μ₂³, γ₁ = μ₃/σ³ — zero for symmetric distributions, positive for a right tail (income, claim size).
• Kurtosis β₂ = μ₄/μ₂², γ₂ = β₂ − 3 (excess). β₂ = 3 is mesokurtic (normal); > 3 leptokurtic (fat tails — equity returns); < 3 platykurtic.
• Moment generating function M_X(t) = E(e^{tX}), when it exists in a neighbourhood of 0. Then μ′ᵣ = M^{(r)}(0), the r-th derivative at t = 0; M uniquely determines the distribution; M_{aX+b}(t) = e^{bt}M_X(at); for independent X, Y, M_{X+Y}(t) = M_X(t)·M_Y(t).
• Cumulant generating function K(t) = ln M(t): κ₁ = μ, κ₂ = σ², κ₃ = μ₃, κ₄ = μ₄ − 3μ₂².
• Characteristic function φ(t) = E(e^{itX}) always exists (|e^{itX}| = 1) — used where the MGF fails, as for Cauchy.
• Factorial moments E[X(X−1)…(X−r+1)] shorten variance work for binomial and Poisson variables.

Several random variables: joint behaviour and conditional expectation

• Joint pmf/pdf p(x,y) or f(x,y); marginals obtained by summing/integrating out the other variable; independence iff f(x,y) = f_X(x)f_Y(y) for all (x,y).
• Covariance: Cov(X,Y) = E(XY) − E(X)E(Y). Cov(X,X) = Var(X); Cov(aX+b, cY+d) = ac·Cov(X,Y).
• Correlation: ρ = Cov(X,Y)/(σ_X σ_Y), −1 ≤ ρ ≤ 1 (from Cauchy–Schwarz), unit-free. ρ = 0 means *uncorrelated*, not independent — independence ⇒ ρ = 0, never the reverse (example: X uniform on {−1,0,1}, Y = X²).
• Variance of a sum: Var(X ± Y) = Var X + Var Y ± 2Cov(X,Y); for n variables Var(ΣXᵢ) = ΣVar(Xᵢ) + 2ΣΣ_{i<j}Cov(Xᵢ,Xⱼ). Independence collapses it to the sum of variances.
• Conditional expectation E(X | Y = y) = Σ x p(x|y) or ∫ x f(x|y)dx — itself a random variable when viewed as a function of Y.
• Law of total expectation (tower property): E(X) = E[E(X|Y)]. Law of total variance: Var(X) = E[Var(X|Y)] + Var[E(X|Y)] — the within/between split, the basis of credibility premiums in actuarial practice and of ANOVA logic.
• Regression interpretation: E(X|Y) is the minimum mean-square-error predictor of X given Y; the best *linear* predictor is μ_X + ρ(σ_X/σ_Y)(Y − μ_Y).

Inequalities and limit links

• Markov's inequality: for X ≥ 0 and k > 0, P(X ≥ k) ≤ E(X)/k. Needs only the mean.
• Chebyshev's inequality: P(|X − μ| ≥ kσ) ≤ 1/k², equivalently P(|X − μ| < kσ) ≥ 1 − 1/k². Distribution-free, hence weak: k = 2 gives only ≥ 75% within two SDs, against about 95% for a normal distribution.
• Cauchy–Schwarz: [E(XY)]² ≤ E(X²)E(Y²); used to prove |ρ| ≤ 1.
• Weak Law of Large Numbers (Chebyshev form): for i.i.d. Xᵢ with mean μ and finite variance, sample mean X̄ₙ → μ in probability. This justifies expectation as a long-run average and underlies sampling in NSO/MoSPI surveys.
• Central Limit Theorem link: E(X̄) = μ and Var(X̄) = σ²/n for a random sample, so the standard error is σ/√n — the single most used consequence of this topic in applied work.
• Limitation of expectation alone: two very different distributions can share a mean; decision-making therefore uses mean *with* variance (Markowitz mean–variance), or utility, since a high expected payoff may carry ruinous downside.

Key points

• Random variable — a measurable function from the sample space to ℝ; not a "variable chosen at random".
• PMF vs PDF — p(x) is a probability; f(x) is a density, can exceed 1, and only its integral is a probability.
• CDF — F(x) = P(X ≤ x), non-decreasing, right-continuous, 0 to 1; works for discrete, continuous and mixed types.
• Jump of F at a — equals P(X = a); zero for a continuous random variable.
• Expectation — E(X) = Σxp(x) or ∫xf(x)dx; the probability-weighted mean, exists only under absolute convergence.
• Linearity — E(aX + bY + c) = aE(X) + bE(Y) + c holds always, independence not required.
• Product rule — E(XY) = E(X)E(Y) requires independence; its converse is false.
• Law of the unconscious statistician — E[g(X)] uses the distribution of X, no need to derive the distribution of g(X).
• Jensen — E[g(X)] ≥ g(E X) for convex g; hence E(X²) ≥ (E X)² always.
• Variance identity — Var(X) = E(X²) − [E(X)]² ≥ 0; Var(aX + b) = a²Var(X).
• Moments — raw μ′ᵣ = E(Xʳ), central μᵣ = E(X − μ)ʳ; μ₂ = σ², μ₃ drives skewness, μ₄ kurtosis.
• MGF — M(t) = E(e^{tX}); r-th derivative at 0 gives μ′ᵣ; multiplies over independent sums; may not exist (Cauchy).
• Covariance and correlation — Cov = E(XY) − E X·E Y; ρ ∈ [−1,1], unit-free; ρ = 0 ≠ independence.
• Variance of a sum — add covariances; only independence allows plain addition.
• Tower property — E(X) = E[E(X|Y)]; Var(X) = E[Var(X|Y)] + Var[E(X|Y)].
• Chebyshev — at least 1 − 1/k² of probability lies within k standard deviations, for any distribution with finite variance.
• Standard error — Var(X̄) = σ²/n, so precision improves with √n, not n.
• Degenerate case — Var(X) = 0 ⇔ X is constant with probability 1.

Formulas and examples

Example 1 — Discrete pmf: constant, mean, variance, CDF. X has p(x) = kx for x = 1, 2, 3, 4.

• Σp = k(1+2+3+4) = 10k = 1 → k = 0.1, so p = 0.1, 0.2, 0.3, 0.4.
• E(X) = 1(0.1)+2(0.2)+3(0.3)+4(0.4) = 0.1+0.4+0.9+1.6 = 3.0.
• E(X²) = 1(0.1)+4(0.2)+9(0.3)+16(0.4) = 0.1+0.8+2.7+6.4 = 10.0.
• Var = 10.0 − 3.0² = 1.0, σ = 1.0.
• F(x): 0.1, 0.3, 0.6, 1.0 at x = 1,2,3,4. P(2 < X ≤ 4) = 1.0 − 0.3 = 0.7.
• Check with Y = 2X + 5: E(Y) = 2(3)+5 = 11, Var(Y) = 4(1) = 4.

Example 2 — Continuous density: mean, variance, median. f(x) = 3x² for 0 ≤ x ≤ 1, zero elsewhere (so F(x) = x³).

• E(X) = ∫₀¹ x·3x² dx = 3/4 = 0.75.
• E(X²) = ∫₀¹ 3x⁴ dx = 3/5 = 0.60 → Var = 0.60 − 0.5625 = 0.0375, σ = 0.1936.
• Median: x³ = 0.5 → x = 0.5^{1/3} = 0.7937 (mean < median here, so the distribution is left-skewed).
• P(X > 0.5) = 1 − 0.5³ = 0.875.

Example 3 — Expected value in a business decision (₹). A Pune component supplier bids for an order. Profit X: ₹8,00,000 with probability 0.3, ₹2,00,000 with 0.5, −₹3,00,000 with 0.2.

• E(X) = 0.3(8,00,000) + 0.5(2,00,000) + 0.2(−3,00,000) = 2,40,000 + 1,00,000 − 60,000 = ₹2,80,000.
• E(X²) = 0.3(6.4×10¹¹) + 0.5(4×10¹⁰) + 0.2(9×10¹⁰) = 1.92×10¹¹ + 2×10¹⁰ + 1.8×10¹⁰ = 2.30×10¹¹.
• Var = 2.30×10¹¹ − (2.8×10⁵)² = 2.30×10¹¹ − 0.784×10¹¹ = 1.516×10¹¹ → σ = ₹3,89,359.
• CV = 3,89,359/2,80,000 = 1.39 — expected profit is positive but risk is larger than the mean, so a mean-only decision hides a 20% chance of a ₹3 lakh loss.

Example 4 — MGF, moments and Chebyshev. X ~ Exponential with f(x) = λe^{−λx}, x > 0, λ = 2 per hour (service completions).

• M(t) = ∫₀^∞ e^{tx}λe^{−λx}dx = λ/(λ − t) for t < λ = 2/(2 − t).
• M′(t) = 2/(2−t)² → M′(0) = 2/4 = 0.5 h = E(X). M″(t) = 4/(2−t)³ → M″(0) = 4/8 = 0.5 = E(X²).
• Var = 0.5 − 0.25 = 0.25, σ = 0.5 h (mean = σ, so CV = 1).
• Chebyshev with k = 2: P(|X − 0.5| ≥ 1) ≤ 0.25. Exact: P(X ≥ 1.5) = e^{−3} = 0.0498 — the bound (0.25) is valid but five times loose, showing why distribution-free bounds are only a safety net.

Example 5 — Covariance, correlation and variance of a sum. Joint pmf: P(0,0)=0.2, P(0,1)=0.3, P(1,0)=0.1, P(1,1)=0.4.

• Marginals: P(X=1) = 0.5 → E(X) = 0.5, E(X²) = 0.5, Var(X) = 0.25. P(Y=1) = 0.7 → E(Y) = 0.7, Var(Y) = 0.7 − 0.49 = 0.21.
• E(XY) = 1×0.4 = 0.4 → Cov = 0.4 − 0.35 = 0.05.
• ρ = 0.05/√(0.25×0.21) = 0.05/0.2291 = 0.218 (weak positive).
• Var(X+Y) = 0.25 + 0.21 + 2(0.05) = 0.56; treating them as independent would wrongly give 0.46.

Where it is used

• Insurance and actuarial work (IRDAI-regulated): expected claim cost per policy fixes the pure premium; variance and higher moments set risk loading and solvency margins. LIC, New India Assurance, HDFC Life and reinsurers use conditional expectation for credibility rating; IAI's CS1 paper is built on this topic.
• Banking and credit risk (RBI-regulated): expected credit loss = PD × LGD × EAD is an expectation; value-at-risk and expected shortfall under Basel-style norms are tail functionals of the loss random variable.
• Analytics and consulting roles: data scientists and business analysts at Mu Sigma, Fractal, ZS, Tiger Analytics and EXL use expected value for A/B test uplift, churn-driven customer lifetime value and expected revenue per user.
• Quality control in manufacturing: mean and variance of a measured characteristic drive control-chart limits (μ ± 3σ) and process capability indices Cp, Cpk in Tata Motors, Bosch and Bharat Forge plants.
• Operations and supply chain: expected demand and its variance set safety stock (z·σ·√lead time) and newsvendor order quantities for e-commerce and FMCG distribution.
• Official statistics and policy: MoSPI/NSO sample surveys (PLFS, HCES) rely on unbiased estimators — expectation equal to the population parameter — and report standard errors σ/√n alongside estimates.

Interview questions

• Q: What exactly is a random variable? A: A real-valued function on the sample space whose pre-images of intervals are events, so probabilities like P(X ≤ x) are defined; it converts outcomes into numbers.
• Q: Can a probability density function take a value greater than 1? A: Yes — a density is not a probability. Uniform(0, 0.5) has f(x) = 2; only the area under f must be 1.
• Q: Does E(X + Y) = E(X) + E(Y) need independence? A: No, linearity holds for any random variables with finite means; independence is needed only for E(XY) = E(X)E(Y).
• Q: Give a distribution with no mean. A: The standard Cauchy, f(x) = 1/[π(1+x²)]: ∫|x|f(x)dx diverges, so E(X) does not exist and its MGF does not exist either.
• Q: If Cov(X,Y) = 0, are X and Y independent? A: Not necessarily. With X uniform on {−1,0,1} and Y = X², Cov = 0 but Y is a deterministic function of X. Zero covariance implies independence only in the bivariate normal case.
• Q: Why is Var(X) = E(X²) − [E(X)]² always non-negative? A: Because it equals E[(X−μ)²], the expectation of a non-negative quantity; equivalently it is Jensen's inequality E(X²) ≥ (E X)².
• Q: What does Chebyshev's inequality buy you over knowing the distribution? A: It bounds P(|X−μ| ≥ kσ) by 1/k² for any distribution with finite variance, so it is safe when the shape is unknown — at the cost of being very loose.
• Q: What is the use of a moment generating function? A: Differentiating it at t = 0 gives all raw moments, it identifies the distribution uniquely, and it multiplies over independent sums, which is how distributions of sums are derived.
• Q: Explain the law of total expectation with a business example. A: E(X) = E[E(X|Y)]: expected claim per policyholder is the average of segment-wise expected claims weighted by segment probabilities, which is exactly how a blended premium is computed.
• Q: Why is E[g(X)] generally not g(E[X])? A: Averaging and a non-linear transform do not commute; Jensen's inequality gives the direction — E(X²) ≥ (E X)², and E(1/X) ≥ 1/E(X) for positive X. Equality only for linear g.
• Q: A fair die is rolled; what is the expected value and variance? A: Mean = 3.5, E(X²) = 91/6, Var = 91/6 − 12.25 = 35/12 ≈ 2.917, σ ≈ 1.708.
• Q: Two investments have the same expected return; how do you choose? A: Compare variance or CV, and the downside tail — equal means with different spreads mean different risk, which is the mean–variance basis of portfolio choice.
• Q: How do you simulate a random variable with a given CDF? A: Draw U ~ Uniform(0,1) and set X = F⁻¹(U); F(X) is then uniform, so X has the required distribution.

Common mistakes

• Treating f(x) as a probability, or writing P(X = c) > 0 for a continuous random variable.
• Using Var(aX + b) = a·Var(X) instead of a²Var(X), or forgetting that adding a constant leaves variance unchanged.
• Applying E(XY) = E(X)E(Y) without independence, and adding variances of dependent variables without the 2Cov term.
• Concluding independence from ρ = 0, or causation from a high correlation.
• Forgetting to check absolute convergence and declaring a mean for the Cauchy or similar heavy-tailed distributions.
• Normalisation slips: not solving Σp(x) = 1 or ∫f(x)dx = 1 for the unknown constant before computing moments, and integrating over the wrong support.
• Reporting variance in the units of X (it is in X²), or comparing CVs when the mean can be zero or negative.
• Confusing μ′₂ with μ₂, or substituting raw moments directly into the β₁, β₂ formulas without converting to central moments.

Related topics

Probability theory and Bayes' theorem · Probability distributions · Sampling distributions and the Central Limit Theorem · Estimation theory and unbiasedness · Correlation and regression · Stochastic processes and Markov chains