Matrices, eigenvalues and diagonalisation

In short

A matrix is a rectangular array of numbers that encodes a linear transformation or a system of linear equations; its eigenvalues and eigenvectors expose the directions the transformation merely stretches, and diagonalisation rewrites the matrix in the basis of those directions as a diagonal matrix. This machinery turns hard problems — matrix powers, systems of differential equations, quadratic forms, stability tests — into arithmetic on scalars. In Indian universities this is the core of B.Sc Algebra / Matrix Algebra papers under the UGC CBCS and NEP-aligned four-year LOCF syllabi, and it carries the largest single block of Linear Algebra marks in GATE MA, CSIR-UGC NET, JAM MA and IIT/CUET-PG entrance papers. It is examined again in placement technical rounds for data science, quantitative analysis and software roles (TCS Digital, Infosys, Mu Sigma, ZS, Goldman Sachs, Quadeye, ISRO/DRDO scientist posts), where PCA, Markov chains and Google-style ranking all reduce to eigen-problems.

Explained

Matrices: definitions, types and operations

• An m × n matrix A = [a_ij] has m rows, n columns; a_ij is the entry in row i, column j. Square if m = n; order of a square matrix is n.
• Special types: diagonal (a_ij = 0 for i ≠ j), scalar (diagonal with equal entries), identity I, zero, upper/lower triangular, symmetric (Aᵗ = A), skew-symmetric (Aᵗ = −A, so all diagonal entries are 0), orthogonal (AᵗA = I), idempotent (A² = A), nilpotent (A^k = O), involutory (A² = I).
• Complex analogues: Hermitian (A* = A, where A* = conjugate transpose), skew-Hermitian (A* = −A), unitary (A*A = I), normal (AA* = A*A).
• Operations: addition and scalar multiplication are entrywise. Product AB is defined only when columns of A = rows of B; (AB)_ij = Σ_k a_ik b_kj. Multiplication is associative and distributive but not commutative, and AB = O does not force A = O or B = O.
• Transpose laws: (A + B)ᵗ = Aᵗ + Bᵗ, (AB)ᵗ = BᵗAᵗ, (Aᵗ)ᵗ = A. Every square matrix splits uniquely as A = ½(A + Aᵗ) + ½(A − Aᵗ) — symmetric plus skew-symmetric part.
• Trace: tr(A) = Σ a_ii. Linear, and tr(AB) = tr(BA) even though AB ≠ BA.
• Determinant (square only): |AB| = |A||B|, |Aᵗ| = |A|, |kA| = kⁿ|A| for order n, |A⁻¹| = 1/|A|. A is singular if |A| = 0, non-singular (invertible) otherwise.
• Inverse: A⁻¹ = adj(A)/|A|, where adj(A) is the transpose of the cofactor matrix. (AB)⁻¹ = B⁻¹A⁻¹. Also A·adj(A) = |A| I.

Rank, elementary operations and systems of equations

• Elementary row operations: R_i ↔ R_j, R_i → kR_i (k ≠ 0), R_i → R_i + kR_j. Each equals pre-multiplication by an invertible elementary matrix, so they do not change the rank or the solution set.
• Rank ρ(A): the number of non-zero rows in row echelon form = number of leading pivots = order of the largest non-vanishing minor = dimension of the row space = dimension of the column space. ρ(A) ≤ min(m, n); ρ(A) = ρ(Aᵗ).
• Normal form: every matrix reduces by row and column operations to [[I_r, O],[O, O]] where r = rank.
• Rank-nullity: for A of size m × n, ρ(A) + (nullity of A) = n, where nullity = dim of the null space {x : Ax = 0}.
• Consistency (Rouché–Capelli) for Ax = b with n unknowns:
  1. ρ(A) ≠ ρ([A|b]) → inconsistent, no solution.
  2. ρ(A) = ρ([A|b]) = n → unique solution.
  3. ρ(A) = ρ([A|b]) = r < n → infinitely many solutions with (n − r) free parameters.
• Homogeneous system Ax = 0 is always consistent (x = 0). It has a non-trivial solution iff ρ(A) < n; for a square matrix, iff |A| = 0. The solution set is a subspace of dimension n − ρ(A).
• Methods: Gauss elimination (forward elimination + back substitution), Gauss–Jordan (reduce to reduced row echelon form; also yields A⁻¹ by reducing [A|I]), Cramer's rule x_i = |A_i|/|A| (valid only for square non-singular systems; impractical beyond order 3–4), LU decomposition for repeated right-hand sides.

Eigenvalues and eigenvectors

• Definition: for a square matrix A of order n, a scalar λ is an eigenvalue if Ax = λx for some non-zero vector x, called an eigenvector. The pair comes from the same transformation leaving the direction of x unchanged (only scaling it).
• Rearranged: (A − λI)x = 0 has a non-trivial solution ⟺ |A − λI| = 0. This is the characteristic equation; the degree-n polynomial p(λ) = |A − λI| is the characteristic polynomial; the set of eigenvalues is the spectrum.
• Expanded form of p(λ): λⁿ − S₁λⁿ⁻¹ + S₂λⁿ⁻² − … + (−1)ⁿ|A|, where S_k is the sum of the principal minors of order k. For order 3: λ³ − tr(A)λ² + (sum of 2×2 principal minors)λ − |A| = 0.
• Two standing checks: Σλ_i = tr(A) and Πλ_i = |A|. Use both on every exam problem.
• Eigenspace of λ: E_λ = null space of (A − λI); its dimension is the geometric multiplicity (GM). The exponent of (λ − λ_i) in p(λ) is the algebraic multiplicity (AM). Always 1 ≤ GM ≤ AM.
• Properties:
  • Eigenvalues of Aᵗ equal those of A; eigenvalues of A⁻¹ are 1/λ_i; of A^k are λ_i^k; of kA are kλ_i; of (A + cI) are λ_i + c (eigenvectors unchanged in all of these except the transpose).
  • A triangular or diagonal matrix has its diagonal entries as eigenvalues.
  • A is singular ⟺ 0 is an eigenvalue. A is invertible ⟺ no eigenvalue is 0.
  • Idempotent: eigenvalues 0 and 1. Nilpotent: all eigenvalues 0. Involutory: ±1. Orthogonal/unitary: |λ| = 1. Symmetric (real): all λ real. Skew-symmetric (real): λ = 0 or purely imaginary. Hermitian: λ real.
  • Eigenvectors belonging to distinct eigenvalues are linearly independent.
  • AB and BA have the same eigenvalues (for square A, B).
  • Real matrices may have complex eigenvalues, which occur in conjugate pairs.
• Cayley–Hamilton theorem: every square matrix satisfies its own characteristic equation, p(A) = O. Uses: compute A⁻¹ (multiply p(A) = O by A⁻¹), reduce high powers of A, express Aⁿ in terms of I, A, …, Aⁿ⁻¹.
• Minimal polynomial m(λ): the monic polynomial of least degree with m(A) = O. It divides p(λ), has the same irreducible factors (same roots), and A is diagonalisable ⟺ m(λ) is a product of distinct linear factors.

Similarity and diagonalisation

• Similar matrices: B = P⁻¹AP for some invertible P. Similarity is an equivalence relation and represents the *same* linear map in a different basis.
• Similarity invariants: characteristic polynomial, eigenvalues with multiplicities, determinant, trace, rank, minimal polynomial. Eigen*vectors* are not invariant — they transform as P⁻¹x.
• Diagonalisable: A is diagonalisable if A is similar to a diagonal matrix D, i.e. P⁻¹AP = D. Then the columns of P are n linearly independent eigenvectors and D = diag(λ₁, …, λ_n) in the matching order.
• Criteria (any one suffices):
  1. A has n linearly independent eigenvectors.
  2. GM = AM for every eigenvalue (the sum of geometric multiplicities is n).
  3. The minimal polynomial has only distinct linear roots.
  4. Sufficient (not necessary) condition: all n eigenvalues distinct. A repeated eigenvalue may still diagonalise — e.g. I.
• Why it matters: if A = PDP⁻¹ then A^k = PD^kP⁻¹, and f(A) = P f(D) P⁻¹ for polynomials and power series (including e^A), reducing matrix functions to scalar functions on the diagonal.
• Defective matrices (GM < AM for some λ) are not diagonalisable; the nearest canonical form is the Jordan canonical form J = diag(J₁, …, J_k), each Jordan block having λ on the diagonal and 1s just above it. Number of blocks for λ = GM(λ); total size of blocks for λ = AM(λ).
• Spectral theorem (real symmetric case): every real symmetric matrix is orthogonally diagonalisable — there exists an orthogonal P (PᵗP = I) with PᵗAP = D, all λ real, and eigenvectors for distinct eigenvalues mutually orthogonal. Build P by normalising eigenvectors (Gram–Schmidt within a repeated eigenspace). For Hermitian matrices the analogue uses a unitary U: U*AU = D. Normal matrices are exactly the unitarily diagonalisable ones.
• Limitations: eigen-decomposition needs a square matrix and can be numerically ill-conditioned when eigenvalues are close; it fails entirely for defective matrices. Singular value decomposition A = UΣVᵗ exists for *every* m × n real matrix, where the singular values are the non-negative square roots of the eigenvalues of AᵗA — the practical substitute used in data work.

Quadratic forms and definiteness

• A real quadratic form is Q(x) = xᵗAx with A symmetric; for Q = ax² + 2hxy + by², A = [[a, h],[h, b]].
• Congruent reduction: x = Py turns Q into yᵗ(PᵗAP)y. Orthogonal P (from the spectral theorem) gives the canonical form Q = λ₁y₁² + … + λ_ny_n² — the principal-axes form.
• Rank, index, signature: rank = number of non-zero λ; index = number of positive λ; signature = (positive count) − (negative count). Sylvester's law of inertia: these are unchanged by any real non-singular congruent transformation.
• Classification by eigenvalue signs: positive definite (all λ > 0), positive semi-definite (all λ ≥ 0, some = 0), negative definite (all λ < 0), negative semi-definite, indefinite (both signs).
• Sylvester's criterion: A is positive definite ⟺ all leading principal minors are > 0. Negative definite ⟺ leading principal minors alternate in sign starting negative. (For semi-definiteness all principal minors, not just leading ones, must be checked.)
• Applications: the Hessian test for maxima/minima in multivariable calculus, conic and quadric classification, stability of equilibria, and covariance matrices in statistics (always positive semi-definite).

Exam framing, Indian syllabi and practice

• Typical syllabus coverage (UGC CBCS / NEP four-year B.Sc Algebra and Matrix Algebra papers): matrix types and algebra, determinants and inverse, elementary operations, rank and echelon forms, systems of linear equations and consistency, eigenvalues and eigenvectors, Cayley–Hamilton, diagonalisation, orthogonal diagonalisation of symmetric matrices, quadratic forms. Jordan form, minimal polynomial and SVD appear in honours, PG and GATE-level courses; check the specific university scheme, since the NEP-aligned syllabi differ by university and are still being revised.
• GATE MA and CSIR-NET pattern: short, computation-light questions that reward the trace/determinant checks, multiplicity counting, and property-based elimination over full eigenvector computation.
• SI units: matrices themselves are dimensionless, but in applied problems entries carry units (stiffness in N/m, flow in ₹ lakh, probability per step), and eigenvalues inherit them — e.g. √λ of a stiffness/mass matrix is a natural frequency in rad/s.
• Computation in practice: by hand, eigenvalues come from the characteristic polynomial; numerically, software uses the QR algorithm (LAPACK), not root-finding, because polynomial roots are unstable. Standard tools in Indian workplaces are NumPy/SciPy (numpy.linalg.eig, eigh for symmetric), R, MATLAB/Octave, and Excel's MMULT/MINVERSE/MDETERM for small matrices.

Key points

• Matrix - rectangular array representing a linear map; product defined only when inner dimensions match, and is non-commutative.
• Rank - number of pivots in row echelon form = largest non-zero minor order = dim(row space) = dim(column space).
• Rank-nullity - ρ(A) + nullity(A) = n (number of columns).
• Consistency rule - compare ρ(A) with ρ([A|b]): unequal → no solution; equal to n → unique; equal and < n → infinite with n − r parameters.
• Homogeneous system - non-trivial solution exists iff ρ(A) < n, i.e. |A| = 0 for a square matrix.
• Eigenvalue equation - Ax = λx with x ≠ 0; λ solves |A − λI| = 0.
• Trace and determinant checks - Σλ_i = tr(A), Πλ_i = |A|; always verify answers with these.
• Multiplicities - 1 ≤ GM ≤ AM; a deficit (GM < AM) makes the matrix defective.
• Derived spectra - A⁻¹ has 1/λ, A^k has λ^k, A + cI has λ + c, Aᵗ has the same λ; eigenvectors unchanged except for Aᵗ.
• Cayley-Hamilton - p(A) = O; used to invert A and to collapse high powers.
• Minimal polynomial - divides p(λ); A is diagonalisable iff m(λ) has distinct linear factors only.
• Similarity - B = P⁻¹AP preserves eigenvalues, trace, determinant, rank and characteristic polynomial.
• Diagonalisation - P⁻¹AP = D with eigenvectors as columns of P; distinct eigenvalues are sufficient but not necessary.
• Power formula - A^k = PD^kP⁻¹ is the main payoff of diagonalisation.
• Spectral theorem - real symmetric ⇒ real eigenvalues, orthogonal eigenvectors, PᵗAP = D with P orthogonal.
• Jordan form - canonical form when diagonalisation fails; blocks per λ = GM, total block size = AM.
• Quadratic form - Q = xᵗAx; classified positive/negative definite, semi-definite or indefinite by eigenvalue signs or Sylvester's leading-minor criterion.
• SVD - A = UΣVᵗ exists for every matrix; singular values = √(eigenvalues of AᵗA); the robust alternative to eigen-decomposition.

Formulas and examples

Example 1 — Eigenvalues, eigenvectors and diagonalisation of a 2×2 matrix. A = [[4, 1],[2, 3]].

• tr(A) = 7, |A| = 4(3) − 1(2) = 10. Characteristic equation: λ² − 7λ + 10 = 0 → λ = 5, 2. Check: 5 + 2 = 7 ✓, 5 × 2 = 10 ✓.
• λ = 5: (A − 5I)x = [[−1, 1],[2, −2]]x = 0 → −x₁ + x₂ = 0 → x = (1, 1)ᵗ.
• λ = 2: (A − 2I)x = [[2, 1],[2, 1]]x = 0 → 2x₁ + x₂ = 0 → x = (1, −2)ᵗ.
• P = [[1, 1],[1, −2]], |P| = −3, P⁻¹ = (1/−3)[[−2, −1],[−1, 1]] = [[2/3, 1/3],[1/3, −1/3]].
• D = diag(5, 2). Verify A³ = PD³P⁻¹: D³ = diag(125, 8); PD³ = [[125, 8],[125, −16]]; (PD³)P⁻¹ = [[125(2/3) + 8(1/3), 125(1/3) + 8(−1/3)],[125(2/3) − 16(1/3), 125(1/3) + 16(1/3)]] = [[86, 39],[78, 47]].
• Direct check: A² = [[18, 7],[14, 11]], A³ = A²A = [[18(4)+7(2), 18(1)+7(3)],[14(4)+11(2), 14(1)+11(3)]] = [[86, 39],[78, 47]] ✓.

Example 2 — A defective (non-diagonalisable) 3×3 matrix. A = [[2, 1, 0],[0, 2, 0],[0, 0, 3]] (upper triangular).

• Eigenvalues are the diagonal entries: λ = 2, 2, 3, so AM(2) = 2, AM(3) = 1. tr(A) = 7 = 2 + 2 + 3 ✓, |A| = 12 = 2·2·3 ✓.
• A − 2I = [[0, 1, 0],[0, 0, 0],[0, 0, 1]], rank 2 → GM(2) = 3 − 2 = 1 < AM(2) = 2.
• Σ GM = 1 + 1 = 2 ≠ 3, so A is not diagonalisable. Its Jordan form is a 2×2 block for λ = 2 plus a 1×1 block for λ = 3: J = [[2, 1, 0],[0, 2, 0],[0, 0, 3]].
• Minimal polynomial: (λ − 2)²(λ − 3) — the repeated factor confirms non-diagonalisability.

Example 3 — Cayley–Hamilton to find the inverse. A = [[1, 2],[3, 4]].

• tr(A) = 5, |A| = 4 − 6 = −2 → p(λ) = λ² − 5λ − 2.
• Cayley–Hamilton: A² − 5A − 2I = O. Multiply by A⁻¹: A − 5I − 2A⁻¹ = O → A⁻¹ = (A − 5I)/2.
• A − 5I = [[−4, 2],[3, −1]] → A⁻¹ = [[−2, 1],[1.5, −0.5]].
• Verify: A A⁻¹ = [[1(−2) + 2(1.5), 1(1) + 2(−0.5)],[3(−2) + 4(1.5), 3(1) + 4(−0.5)]] = [[1, 0],[0, 1]] ✓.

Example 4 — Consistency of a linear system. Solve 2x + 3y + z = 9, x − y + z = 2, 3x + 2y + 2z = 11.

• Augmented matrix [[2, 3, 1 | 9],[1, −1, 1 | 2],[3, 2, 2 | 11]]. Swap R₁, R₂ → [[1, −1, 1 | 2],[2, 3, 1 | 9],[3, 2, 2 | 11]].
• R₂ → R₂ − 2R₁ = (0, 5, −1 | 5); R₃ → R₃ − 3R₁ = (0, 5, −1 | 5). R₃ → R₃ − R₂ = (0, 0, 0 | 0).
• ρ(A) = ρ([A|b]) = 2 < n = 3 → infinitely many solutions with 3 − 2 = 1 parameter.
• Let z = t: 5y = 5 + t → y = 1 + t/5; x = 2 + y − z = 2 + 1 + t/5 − t = 3 − 4t/5. Solution: (3 − 4t/5, 1 + t/5, t).
• Check with t = 5: (−1, 2, 5) → 2(−1) + 3(2) + 5 = 9 ✓; −1 − 2 + 5 = 2 ✓; −3 + 4 + 10 = 11 ✓.

Example 5 — Orthogonal diagonalisation and definiteness of a quadratic form. Q = 5x² + 5y² + 6xy, so A = [[5, 3],[3, 5]].

• tr(A) = 10, |A| = 25 − 9 = 16 → λ² − 10λ + 16 = 0 → λ = 8, 2.
• λ = 8: (A − 8I) = [[−3, 3],[3, −3]] → x = y → unit vector (1/√2)(1, 1)ᵗ. λ = 2: [[3, 3],[3, 3]] → x = −y → (1/√2)(1, −1)ᵗ. The two are orthogonal, as the spectral theorem guarantees.
• P = (1/√2)[[1, 1],[1, −1]]; PᵗP = I, and PᵗAP = diag(8, 2).
• Canonical form Q = 8u² + 2v². Both eigenvalues positive, and the leading principal minors 5 > 0 and 16 > 0 agree — Q is positive definite; rank 2, index 2, signature 2.

Where it is used

• Data science and analytics (Mu Sigma, ZS Associates, Fractal, Tiger Analytics, Flipkart, Swiggy): principal component analysis diagonalises the covariance matrix so the top eigenvectors become the retained features; eigenvalue shares give variance explained.
• Quantitative finance and risk (Goldman Sachs Bengaluru, Quadeye, Tower Research, bank treasury desks): correlation-matrix eigen-decomposition for portfolio risk factors, positive-definiteness checks on covariance matrices, and Value-at-Risk computation in ₹ crore exposures.
• Structural and mechanical engineering (L&T, BHEL, ISRO, NPCIL): natural frequencies and mode shapes come from the eigenvalues of K − ω²M in finite-element analysis; seismic design to IS 1893 (Part 1): 2016 uses modal analysis, and buckling loads are eigenvalues of a stability matrix.
• Control systems and power systems (ISRO, DRDO, BEL, POSOCO/Grid-India): eigenvalues of the state matrix A in ẋ = Ax decide stability (all real parts negative), and small-signal stability of the national grid is an eigenvalue study.
• Computer science and machine learning: PageRank is the dominant eigenvector of a stochastic link matrix; Markov chains reach a steady state given by the λ = 1 eigenvector; graph Laplacian eigenvalues drive spectral clustering; SVD powers recommender systems and image compression.
• Teaching, research and competitive exams: B.Sc/M.Sc teaching posts, CSIR-UGC NET/JRF, GATE MA, ISI/CMI entrance and IIT PhD interviews all test this topic directly.

Interview questions

• Q: What does an eigenvector mean geometrically? A: It is a non-zero direction that the linear map does not rotate — the map only scales it by the eigenvalue λ; negative λ flips it, |λ| < 1 shrinks it.
• Q: Can a matrix have fewer eigenvectors than eigenvalues? A: It can have fewer *independent* eigenvectors than n if some eigenvalue has geometric multiplicity below its algebraic multiplicity; such a defective matrix cannot be diagonalised.
• Q: Are distinct eigenvalues necessary for diagonalisation? A: No — they are sufficient but not necessary. The identity matrix has one eigenvalue repeated n times and is already diagonal.
• Q: How do you check diagonalisability quickly without finding every eigenvector? A: For each eigenvalue compute ρ(A − λI); the geometric multiplicity is n − ρ(A − λI), and A is diagonalisable iff these sum to n. Equivalently, check that the minimal polynomial has no repeated factor.
• Q: Why are the eigenvalues of a real symmetric matrix always real? A: From Ax = λx with x ≠ 0, x*Ax = λx*x; x*Ax is real because A = A* and x*x > 0, so λ = x*Ax/x*x must be real.
• Q: State Cayley–Hamilton and give one practical use. A: Every square matrix satisfies its own characteristic equation, p(A) = O; it lets you write A⁻¹ and any high power A^k as a polynomial in A of degree below n.
• Q: Do similar matrices have the same eigenvectors? A: No. They share eigenvalues, trace, determinant, rank and characteristic polynomial, but if B = P⁻¹AP then the eigenvector of B corresponding to λ is P⁻¹x, not x.
• Q: A 3×3 matrix has eigenvalues 1, 2 and 3. What are its trace, determinant and the eigenvalues of A⁻¹ and A²? A: Trace 6, determinant 6; A⁻¹ has 1, 1/2, 1/3 and A² has 1, 4, 9.
• Q: When is a matrix invertible in eigenvalue language? A: Exactly when 0 is not an eigenvalue, since |A| = Πλ_i and |A| = 0 would mean a non-trivial null space.
• Q: How does PCA relate to eigenvalues? A: PCA orthogonally diagonalises the covariance matrix; the eigenvectors are the principal directions and each eigenvalue is the variance along that direction, so eigenvalue/Σeigenvalues is the variance explained.
• Q: What if a matrix is not square or not diagonalisable, but you still need a decomposition? A: Use the singular value decomposition A = UΣVᵗ, which exists for every real matrix; its singular values are the square roots of the eigenvalues of AᵗA.
• Q: How do you test a quadratic form for positive definiteness? A: Either show all eigenvalues of its symmetric matrix are positive, or apply Sylvester's criterion that every leading principal minor is positive.
• Q: Why does a numerical library not find eigenvalues by solving the characteristic polynomial? A: Polynomial root-finding is numerically unstable for larger matrices; libraries use the QR algorithm with similarity transformations instead.

Common mistakes

• Treating x = 0 as an eigenvector — it satisfies Ax = λx for every λ and so is excluded by definition; only the *eigenvalue* may be zero.
• Writing |λI − A| = 0 and |A − λI| = 0 as if the polynomials are identical for odd n — they differ by a factor (−1)ⁿ, which flips signs of coefficients (the roots are the same).
• Assuming a repeated eigenvalue automatically means non-diagonalisable, or assuming every matrix can be diagonalised; the GM = AM test must actually be done.
• Forming P with eigenvectors in one order and D with eigenvalues in another — the k-th column of P must match the k-th diagonal entry of D.
• Forgetting to normalise (and orthogonalise within a repeated eigenspace) when an *orthogonal* P is demanded for a symmetric matrix, and then using Pᵗ in place of P⁻¹ when P is not orthogonal.
• Comparing ρ(A) with ρ([A|b]) carelessly, or reducing the augmented matrix with *column* operations — column operations change the solution set and must not be used on [A|b].
• Skipping the Σλ = tr(A) and Πλ = |A| checks, so a sign slip in the characteristic polynomial goes undetected through the whole eigenvector computation.
• Assuming eigenvalues of A + B or AB are sums or products of those of A and B — they are not, unless the matrices commute and share eigenvectors.

Related topics

Vector spaces and linear transformations · Systems of linear equations and numerical methods · Quadratic forms and conic sections · Group theory · Multivariable calculus: Hessian and optimisation · Probability and statistics: covariance and PCA