Every matrix has a direction it drags everything toward
Micro-lessons
Multiply almost any vector by a matrix, over and over, and it swings to one special direction and stays there. That direction is the dominant eigenvector — and the dead-simple loop that finds it, power iteration, is the seed of PageRank, PCA, and Lanczos. Full page →
Eigenvectors are the directions a matrix doesn’t rotate
A matrix generally turns a vector. The exceptions are its eigenvectors: for those, A v = λ v — the vector keeps its direction and only gets scaled by λ. Most inputs get rotated; eigenvectors are the axes the transformation leaves alone.
Watch every arrow fall into the same line
Pick any starting direction, then press Apply A repeatedly. Each press multiplies by A and rescales to unit length. The arrow homes in on the dominant eigenvector (dashed), and the Rayleigh quotient v·Av climbs to the largest eigenvalue:
Each press multiplies by A and renormalizes. The arrow swings toward λ₁’s eigenvector (dashed), and v·Av climbs to 3.00. The other eigen-direction shrinks by λ₂/λ₁ = 0.33each step — that ratio is the convergence rate.
Write your start vector in the eigenbasis: v = c₁e₁ + c₂e₂ + …. Applying A scales each piece by its own λ, so after k steps the i-th piece carries λᵢᵏ. The largest eigenvalue outruns all the others exponentially — everything else fades like (λ₂/λ₁)ᵏ. That single ratio is the convergence rate: a big spectral gap converges in a few steps, a small one crawls.
Why this is a foundation, not a trick
The same three lines are everywhere. PageRank is the dominant eigenvector of the web’s link matrix. PCA keeps the top eigenvector of the covariance matrix. And Lanczos / Krylov methods — the workhorses behind the site’s spectral and quantum-chemistry pages — are what you get when you keep the whole trail {v, Av, A²v, …} instead of throwing it away. Power iteration is the atom those methods are built from.
Say it, don’t just nod
A has eigenvalues 3 and 1. You run power iteration from a generic start. What does it converge to, and how fast?
It converges to the eigenvector belonging to λ₁ = 3 (the largest in magnitude). The Rayleigh quotient approaches 3.
The error shrinks by the ratio λ₂/λ₁ = 1/3 every step — geometric convergence. Ten steps kill the stray component by (1/3)¹⁰ ≈ 1.7 × 10⁻⁵. Shrink the gap (say eigenvalues 2.9 and 3) and the same ten steps barely move — the spectral gap is everything.