How does your phone recognize your face in under a second?
It's not doing what you'd guess. It isn't laying your photo on top of a saved one and checking if the pixels match. Pixels are way too fragile for that, change the lighting a little and every single one shifts. Instead, it boils your face down to a short list of numbers, maybe 20 or 30 of them, and compares those.
That trick has a name: eigenfaces. And once you see how it works, it stops looking like machine learning magic and starts looking like something you already know: baking.
A photo is secretly a giant list of numbers
Zoom into any grayscale photo and you'll find it's just a grid of brightness values. Black is 0, white is 1, gray is somewhere in between. Line those values up in a row and you get a single long list.
That list is what mathematicians call a vector. Not an arrow this time, just an ordered stack of numbers. An 8×8 thumbnail gives you 64 numbers. A modest 100×100 photo gives you 10,000. Every one of those numbers is its own "direction," its own dimension.
So a tiny photo isn't just a picture. It's a single point sitting in a space with thousands of dimensions.
Drag the resolution and watch the same face turn into more, smaller numbers. Hover a cell to read its exact value.
Drag the resolution up and watch the pixel count climb. That's the first uncomfortable fact about images: even a small photo lives in an enormous space. Comparing two faces by checking all 10,000 numbers one by one would be slow, and worse, it would be wrong, because tiny shifts in lighting or pose scramble individual pixels without changing who the person is.
We need a smarter set of numbers to compare. That's what eigenfaces give us.
Most of that giant space is wasted
Here's the saving grace: real faces don't scatter randomly through that 10,000-dimensional space. Every face has two eyes, a nose, a mouth, all roughly in the same places. That means real face-photos cluster tightly into a much smaller, much more useful region.
Think of it like recipes. Every cake recipe is technically a huge list of possible ingredients and amounts, flour, sugar, butter, seventeen kinds of extract nobody uses. But real cakes only vary along a few meaningful "dials": how sweet, how moist, how rich. Nobody needs the full ingredient list to describe the difference between two cakes. A handful of dial settings does it.
Faces work the same way. Before we can find those dials, though, we need a base recipe to dial away from. We get that by averaging a stack of real faces together, pixel by pixel.
Each square below is one training face. Slide n and watch the average below settle into a smooth, featureless blur, that blur is the mean face.
Watch the average blur into something smooth and generic as more faces join. That blur is the mean face, and it's the plain, unflavored starting point every other face gets compared against. In the math, we call it , and finding it is the very first step of the whole eigenface technique:
Once we have that base recipe, every individual face gets described not by its raw pixels, but by how it differs from the average. That difference is where the interesting variation lives.
Finding the directions that matter
So we've got a bunch of "face minus average" vectors, each one describing how one person's face deviates from the mean. But here's the question that actually matters: which deviations are common across many faces, and which are just noise?
That's exactly what PCA (principal component analysis) answers. Feed it that pile of difference-vectors, and it hands back a small set of directions, ranked by how much of the total variation each one explains. Each direction is itself a face-shaped pattern (which is why they're called eigenfaces, not eigenvectors), and instead of describing a face with 10,000 pixel values, you describe it with a handful of numbers saying how strongly each dial is turned.
The math behind PCA is an eigenvalue problem on the covariance matrix of all those difference-vectors:
Solve it and each eigenvector is one eigenface, and its eigenvalue tells you how much of the total variation across all faces that direction accounts for. Bigger eigenvalue, more important dial.
One neat wrinkle: with only a handful of training photos but thousands of pixels, computing directly is overkill. There's a trick: build the much smaller matrix from your training differences instead (one row/column per photo, not per pixel), find its eigenvectors, and map them back. Same eigenfaces, a fraction of the work. That's the trick that made this practical back in 1991, and it's exactly what computed the eigenfaces used in every visualization on this page.
One dial, one expression
Let's turn just one of those dials and watch what happens.
This single number dials one PCA direction up or down, the reconstructed face changes continuously as you drag it.
That grid up top is the reconstructed face: mean face, plus one single number times one single eigenface. Drag the slider and the face doesn't glitch or jump between discrete states, it morphs continuously, because you're sliding smoothly along one real direction of variation.
On this toy dataset, one of the directions PCA discovered on its own happens to line up almost exactly with smiling versus frowning, no one told it to look for a mouth. It found that pattern purely from which pixels tend to move together across many faces. Hit the jump buttons and you'll land on the exact coefficient our two example faces sit at along this axis.
We write a reconstructed face as the base recipe plus a weighted eigenface:
where is that one coefficient. One number, one continuously adjustable expression.
Turning a face into a handful of numbers
Real eigenface systems don't use just one direction, they stack several, each contributing its own coefficient:
More directions, better reconstruction... and here's the part that actually matters for recognition.
Add eigenfaces one at a time and watch the reconstruction close in on the target, using only k numbers instead of all 64 pixels.
Slide k up and watch the reconstruction close the gap with the target, using only k numbers the whole time, not the 64 raw pixels sitting right next to it. Notice how fast the error drops: the first couple of eigenfaces already capture most of the variance, because they're ranked by how much they matter.
This is the entire point. Once every face is reduced to a short coefficient list like , recognizing someone stops being an image-comparison problem and becomes a simple distance calculation between two short lists of numbers. Two faces are "the same person" if their coefficient lists sit close together, and that comparison takes microseconds instead of comparing every pixel under every possible lighting condition.
The short version
A photo is a giant vector, one number per pixel, living in a space with thousands of dimensions. Real faces don't fill that space randomly, they cluster around a much smaller set of meaningful directions. PCA finds those directions automatically by averaging a bunch of faces (the mean face), then discovering which deviations from that average repeat most often across the group (the eigenfaces). Any face can then be rebuilt, approximately, as the mean face plus a short list of coefficients along those directions.
That short list, not the raw pixels, is what your phone actually compares. Fewer numbers, faster comparisons, and shockingly robust to the lighting changes that would break pixel-by-pixel matching completely.
The technique dates back to 1991 (Turk and Pentland), decades before "AI" was a headline word, and it still underlies the intuition behind modern face recognition today: reduce, don't compare raw pixels.
All visualizations are interactive React components computed entirely in your browser. The eigenfaces shown are real PCA output (power iteration via the Turk–Pentland Gram-matrix trick) on a synthetic 8×8 face dataset, not illustrative fakes. No libraries beyond React.