How Calculus Is Used in Machine Learning

How does a machine learning model actually learn anything?

Not metaphorically. Mechanically. What is the model doing, line by line, that makes it get better at its job?

The honest answer is a little deflating: it's not reasoning, it's not understanding, it's rolling downhill. And the tool it uses to feel which way is downhill is calculus. Specifically, one calculus concept, applied over and over, at a scale that would make your high school teacher's head spin.

A hiker in thick fog

Picture a hiker standing on a hillside, trying to reach the bottom of a valley. Except there's a catch: thick fog. She can't see the valley, can't see the bottom, can't even see more than a few feet in any direction.

What can she do? She can feel the ground under her feet. Is it sloping left? Sloping right? Steep or gentle? So she picks the steepest way down, takes a step, and feels again.

That's it. That's the whole strategy. Feel the local slope, step downhill, repeat.

Now replace "hiker" with "machine learning model," and replace "valley" with "how wrong the model's predictions are." That quantity, how wrong the model is, is called the loss. Training a model means finding the dial settings, the weights, that make the loss as small as possible. And just like the hiker, the model can't see the whole landscape at once. All it can do is feel the local slope and take a step. That local slope is a derivative.

What is a derivative, really?

A derivative is just a slope. Not a slope of a straight road, but the slope of the ground right under your feet, at one exact point, on a curve that's constantly bending.

Here's a toy version of a loss. Say a model has one adjustable number, a weight $w$ , and its loss is:

L(w) = (w - 3)^2 + 1

This bottoms out at $w = 3$ , where the loss is $1$ . Everywhere else, the curve tilts one way or the other. The derivative, written $L'(w)$ , tells you exactly how much it tilts, at that specific $w$ .

Drag the dot along the curve. The tangent line is the derivative, and its sign tells you which way is downhill.

Slope is negative → step right (increase w) to go downhill.

w = 1.10

Loss L(w)4.610

Slope L′(w)-3.800

Drag that dot around. Notice two things. First, the tangent line's steepness is the slope value shown below, nothing mystical about it. Second, and this is the part that matters: whenever the slope is positive, the ground goes up to the right, so downhill is to the left. Whenever it's negative, downhill is to the right. The sign of the derivative points you toward the minimum, automatically.

We write the derivative of $L(w) = (w-3)^2 + 1$ as:

L'(w) = 2(w - 3)

You don't need to memorize where that formula comes from right now. What matters is what it does: plug in any $w$ , and it hands you the direction and steepness of the ground there.

Turning a slope into a strategy

Knowing the slope is nice. But a hiker doesn't just want to know which way is down, she wants to actually get to the bottom. So here's the rule, called gradient descent:

w_{\text{new}} = w_{\text{old}} - \eta \cdot L'(w_{\text{old}})

In words: take a step opposite the slope, and scale that step by a number $\eta$ (the Greek letter eta), called the learning rate. Big slope, big step. Small slope, small step. Wrong direction is never an option, because you always subtract the slope, never add it.

Each step moves w opposite the slope, scaled by the learning rate. Push the rate too high and the ball overshoots the valley entirely.

drag the ball, then press play

lr = 0.25

Step0

Current w-1.500

Loss L(w)21.250

Play with the learning rate. At a sensible value, the ball glides down and settles at the bottom, exactly where $L'(w) = 0$ . But crank the learning rate too high, and watch what happens... the ball doesn't creep toward the minimum, it launches past it, lands somewhere worse, and launches again, further this time. Too big a step size turns "walk downhill" into "fly off the map."

This single update rule, repeated millions of times over millions of weights, is what "training" a neural network means. There's no other secret ingredient. It's the hiker's rule, at scale.

But models don't have just one dial

Real models aren't one weight, they're chained stacks of simple functions, layer feeding into layer. The output of layer one becomes the input of layer two, and so on. So here's the question that should be bugging you: if the loss depends on a weight buried three layers deep, how do you find that weight's slope?

You can't just look at layer three in isolation. A nudge to a weight in layer one ripples forward through every layer after it. Fortunately, there's a rule for exactly this: the chain rule. It says that the slope of a chain of functions is the product of each stage's local slope.

Two tiny layers, chained together. Each has its own local slope — multiply them and you get the slope of the whole thing.

x = 0.40

Chain rule: f′(h)·g′(x)7.200

Numerical dz/dx7.200

Slide $x$ back and forth. Watch the two local slopes, $g'(x)$ for the first layer and $f'(h)$ for the second, multiply together into one combined number. Then look at the bottom graph: that combined number is exactly the tangent slope of the whole chained function, $z(x)$ . Two completely different ways of computing the same slope, and they agree every time.

This is precisely what backpropagation does inside a neural network. It walks backward through the layers, multiplying local slopes stage by stage, until every single weight, no matter how deeply buried, has a slope assigned to it. It's the chain rule, running at the scale of billions of weights.

What if there's more than one dial at once?

Real models don't adjust weights one at a time either. They adjust all of them simultaneously, every training step. So the "slope" isn't a single number anymore, it's a whole bundle of them, one per weight. That bundle has a name: the gradient, written $\nabla L$ .

\nabla L = \left( \frac{\partial L}{\partial w_1}, \frac{\partial L}{\partial w_2} \right)

Each piece, $\frac{\partial L}{\partial w_1}$ , just asks "if I nudge $w_1$ a tiny bit and freeze everything else, how much does the loss change?" Do that for every weight, and you get a vector, an arrow, not just a number.

Now there are two dials, w₁ and w₂. Drag the point around the bowl. The gradient (orange) always points uphill — so we step the opposite way (green).

Position (w₁, w₂)(5.50, 6.00)

Loss22.250

Gradient ∇L(7.00, 6.00)

|∇L|9.220

Drag the point around this bowl-shaped loss surface. That orange arrow, the gradient, always points toward higher loss, straight up the steepest slope. So gradient descent, in the multi-dial world, is still the same idea: step opposite the gradient. That's the green arrow. Notice it shrinks as you approach the center. Near the bottom, the ground is nearly flat in every direction, so the steps naturally get smaller too, right when you want them to.

This is exactly how a real network trains, except instead of two weights, there might be two billion. The gradient is still just a list of partial derivatives, one per weight, and gradient descent still just walks opposite it, over and over, in every dimension at once.

The short version

Machine learning models don't understand anything. They minimize a number, the loss, by repeatedly asking "which way is downhill from here?" That question is answered by a derivative. When the loss depends on a long chain of layers, the chain rule lets you compute every layer's derivative efficiently, that's backpropagation. When there are many weights at once, those derivatives bundle into a gradient, and the model steps opposite it, again and again, until the ground goes flat.

Underneath every model you've ever used, that's the whole engine: slopes, chained together, followed downhill.