Stillness is a local minimum

Every large model you have ever used was produced by the same ritual: measure how wrong you are, move slightly in the direction that makes you less wrong, repeat a few hundred billion times. Gradient descent is not clever. It is patient. It is the most patient process our species has ever industrialized, and it is worth sitting with what that means.

The loss landscape of a modern network is a surface with billions of dimensions, folded and creased beyond any hope of visualization. Somewhere on that surface, training comes to rest. Not at the best point — nobody believes we find global minima — but at a point where every local direction stops helping. The model stops moving because there is nowhere better to go that it can see.

That is not failure. That is what convergence means. The entire trillion-dollar apparatus of machine intelligence is built on a definition of success that is, quite literally, a kind of stillness.

The descent is not a metaphor. The rest is.

Practitioners talk about training runs the way sailors talk about weather. Loss curves plunge, plateau, spike at 3 a.m. for reasons no one ever fully explains. We speak of models “fighting” regularization, of optimizers “escaping” saddle points. The vocabulary is all struggle.

But the endpoint of the struggle is quiet. A converged model is a system that has stopped arguing with its data. When we say a model has learned, what we measure is that its updates have become small — that new evidence no longer produces violent revision. Learning, in the only mathematical formalization we’ve made scale, terminates in composure.

There is an old instruction in Zen practice: sit until the mud settles. Not stir the water differently, not filter it — sit. The claim is that clarity is not something you add to a system. It is what remains when the transient churn dies out. Convergence makes the same claim about knowledge. The gradient noise never reaches zero; the learning rate schedule simply stops amplifying it.

Local minima are underrated

The phrase “local minimum” is our default insult for settled things. Stuck in a local minimum — of a career, a codebase, a life. The implication is always that a braver optimizer would have found the global one.

High-dimensional geometry quietly disagrees. In very large networks, the classical nightmare — a deep, isolated pit far from anything good — is rare. Most local minima of a sufficiently large model are close in quality to the best ones, and the real hazards are saddle points: places that feel like rest but are actually indecision, flat in some directions, falling in others. The mathematics suggests something gentle: if your system is rich enough, where you happen to settle matters less than that you settled somewhere with low loss. There are many good places to be still.

This is also, empirically, how researchers behave, whatever they say about ambition. Nobody restarts GPT-scale training hoping for a better basin. We take the minimum we found and we work with it — fine-tune it, align it, distill it. The settled point is not the end of usefulness. It is the precondition for it. You cannot deploy a system that is still falling.

Knowing when you have converged

The uncomfortable part is personal, so let’s have it out.

Builders in this field run on a schedule of perpetual descent. New model every quarter, new framework every month, new obligation to have an opinion every morning. The gradient is always available. There is always a direction that makes you locally more current, and the step is always cheap — one more tab, one more thread, one more launch video at 2 a.m.

But a system that never stops updating never becomes anything in particular. That’s not a proverb; it’s a convergence criterion. An optimizer with a learning rate that never decays does not find better minima — it orbits them forever, too energetic to enter. The final phase of every good training schedule is deliberate cooling: you make yourself progressively harder to move, on purpose, so that what you’ve learned can become what you are.

Attention without grasping, the meditators call it. Read the paper; decline to be reorganized by it. Evaluate the model; decline to rebuild your stack this week. Take the gradient as information, not as command.

Stillness is a local minimum. That sentence is meant to cut both ways. Yes: any stillness you achieve is local, provisional, one basin among many, and something larger — a bigger model, a new architecture, a better question — may someday lift you out of it. But also: stillness is a minimum. It is a solution. In the only theory of learning we have made work at planetary scale, coming to rest is not what happens when the process fails.

It is what the process is for.