The Bounded-Choice Cascade

The first time Grit Mile's plan generator ran against live production Bedrock, it took eighteen minutes and thirty-one seconds to build a training plan. Twenty weeks, a hundred and forty-four workouts. Sixty-nine seconds elapsed before even week one appeared. Eighteen and a half minutes is not a latency budget. It's an outage with a progress spinner.

The architecture looked right on paper, which is what made it interesting. Deterministic orchestration in code, agentic reasoning at the leaf — exactly the shape I'd argue for. So my first reaction wasn't "the model is slow," it was worse: did we build plan-gen wrong? That question is expensive enough that I didn't want to answer it from a gut feeling, so I ran a four-lens latency audit before touching a line of code.

One detail from an earlier prototype comes back later, so set it here. A Grit Mile experiment had an AI "coach voice" narrate every workout in a fully pre-computed plan. On days the plan said hold steady, the voice layer produced an empty string. Nothing. Blank narration. Commit 051635d in the history is me decoupling move-proposal from the coach voice and fixing the empty-on-holds bug. It's the whole essay in one blank text box.

The villain is a repair loop

The audit found this: 57% of the entire wall-clock — 631 seconds across twenty-nine calls — was a repair loop. The micro tier would ask the LLM to propose the numbers for a week. Eleven code validators would check those numbers against the rubric. When they failed, the loop re-prompted the model to try again, and again, up to its budget. Twenty-nine of those re-invocations, more than ten minutes of clock, spent re-prompting the LLM back toward numbers the macro layer and the rubric formulas had already computed.

That's the one-line summary of the miss: deterministic code fighting the LLM over values the code already knows.

And the spend bought nothing. After all that repair traffic, the plan still shipped with twenty-seven soft-fail violations. The prompt fails validation 59% of the time — more than half the time the model proposes a week, the code rejects it — and only three of the twenty weeks validated on the first try. The non-LLM layer — all the orchestration, persistence, the actual code — summed to about six and a half seconds. 99.4% of the run was inside fifty strictly-serial Bedrock calls, most of them arguing.

The reflex fix is to swap the model. I tried it, because I wanted the number. Haiku 4.5 through the same pipeline was only 1.6× faster and shipped 2.3× the violations. A weaker model in a broken harness ships a worse plan faster, because the repair loop punishes it twice — once in latency and once in quality. Its softer instruction-following inflated the repair traffic, which is the exact thing eating the clock. The model was never the variable. This is the orchestration-failure bucket from It's Not the Model, made concrete enough to measure in seconds.

Invert it: the reasoner picks, the code binds

The fix is an inversion, and it's small enough to say in one breath.

Today's micro tier is "LLM proposes numbers, eleven validators reject, LLM re-proposes numbers." Invert it to "code composes the numbers, the LLM picks from a bounded menu, code binds the arithmetic."

The primitive I landed on I call the Pick. The LLM emits an archetype_id, some enum slot_choices, and a rationale — and never a number. It selects an archetype like a tempo-interval session from a catalog of named archetypes; it selects a distance from a range the code computed; a function called VdotResolve binds the actual paces, and code binds every rep, every duration, every weekly volume. The model chooses; the code does the math.

This collapses the repair loop at its source, and the reason is almost tautological once you see it. Everything the eleven hard validators police is, by definition, code-computable — that's why code can validate it. So if code composes the numeric scaffold in the first place, the scaffold is born compliant. It cannot breach the rules it was generated from. A wrong Pick becomes a cheap set-membership miss — "that archetype isn't in the catalog" — not a 1,900-token re-emission of an entire week that then has to survive eleven more checks. The thing you were spending 57% of your clock repairing can no longer occur.

One image carries the whole pattern: the reasoner only ever holds the pen on a choice, never on a value.

Two structural invariants fall out of that. Every handoff from a reasoner to the executor is a selection from a curated closed set — or a raw value the executor clamps to a supplied range, never one it repairs. And every handoff from the executor back to a reasoner is a hard envelope the next reasoner can only choose within, never violate. Selection down, envelope up. That pair is what kills the propose-reject-re-propose loop: there's no free value left anywhere for a validator to police.

Four rungs, one shape

The Pick wasn't a leaf trick. Once I had it, I saw the same shape at every altitude of the pipeline. Plan generation has four rungs, and they are all the same move: reason down to a small bounded set, let the LLM pick within it, let deterministic code execute the pick.

The bounds are real and deliberately small. The selection surface is about thirty-five named archetypes — biased small, to the 25–35 range, specifically because that's where LLM selection stays reliable. The adaptation catalog is nine moves, enumerated: load up minor, load down, load up major, swap quality to easy, add a rest day, repeat-or-extend the block, re-anchor pace zones, anchor zones up, re-plan. The macro ramp is +8% a week with a hard 15% week-over-week clamp the code owns. None of those are things I let the model invent. They're things I let the model choose among.

And correct-by-construction buys you more than a dead repair loop — it buys parallelism. The old micro tier was strictly serial because each week's proposal leaned on the prior week's committed actuals. Anchor each week to the macro targets instead, and that serial dependency dissolves; the weeks can fan out concurrently. Projected time-to-first-week drops from sixty-nine seconds to single digits, at which point the bottleneck isn't the model at all — it's how fast the infrastructure can deliver the event.

The tell that this was a real pattern and not me talking myself into one: a separate design effort, the workout-catalog debate, had independently crowned the same Pick primitive "the single most load-bearing mechanism in the whole debate." Two efforts, different rooms, converged on the identical move.

Why moving numbers to code isn't the parrot

There's an obvious objection here, and it's the important one. Isn't a fully deterministic plan with the LLM demoted to picking from menus just... a fancier deterministic plan? Haven't you built a spreadsheet with a chatbot stapled to it?

This is where the empty text box comes back. That prototype's blank narration on holds was the parrot exposed. The voice had nothing to say on a hold day because there was no judgment underneath it to narrate — the plan was fully pre-computed, the LLM was decoration, and decoration over a null decision renders as an empty string. Strava's "Athlete Intelligence" is the shipping version of this: generic comments on your run, valueless precisely because the AI had no hand in shaping the workout it's commenting on. A principle we've written down names the permanent lesson: the prose is never the product; the judgment is.

So the anti-parrot guarantee does not live in the prose layer. It lives in the code/LLM boundary. Moving the numbers into code is correct only because the LLM's judgment moves up — into archetype selection, distance choice, shaping the plan skeleton from a real conversation. Get that backward and you haven't fixed anything; you've rebuilt the valueless prototype at lower latency. A faster parrot is still a parrot.

Which is why the two wins are actually one fact. The rewrite is fast because it kills the repair loop, and it's non-parrot because the judgment moved up to genuine bounded picks instead of being deleted. You don't trade one for the other; the same structural move buys both.

The executor is a role, not a thing

The executor doesn't have to be code — and that's what makes this generalize past one running app.

The thing filling the envelope — the "executor" — doesn't have to be code. It can be a cheaper model. The exact same shape shows up in how I build software, and I've caught myself saying it as a one-liner: don't use Fable 5 to write code — use Fable to tell Opus to tell Sonnet to write code. The frontier model does the sparse, high-judgment reasoning. It narrows to a bounded spec — the groom. The cheaper model executes within that spec — the dispatch. Which, if you've read The Player-Coach Decade, you'll recognize as literally my own control-plane → groom → dispatch workflow. Bounded choice at the top, materialization below, recursing down.

Two substrates — deterministic code in Grit Mile, a cheaper LLM in the build pipeline — identical shape. That's how I know it's a shape and not a trick.

It also reframes what an orchestration failure is. It's symmetric. Under-powering a rung — handing an architecture call to a model too small to make it — is a failure. Over-powering a rung — burning a frontier model on boilerplate — is the same failure: a model working off its rung. I watch Amazon engineers fight over scarce Bedrock capacity, and a lot of the time the real diagnosis is: capacity misallocation — Fable-grade reasoning spent on Sonnet-grade work. The cascade fixes both directions at once, because its whole discipline is matching reasoning cost to judgment density at every rung — so nothing works above or below its level.

The pattern the last two posts were circling

I've written the perimeter of this idea twice without naming the center.

It's Not the Model named the disease: most AI failures aren't capability failures, they're context or orchestration failures. But "orchestration failure" stayed abstract — a bucket you couldn't act on. The Player-Coach Decade named the human's new job: orchestration, decomposition, inspection, running a stack of agents. But it stopped at the job, without the structure it produces.

The Bounded-Choice Cascade is the mechanism sitting under both. It's what an orchestration fix actually looks like when you draw it — selection down, envelope up, recursing through the altitudes. And it's what the player-coach is doing at the top rung: writing intent, and curating the closed sets everything below chooses from. The judgment moved up — into the picks and the set-design — while the factory materializes and recurses beneath it.

It's only as good as the sets

This is not a free lunch. The cascade is exactly as good as the curation of its bounded sets, and it fails in both directions. Make a set too small and you strangle judgment — the model produces something compliant and mediocre because the good answer wasn't in the menu you handed it. Make it too loose and you've reintroduced guess-mode and invited the repair loop right back in the door you just closed it through. There is no setting where the sets curate themselves.

But I'd argue that curation cost is the point, not the tax. Deciding it's about thirty-five archetypes and not two hundred, deciding the nine adaptation moves, pinning the +8%/15% ramp constants — that is the domain work. That's the coaching expertise, encoded. It's precisely where the moat lives, so it's effort spent exactly where you'd want to spend it. The cascade doesn't eliminate the hard thinking; it relocates it from per-request firefighting to up-front set design, which is a much better place to keep it.

And I want to be careful about the size of the claim. Two worked examples across two substrates is enough to name a shape — it is not a proven universal law, and I'm not going to dress it up as one. This is "here's a shape I keep seeing," not a theory of everything.

The move

The reusable instinct is a single question:

When you catch a reasoner and a validator fighting over a number — one proposing, the other rejecting, round after round — stop. That fight is the tell. Ask what the executor below could compose instead, and hand the reasoner a menu rather than a blank.

And at any rung, ask: is the model holding the pen on a value, or on a choice? If it's a value, you've got a repair loop or a parrot queued up somewhere downstream, whether you can see it yet or not.

I keep finding these in the same place: anywhere judgment and execution got collapsed into a single call, instead of interleaved down the altitudes. Pull them apart, and the eighteen-minute plan should collapse to single-digit seconds — because you finally stopped asking the model to hold a pen that was never its to hold.