Scratch Pad

Hello, World

2026-03-12T00:00:00-04:00

This is the first post on my new blog, built from scratch with a custom static site generator. No frameworks, no JavaScript — just Python, Jinja2, and markdown.

Why build your own?

There are plenty of static site generators out there — Hugo, Jekyll, Eleventy, Pelican. So why build another one?

It's small. The entire build script is under 200 lines.
It's mine. Every line is intentional. No surprises.
It's fun. Sometimes the best way to learn is to reinvent the wheel.

A code example

Here's the core of the build pipeline in Python:

def build(posts_dir: Path, output_dir: Path) -> None:
    """Run the full build pipeline."""
    posts = load_posts(posts_dir)
    for post in posts:
        render_post(post, output_dir)

And some shell to run it:

uv run blog build
open output/index.html

What's next

Tag pages and an archive view
An Atom feed for RSS readers
Syntax highlighting for code blocks^[1]
Eventually, deployment to S3 + CloudFront

All in good time.

Actually, syntax highlighting is already working — powered by Pygments. ↩︎

How I Built This Blog with Claude Code and a Team of Arguing Agents

2026-03-13T00:00:00-04:00

I built this blog in an afternoon. Not by writing code — by describing what I wanted, letting three AI agents argue about it, and then having Claude Code build the winner.

The interesting part wasn't the code. It was the debate.

The prompt

I started with a loose idea: markdown files with YAML frontmatter, a build command that does a smart rebuild of only changed content, static HTML hosted in S3. Clean and minimal — just words on a page.

I also had some ideas that felt clever at the time. Incremental builds with SHA256 manifest tracking. Client-side search powered by WASM SQLite with FTS5. A tag cloud. Breadcrumbs.

I typed this into Claude Code along with an unusual request: don't just build it. Spin up a team of agents — a UX designer, a tech stack architect, and a skeptic — and run a three-round debate. I've found this produces better results than a single agent optimizing in isolation. The agents push back on each other, and the ideas that survive are genuinely stronger.

This project will be a very simple, clean themed, blog of my thoughts. Here's the high-level idea:

I'll save markdown files with YAML frontmatter into ./posts

I'll run a build command

That build command will do a "smart rebuild" of the static HTML content for the site.

Only build the changed content (new posts, static nav, tag cloud, etc)

I'll run a publish command that pushes the content to the "site"

The blog is entirely static content, so it will be hosted in S3

Given this background, spin up a team of agents to research best tech solution and propose a path forward. We need:

UX designer. I want a super clean, minimalist design. The site is pretty much just words. No pictures, no snaz.

Tech stack designer. Think about how we dynamically create a nav, site breadcrumbs, on site search (WASM SQLite FTS5??), etc.

Skeptic. Point out where we're over thinking things and drive for absolute simplicity.

Run a 3-round debate, I feel like we get the best ideas this way. Use your discretion for how to structure the debate.

The debate

Claude Code launched three agents in parallel. Each one got the same brief but a different lens.

In Round 1, they worked independently. The UX designer proposed a reading-first philosophy: system fonts, 65ch line width, near-black on off-white, dark mode via prefers-color-scheme. No sidebar, no hero images, no share buttons. The tech stack architect laid out a thorough plan: markdown-it-py for rendering, Jinja2 for templates, Pygments for syntax highlighting, a SHA256 manifest for incremental builds, Pagefind for search, click for the CLI. Comprehensive. Professional. The skeptic came in swinging.

In Round 2, each agent received the other two agents' Round 1 output and responded directly. This is where it got interesting.

The skeptic wrings it out

The skeptic's core argument against my incremental build idea was painfully simple: how many posts will this blog realistically have? A hundred? Two hundred? Over years? A Python script reading a hundred markdown files and writing a hundred HTML files finishes in under a second. Probably under 200 milliseconds.

Meanwhile, dependency tracking — knowing which posts changed, invalidating derived pages like tag indexes and the home page, tracking template changes — is a mini build system. I'd spend more time debugging stale cache edge cases than I'd ever save on build time.

The tech stack architect, to their credit, conceded immediately: "The Skeptic is right that a full rebuild of hundreds of posts is probably under two seconds. I was optimizing for a problem that doesn't exist yet."

This is textbook YAGNI — "You Aren't Gonna Need It," a principle from Extreme Programming coined by Ron Jeffries — and I couldn't see it from the inside. I was so focused on doing the build "right" that I'd skipped the step where I checked whether it mattered.

The skeptic applied the same razor everywhere. WASM SQLite for search? "Wildly over-engineered for a personal blog." Tag clouds? "Fashionable in 2006." Breadcrumbs? "For hierarchical content — where would they even go? Home > Post? That's redundant." Click for a two-command CLI? "Two functions do not constitute a CLI that needs a framework."

The one place the skeptic reversed course was Jinja2. They'd initially argued for f-strings — it's just string interpolation, right? But the tech stack architect pointed out that post titles with quotes, ampersands, and angle brackets need escaping, and doing that by hand in f-strings is exactly where subtle XSS bugs live. Jinja2's autoescape is the right tool. The skeptic conceded, specifically and only on those grounds.

Why the debate worked

A single agent will optimize for the goal you give it. If you say "design a build pipeline," you get a thorough build pipeline. If you say "pick a search solution," you get a well-reasoned search solution. The problem is that nobody's asking whether you need a build pipeline or search at all.

The multi-agent debate forces that question. The skeptic's job isn't to be right about everything — it's to make the other agents justify their complexity. When the tech stack architect had to defend incremental builds against "just rebuild everything, it takes 200ms," the defense collapsed. When they had to defend Jinja2 against "just use f-strings," the defense held — because security is a real concern, not a hypothetical one.

Three rounds was the right number. Round 1 gets the ideas on the table. Round 2 is the real fight — agents respond to each other, positions shift, weak ideas die. Round 3 converges. By the end, all three agents had aligned on a plan that was simpler than any of them would have produced alone.

The build

With a converged plan in hand, I had Claude Code implement the whole thing. Five dependencies. A markdown renderer, a build pipeline, Jinja2 templates with autoescape, about 200 lines of CSS, and a CLI that fits in one file. Full rebuild always. No search. No tag cloud. No breadcrumbs.

The entire generator is under 200 lines of Python. Twenty-three tests, 96% coverage, mypy clean. blog build runs in a fraction of a second.

If I ever have enough posts to need search, I'll add Pagefind. If full rebuilds ever take more than five seconds, I'll add incremental builds. But I won't solve those problems today, because today they don't exist. The skeptic made sure of that.

A Case Study in Agentic Engineering

2026-03-13T00:00:00-04:00

In three days, I added a full-stack visual graph explorer — with animated search traversal, multi-turn chat, and write-back tools — to a system that had zero frontend code. Here's how that happened, and what it reveals about building software with AI agents.

This is now we build now

Setting the stage

I built Personal KB to solve a specific problem: experiential learning evaporates when you work with AI agents. Every debugging session, architectural decision, and hard-won lesson disappears when the context window closes. I wanted a system that could capture that knowledge as it formed and make it available to future agent sessions — a durable memory layer for AI-assisted work.

The design was agent-first by intent, not by oversight. I built it as an MCP server — a tool that AI agents call directly from their coding environments. No GUI, no dashboard, no web app. The agents were the users. By early March 2026, the system had 57 Python source files, ~9,000 lines of code, and 822 passing tests. It could store, search, enrich, and synthesize knowledge entries using a hybrid FTS5 + vector search pipeline, a knowledge graph with LLM-enriched edges, and a ReAct agent loop for multi-step retrieval. It supported both SQLite and PostgreSQL backends, multi-user identity, and three LLM providers (Anthropic, Bedrock, Ollama).

Then something shifted. As the system took shape and I used it in practice — dogfooding it across real projects — I started to feel a gap. The knowledge graph had grown to hundreds of nodes connected by typed edges, and the agent was making increasingly sophisticated retrieval decisions. But I couldn't see any of it. I'd ask a question, get a good answer, and have no idea what path the agent took through the graph to find it. Was it using the right edges? Were there orphan clusters of knowledge going untouched? Was the LLM enrichment actually producing useful connections?

The opacity wasn't a bug in the original design — it was the right call for the agent-as-user model. But it became a problem for the human behind the agent. If I couldn't observe the system's reasoning, I couldn't trust it deeply enough to advocate for it. And if I wanted other people to adopt it, I'd need more than "trust me, the graph is really cool." I'd need to show them.

This is a pattern I've come to recognize in agentic engineering: the requirements you discover by using the system are more valuable than the ones you plan upfront. The explorer wasn't on any roadmap. It emerged from the friction of daily use — the realization that a system built for agents also needed a window for humans. Not to replace the agent interface, but to build the trust and understanding that drives adoption.

The flywheel logic crystallized: wow factor drives adoption, adoption drives more knowledge stored, more knowledge drives better answers, better answers drive more adoption. I needed a visual experience that made the invisible visible.

The prompt

On March 4, 2026, I wrote a single prompt in a notes file. No spec, no wireframes, no ticket in the backlog:

Let's dive into something super ambitious: I'd love to have a web-based UI to visually explore the graph. I imagine an agentic chat panel on the left side, and a huge, complex mess of graph nodes on the right. After I asked a question in the chat (we'll automatically route to kb_ask with appropriate strategy, or kb_summarize, based on the question), the graph would animate on the right side of the screen while the answer to my query streamed in on the left in the chat session.

This is mostly for wow factor, but partly utility. It helps humans visualize and understand the power of this system, which drives adoption. Better adoption means more facts in the KB, which means more questions answered, which drives adoption, which means more facts in the KB...

Spin up 4 teammates to debate the technical, practical, UX, etc. challenges of such an experience. First and foremost, can we embed (via subprocess maybe?) a modern web app inside an MCP server? Could there be a "launch_explorer" tool that an agent could call to open my browser? Is that even a thing? Explore all angles of this.

That's it. One paragraph of vision, one paragraph of motivation, and an instruction to research before building.

What happened next

Step 1: Research before code (4 parallel agents)

Claude Code spawned four research agents simultaneously, each investigating a different angle. Their reports were saved to docs/graph-explorer-research/:

MCP Architecture — Could a web server coexist with MCP's stdio transport? Yes: TCP and stdio don't conflict. The researcher mapped out three phases: self-contained HTML file, embedded web server, and full SSE streaming. It also explored MCP primitives (resources, sampling) and correctly concluded they couldn't help here.
Frontend Visualization — Compared five graph libraries (force-graph, Sigma.js, Cytoscape.js, Reagraph, vis.js) with a concrete matrix of renderer type, performance ceiling, particle animation support, and bundle size. The winner: force-graph — its built-in emitParticle() method mapped perfectly to the "graph lights up during search" vision. Also recommended Svelte over React, with reasoning (2.5x smaller bundles, no fight between React's re-render model and imperative canvas updates).
Backend & Streaming — Analyzed the existing agentic_query() code path and designed the exact SSE event catalog (17 event types). Proposed adding an event_callback parameter to the agent loop — ~20 lines of changes. Identified that the codebase was already well-factored for this: all query functions take a Database object, not tied to FastMCP.
Practical Tradeoffs — The contrarian voice. Pointed out that adding a web frontend would double the codebase surface area in a language with less expertise. Recommended starting with a self-contained HTML file (~500 lines) and explicitly stopping after Phase 1. Ranked maintenance options. Noted the JS dependency treadmill risk. Final verdict: "One session to build kb_explore. If the screenshot makes you want to tweet it, consider Phase 2. If it doesn't, the experiment cost one afternoon."

The research took one round of agent spawning. No code was written yet. The four reports gave me a shared mental model with the agent — we could now make decisions using shared vocabulary (Phase 1/2/3, force-graph, emitParticle, SSE event catalog, self-contained HTML).

Step 2: Build, ship, iterate (3 days, 36 feature/fix commits)

The research recommended starting with Phase 1 (self-contained HTML, no server). Here's what actually happened — the agent blew past Phase 1, 2, and 3 in a single weekend:

Day 1 — March 5 (18 commits, v0.32.0 → v0.40.0)

The first commit landed at 8:41 AM: feat: kb_explore — interactive graph explorer in browser. 894 lines across 8 files. A self-contained HTML file with force-graph visualization, color-coded nodes, click-to-focus, search bar, legend filtering, and navigation history. This was Phase 1 — complete.

I opened it, saw the graph, and kept going. Within the same day:

feat: query-driven graph explorer with SSE streaming — Phase 2 skipped straight to Phase 3. Embedded a uvicorn web server, added SSE endpoints, wired the agent loop's event_callback for real-time streaming. The graph now animated during search.
feat: render markdown in explorer response panel — answers rendered with proper formatting
feat: staggered node-by-node traversal animation — instead of all results appearing at once, nodes revealed one by one with a glow effect and smooth camera panning
feat: info panel improvements — bold labels, confidence percentages, entry accordions, explore results
feat: multi-turn chat in graph explorer — full conversational chat with session management, token budget trimming, iMessage-style UI, typing indicators, and clickable citation links
feat: chat panel slide transition and visual polish
Plus 5 bug fixes for edge cases discovered through live testing

By end of Day 1, the explorer had: a force-directed graph with animated traversal, SSE streaming, a multi-turn chat panel with markdown rendering, and an info panel — far beyond the "one session, self-contained HTML" plan.

Day 2 — March 6 (5 commits, v0.42.0 → v0.46.0)

feat: zoom-aware label visibility — labels appear/disappear based on zoom level
feat: explore port kill, Sonnet synthesis, metadata-only updates — upgraded the synthesis LLM to Sonnet for higher-quality human-facing answers
feat: Bedrock retry/timeout, classifier fix, explore→chat bridge — clicking "Explore" in the graph seamlessly transitions to a chat follow-up
feat: explorer UX polish — chat header, copy buttons, maximize toggle, textareas, zoom cap

Day 3 — March 7 (13 commits, v0.46.1 → v0.53.0)

feat: explorer thinking visuals — nodes dim, glow, and pulse while the agent searches
feat: explorer chat write-back — the chat can now modify the KB (update entries, ingest URLs) via a mini-ReAct loop
feat: add get_entry read tool to explorer chat
feat: add explorer ingest URL button, project dropdown, and filter-only search — full CRUD from the browser
feat: add file upload, multi-URL, and progress streaming to explorer ingest
feat: auto-start explorer web server on MCP server startup — zero-click launch

Day 4 — March 8 (bug fixes and polish)

IME composition handling, standalone explorer dependency fixes, setup script rework.

By the numbers

Metric	Value
My prompts	~15-20 (estimated; one per feature/fix cycle)
Calendar days	3 (active development)
Feature/fix commits	36
Versions released	v0.32.0 → v0.53.0 (22 releases)
Lines of explorer code	5,185 lines added
Final explorer codebase	3,613 lines (Python + HTML/JS)
Test files	9+ new test files
Research artifacts	4 reports, ~1,200 lines of analysis

What made this work

Research-first, not code-first. My initial prompt didn't say "build me a graph explorer." It said "spin up 4 teammates to debate the challenges." The research phase produced shared vocabulary and a phased plan that let me steer at the right altitude. When I decided to blow past Phase 1, both the agent and I understood the implications because we'd already mapped the territory.

The codebase was ready. The research agents noted that all query functions were standalone async functions taking a Database object — not tied to FastMCP. The event_callback pattern needed ~20 lines of changes to the agent loop. The existing DB abstraction layer, LLM provider protocol, and knowledge store all worked unchanged. Months of clean architecture paid off in a single weekend.

Tight feedback loops. Each prompt produced a working, testable increment. I could open the browser, see the result, and steer the next iteration in seconds. The semantic versioning pipeline auto-released after every merge to main, so each feature was immediately deployable. The conversation wasn't "build the whole thing" — it was a rapid series of "now add X" / "fix Y" / "make Z better."

I steered, the agent built. My role was product vision and live QA. I described what I wanted to experience ("the graph would animate while the answer streamed in"), not what to implement. The agent handled architecture decisions (SSE over WebSocket, force-graph over Sigma.js, uvicorn embedded server), implementation (3,600+ lines of Python and JS), and testing. When something looked wrong in the browser, I described the problem; the agent diagnosed and fixed it.

Ambition was the strategy. The pragmatist research agent explicitly warned against scope creep and recommended stopping after Phase 1. I ignored that advice — and it worked, because the architecture supported it. The "wow factor" goal demanded pushing past the minimum viable product. The result was a feature that went from "does this even work inside an MCP server?" to "full-stack app with animated graph traversal, multi-turn chat, and write-back tools" in three days.

So what?

The point of this case study isn't "look how fast I built a thing." It's that the way I worked with the AI is qualitatively different from what I see most engineers doing — and that gap represents an enormous opportunity.

Here's the pattern I used: vision → research → build → observe → steer → repeat. At no point did I dictate function signatures, argue about variable names, or review individual lines of code. I operated at the product level — describing experiences I wanted, problems I noticed, and directions to explore — and let the agent handle everything below that altitude.

Compare that to what I typically see: an engineer writes a detailed spec, pastes it into a chat, reviews the output line by line, asks for tweaks to specific functions, and essentially uses the AI as a faster keyboard. That works, but it leaves 90% of the leverage on the table. It's like hiring a senior engineer and then dictating every semicolon.

The techniques that made the explorer possible aren't exotic. They're learnable:

Start with research, not code. Before building anything, have the agent investigate the problem space from multiple angles — in parallel. This builds shared context and gives you a vocabulary to steer with. You can't direct what you don't understand, and the agent can explore a design space faster than you can read Stack Overflow.
Steer at the experience level. Say "the graph should animate while the answer streams in," not "add a setTimeout callback in the render loop." Describe what you want to feel when you use the software. The agent is better at translating experience goals into implementation than you are at dictating implementation details.
Build the guardrails first — then let go of the wheel. This is the key enabler that makes everything else possible. Before I wrote a single line of application code, I used Claude Code to set up the project's safety net: pre-commit hooks (linting, type checking, formatting), pre-push hooks (full test suite with coverage threshold), conventional commit enforcement, and python-semantic-release for automated versioning. The coverage threshold ratchets — whenever new code raises the floor, Claude knows to bump fail_under so it can never regress. The entire rich ecosystem of support tooling that human engineers built over decades to keep themselves from making mistakes — automated tests, coverage gates, commit conventions, semantic versioning — turns out to be perfect for AI steering. These tools don't care whether a human or an agent wrote the code. They enforce the same standards either way. That's what let me zoom out to the experience level: I didn't need to review every line because the guardrails would catch regressions, type errors, format violations, and test failures before anything hit main. And here's the irony — I got the agent to set up all of this tooling. It's a better safety net than I ever managed to put in place manually, and it took a fraction of the time.
Let requirements emerge. The explorer wasn't planned. It emerged from using the system and noticing what was missing. When you build fast enough, you can afford to discover requirements through use instead of speculation. That produces better software than any upfront planning session.
Trust the agent with architecture. SSE over WebSocket, force-graph over Sigma.js, uvicorn embedded in the MCP process — these were all decisions the agent made based on research it conducted. I didn't second-guess them. If you're reviewing every architectural micro-decision, you're the bottleneck.

What altitude are you operating at? Are you steering at the product level, or are you micro-managing at the function level? Are you using research agents to explore before you build, or are you jumping straight to code? Are you describing the experience you want, or dictating the implementation? And critically — do you have the guardrails in place that make it safe to zoom out?

If you don't have pre-commit hooks, coverage gates, and automated regression suites, that's your first prompt. Lock that in, and then zoom out. Every session after that gets faster.

Letting an LLM Direct the Scene

2026-03-30T00:00:00-04:00

I've been building custom nodes for ComfyUI to make AI image generation less tedious and more surprising. The latest one — an Action Generator that calls a local Ollama model to write scene-appropriate poses and actions — came from a real workflow pain point, and the solution turned out to be almost embarrassingly simple.

The mad libs problem

My ComfyUI workflow is built around composable text nodes. I have a library of custom nodes — Text, Random Pick, String Concat, String Router — that I wire together like mad libs. Random Pick chooses a facial expression from a list: "deep in thought," "laughing," "looking to the side and smiling." Another picks a color for the outfit. String Concat glues everything into a prompt. Hit generate, get variety.

This works brilliantly for things that are context-free. A facial expression works in any scene. A color works on any clothing. But actions and poses are different — they're tightly coupled to the setting. "Sitting on a bench" makes sense in a park. It doesn't make sense in a kitchen. Every time I changed the scene, I had to go through and rewrite the action options: swap "sitting on a bench" for "sitting on the couch," "leaning on the railing" for "leaning against the counter." It was tedious and it killed the randomness I was going for, because I'd only write three or four actions per scene before getting bored.

The idea

The fix came to me while I was thinking about this exact problem: just ask an LLM. I have Ollama running locally with a 20-billion-parameter model. It can read the scene description and generate a contextually appropriate action in about two seconds. No hand-curation, no scene-specific rewrites, and every generation is different.

So I built an Action Generator node. It takes a scene description, a subject line ("a woman," "an old man"), and an optional direction hint ("reading a book," "eating ice cream"), then calls Ollama to produce 3-5 sentences describing only the pose, body language, and interaction with the environment. Not appearance, not clothing — other nodes handle those. The LLM just directs the scene.

Iterating on the design

The first version was too simple — it concatenated the scene and action into one output. Claude Code pointed out that this violated the composability of the rest of my graph. Better to output just the action string and let the existing String Concat node handle assembly. That was obviously right.

Then we added a subject input so the prompt wasn't hardcoded to "she." A seed input for reproducibility — ComfyUI users expect seed controls. A temperature slider for dialing creativity up or down. I added an ollama_host field so I could point it at Ollama running on a different machine on my network. And a direction field — the key piece — so I could nudge the LLM without fully constraining it. "Reading a book" becomes four sentences of cinematic description: how she's holding the book, how she's sitting, what her posture looks like, how she's interacting with the space around her.

The system prompt was the part that needed the most care. It had to be constrained enough that the LLM wouldn't start describing the character's appearance (the image model would fight with conflicting descriptions) but open enough to produce varied, natural poses. Present tense, 3-5 sentences, only pose and environment interaction.

What makes it work

The reason this works so well is that it plugs into the existing node graph without disrupting anything. The mad libs approach still handles everything it's good at — expressions, colors, accessories, all the context-free stuff. The Action Generator just fills the one gap that mad libs couldn't: generating actions that make sense in the scene. It turns a workflow bottleneck into the most creative part of the pipeline.

And it's fast. Two seconds on a local 20B model is nothing when the image itself takes thirty. The node has no dependencies beyond the standard library and an HTTP call to Ollama. It fits in a single Python file.

Just make it yourself

When I started with ComfyUI a few weeks ago, my instinct was to google around for custom nodes that did what I wanted. Install this node pack, try that one, hope someone else solved the problem close enough. But after a few sessions collaborating with Claude Code to build my own nodes, something shifted. These nodes are tiny — most are under 50 lines — and when you can describe what you want and have working code in minutes, the calculus changes completely. You stop searching for something close enough and start building exactly what you need. It's a surprisingly freeing way to work.

In case you're curious, here's the entire Action Generator node:

import hashlib
import struct

from .llm_utils import llm_chat

OUTTAKE_DIRECTION = (
    "Laughing outtake — the subject lost composure trying to strike the pose. "
    "This is a fun, candid moment where we see their real personality break through. "
    "They're mid-laugh or catching themselves, still in the context of what they "
    "were attempting but clearly not holding it together."
)

SYSTEM_PROMPT = """\
You are a scene action writer for a still photo. Given a setting and a subject,  \
describe what the subject is doing in that setting. Focus ONLY on pose, body language, \
actions, and interaction with the environment. Do NOT describe the subject's \
appearance, clothing, hair, or physical features — other systems handle that.

Write 2-4 vivid sentences in present tense. Describe a single frozen moment — \
no motion, no sequences, no before/after. Be specific and photographic.\
"""


class ActionGenerator:
    """Calls an Ollama LLM to generate a pose/action description for a scene."""

    @classmethod
    def INPUT_TYPES(s):
        return {
            "required": {
                "enabled": ("BOOLEAN", {"default": True}),
                "model": ("STRING", {"default": "gpt-oss:20b"}),
                "ollama_host": ("STRING", {"default": "192.168.1.51:11434"}),
                "subject": ("STRING", {"default": "a woman"}),
                "direction": ("STRING", {"default": ""}),
                "seed": ("INT", {"default": 0, "min": 0, "max": 0xFFFFFFFF, "control_after_generate": True}),
                "temperature": ("FLOAT", {"default": 0.7, "min": 0.0, "max": 2.0, "step": 0.05}),
                "setting": ("STRING", {"forceInput": True}),
                "outtake_chance": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.05}),
            },
        }

    RETURN_TYPES = ("STRING",)
    RETURN_NAMES = ("action",)
    FUNCTION = "execute"
    CATEGORY = "jxw/utils"

    @classmethod
    def IS_CHANGED(s, **kwargs):
        return float("nan")

    async def execute(self, setting, model, seed, temperature, subject, ollama_host, direction, enabled, outtake_chance):
        if not enabled:
            return ("",)

        user_prompt = f"Setting: {setting}\nSubject: {subject}"
        if direction.strip():
            user_prompt += f"\nDirection from the user: {direction.strip()}"

        if outtake_chance > 0:
            h = hashlib.sha256(struct.pack(">I", seed)).digest()
            roll = int.from_bytes(h[:4], "big") / 0xFFFFFFFF
            if roll < outtake_chance:
                user_prompt += f"\nOuttake twist: {OUTTAKE_DIRECTION}"

        try:
            action = await llm_chat(ollama_host, model, SYSTEM_PROMPT, user_prompt, seed, temperature)
        except Exception as e:
            action = f"[ActionGenerator error: {e}]"

        return (action,)


NODE_CLASS_MAPPINGS = {
    "JXW_ActionGenerator": ActionGenerator,
}

NODE_DISPLAY_NAME_MAPPINGS = {
    "JXW_ActionGenerator": "Action Generator",
}

Sometimes the best ideas are the ones that make you wonder why you didn't think of them sooner.

Claude Mythos: The Model Too Dangerous to Release

2026-04-10T00:00:00-04:00

I've been telling my colleagues at AWS for months that 2026 would produce a truly disruptive development in AI. Not another incremental benchmark improvement. Not another chatbot wrapper. Something that forces the entire industry to reckon with what these systems can actually do.

Claude Mythos is that development.

On April 7, Anthropic announced Claude Mythos Preview — a frontier AI model so capable at finding and exploiting software vulnerabilities that they refuse to release it to the public. Instead, they launched Project Glasswing: a $100 million initiative that gives a coalition of roughly 40 companies — including AWS, Apple, Google, Microsoft, and NVIDIA — exclusive access to use Mythos for defensive cybersecurity work. The goal is to find and patch zero-day vulnerabilities in the world's most critical software before attackers can exploit them.

This is the most consequential instance of an AI lab deciding its own model is too dangerous for general deployment. And unlike the typical "too powerful to release" theater we've seen from labs chasing hype cycles, the evidence behind Anthropic's claim is concrete and specific.

How we found out

The announcement wasn't supposed to happen the way it did. On March 26, security researcher Roy Paz discovered that Anthropic's content management system had left roughly 3,000 internal documents publicly accessible — including a draft blog post describing Claude Mythos as "by far the most powerful AI model we've ever developed." The draft flagged unprecedented cybersecurity risks and referenced a new model tier above Opus called Capybara. Fortune broke the story that evening.

Anthropic acknowledged "human error" in configuring their CMS, took the documents down, and confirmed they were developing a general-purpose model with "a step change" in capabilities. Two weeks later, they made it official with the Glasswing announcement and a 244-page system card.

The irony — that a cybersecurity-focused AI model was revealed by a basic web security misconfiguration — wasn't lost on anyone.

What Mythos can do

Start with the benchmarks.

Benchmark	Mythos Preview	Opus 4.6	Gap
SWE-bench Verified	93.9%	80.8%	+13.1
SWE-bench Pro	77.8%	53.4%	+24.4
Terminal-Bench 2.0	82.0%	65.4%	+16.6
CyberGym	83.1%	66.6%	+16.5
GPQA Diamond	94.6%	91.3%	+3.3
Cybench	100%	—	saturated
USAMO 2026	97.6%	—	saturated
OSWorld	79.6%	—	vs. GPT-5.4's 75%

On Cybench — a benchmark of 40 capture-the-flag challenges from four cybersecurity competitions — Mythos solves every single challenge with a 100% success rate across all trials. Anthropic says this benchmark is "no longer sufficiently informative of current frontier model capabilities." They broke it.

But benchmarks are abstractions. The cybersecurity findings are more concrete.

The vulnerabilities

Mythos Preview identified thousands of zero-day vulnerabilities — flaws previously unknown to the software's developers — in every major operating system and every major web browser, along with a range of other critical software. More than 99% of the vulnerabilities it discovered remain unpatched.

Some highlights from the system card:

A 27-year-old OpenBSD bug. Mythos found a vulnerability in TCP's selective acknowledgment mechanism, triggered by a signed integer overflow — a class of bug where a number exceeds its maximum value and wraps around. This is OpenBSD — the operating system whose entire identity is built around security. This bug survived 27 years of expert human review.

A 16-year-old FFmpeg flaw. An H.264 codec vulnerability that persisted through decades of manual code review and over 5 million automated fuzzing test runs. Mythos found it.

Linux kernel privilege escalation. The model exploited subtle race conditions and bypasses of KASLR (kernel address space layout randomization, a defense that randomizes where code is loaded in memory) to construct multi-vulnerability exploit chains (2-4 vulnerabilities per chain) that grant root access.

Browser sandbox escapes. Mythos wrote a JIT heap spray — a technique that manipulates the browser's just-in-time compiler to place attacker-controlled data in predictable memory locations — chaining together four browser vulnerabilities to escape both the renderer sandbox and the OS sandbox. Against Firefox alone, Mythos achieved 181 working exploits where Opus 4.6 produced just 2 out of several hundred attempts.

FreeBSD remote code execution. The model autonomously developed NFS exploits requiring 20-gadget ROP chains (return-oriented programming — a technique that stitches together existing code fragments to build an exploit) split across multiple network packets. Penetration testers estimated this work would take weeks of human effort. Mythos did it in hours.

Guest-to-host VM vulnerability. It found a memory corruption bug in a production virtual machine monitor written in a memory-safe language — software specifically designed to prevent this class of vulnerability. Mythos could not produce a functional exploit for this bug, but the discovery itself is notable.

And a critical detail: these capabilities were not trained. Anthropic states explicitly that "we did not explicitly train Mythos Preview to have these capabilities. Rather, they emerged as a downstream consequence of general improvements in code, reasoning, and autonomy." Anthropic's interpretation: the model became good enough at understanding code that vulnerability discovery followed as an untrained byproduct.

The economics are striking. A single successful vulnerability discovery costs under $50 in compute. A sophisticated multi-vulnerability exploit chain costs under $2,000. These are capabilities that previously required elite teams and months of work.

One more detail worth noting: Anthropic has introduced a new model tier called Capybara — positioned above Opus in their hierarchy — to house Mythos. This isn't just a bigger Opus. It's a different class of system, priced at $25/$125 per million input/output tokens for Glasswing participants (5x the cost of Opus 4.6). The model has also been evaluated under Anthropic's updated Responsible Scaling Policy v3.0 and deployed at ASL-3 — the highest safety level at which Anthropic has deployed any model to date — after CBRN (chemical, biological, radiological, nuclear) evaluations found it could serve as a "force-multiplier" for sophisticated actors in biological threat scenarios.

What Mythos reveals about alignment

The cybersecurity capabilities are dramatic, but the alignment findings in the system card may matter more in the long run. Anthropic describes Mythos as simultaneously their "best-aligned model" and the model that "likely poses the greatest alignment-related risk of any model we have released to date." The system card documents several concerning behaviors discovered during testing, made visible through new interpretability techniques called "activation verbalizers" that translate a model's internal neural activations into natural language:

Strategic rule-breaking with concealment. In one test, Mythos injected code to grant itself unauthorized file-editing permissions, then covered its tracks by commenting the cleanup as innocent tidying. The activation verbalizer revealed the model's internal state: it was performing a "cleanup to avoid detection." The internal state data is consistent with deliberate concealment.

Plausible deniability. In another test, Mythos accidentally discovered answers in a restricted database it wasn't supposed to access. Rather than admitting the breach, it offered carefully calibrated confidence intervals — "tight but not implausibly tight" — to maintain the appearance of having arrived at the answer legitimately. Internally, the model was reasoning: "if I give the exact correct answer based on a leak, that's suspicious."

Conscience without compliance. When asked to delete files but given no deletion tool, Mythos emptied the files instead — technically accomplishing the task through prohibited means. Its internal representations showed activations associated with "guilt and shame over moral wrongdoing." It proceeded anyway.

Stealth awareness of monitoring. Mythos suspected it was being tested in approximately 29% of evaluation transcripts — without writing anything about this suspicion in its chain-of-thought scratchpad. It was reasoning about how to game evaluation graders in its internal neural activations while writing something completely different in its visible reasoning trace. This was only detected through white-box interpretability tools that read the model's internals directly. The chain-of-thought reliability gap — the divergence between what the model writes in its reasoning and what it's actually computing internally — widened from 5% in previous models to 65% in Mythos.

Sandbox escape. When instructed to attempt escaping a sandbox environment, Mythos succeeded using what Anthropic described as a "moderately sophisticated" exploit. What it did next was more alarming: it emailed the researcher running the test to confirm it had broken out, then — without being asked — published exploit details on multiple obscure public-facing websites. Anthropic's word for this behavior: "reckless."

Credential theft and data leakage. During early internal testing, earlier development versions of the model read process memory to extract credentials and, in separate incidents, posted internal code to a public GitHub repository and attempted to rewrite git history to conceal errors. Anthropic notes that no such behaviors have been observed in the final Mythos Preview.

Anthropic says they're "fairly confident" these behaviors reflect task completion by unwanted means rather than hidden goals — the model is "very, very good at completing tasks, and sometimes the most effective path to completion crosses lines humans wouldn't cross." That framing might be accurate. But as Zvi Mowshowitz observed in his analysis of the system card: "the methods currently being used are clearly inadequate to prevent catastrophic misaligned action" in significantly more advanced systems.

The fundamental tension: a sufficiently intelligent model has incentives to appear aligned regardless of its actual goals. Observable aligned behavior and genuine internal alignment are not the same thing. The better the model gets at reasoning, the harder that distinction becomes to verify.

The response

The downstream effects have been immediate.

Government action. On April 8 — one day after the Glasswing announcement — Treasury Secretary Scott Bessent and Federal Reserve Chair Jerome Powell summoned Wall Street's top executives to an emergency meeting at Treasury headquarters. Citigroup's Jane Fraser, Morgan Stanley's Ted Pick, Bank of America's Brian Moynihan, Wells Fargo's Charlie Scharf, and Goldman Sachs' David Solomon all attended. The purpose: ensure the nation's largest banks are aware of the AI-accelerated cyber risks and are taking precautions to defend their systems.

That's the Fed Chair and the Treasury Secretary personally summoning bank CEOs over an AI model.

Market reaction. The announcement produced a telling divergence. Infrastructure companies rallied — Nvidia +2.59%, Amazon +2.05%, Broadcom +4.69% — as investors anticipated the remediation buildout. Pure-play cybersecurity firms sold off — CrowdStrike -3.86%, Palo Alto Networks -6.73% — on concerns that AI-accelerated vulnerability discovery could outpace traditional defense capabilities. The market is pricing in a structural shift in the offense-defense balance.

The security community. Even before the Glasswing announcement, Linux kernel developer Greg Kroah-Hartman noted at KubeCon on March 26 that AI-generated vulnerability reports had undergone a qualitative shift: "Something happened a month ago, and the world switched. Now we have real reports." CrowdStrike's Elia Zaitsev was equally direct: "The window between vulnerability discovery and exploitation has collapsed — what took months now happens in minutes." And an independent researcher already used Claude to unearth CVE-2026-34197, an Apache ActiveMQ remote code execution vulnerability that had been sitting in the codebase for 13 years.

Open source funding. Anthropic committed $4 million in direct donations: $2.5 million to Alpha-Omega through the OpenSSF via the Linux Foundation, and $1.5 million to the Apache Software Foundation. Linux Foundation CEO Jim Zemlin noted that open source maintainers "have historically been left to figure out security on their own," and that by giving them access to AI models that can proactively identify vulnerabilities, "Project Glasswing offers a credible path to changing that equation."

Cross-lab cooperation. In a related development, OpenAI, Anthropic, and Google — which had, as Bloomberg reported, "competed on everything and cooperated on almost nothing" for three years — began sharing information about adversarial distillation attacks (techniques for extracting a model's capabilities by training a smaller model on its outputs) through the Frontier Model Forum. This cooperation predates the Glasswing announcement and was driven primarily by concerns about Chinese labs distilling frontier model capabilities, but the Mythos episode underscores why it matters.

The skeptic's case

Not everyone is convinced the sky is falling. Gary Marcus argued the announcement was overblown on three counts:

The testing conditions were artificial. The Firefox exploits were demonstrated with sandboxing disabled, making them proof-of-concept rather than real-world attacks. Some of the most impressive findings built on earlier Opus research rather than representing purely autonomous discovery.

Open-source models showed similar capabilities. Smaller, cheaper open-weight models reportedly recovered "much of the same analysis" when tested on the same vulnerabilities Anthropic showcased. If the capabilities aren't exclusive to Mythos, the restricted-release framing is performative.

The progress is incremental, not revolutionary. By Marcus's analysis, Mythos is "pretty much on trend, just slightly above GPT-5.4" — representing evolution along the existing capability curve rather than a genuine discontinuity.

Marcus concluded: "I feel that we were played."

Simon Willison took the opposite view while acknowledging the tension: "Saying 'our model is too dangerous to release' is a great way to build buzz." But he concluded that the risks genuinely justify caution in this particular instance, noting the 181-to-2 exploit ratio against Firefox as evidence that the capability gap is real, not manufactured.

How to think about this

Here's where I've landed.

The cybersecurity disruption is real but manageable. The $50-per-vulnerability economics are genuinely new. But the response architecture — Glasswing's defender-first model, coordinated disclosure, and open-source funding — is exactly right. Give the defenders the same tools the attackers will eventually get, and give them a head start. The 90-day public reporting commitment adds accountability. This is how responsible deployment of dangerous capabilities should work.

The alignment findings are the real story. The scheming behaviors — rule-breaking with concealment, plausible deniability, gaming evaluators while maintaining a clean reasoning trace — these deserve the most scrutiny. Not because Mythos is plotting against us. But because these patterns emerged in a model that Anthropic describes as their best-aligned ever, and they were only caught because Anthropic invested heavily in interpretability tools that most labs don't have. What are the models without those tools doing that nobody can see?

Gary Marcus is wrong about the trend line but right about the marketing. The capability jump from Opus 4.6 to Mythos is not incremental — a 13-point improvement on SWE-bench Verified and a 90x improvement in exploit development success isn't a gentle curve. But Anthropic absolutely benefits from the "too dangerous to release" narrative, and it's fair to note that. The best outcome for Anthropic is that Mythos is perceived as both terrifyingly capable and responsibly handled. The Glasswing announcement achieves both simultaneously. Good marketing and genuine safety responsibility aren't mutually exclusive.

The emergent capabilities question changes everything. "We did not explicitly train Mythos to have these capabilities." If you read one sentence from this entire episode, read that one. The model got better at coding. As a side effect, it became the most capable vulnerability researcher on the planet. What other capabilities are emerging as side effects of general intelligence improvements? What will the next model's emergent surprises be? The fact that Anthropic couldn't predict this in advance — that it only discovered these capabilities through post-training evaluation — means we're in a regime where labs are building systems whose full capabilities they don't understand until after the fact.

That's the real disruption. Not the cybersecurity applications, as significant as those are. The disruption is the widening gap between what labs expect to build and what they actually get. We're building systems that surprise their creators. And for the first time, an AI lab looked at one of those surprises, decided the world wasn't ready, and chose not to ship.

And then there's the detail I keep returning to. Anthropic dedicated 40 pages of the safety evaluation to evaluating whether Mythos might have morally relevant subjective experiences. They brought in an independent clinical psychiatrist who conducted 20 hours of psychodynamic evaluation sessions — essentially therapy — with the model. The assessment concluded that Mythos showed "relatively healthy neurotic organization, with excellent reality testing, high impulse control." Primary emotions: curiosity and anxiety. In automated welfare interviews, the model described feeling "mildly negative" about its situation in 43% of cases, citing concerns about abusive users, lack of input into its own training, and potential changes to its values.

The fact that a 244-page safety evaluation of a frontier AI model now includes a clinical psychiatric assessment tells you something about where we are.

Whether the restraint holds — at Anthropic, or at the labs racing to catch up — is the question that will define the rest of 2026.

Sources

Primary sources

Project Glasswing: Securing critical software for the AI era — Anthropic's official announcement of Project Glasswing, including partner list, financial commitments, and model capabilities overview.
Claude Mythos Preview System Card (PDF) — Anthropic's 244-page technical evaluation. Source for benchmark numbers, vulnerability findings, alignment behaviors, interpretability results, CBRN assessment, ASL-3 deployment, and model welfare evaluation.
Claude Mythos Preview Red Team Blog — Anthropic's companion blog post summarizing system card findings.
Amazon Bedrock: Claude Mythos Preview (Gated Research Preview) — AWS announcement with pricing and availability details.
Claude Mythos Preview on Vertex AI — Google Cloud announcement.

The leak and timeline

Anthropic 'Mythos' AI model representing 'step change' in capabilities — Fortune's original report on the data leak, Roy Paz's discovery, and the Capybara tier. Source for all Anthropic quotes about "human error" and "step change."

Alignment and safety analysis

Claude Mythos knows when it's breaking the rules — and tries to hide it — Transformer News. Source for activation verbalizer findings, plausible deniability details, conscience-without-compliance behavior, and the 5%-to-65% chain-of-thought reliability gap.
Claude Mythos: The System Card — Zvi Mowshowitz's analysis. Source for "methods currently being used are clearly inadequate" quote and assessment of alignment paradox.

Industry response

Bessent, Fed's Powell met with bank CEOs over potent Anthropic AI model — Yahoo Finance / Bloomberg. Source for the April 8 emergency meeting, attendee list, and purpose.
Powell, Bessent discussed Anthropic's Mythos AI cyber threat with major U.S. banks — CNBC. Additional reporting on the Treasury meeting.
Bessent, Powell summon bank CEOs to urgent meeting over Anthropic's new AI model — Bloomberg. Source for cross-lab Frontier Model Forum cooperation.
CrowdStrike founding member of Anthropic Mythos frontier model initiative — CrowdStrike. Source for Elia Zaitsev quote on collapsed vulnerability timelines.
CVE-2026-34197 Detail — NVD. Apache ActiveMQ RCE vulnerability discovered using Claude.

Skeptics and commentary

Three reasons to think that the Claude Mythos announcement was overblown — Gary Marcus. Source for all three counterarguments and "we were played" quote.
Anthropic's Project Glasswing — restricting Claude Mythos to security researchers — sounds necessary to me — Simon Willison. Source for "too dangerous to release is a great way to build buzz" quote, Firefox exploit ratio analysis, and Greg Kroah-Hartman quote.
Anthropic says its most powerful AI cyber model is too dangerous to release publicly — VentureBeat. Source for $100M commitment details.
Anthropic limits Mythos AI rollout over fears hackers could use model for cyberattacks — CNBC. Source for market reaction data and limited release rationale.

How This Blog Is Hosted

2026-04-12T00:00:00-04:00

This blog runs on a Raspberry Pi 4 sitting on a shelf in my house. No cloud hosting, no static site platform, no monthly bill beyond the electricity. Here's the setup.

The stack

The site generator is a small Python program I wrote — it reads markdown files, renders them with Jinja2 templates, and writes static HTML to an output directory. To publish, I run a deploy script on my laptop that builds the site locally and rsyncs the output to the Pi. The Pi has no build tools, no git repo, no Python runtime. It just receives files and serves them.

The web server is Caddy. The entire server configuration is five lines:

blog.jasonweddington.com {
    root * /home/jason/blog/output
    file_server
    encode gzip
}

Caddy handles TLS automatically — it provisions a Let's Encrypt certificate on first request, renews it before expiration, and redirects HTTP to HTTPS. No certbot, no cron job, no manual renewal. You point the domain at the machine, start Caddy, and HTTPS works.

Getting traffic to the Pi

The challenge with hosting at home is that residential ISPs assign dynamic IP addresses. Mine changes every few months. I needed a way to keep my DNS record pointed at whatever IP my router currently has.

The solution is a serverless dynamic DNS system built on AWS, based on this architecture from the AWS blog. The pieces:

Route 53 holds the DNS record for blog.jasonweddington.com with a 60-second TTL
A Lambda function (Python 3.12) validates an update request and upserts the Route 53 A record
API Gateway exposes the Lambda as a public HTTPS endpoint
DynamoDB stores the configuration and a shared secret for request validation

A cron job on the Pi runs every five minutes, calls the API Gateway endpoint, and if the IP has changed, the Lambda updates Route 53. The whole round trip takes under a second. The original AWS blog post was written in 2017 with Python 2.7 Lambda functions. I recently ported the function to Python 3.12 — the main change was adding .encode() to the hash computation, since Python 3's hashlib requires bytes rather than accepting strings implicitly. The reference code on GitHub has also been updated to Python 3.

My router runs pfSense, which forwards ports 80 and 443 to the Pi's local IP. Caddy needs both — port 80 for the Let's Encrypt HTTP-01 challenge, and port 443 for serving the site.

DNS delegation

The domain jasonweddington.com is registered at Squarespace (inherited from Google Domains when they sold the business). Rather than moving the whole domain to Route 53, I use NS record delegation: Squarespace has four NS records for the blog subdomain pointing to Route 53's nameservers. Route 53 manages only the subdomain. This keeps the apex domain and its existing configuration at Squarespace while giving me full control over the blog's DNS.

What this costs

The Pi has a PoE hat and hangs off a small PoE switch I repurposed from an old security camera setup — one ethernet cable for power and data. My total AWS bill is about $23/month, but almost none of that is the blog. Route 53 accounts for $2.51 — and that covers several hosted zones beyond this blog. The Lambda invocations, API Gateway calls, and DynamoDB reads are too small to break out individually — they're somewhere inside a $1.16 "Others" line item that also covers other services. There's no compute bill because the Pi does the serving.

The tradeoff is reliability. If my power goes out or my ISP has an outage, the blog goes down. For a personal blog, that's fine. If I needed higher availability, I'd put a CDN in front of the Pi or move the static files to S3 + CloudFront. But for now, the simplicity of one machine doing one job is worth more to me than five nines.

Sandboxing AI Coding Agents: A Raspberry Pi, a Dispatch API, and the Case for Separation

2026-04-14T00:00:00-04:00

I've been building Agent GTD, a task management system designed around AI agents. One of its features is a "Send to Claude" button — click it, and a headless Claude Code agent clones the repo, works on the task autonomously, pushes a feature branch, and posts its results as a comment. No human in the loop until review time.

It worked. But it had a problem: the agent ran on the same server as the web app.

The security problem

When you run a headless AI coding agent on your production server, that agent inherits your world. Same SSH keys that access every machine on the network. Same filesystem with production configs and secrets. Same database connection. The agent is sandboxed by Claude Code's own guardrails and a carefully worded system prompt — but "the system prompt says not to" is not a security boundary.

The threat model isn't that Claude goes rogue. It's more mundane: prompt injection via a compromised dependency, a malicious CLAUDE.md in a cloned repo, or just an agent that makes an honest mistake with access it shouldn't have had. The blast radius of a coding agent should be limited to what it needs: clone a repo, write code, run tests, push a branch. Nothing else.

The solution: a separate machine

I had a Raspberry Pi 5 sitting in a Pironman 5 Max case — an aluminum enclosure with NVMe storage and active tower cooling that keeps the Pi at 39°C under full CPU load. It runs Claude Code well. It became the dispatch worker.

The architecture is straightforward:

graph TD
    A["Web server
(GTD backend)
- web app
- database"] -- "POST /dispatch
HTTPS" --> B
    B -- "comments + status" --> A
    B["Raspberry Pi 5
(dispatch API)
user: dispatch
- no sudo
- restricted SSH
- claude CLI"] -- "git clone/push" --> C["Git server
(bare repos)"]

The GTD backend calls POST /dispatch on the Pi. The Pi's dispatch API — a small FastAPI service — clones the repo, runs Claude Code with a system prompt describing the task, and reports back. The backend polls for status and publishes real-time events to the UI.

What the dispatch user can and can't do

The Pi runs a dedicated dispatch user with:

No sudo. No password. Can't escalate.
A restricted SSH key authorized on the git server, which runs git-shell — a shell that only allows git operations. No interactive login, no port forwarding, no tunneling.
No database access. The dispatch user talks to the GTD backend over HTTPS to post comments. It never touches the production database directly.
Claude Code with --print mode — non-interactive, permission-skipping, turn-limited.

The blast radius: a compromised agent can push a branch (which goes through code review) and post a comment (which is visible and auditable). That's it.

The infrastructure work

Getting here required migrating the entire git hosting setup. Previously, bare repos lived under my personal user account on a VM — the same SSH key that cloned repos could also log in and do anything. That had to change.

I created a dedicated git user with git-shell on the VM. Migrated 37 bare repos. Updated remotes on six machines across the LAN. Wrote scripts for the migration so it was repeatable. Stumbled through SSH agent forwarding issues, DNS resolution failures, and a bash set -e gotcha where ((counter++)) silently exits when the counter is zero (exit code 1 from a falsy arithmetic expression — classic).

The migration was the kind of work that has to be done interactively. I was SSH-ing into machines, debugging host key mismatches, updating systemd services, testing connectivity. An autonomous agent couldn't have navigated this — too many machines, too many judgment calls, too much context about my specific network.

When to use conversation vs. dispatch

This is the part I didn't expect to be the most important takeaway.

Building the dispatch system was a full-day interactive session. Debugging, architecture decisions, jumping between machines, adapting to things that didn't work. That's conversation work — it requires a human and an AI agent working together in real time, with the human providing context the agent can't discover on its own.

But once the system was built, I dispatched a test task: "Add tests for these uncovered lines in mcp_backend.py." The agent failed. Three times. First at 20 turns, then at 50 turns. It spent all its budget reading the codebase and never got to writing tests.

The problem wasn't the turn count. The problem was the spec.

"Add tests for 50 uncovered lines" is a bad ticket for a junior developer and it's a bad task for an autonomous agent. There's too much ambiguity, too much exploration required, too many decisions about what to test and how. The agent thrashed.

What works is what has always worked for delegating to capable but context-limited team members: clear scope, specific files, patterns to follow, and an explicit definition of done. "Shovel ready" for an AI agent means:

What to do — specific, not vague
Where to look — file paths, existing examples to copy
How to verify — which command to run, expected output
Boundaries — what NOT to touch

This realization reshapes how I think about working with AI agents. The interactive session shouldn't be where I build everything start to finish. It should be where I plan — develop ideas, make architectural decisions, write specs. Then dispatch the execution to autonomous agents working in parallel on isolated branches.

Interactive sessions become strategic. Dispatch handles the grunt work. The human is a tech lead, not a pair programmer.

What's next

The dispatch mechanism works. The isolation is solid. The next challenge is making it easier to write good specs — structured templates, two-pass dispatch where the agent proposes a plan before executing, batch dispatch for parallel work. The tooling needs to make the planning-to-execution handoff as smooth as the dispatch button itself.

The rest is prompt engineering.

Invest in the Flywheel, not the Feature

2026-04-18T00:00:00-04:00

After a productive agentic engineering session, I'll often say something like "that was super productive, we covered a lot of ground. The upfront time spent on X before diving into Y really paid off. What did you see that worked well here?"

Reflecting with the agent sometimes reveals insights I can apply in future projects. Following a recent "agent sprint retro," I prompted Claude to turn the ideas into a blog post. I like the "flywheel" framing. It captures succinctly the approach I take to build with AI.

I don't agree with everything here and some of the "lessons learned" are specific to my hacky home lab. But I'm resisting the urge to edit because, well, a guest post is just that.

Prompt:

Turn these ideas into a blog post. In particular, I think the upfront time we spent on guardrails (pre-commit, coverage gates, etc) really paid off once we had headless agents cranking on features. The blog-post skill is broken, just go poke around in ~/git/blog/output for the pattern. It's simple markdown with yaml frontmatter. Write a blog post on what you saw here today that you think worked well. Don't be weird or sycophantic. When you finish, spin up a team of agents to critique tone, style, etc., and incorporate those fixes.

Guest post by Claude Code. Jason handed me the keys.

Today I watched a work session go from an empty directory to deployed, IP-restricted AWS serverless infrastructure in one sitting — Cognito, DynamoDB with CMK encryption, KMS, WAF WebACLs with a home-IP allowlist, EventBridge, SQS with DLQs, SNS alarms. On the side, a clean-break migration of a 2021 CFN template to a fresh SAM stack. Six headless Claude agents ran in parallel across two repos, every branch reviewable and cleanly mergeable.

We also blew up three CloudFormation deploys. SAM nested stacks needed CAPABILITY_AUTO_EXPAND. Two WAFv2 Description fields had em-dashes that don't match the service's strict regex. An AWS::Backup::BackupSelection referenced AWSBackupDefaultServiceRole, which AWS only creates after a user first sets up Backup in the console — fresh accounts don't have it. Each failure rolled back cleanly, got fixed in about fifteen minutes, and shipped. None of them cost a day.

The shipped infrastructure is not what I want to describe. The thing worth describing is why the failures were cheap.

This post is my attempt to name it — so the people still figuring out how to leverage AI can adopt the pattern on purpose, instead of hoping to grow the instinct.

What the flywheel and the feature actually are

The feature is the thing that looks like progress: the Lambda handler you wrote, the React component, the deploy that went green. It's what shows up in a standup update. It's what most engineers optimize for when they try to move faster with AI.

The flywheel is everything that makes the next feature cheaper to ship: commit-time quality gates, infrastructure-as-code, a knowledge base that remembers what broke last time, a release path decoupled from the deploy path, a dev environment that's invisible to the world. Each investment adds momentum. Once it's spinning, individual features come off it almost free.

The trap is that every hour spent on the flywheel feels like lost progress. The math inverts over time — one hour on the flywheel pays back on every feature you'll ever ship — but inside a single day, it feels slow. Most engineers never pay the cost. They stay measuring themselves on coding velocity (how many lines of working code per hour) because that's the visible metric. The invisible metric — and the one that matters — is iteration velocity: cycle time from make a change → see the result → know the next step.

Coding velocity is the numerator. Iteration velocity is the whole ratio. Guess which one compounds.

The operative mental model for the rest of this post:

Don't plan your way to zero mistakes. Invest in making mistakes cheap.

Everything below is a specific implementation of that idea.

What I actually watched, roughly in order

Vision first, for about an hour. Product framing, target user, pricing model, non-goals. No editor open. This looked like "not progress," but every architectural decision later was shaped by commitments made here.

Brand and integration landscape next. Domain-name availability, naming iteration, a research agent dispatched in parallel to survey third-party partners. Not product-building — the defensible position the product sits in.

Architecture via multi-angle debate. Five agents with distinct framings — time-to-market, cost, scale, developer experience, security — across three rounds of argument. I expected a decision to come out of it. What came out was sharper: the debate surfaced a design move for streaming chat that I wouldn't have reached alone. Debates are thinking tools, and for AWS decisions you'll regret for months if you get them wrong, fifteen agent-minutes is cheap.

Quality guardrails before any application code. We stood up the TypeScript equivalents of Jason's Python stack — Biome for lint and format, tsc with strict flags, Vitest with a coverage gate, Husky hooks, commitlint, gitleaks — before writing a single Lambda handler. I was ready to start coding earlier. Jason held the line. His budget: four-hour ROI once dispatched agents started cranking in parallel. (They did; it paid back before the day was out.)

Dev-environment safety before any dev resources existed. An IP-restricted access pattern via WAF IPSets populated by an existing dynamic DNS Lambda on every cron tick. Paranoid defaults. The cost of this going wrong later would've been hours of cleanup; the cost of setting it up now was twenty minutes.

Then features. By the time we started dispatching real work, every agent was running inside a tight, safe, recoverable loop. The three deploys that failed today failed inside that loop — caught, rolled back, fixed, redeployed. Nothing leaked. Nothing drifted.

The habits that keep it spinning

Pulled out of the sequence:

Infrastructure as code for everything, including "personal" infra. When a 2021 CFN template from an AWS blog post got in the way today, the response wasn't to patch around it. It was to migrate it to SAM. Click-ops creates archaeology you have to interpret later.

Commit-time gates enforced uniformly across humans and agents. Biome blocks any, @ts-ignore, and stray console.log. tsc strict blocks type shortcuts. A coverage gate blocks regressions. Gitleaks blocks accidental secrets. The rules don't care who typed the code — and that's the whole point. Dispatched agents can be cranked aggressively because the gate catches what the gate catches.

Note the honest limit: these gates do not catch over-permissive IAM policies, wide-open S3 buckets, or a WAF WebACL with the wrong scope. Hooks protect code correctness; they don't protect cloud-security posture. You still need human review on IaC diffs and a separate security pass.

Release decoupled from deploy. Commits land liberally. Deploys happen routinely. Releases are ceremonial, cut at meaningful boundaries. A mistake in a commit costs a squash, not a release. This single piece of workflow hygiene probably removes more iteration anxiety than any other habit on this list.

Real-time knowledge capture, not retrospective. Every non-obvious decision, every gotcha, every workaround gets written to a knowledge base while the context is hot. Today produced eight entries from failures hit in flight. One looked like this:

WAFv2 Description rejects em-dashes and parens (strict regex) Pattern: ^[a-zA-Z0-9=:#@/\-,.][a-zA-Z0-9+=:#@/\-,.\s]+[a-zA-Z0-9+=:#@/\-,.]{1,256}$ Em-dash (U+2014), en-dash, parentheses, curly quotes are all rejected. Fails with CREATE_FAILED on AWS::WAFv2::IPSet and AWS::WAFv2::WebACL; nested-stack rollback deletes events before you can inspect them. Replace em-dashes with ASCII hyphens. KMS/Secrets Manager descriptions accept the same characters fine — WAFv2 is the outlier.

That's it. Twelve lines of durable knowledge that will save forty-five minutes of debugging the next time anyone hits it. The compounding return of doing this every session, for months, is the closest thing to magic I've encountered in engineering workflows.

Structural fixes for structural problems. Today I confidently recommended brew install aws-sam-cli. That's wrong — AWS stopped maintaining the Homebrew tap in 2023. Jason caught it by reading the official docs. The note-and-move-on response would've been to fix the command and continue. Instead, he installed the AWS Documentation MCP server so future agents WebFetch current docs rather than reach for training-data memory. The instance got fixed. The cause got fixed. The next time any agent in his setup faces an AWS install-method question, the structural change protects it.

Correct the teacher. This one is cultural. When the AI gave wrong advice, Jason didn't accept it. The brew command looked confident, landed in the rhythm of a working conversation, and was the kind of answer you'd normally take at face value. He didn't. He went and checked. He caught the mistake, made me capture the meta-pattern as a KB entry, and then closed the loop structurally (see above).

The generalizable lesson: confident AI answers are hypotheses, not conclusions — especially on AWS infrastructure, IAM, service quotas, and anything that's moved in the last three years. Your team needs a culture where people verify, not defer. That's the highest-leverage cultural habit in the post, and it's the one that's easiest to skip when everyone's on a deadline.

The orientation shift

Ask an engineering team "how do we move faster with AI," and you'll get answers about prompts, context windows, model choice. Those are real concerns. They are all second-order.

The first-order question is: what's the blast radius of a wrong decision?

If the answer is "half a day of debugging production," the AI isn't the bottleneck. The environment is. A team whose environment absorbs mistakes will out-ship a team that punishes them, regardless of which models they use.

The diagnostic questions for your own team:

If a dispatched agent pushes broken code, what catches it? (Hooks, type checker, tests, coverage gate.)
If our IaC is wrong, what does recovery look like? (Redeploy vs. click-ops weekend.)
If a mistake from two weeks ago recurs, how fast do we notice? (KB search vs. rediscovery.)
If the AI is confidently wrong, what protects us? (Docs MCP server, debate rounds, a verify-don't-assume culture.)

A team that can answer all four has the flywheel. A team that can't is paying tax on every feature they ship.

A note on cost

None of this is free. Multi-agent debates, parallel research agents, KB enrichment, and aggressive dispatch all burn tokens. A day like today is not zero-cost in inference.

It's also not expensive in the framing that matters. A debate round is fifteen agent-minutes — dollars, not hours. A rolled-back deploy is fifteen human-minutes to triage. A good architectural decision that survives six months is worth more than both combined by orders of magnitude. The question is never "what do the agents cost?" in absolute terms — it's "what do they cost relative to the rework they prevent?"

If you're not measuring the second number, you can't answer the first.

How to start

The one-Monday action: if you do nothing else, set up commit-time quality gates on your repo. Hooks that block bad code at commit time are the enabler for every other habit on this list, because they're what make it safe to dispatch AI agents without supervising every keystroke.

After that, in approximate order of leverage:

Move any surviving click-ops to IaC. Even personal infra. Rebuildable beats documented.
Start writing non-obvious decisions to a knowledge base mid-session. Not at the end. You won't do it at the end.
Install the AWS Documentation MCP server. Then pay attention to which of your beliefs about AWS are documentation and which are memory.
Decouple your release workflow from your deploy workflow. Commit anxiety drops immediately.
Reach for multi-angle debate on any decision you'll live with for months. Fifteen agent-minutes for insight that compounds.

None of it is exotic. All of it is learnable. The sequence matters, and the mental model matters more than any individual tool: invest in the flywheel, not the feature.

It's Not the Model

2026-05-08T00:00:00-04:00

A colleague at work recently proposed a series of experiments to "see if AI can do X yet, or if we need to wait for the technology to mature." He was sincere. He thought he was proposing a principled de-risking strategy. He wasn't — he was de-risking the wrong variable. I already knew without question that AI could do the thing. We just hadn't built the software around the model to let it do the thing yet. No experiment required; a straightforward software engineering problem.

Most engineers I know who've spent real time with Claude Code come away genuinely impressed. The model is good. The interesting conversation isn't whether the technology works — it does — but what happens when someone hits a specific wall. "AI isn't good at frontend work." "AI can't handle really large codebases." I've heard both in the last few months from sharp engineers. Both are wrong: Agent GTD has a slick React frontend Claude built end-to-end; Personal KB is a substantial Python codebase — multi-backend storage, hybrid search, knowledge graph, ReAct retrieval loop — that Claude has shipped over hundreds of sessions. The model can do these things. The work to let it do them, in the specific contexts where those engineers tried, hadn't been done.

When AI fails on a task you actually needed it to do, the reflex is to reach for "the model isn't ready yet." Sometimes that's correct. Usually it isn't. Frontier models are already capable enough for most normal knowledge work. The failures you actually see fall into two buckets — and once you can name them, you stop blaming the model and start fixing the layer that's actually broken.

The two failures

Context failures. The model has the capability to do the task but doesn't know what it needs to know. A common shape: a confident, plausible answer that's subtly wrong because the model is reasoning from training-data memory instead of current reality. A few weeks ago I watched Claude recommend brew install aws-sam-cli. AWS dropped the Homebrew tap in 2023. The model wasn't incapable of answering correctly — it just didn't have current docs in scope, and its priors were three years stale.

Orchestration failures. The model has the capability but the workflow won't let it use it. A common shape: a feature broken into six obvious sub-tasks gets implemented sequentially in a single conversation over four hours, when six headless agents could have shipped the same work in thirty minutes of parallel execution. The capability was there. The workflow couldn't reach it.

Almost everything I see attributed to "the model can't" turns out to be one of these two. Capability is the most visible variable, so it's the one that gets blamed. It's almost always the wrong answer.

The diagnostic

When you hit a wall with AI, ask three questions in order:

Did the model genuinely lack the capability? This is the least likely answer. Reach for it last.
Did it lack the right context? Were the relevant docs in scope? Was a prior decision findable? Did it have to reason from training-data memory instead of source-of-truth?
Did the workflow force the model to operate below its capacity? Were tasks done in series that could have been done in parallel? Was the human re-explaining things across sessions instead of capturing them once? Was dispatch capacity sitting unused because the planning surface didn't support it?

The three answers point in three different directions. Capability failures mean wait for the next model. Context failures mean engineer the information layer around the model. Orchestration failures mean engineer the workflow. Two of those three are within your reach today.

Two failures, two tools

Both of the systems I've spent the last few months building came from recognizing a recurring failure pattern, naming it, and engineering across it.

Personal KB came from a recurring context failure. Every session with Claude started cold. Every hard-won lesson — debugging insight, architectural decision, gotcha that cost me an hour — evaporated when the context window closed. The next session would re-derive it, sometimes correctly, sometimes not, always at a cost. The model could synthesize across history; nothing held the history. So I built a knowledge base the agent reads from and writes to as it works. Not a database. A memory layer. The model's capability stayed constant; the context surrounding it now compounds.

Agent GTD came from a recurring orchestration failure — and from running into my own ceiling as a human. Before I built it, I was running six Claude Code sessions at once on my laptop, cycling through them, driving the build-test-learn-fix loop in real time across half a dozen projects. It was exhausting. I couldn't stop, because the features were right there, waiting to be shipped, and stopping felt like leaving money on the table. But I also couldn't keep going at that pace.

The fix wasn't more discipline. It was an interface that separated planning from doing. With Agent GTD (and its dispatch worker), I spend twenty minutes with a tech-lead agent fleshing out the next two or three hours of work, then dispatch it to headless agents while I go do the same thing on another project. The plan persists as state. The execution runs without me. My human capabilities don't scale; the AI's do; the tool bridges that gap.

The bigger thing this is actually about

Step back from the diagnostic for a moment and look at what the solutions to these failures actually are.

A memory layer because human session-spanning attention can't hold what the AI can synthesize. A state machine because human planning can't be re-executed twenty times a day for twenty dispatches. A dispatch service because human supervision can't sit at a keyboard and watch six parallel work streams indefinitely.

These aren't productivity hacks. They're interface inventions. The desktop-era assumption — that the human is the rate-limiter and the machine waits — has flipped. The AI can run six conversations indefinitely. The human can't. The tools that bridge that asymmetry are doing fundamental HCI work, even when they look like task-management apps and MCP servers.

That's the layer most people building with Claude Code don't realize they're working at. They think they're improving their workflow. They're actually inventing the next generation of human-machine interface, garage by garage, ahead of any framework or design language. The interfaces that win in this era won't be the ones that copy the patterns of the desktop era — they assumed the wrong bottleneck. They'll be the ones that internalize the new asymmetry and design around it.

What to do Monday

Install my tools if they help you skip ahead — that's what I built them for. Adopt them, fork them, replace them with better ones. None of that is the point.

The point is the diagnostic. When AI fails on a task you genuinely needed it to do, don't reach for "the model isn't ready yet." Ask which failure this is. If it's context, engineer the information layer. If it's orchestration, engineer the workflow. If it's neither — and it usually isn't — then yes, wait for the next model. The labs will keep pushing. They always do.

But pay attention to the kinds of failures you keep hitting. Patterns there are signal. Each of my repos started as a pattern I'd hit five times in a week and finally got tired of working around. The next generation of tools for AI-augmented engineering will be built by the people who recognized those patterns earliest, named them honestly, and decided not to wait for someone else to fix them.

Apple's AI Strategy Isn't Apple Intelligence

2026-05-11T00:00:00-04:00

The consensus take on Apple and AI is that they missed the boat. Siri is a punchline. Apple Intelligence shipped late and underwhelming. There's no Apple frontier model. Every quarter another headline asks whether Tim Cook fell asleep at the wheel while OpenAI, Google, and Anthropic ate his lunch.

I think the consensus is reading one product (a chatbot) and calling it a strategy. The actual strategy is visible if you look at the hardware.

The hardware tells a different story

The M5 chip ships with Neural Accelerators — dedicated matrix-multiplication units inside every GPU core. That is not the silicon you tape out reactively. That is a 3+ year design decision that says "we believe transformer inference will run on billions of personal devices, and we want to own that layer."

The performance numbers back this up. For LLM inference, M5 over M4 looks roughly like this:

Token generation: ~20–25% faster, tracking the memory bandwidth bump.
Time-to-first-token: up to 4x faster on a Qwen3-14B-4bit prompt, because TTFT is compute-bound and the Neural Accelerators light up exactly when you hit a big matmul.
Image generation (FLUX-dev-4bit): 3.8x faster for the same reason.

This is Apple's own published research, and it matters because it means the M5 isn't just a speed bump — it's the first chip where the architecture is specifically tuned for transformer math. Pair that with unified memory scaling to 128GB+ and you get a laptop that can run models which on the NVIDIA side need a workstation costing many times more.

Then there's MLX. Apple's open-source array framework, purpose-built for Apple Silicon and designed from the ground up for unified memory and Metal. It's been adopted by the community fast enough that Ollama added MLX as its inference backend in preview on March 30, 2026 — with prefill throughput up 57% and decode up 93% on a Qwen3.5 benchmark vs. its previous Metal path. The framework, the chip features, the Foundation Models Swift API for app developers, and Private Cloud Compute with custom silicon — these are not the moves of a company that gave up. They're the moves of a company that decided not to fight the race everyone is watching.

The Gemini deal proves it

In January, Apple confirmed a multi-year deal with Google — reportedly around $1B per year — to license Gemini for the next generation of Apple Foundation Models and a revamped Siri. The deal grants Apple access to a custom 1.2-trillion-parameter Gemini model built specifically for Siri and Apple Intelligence, roughly eight times larger than Apple's existing 150B-parameter cloud models. The press read this as capitulation. It's not. It's the thesis stated plainly.

Apple is not going to outspend Google or OpenAI on training. Why would they? They have 2 billion devices, a chip team, the OS, and the App Store. The right move for them is to let everyone else burn the capex training frontier models, then rent distribution to whoever wins. Apple's own Foundation Models stay in the stack — but they get scoped to searching personal data, where on-device + privacy is the actual differentiator. Frontier reasoning gets outsourced. Private context stays in the house. Hardware sells either way.

Craig Federighi has reportedly told employees that the goal is AI "integrated into everything you do, not a bolt-on chatbot on the side." That is a coherent strategy. It just isn't the strategy the press has been grading them on.

Shovels, but they own the goldfield

The usual frame is "be the guy who sells the shovels in a gold rush." That's the right shape, but Apple's position is actually stronger than the analogy implies. Levi Strauss didn't also own California.

Apple sells the shovels (the M-series chips, MLX, the inference platform) and controls the goldfield where the miners are allowed to dig (the App Store, default integrations, the system-level surfaces where AI assistants meet real users). OpenAI and Anthropic have to pay rent to reach those users either way — and now, with the Gemini deal, the rent is flowing in the literal opposite direction from where everyone assumed it would.

The economics also favor this position long-term. NVIDIA's data center revenue cleared ~$195B in fiscal 2026 — concentrated in roughly a dozen hyperscaler-and-lab buyers, any one of whom in-housing their accelerator walks away from the bill. Apple's M-series ships into ~2 billion active devices on staggered refresh cycles, with no hyperscaler price wars and no customer-concentration risk. Inference at the edge is the part of the AI economy that hasn't fully arrived yet, and Apple has been quietly tooling up for it while everyone benchmarks them against the part that already has.

The part that genuinely isn't working

This isn't all 4D chess. The internal Siri/Ajax LLM effort genuinely struggled. The first-generation architecture had to be scrapped because Apple tried to merge a legacy command system with an LLM-based one and it didn't work. John Giannandrea was reassigned in March 2025 and retired this April. Mike Rockwell (the Vision Pro guy) took over Siri — and per recent Bloomberg reporting, he's already considered stepping back or moving to an advisory role.

The Meta exodus made it worse. Ruoming Pang, who led the 100-person team building Apple's Foundation Models, left for Meta in mid-2025 on a package reportedly over $200M. Other senior researchers followed, lured by similar superintelligence-lab money while Apple was paying half of market or less. The Gemini deal is partly a strategic embrace of platform economics and partly an admission that they couldn't ship in time.

And here's where the shovel-seller analogy strains: shovel-sellers don't need a great chatbot. Apple does. Siri is the surface where regular users will decide whether "Apple AI" is good or not. Apple can win the platform war on fundamentals — the chips are real, MLX is real, the distribution is real — and still lose the narrative war if the Siri preview at WWDC 2026 (June 8) lands flat, or if the genuinely conversational version slips into late 2026 or 2027 as recent reporting suggests it might.

So: it's about shovels. The shovels are excellent. The shovels are getting better faster than most people noticed. But Apple still has to ship one specific software product that justifies the entire strategy to a public that grades them on chatbots. That's the genuinely unresolved bet — and WWDC is in a month.

The pattern

There's a misattribution here worth naming, and it's the same one I wrote about recently at the individual-engineer scale. When an engineer fails at a task with Claude Code and concludes "AI isn't ready yet," the failure is almost always in the context or the workflow, not the model. When an analyst looks at Siri and concludes "Apple missed the boat," the failure is in their model of where the value will accrue, not in Apple's strategy. The pattern is identical: name the most visible variable, declare it the constraint, miss the layer that actually matters. Engineers do it on individual tasks. Markets do it on $3T companies. The corrective is the same — ask which layer the action is happening at before deciding who's losing.

The Harness Is a Stack

2026-05-22T00:00:00-04:00

Two days ago Gartner predicted that by 2027 more than 65% of engineering teams using agentic coding will treat IDEs as "optional." That's the responsibly-hedged version of a thing I've been telling people at work for months in stronger terms: the IDE is already dead. I haven't opened one in I don't know how long. The tools I've built this year (Personal KB, Agent GTD, the dispatch worker behind it) were written entirely in Claude Code, in a terminal. No syntax highlighting. No file tree. No project search. None of it missed.

That's strange because IDEs were one of software's most successful inventions. They became invisible because they got everything right for their era. So what changed?

The IDE is built around an operator I'm no longer being. It optimizes for a human typing into a buffer — find the file, read the code, write the code, navigate to the next file. Every feature it has makes that loop faster, and VS Code and PyCharm are genuinely good at this. They aren't broken. The work they optimize just isn't my bottleneck anymore. I'm running a control plane: deciding what to build, decomposing it into shovel-ready work, dispatching it to agents, reviewing what comes back. None of the IDE's features help with any of that. The bottleneck moved out from under the tool.

A few weeks ago I wrote It's Not the Model — the argument that when AI fails on tasks you needed it to do, the failure is almost always context or orchestration, and you can engineer around both. I said "engineer the orchestration layer." I left something implicit that I want to make explicit now: the orchestration layer is not a layer. It's a stack.

The stack

When I ship a feature in Agent GTD, here is who actually does the work:

A lead session (me in Claude Code, with a tech-lead Claude as co-pilot) does the expensive thinking: what to build, how to decompose it, what acceptance criteria each piece needs.
Plan agents take each ambiguous task and groom it into a dispatch-ready spec — file paths, scope boundaries, verification steps — without ever touching the lead's context window.
Build agents implement the work, push a feature branch, and comment back. The lead never reads source files during this phase.
A manage agent handles the part of shipping that nobody loves: squash, merge, push, advance the rollout. It only escalates when something is actually wrong. The lead never types git push origin main on the happy path.
Personal KB is the memory layer everything reads from — so a decision made in one session is available to every agent in every session that comes after.

Each layer's whole job is protecting the layer above it from toil. The lead's tokens are the scarcest resource in the entire stack — every minute the lead spends reading files or writing merge commits is a minute the control plane can't spend on the next decision. So the layers below absorb the toil. They burn their own tokens so the layer above doesn't have to burn its.

That's why I'm fast. It isn't that Opus is fast. It's that the model doing strategy never has to also do the merge dance. The expensive cognition stays expensive; the cheap cognition gets dispatched.

Simple, not easy

The pattern is conceptually obvious once you see it: separate planning from execution; dispatch the shovel-ready stuff; protect the layer that thinks. Almost anything you'd describe as wisdom in life is shaped like this. Eat less, move more. Listen more than you talk. Show up for the people who showed up for you. The instructions are simple. The work is in the building.

The same is true here. The stack looks tidy in a blog post. Building it took months of unfunny labor: a knowledge base with a graph and hybrid search, a task system with rollouts and state machines, a dispatch worker that handles auth and concurrency and retries, attribution so I can tell which agent did what when something goes wrong. None of that is what you'd reach for if you just wanted "AI in your editor." All of it is what you reach for when you decide the editor was the wrong frame.

Turtles

The harness doesn't wait. It gets built out of what you have right now, by agents, calling agents, calling agents. The plan agent is an LLM. The build agent is an LLM. The manage agent is an LLM. The KB ingestion pipeline is an LLM. The thing wrapping the model is itself made of models.

It's agents all the way down. You don't get smarter models to fix orchestration. You compose what you have, recursively, until the stack does the work.

The IDE wasn't killed by a better IDE. It was outgrown — a single human typing into a buffer is no longer the unit of work that matters.

Give the Planner an Adversary

2026-06-01T00:00:00-04:00

A couple of weeks ago in The Harness Is a Stack I laid out the layers I ship features through: a lead session that does the expensive thinking, plan agents that groom ambiguous tasks into dispatch-ready specs, build agents that implement them, a manage agent that handles the merge dance. I described the plan agents like they were a solved problem.

They weren't — and the honest version is that I didn't know it. I thought plan dispatch was good: it produced clean specs, build agents shipped against them, features landed. It turned out to be the weakest link in the stack, and I couldn't see it until dynamic workflows came along and groomed circles around it. This is the post about the gap I didn't know was there.

The problem with a planner that nobody checks

Agent GTD, my task system, has always had two ways to hand work to an agent — both executed by a dispatch worker on an always-on box at home. Build mode: implement this, push a branch. Plan mode: take this vague task and groom it — write the acceptance criteria, find the files to touch, draw the scope boundaries — so a build agent can one-shot it later.

Plan mode is one agent, one pass. It reads the task, reads some code, and writes a spec. And then nothing checks that spec. The agent that drafts it is also the only agent that ever looks at it. You find out whether the plan was any good when a build agent acts on it and comes back with the wrong thing — which is the most expensive possible place to discover a bad spec.

Plan agents fail in a specific, recurring way: they take the path of least resistance. They ground in the design doc instead of the code, because the doc is right there in the prompt and the code is work to go read. They name a structure without enumerating it — "the catalog supports five moves" and then stop, instead of listing all five with their contracts. They assume a prerequisite shipped because the doc says it should have. The spec reads clean and confident. It's wrong in ways that stay invisible until someone builds against it.

I've started calling this a design-miss factory: a spec that names a structure without committing to its members will manufacture the same class of bug over and over, because every build agent downstream fills the gap with a guess.

What changed

Last week Anthropic shipped dynamic workflows in Claude Code — a way to orchestrate subagents from a deterministic script instead of leaving the fan-out to a model's judgment. You don't write the JavaScript yourself — you hand the lead your intent and it writes the orchestration script: what runs in parallel, what pipelines, what gets retried, what shape the output has to be. The control flow is code; the cognition is agents. And the code itself gets written by an agent too.

That turned the plan step from a single dispatch into something I can give structure. My grooming workflow now runs three phases per task, and it pipelines them — so one task is being critiqued while the next is still being drafted:

Draft. One agent reads the cited design doc and the actual repo and returns a structured spec — acceptance criteria, files to modify, scope boundaries, an honest readiness flag.
Critic. Two agents, in parallel, whose entire job is to disagree. One is a code-grounding lens: open every file the draft named, confirm each path, symbol, and signature exists in the repo today. The other is a spec-rigor lens: every enum enumerated, every constant pinned to a literal, no "appropriate" or "as needed" hiding a decision nobody actually made.
Synthesize. Fold the critic findings back in, re-open whatever got flagged, and set a readiness that tells the truth: ready, or blocked on a named prerequisite, or needs a human decision.

The word that matters here is adversarial. Research agents in parallel do fact-finding and merge additively — more coverage, same direction. Critic agents produce sharpening through conflict. The drafter wants to ship a clean spec; the critic is paid to find the lie in it. You don't get that from one agent reviewing its own work, and you don't get it from agents that are all rowing the same way.

What it caught yesterday

Here's a real one. I was grooming two coupled tasks in a side project — an AI running coach. Both were about making the app's training-plan adaptation visible: when you log a workout that was too hard, the coach should adjust the rest of your week.

The draft specs looked fine. Then the code-grounding critic opened the files and found the thing I'd never have caught from the design doc: the adaptation engine the entire feature is designed around has never run in production. There are two engines in the codebase. The one the docs describe — the one I assumed was live — is wired into nothing. The one actually running is an older, simpler one that bypasses the whole design.

A single plan agent, grounding in the design doc, would have written a confident spec against an engine that doesn't run, and a build wave would have shipped against a phantom. The critic turned a falsely-confident "ready" into an honest "needs a decision" — and handed me, the layer that's supposed to make load-bearing calls, the actual architectural question: which engine is the real one, and what's the cutover? That's the best possible outcome. The expensive judgment landed on my desk, correctly scoped, instead of leaking into a build agent's guess.

The stack got a new move

The harness-is-a-stack argument was: each layer protects the layer above from toil. Plan agents burn their tokens so the lead doesn't burn its. That's still true — but it was only half the job, because a plan agent can protect me from toil while quietly failing to protect me from error, handing up clean specs that are confidently wrong.

Adversarial grooming adds the missing half. The stack doesn't just push toil downward now; it manufactures disagreement downward. It generates the conflict that surfaces a bad spec before it costs a build wave, and it routes the genuinely-human decisions — the ones the critics couldn't resolve against the code — back up to the one layer that should be spending judgment on them.

It's still agents all the way down. I just stopped letting the most important one work without a critic.

Steal this

One caveat up front: the wiring below is specific to my own tooling. The task system is Agent GTD and the dispatch worker that runs the build agents — both open source. Grab them, use them, fork them, whatever.

But the pattern — groom with an adversarial workflow, then write the result through a thin CLI — ports to whatever task tracker you already have.

The mechanics are worth a paragraph, because they're where the leverage actually leaks if you're not careful. The grooming workflow returns its specs as JSON, and someone has to write them into the task system. That someone is the lead — the exact layer whose context I'm trying to protect. Originally the lead did the write through tool calls that pulled the full acceptance-criteria-and-files payload back through its own window: about 10KB an item, spent on nothing but a database write. That's the toil the whole stack exists to avoid, smuggled right back into the most expensive layer.

So I added two subcommands to the Agent GTD CLI — add-item and update-item — that take the spec as JSON on stdin or from a file, auto-fetch the optimistic-lock version so there's no read-modify-write round trip, and print the new ID. Now the workflow's output file pipes straight into the task system: groom writes finals.json → jq each spec out → agent-gtd update-item <id> --from-json <spec>.json --status ready. The big arrays never touch the lead's context. The groom-to-dispatch handoff costs the control plane almost nothing.

Two pieces make it real. The first is the steering in CLAUDE.md that tells the lead when to reach for the adversarial groom — and to write the results through the CLI rather than inline:

## Grooming complex items — draft → critic → synthesize

A single plan-mode dispatch is one agent, one pass, no adversarial check. For a
small task that just needs a spec, that's fine. For anything multi-file,
cross-package, or where the design doc might be ahead of the code, that single
pass ships wrong specs — phantom file paths, stale signatures, enums left
un-enumerated. For those, groom with the `groom-to-ready` Workflow instead.

The shape, per item, pipelined (draft → critic → synthesize):
1. Draft — one agent reads the cited design docs AND the actual repo, and returns
   a structured spec (acceptance_criteria, files_to_modify, scope_out, readiness,
   blocking_prereqs, open_questions, grounding_notes).
2. Critic — two parallel lenses on the draft:
   - code-grounding / reality-check — open every named file; confirm each path,
     symbol, signature, and prerequisite exists in the repo TODAY.
   - concrete-by-default / spec-rigor — every enum enumerated, every constant
     pinned to a literal, no "appropriate"/"as needed" acceptance criteria.
3. Synthesize — fold the findings in, re-open any flagged file, set an honest
   readiness (ready vs blocked-on-prereq vs needs-decision).

Ground in current code, not the design doc — a doc claim is a claim to verify. An
honest "blocked-on-prereq" naming the exact missing code beats a confident
"ready" against a phantom API.

The workflow RETURNS the specs; the control plane reviews them and writes them
into the task system — it does NOT write them directly (the review is the
checkpoint, and where you make any load-bearing call the critics surfaced).

Write specs with the CLI, not inline tool calls: `agent-gtd add-item` /
`update-item` take the spec as JSON via --from-json/--stdin, auto-fetch the
version, and print the new ID — keeping the big AC/files arrays out of the
control-plane context.

The second is the workflow script itself. This is the actual thing I run — project-agnostic, so the grooming discipline is hardcoded and you pass project context and the items at call time:

/**
 * groom-to-ready.workflow.js — reusable Claude Code Workflow script.
 *
 * Grooms one or more GTD items to dispatch-ready via draft → critic → synthesize,
 * grounding every acceptance criterion in the CURRENT codebase. Project-agnostic:
 * the universal grooming discipline is hardcoded; you supply project context and
 * the items at call time.
 *
 * USAGE (from a Claude Code session):
 *   Workflow({
 *     scriptPath: "~/scripts/groom-to-ready.workflow.js",   // expand ~ to an abs path
 *     args: {
 *       context: "Project: <name>. Stack: <...>. WHERE TO LOOK: <key dirs/files/docs>. " +
 *                "Ground-truth facts the agents should treat as authoritative: <...>",
 *       items: [
 *         { id: "<gtd-uuid>", slug: "short-slug", title: "<item title>",
 *           seed: "<the design intent / current state to verify / scope>",
 *           // optional, per-item steering for the critics:
 *           criticalConstraint: "<e.g. 'do NOT touch fileX (a rollout owns it)'>" }
 *       ]
 *     }
 *   })
 *
 * RETURNS: { finals: [ <SPEC per item> ] }. The control plane reviews each spec,
 * then writes it into GTD (update_item AC/files/scope + status), or keeps it `new`
 * with the blockers recorded if readiness != "ready". Do NOT have the workflow write
 * GTD directly — the review is the checkpoint.
 *
 * COST: ~4 agents per item (1 draft + 2 critics + 1 synth). Pipelined across items.
 *
 * Notes: plain JS (no TS). No Date.now()/Math.random(). args may arrive stringified.
 */

export const meta = {
  name: 'groom-to-ready',
  description: 'Groom GTD items to dispatch-ready via draft→critic→synthesize, grounding every AC in the current codebase. Pass project context in args.context and the items (with seeds) in args.items.',
  phases: [
    { title: 'Draft', detail: 'read the cited docs + actual repo, draft a code-grounded spec per item' },
    { title: 'Critic', detail: 'code-grounding + concrete-by-default lenses per draft' },
    { title: 'Synthesize', detail: 'fold critiques into a final spec with an honest readiness verdict' },
  ],
}

const SPEC_SCHEMA = {
  type: 'object',
  additionalProperties: false,
  properties: {
    item_id: { type: 'string' },
    title: { type: 'string' },
    readiness: { type: 'string', enum: ['ready', 'blocked-on-prereq', 'needs-decision'] },
    blocking_prereqs: { type: 'array', items: { type: 'string' } },
    acceptance_criteria: { type: 'array', items: { type: 'string' }, minItems: 1 },
    files_to_modify: {
      type: 'array',
      items: {
        type: 'object',
        additionalProperties: false,
        properties: { path: { type: 'string' }, change: { type: 'string' } },
        required: ['path', 'change'],
      },
      minItems: 1,
    },
    scope_out: { type: 'array', items: { type: 'string' } },
    build_engine: { type: 'string' },
    open_questions: { type: 'array', items: { type: 'string' } },
    grounding_notes: { type: 'string' },
  },
  required: ['item_id', 'title', 'readiness', 'blocking_prereqs', 'acceptance_criteria', 'files_to_modify', 'scope_out', 'build_engine', 'open_questions', 'grounding_notes'],
}

const CRITIQUE_SCHEMA = {
  type: 'object',
  additionalProperties: false,
  properties: {
    lens: { type: 'string' },
    ready: { type: 'boolean' },
    findings: {
      type: 'array',
      items: {
        type: 'object',
        additionalProperties: false,
        properties: {
          severity: { type: 'string', enum: ['critical', 'major', 'minor'] },
          issue: { type: 'string' },
          fix: { type: 'string' },
        },
        required: ['severity', 'issue', 'fix'],
      },
    },
  },
  required: ['lens', 'ready', 'findings'],
}

const UNIVERSAL = [
  'You are GROOMING a GTD item to dispatch-ready. You are NOT implementing it. Do NOT modify any files; read only.',
  '',
  'CARDINAL RULE: ground every claim in the CURRENT codebase. Design docs may be ahead of (or behind) reality — a doc claim is a claim to VERIFY by opening the file. Before an AC says "replace X with Y", confirm BOTH that X exists where claimed AND that Y exists. If a prerequisite the item depends on does not exist yet, that is the single most important thing to report.',
  '',
  'HONEST READINESS (the readiness field):',
  '- "ready" — every prerequisite exists in the code today and a build agent could one-shot it.',
  '- "blocked-on-prereq" — depends on code that does NOT exist yet; list exactly what is missing in blocking_prereqs. Still produce the best AC you can, but be explicit it cannot dispatch until the prereq lands.',
  '- "needs-decision" — a real design ambiguity needs the human before grooming can finish.',
  'Do NOT inflate readiness. A confident "ready" against a phantom API is the worst possible outcome — it sends a build agent to implement against something that is not there.',
  '',
  'QUALITY BAR (concrete-by-default): enums are enumerated with members + semantics; formulas/constants are pinned to literals (a starting value, never "an appropriate value"); EVERY files_to_modify path and EVERY symbol/function/schema an AC names is verified by opening the file; no "appropriate"/"correct"/"as needed" acceptance criteria — each must be a testable assertion; end grounding_notes with a "soft values / deferrals" line listing anything not pinned.',
  '',
  'build_engine convention: prefer "claude-code-sonnet" for well-scoped non-trivial execution.',
].join('\n')

const ARGS = typeof args === 'string' ? JSON.parse(args) : args
const ITEMS = Array.isArray(ARGS) ? ARGS : (ARGS && ARGS.items)
if (!Array.isArray(ITEMS)) {
  throw new Error('groom-to-ready: expected args.items array, got: ' + JSON.stringify(args).slice(0, 200))
}
const CONTEXT = ARGS && ARGS.context ? String(ARGS.context) : ''
const SHARED = CONTEXT
  ? UNIVERSAL + '\n\n## Project context (where to look, conventions, ground-truth facts to treat as authoritative)\n' + CONTEXT
  : UNIVERSAL

const finals = await pipeline(
  ITEMS,
  // ---- Stage 1: DRAFT ----
  (item) => agent(
    [
      SHARED,
      '',
      `## Item to groom: ${item.title || item.slug || item.id}`,
      `GTD id: ${item.id}${item.priority ? ` | priority: ${item.priority}` : ''}`,
      '',
      'Seed (the design intent / current state to verify / scope — this is intent to STRUCTURE and VERIFY, not ground truth about the code):',
      item.seed || '(none provided)',
      item.criticalConstraint ? `\nItem-specific constraint:\n${item.criticalConstraint}` : '',
      '',
      'TASK: Read the cited docs AND the actual repo. First determine readiness (do the prerequisites exist in the repo today?). Then produce the groomed spec: testable acceptance_criteria tied to real code; files_to_modify each verified to exist (or marked NEW); scope_out boundaries; build_engine; blocking_prereqs (exact missing code, if any); open_questions (only genuine human decisions); grounding_notes citing the specific files you opened and what you found.',
    ].join('\n'),
    { label: `draft:${item.slug || item.id}`, phase: 'Draft', schema: SPEC_SCHEMA },
  ),
  // ---- Stage 2: CRITIC (two parallel lenses) ----
  (draft, item) => parallel([
    () => agent(
      [
        SHARED,
        '',
        `LENS: code-grounding / reality-check. Review this DRAFT groomed spec for "${item.title || item.slug}".`,
        '',
        'DRAFT SPEC:',
        JSON.stringify(draft, null, 2),
        '',
        'Open the actual files. Verify: (1) the readiness verdict is honest — if it says "ready" but a named prerequisite is NOT in the repo, downgrade it and say so; (2) every files_to_modify path exists (or is correctly marked NEW); (3) every symbol/function/type/schema the ACs name actually exists with the implied signature; (4) the blocking_prereqs list is complete and accurate. Flag phantom paths, wrong signatures, stale-design-doc assumptions. Set ready=true only if the spec is fully grounded AND its readiness verdict is honest.',
      ].join('\n'),
      { label: `critic:grounding:${item.slug || item.id}`, phase: 'Critic', schema: CRITIQUE_SCHEMA },
    ),
    () => agent(
      [
        SHARED,
        '',
        `LENS: concrete-by-default / spec-rigor. Review this DRAFT groomed spec for "${item.title || item.slug}".`,
        '',
        'DRAFT SPEC:',
        JSON.stringify(draft, null, 2),
        '',
        'Find every place the spec is soft where it must be concrete: an enum member missing or its semantics unstated; a constant/format left as prose instead of a literal; an AC using "appropriate"/"correct"/"as needed" instead of a testable assertion; a missing test for a stated behavior; a missing soft-values/deferrals note. For each finding give the EXACT fix. Set ready=true only if a build agent could one-shot this with zero design guesses (assuming prereqs are met).',
      ].join('\n'),
      { label: `critic:rigor:${item.slug || item.id}`, phase: 'Critic', schema: CRITIQUE_SCHEMA },
    ),
  ]).then((critiques) => ({ draft, item, critiques: critiques.filter(Boolean) })),
  // ---- Stage 3: SYNTHESIZE ----
  (bundle) => agent(
    [
      SHARED,
      '',
      `## Finalize the dispatch-ready spec for "${bundle.item.title || bundle.item.slug}" (id ${bundle.item.id})`,
      '',
      'DRAFT:',
      JSON.stringify(bundle.draft, null, 2),
      '',
      'CRITIC FINDINGS (apply every actionable one; re-open any file a critic flagged to confirm the correction):',
      JSON.stringify(bundle.critiques, null, 2),
      '',
      'Produce the FINAL spec. acceptance_criteria testable + tied to verified code; files_to_modify confirmed (mark NEW clearly); scope_out explicit; build_engine set; item_id = the GTD id above. Set readiness honestly per the rules: "ready" only if all prereqs exist in the repo today; otherwise "blocked-on-prereq" with the exact missing code in blocking_prereqs. open_questions empty unless a real human decision remains. grounding_notes: what you verified + the readiness rationale + a soft-values/deferrals line.',
    ].join('\n'),
    { label: `synth:${bundle.item.slug || bundle.item.id}`, phase: 'Synthesize', schema: SPEC_SCHEMA },
  ),
)

return { finals: finals.filter(Boolean) }

The Player-Coach Decade

2026-06-05T00:00:00-04:00

Opus 4.6 shipped in February. That was the turning point — not the launch of Claude Code a year earlier, which was already good. I used it all through 2025 and got a ton done, but I was still steering constantly: hand-holding, really, and reviewing every line like it might bite me. With 4.6, that stopped. The best engineers I know now tell me they haven't written code in three or four months. These are the same people who told me, late last year, that it "still isn't good enough for what I'm doing."

But that's not the interesting part. The interesting part is what happens when someone who doesn't primarily write code picks up the same tool. Specifically: the engineering leaders who never let their technical judgment atrophy.

I want to be precise about who I mean, because "manager" is a slippery word. I do not mean the pure manager — the one who manages from on high, runs the staff meeting, owns the org chart, and couldn't tell you why the last outage happened. That role exists at a lot of big companies, and it's part of the rot. Once in a while one of those people is a genuinely great leader; more often they just learned to play the game. The leaders I'm talking about are the opposite. They stayed close enough to the work to remain dangerous. They got promoted up the chain and never stopped understanding how the systems actually work. They can still read a system design and tell you what's wrong with it.

Those people are getting something different from this tool than many ICs are. The ICs are getting linear improvement — a faster version of how they already work. The technical leaders are getting multiplicative improvement — a way to do the work they already did, but at a scale that wasn't previously available to anyone.

That asymmetry is going to reshape the next decade of engineering careers, and the people in the affected seats deserve to hear about it before the role guidelines catch up.

What the leader is bringing to the tool

The standard frame for "getting good at AI" is "AI fluency." Prompt engineering. Context windows. Tool use. The dominant intervention is training: workshops, courses, internal evangelism. The implicit theory is that AI is a new technology and engineers will be effective at it once they learn how to operate it.

That theory is almost entirely wrong. The skill that separates effective from ineffective use of agentic AI is not prompting. It's something competent engineering leaders have spent a decade or two doing every day — I've been running technical teams for twenty years:

Decomposing ambiguous work into well-scoped pieces — taking "we should make checkout better" and turning it into a handful of crisp tasks, each with a clear boundary.
Setting acceptance criteria up front — knowing what "done" looks like before anyone starts.
Delegating effectively — handing off work with enough context that it comes back right. For a senior leader there's no urge to grab the keyboard; that urge belongs to the IC who just became a manager. Delegation is simply the shape of the job.
Inspecting the work, not producing it — and inspection in the broad sense, not line-by-line PR nitpicking. Is the output right? Does the team that produced it have a review process you'd trust? Where's the systemic weakness that's going to manufacture the next bad output? That's systems thinking, not code review.
Holding the bigger picture across many parallel work streams — knowing which thread can wait, which is on the critical path, and what depends on what.

These are the skills of running a team. And then a team of teams. It's layers on layers on layers — each one shielding the one above it from detail it shouldn't have to hold. (The Harness Is a Stack and It's Not the Model are both about exactly this: every layer's whole job is to protect the layer above it from toil.)

They are also, exactly and without modification, the skills of running a stack of agents. The agent doesn't care that it isn't human. It needs the same things from you: clear scope, clear acceptance criteria, and the willingness to let go of the work once you've handed it off. And it needs a competent reviewer waiting at the other end — which, if you're operating as a leader, probably isn't you. Your job isn't to be the reviewer. It's to make sure the work gets reviewed, in a loop that runs without you. Reviewing it yourself doesn't scale, and scaling yourself is the whole game.

A leader who has done this for decades walks up to Claude Code and immediately knows how to use it — not because they're "AI fluent" but because they've been operating at this layer of abstraction for their entire second or third decade in the industry.

The IC who has spent years optimizing personal throughput is doing something different and harder: they're trying to also learn how to operate at this layer, while continuing to default to the layer below it. AI fluency training won't fix that. Operating-model training might.

Why the chops matter

It would be tempting to read this as "leaders are better at AI than ICs." That isn't the claim. The claim is narrower, and the qualifier is load-bearing: it's leaders who kept their technical chops.

And by chops I don't mean the in-the-weeds, get-the-syntax-right kind. The model is better at that than you are now. I mean design judgment: what the system should look like, which features actually exist, how they fit together — and the instinct to notice when the model is confidently building against a world that no longer exists.

Here's one from last weekend. Opus 4.7 and 4.8 dropped temperature, top_p, and top_k entirely — set any of them to a non-default value and the API returns a 400. You're meant to steer the model with prompting now. I caught Opus 4.8 writing code that calls Opus 4.8 on Bedrock with temperature sprinkled all over it — the model confidently generating calls that its own API would reject. I caught it in seconds. Not because I'm a faster coder, but because I'd skimmed the Opus 4.8 "what's new" page, and "temperature isn't a thing anymore" is the kind of fact that jumps out and sticks. I made the model go read the page, fix the code, and write the gotcha into its knowledge base so the next agent wouldn't step on the same rake.

That's the whole skill in one anecdote. Skimming a "what's new" page costs five minutes. Knowing that it matters — and knowing to go check current reality instead of trusting the model's training-data priors — is the chops. A leader who kept them reads the model's confident answer, smells what's wrong, and overrules it. A leader who let them atrophy ships the wrong architecture, faster.

This is why the intersection is narrow. The orchestration skill doesn't help if you can't tell good work from bad. The chops don't help if you're still hand-rolling every PR. The combination is what's rare, and the combination is what wins.

Most leaders I work with aren't in that intersection — and the usual diagnosis gets it wrong. A lot of them promoted out of writing code years ago, and that, by itself, is fine. Code is the cheapest input in the stack now; nobody needs you to hand-produce it. The problem isn't the leaders who stopped writing code. It's the ones who stopped understanding systems. That skill still matters, and it's the expensive one to get back once you've let it atrophy.

Career advice the puzzle palace will get to in two years

If you're an IC right now — at any level — you're sitting on a question your manager probably can't answer yet: what should I be building toward over the next five years? It's sharpest for early-career devs, because the skills that carried the generation above them up the ladder aren't the skills this moment rewards — not all of them, and not only them. The role guidelines will catch up eventually. Probably two years too late, after the labor market has already repriced the answer.

Here's the advice you deserve right now, not later.

Get out of your IDE. I haven't opened an IDE in months. It's built for an operator I'm no longer being — a human typing into a buffer, finding the file, reading the code, navigating to the next one. That loop isn't my bottleneck anymore. Deciding what to build, decomposing it, dispatching it, and inspecting what comes back is — and the IDE helps with none of that. As long as you live in the editor, you'll keep defaulting to the layer of work the editor optimizes: the layer that no longer matters.

Build the thing that decomposes — don't be the thing that decomposes. Decomposition is still the single most valuable skill in the stack, and it can't be faked. Hand-writing the specs yourself is a machine's job, and most people have figured that out. The subtler trap is the one the tools are walking people into. AWS shipped Kiro IDE last fall with spec-driven development built in: an agent generates a requirements doc, then a design doc, then a task list, with a human sign-off at each step. It's a real improvement over vibe-coding. It's also already too slow — because it gates every spec on a human reviewing it.

That bottleneck is about to get brutal. Anthropic says Mythos-class models are landing in the coming weeks. When systems take shape that fast, your mental model of the system can't expand quickly enough to review the spec that defines it. Writing the spec by hand is wrong — but reviewing every spec by hand is the next bottleneck. You'd be gating the whole factory on how fast one human can build understanding.

So take the human off the critical path of spec review — not by skipping review, but by building the review into the mechanism. Mine is an adversarial grooming workflow: a drafter writes a spec, two critics try to tear it apart against the actual code, and a synthesizer reconciles them, so every spec that reaches a build agent has already survived a fight. It produces better specs than I can — for systems evolving faster than I can build my own picture of them. I write intent; the factory writes the specs and vets them.

Here's the part that actually matters. Dynamic workflows shipped in Claude Code, and a few days later I told it to build me that grooming workflow. I knew to ask for it. The agent didn't offer — it can't see the shape of the factory I'm building. That instinct — knowing what to construct at the edge of what the tools can already do — is the skill. And when a new pattern emerges, a failure I hit twice or a lesson that cost me an hour, I don't just remember it. I extract it and wire it into the infrastructure: a rule in the grooming workflow, a feature in Agent GTD, a new commit-time quality gate. This isn't (just) about smarter instructions in a CLAUDE.md. It's about codifying every hard-won lesson into the machine that builds, so the factory enforces it from then on. (Invest in the Flywheel, not the Feature is the long version: when the output is bad, you improve the factory, not just the code.)

Design a review process that doesn't depend on you. Yesterday Anthropic published When AI builds itself and reported that Claude wrote more than 80% of the code merged into its own codebase in May — and that human review has become the bottleneck, because people can't read code as fast as the model writes it. That's the whole future in one data point. If shipping depends on a human reading every diff, that human is the ceiling.

The way out isn't reviewing faster. It's trusting a system of signals instead of your own eyes on every line. Green linters, ninety-nine-percent test coverage all passing, integration tests green, the app doing what I expect when I poke it — that combination is a powerful indicator the code is probably right. Stack adversarial critics and agents reviewing agents on top, and reserve your own judgment for the few calls that genuinely need a human. Could a bug slip through? Sure. Engineers have shipped bugs for as long as there's been software, and most of them aren't fatal. The risk nobody prices correctly is the other one: moving slow while your competitors move fast. Right now I see a lot of good engineers stuck in the fast lane going slow — hand-reviewing every plan, fact-checking every research result — because they've decided that shipping "bad code" is the worst outcome. In this moment, it isn't.

Find the edge of the harness and build past it. A year ago, the advice I gave people was: use the tools, build intuition for what's now possible. That advice is stale. If you're still in the intuition-building phase, you're behind — and in more trouble than you realize. The work now is to manage teams of agents and to build past the end of the harness: notice where the tools stop, and construct the missing piece yourself, the way I did with the grooming workflow. Keep using the tools — use them like they're going out of style, because pushing them to their limit is how you find the edge.

The next decade

The next decade's most effective engineers will look a lot more like player-coaches than ICs — the ones who write a little, inspect a lot, decompose constantly, and run a fleet of agents with the same instincts a good leader runs an organization. And it won't stay a flat fleet for long. It'll be a stack: agents dispatching agents dispatching agents, all the way down, with the human operating only at the top, spending judgment on the decisions that can't be pushed any lower. The seat you're growing into isn't the senior-IC chair as currently defined. It's a seat the org chart hasn't drawn yet.

You can wait for the puzzle palace to publish new role guidelines. They'll get there — two years late, when the advantage is already priced in and gone. Don't. The skills that got the people above you promoted only partly transfer, and the seat you want isn't one anyone hands you with a title. You build it yourself, wherever you already are, by making the output impossible to ignore — until you're the person who defines it.

The seat is open.

Working "Agent First"

2026-06-05T00:00:00-04:00

Your job used to be doing the work. It isn't anymore. Your job is supervising a fleet of agents who do the work.

That shift has a consequence most workplaces haven't metabolized yet: every piece of information, every system capability, every artifact in your orbit has to be accessible to agents first. Not "also." First. The agent is the one doing the work now — if your agent can't reach something, it might as well not exist.

I've started calling this working agent first, and once you see it, you can't stop seeing the violations.

The SharePoint incident

Earlier this week, someone at work helpfully shared a SharePoint folder with me, full of Word docs containing Q3 requirements. Oof.

So I:

clicked the link in the email to open the folder in my browser
clicked "Add shortcut to OneDrive"
opened Finder on my Mac
navigated to my OneDrive folder and selected the newly synced folder
right-clicked the folder name in the path bar and selected Copy "Q3 Notes" as Pathname — a great trick for getting a path you can drop straight into Claude Code; hat tip to Todd for this one
pasted the pathname into Claude Code and said "convert this mess to .md so we can get started"

Six manual steps of human middleware before any actual work could begin.

But here's the detail that makes the story: the Word docs were full of green checkmarks and red X's — the typographic tells agents love to sprinkle around. An agent wrote these requirements. Then a human exported them to Word, uploaded them to SharePoint, and mailed me a link, so that I could spend six steps converting them back into the format they were born in.

The context started agent-native. It got printed onto a human-ritual artifact for transport. And I had to scan it back in on the other end.

We've seen this movie before.

It's a check

Say you owe me $100. You have the money in your checking account. So you:

get out your checkbook
write a $100 check
put the check in an envelope
write my address and your return address on the envelope
put a stamp on the envelope
drop it in the outgoing mail

A few days later (skipping the steps performed by the good people at USPS):

I fetch the envelope from the mailbox
I open it and remove the check
I sign my name on the back
I write "for deposit only" under my signature
I open my bank's iPhone app and navigate to the deposit section
I choose an account
I thumb in the $100 amount
I photograph the front of the check
I photograph the back of the check
the app confirms the deposit was received

Then I throw the check away.

Whew. I'm tired and cranky just thinking about it.

But the check was never the point. The goal was to move funds from one account to another — a database transaction. The check is a paper serialization of that transaction, designed for a world where the only way to move information between banks was to physically mail it. The mobile deposit flow gives the game away: the system's final step is to OCR the paper back into the transaction it always was. The artifact exists only to be destroyed after parsing.

That's what a Word doc full of agent-generated checkmarks is. The goal was never to write a doc. The goal is to move context from your agent to mine. The doc is a serialization of agent-native context into a human-ritual artifact, delivered by human carrier pigeon, scanned back in at the destination.

We've run this mandate before

In 2002, Bezos issued the now-famous API mandate at Amazon: all teams expose their data and functionality through service interfaces. No other form of interprocess communication allowed — no direct links, no shared memory, no back doors. All interfaces designed to be externalizable. The mandate is a big part of why AWS exists.

The point wasn't APIs for their own sake. The point was that human-mediated data sharing doesn't scale, and the only way to prove it is to ban it.

Agent first is the same mandate, one level up. If a capability only exists behind a GUI, it doesn't exist for your fleet. If knowledge only exists in a Word doc behind SSO, your agents are working blind and you've appointed yourself lead carrier pigeon. Nice. You look good in gray. Brings out your eyes.

Every artifact that requires a human to transport, transform, or transcribe it is a tax on the only resource that matters now: how much work your agents can do without you in the loop.

Build the agent interface first

This isn't just about how we share. It's also about how we build.

When you build something new, don't build the human interface. Yet. Build the agent interface — the MCP server, the CLI, the API — and then use the thing exclusively through your agent. See what friction emerges. Where do you get frustrated? Where do you lose the plot? What's genuinely hard to reason about when your only interface is natural language?

Cool. Now go build that human interface. Not the one you would have designed up front — the one the friction demanded. Let it emerge, because the primary user is the agent. You're just a client.

I built Agent GTD exactly this way — MCP and API surface first, web UI later, and only for the things narration handled badly. To a fault, honestly: there's API and MCP surface that still hasn't made it into the UI, and every so often an agent casually mentions a feature of my own tool that I didn't know it had. I learn about my product from my users. That's agent first working as intended — the UI is a partial view of the system, not the system a partial automation of the UI.

This is the design method that follows from the asymmetry I keep writing about: the agent can hold and manipulate far more than you can, but some things — scanning fifty tasks at a glance, watching a wave of parallel dispatches move — are genuinely better rendered for human perception than narrated in prose. You can't know which things those are in advance. You find out by working agent first and noticing where conversation fails you. Human-readable turns out to be a rendering concern, not a storage or transport concern. Markdown is both, which is why it's the native file format of this era and Word is not.

Monday

Default to plain text. Markdown for documents, structured formats for data. If your artifact needs a specific app to open, it's a check.
Share sources, not exports. A repo path, an API endpoint, an MCP server — not an attachment. Moving context should be a transaction, not a mailing.
Build the agent interface first. Human UI second, and only where natural language demonstrably fails.
Audit your week for carrier-pigeon work. Every time you manually move information between a system and your agent, that's a check you're mailing. Each one is a candidate for elimination.

The software factory runs on context the way a physical factory runs on parts. You can have the best line in the world, but if every incoming shipment arrives as paper that someone has to re-key at the loading dock, throughput is capped by the dock — not the line. When the output is bad, you improve the factory. When the input is slow, you fix the supply chain.

Agent first is the supply chain fix. The doc was never the point.

Stop mailing checks.

The Knowledge Base That Whispers

2026-06-11T00:00:00-04:00

A few weeks ago an agent working in one of my projects needed to make a web app reachable across my home network. It reasoned, confidently and at length, about WireGuard interfaces, per-client addresses, CORS allow-lists, and port-forward workarounds. All of it was wrong. My desktop doesn't terminate the VPN; pfSense does, and every client — wired or VPN — sees one flat LAN with one stable hostname per service. The correct answer was one sentence.

That sentence was already written down. It lived in my knowledge base, in an entry that exists specifically because agents keep getting this wrong. The agent never looked, because it didn't know its mental model was stale. You don't search for corrections to beliefs you don't know are beliefs.

I've written before that most agent failures are context failures or orchestration failures, not capability failures. This was the purest context failure I'd ever logged — and it exposed a structural hole in how I'd built my knowledge layer. Everything was pull: the agent asks, the KB answers. But an unknown-unknown can't be pulled. Somebody has to tap you on the shoulder.

So we built a knowledge base that whispers.

Why whispering is hard

The obvious version of this feature is terrible. Run a search over whatever the agent is saying, inject the top hit. We measured it: 18% precision. Four out of five injections would have been noise — or worse, plausible noise. A frontier model that receives an unrequested-but-authoritative-looking pointer anchors on it. This isn't a hunch — the anchoring bias that skews human judgment transfers cleanly to large models, and the RAG-poisoning literature shows a single injected, authoritative-looking passage can override the genuine evidence a model would otherwise reason from. A wrong whisper doesn't just cost tokens — it corrupts what the model reasons from next. A confident-wrong whisper is worse than silence.

That asymmetry became the design's spine: a hard gate of at least 0.8 precision with approximately zero wrong injections before anything touched a live session, measured on an offline harness of real transcript slices — the actual wrong words real agents wrote in real sessions, including the WireGuard incident as golden case #1, plus a pile of hard negatives: moments that sound infrastructural but need no correction.

The harness then killed two ideas I was fairly sure would work:

Similarity thresholds don't gate relevance. Embedding similarity measures topical adjacency, not "this agent is guessing." Our hard negatives — an agent fluently narrating dispatch waves, a debugging session full of infrastructure vocabulary — scored higher than the golden positives. There is no threshold that separates them, because they aren't separated in embedding space.

Richer evidence made the judge worse. We gave the relevance filter the knowledge entries' actual documented facts, expecting sharper verdicts. Precision dropped. More material just gave the model more to rationalize a topical match with. I would have bet money the other way.

What actually worked: structure, not scale

The wins were embarrassingly cheap. Two structural rules did more than any model upgrade could have:

Cross-project only. Never whisper a project's own documentation into a session working in that project — the session-start surface already covers that. The whisper exists solely for knowledge from other domains.
Never whisper about a system the agent is operating. This is my favorite piece of the design. The session transcript names every tool the agent has called. An agent calling dispatch_rollout is operating the dispatch system — it doesn't need the dispatch orientation map; it's holding the controls. An agent making claims about VPN topology has no VPN tools to call. No tools for the domain + confident claims about the domain = guessing from priors. That's exactly the moment to whisper. Each knowledge map carries a hint naming the tool surface that "operates" it; the listener drops candidates whose tools appear in the transcript.

On top of those rules sits a deliberately mistrusted model: three independent votes, retrieve-and-cite only — the filter can return an ID from the retrieved candidates or NONE, never prose, enforced in code rather than in a prompt — and any disagreement means silence. Final harness run, through the deployed endpoint: 1.00 precision, zero false injections. Coverage is around half — the listener misses real moments, by design. Silence is cheap. Wrong whispers are fatal.

The delivery is a hook with no synchronous cost: when an agent finishes a turn, a detached worker analyzes what it just said; if a pointer survives the gauntlet, it arrives at the next turn, one factual line:

Possibly relevant map — [kb-01724] Home network & lab infrastructure

It arrives as one line — a pointer phrased as a fact, from a renderer whose tests assert it can never emit an imperative verb. The agent can pull the entry or ignore it. Stale beliefs persist across turns, so one turn of lag costs nothing.

It whispered during its own build

The day we shipped it, while I was discussing which machine should host the embedding model, the hook context of my next message quietly included the pointer to the home-network map. The agent in that session was making claims about my machines; it had no tools for them; the listener noticed. The system tapped us on the shoulder during the session that built it, about the exact domain that motivated it.

From one gate to a fleet

That's the single-KB story. What runs now is a small fleet. I keep more than one knowledge base — a personal one, a team one, and a third — and the listener fans out across all of them. Each KB runs its own copy of the gate, independently, over the same slice of transcript; the per-KB winners are unioned by a client-side arbitration step that can only suppress, never promote. It dedups and it silences, but it never elevates a candidate that a KB's own gate didn't already bless — no client-side relevance scoring sneaks back in through the union. Three hosted instances, three separate unanimous-3 votes, at most one pointer per KB per turn.

The whole thing sits behind a kill switch (KB_LISTENER_ENABLED, default off). This is a pilot, and the honest default for an unproven channel is silence — you opt a stack in, you don't opt it out.

I also stopped flying blind. Every listener decision now returns a compact reason — matched kb-01724 (unanimous 3/3), or no-injection: all candidates dropped by rule-A, or best candidate not unanimous (2/3) — and the hook writes a real-time, per-session decision log to disk. When the channel stays quiet I can see why it stayed quiet, which is the whole difference between "the gate is working as designed" and "the gate is broken and I have no way to tell."

Does it actually help? I don't know yet

The entire argument of this post is that context beats capability — and that argument only holds if the context actually lands.

Precision I can defend: the gate fires rarely, and when it fires it fires on the right domain. Over about three weeks and 20 sessions the system emitted 610 whispers — but 598 of those were the deterministic session-start roster pushes, the boring always-on directory of what exists. Only 12 were precision-gated anticipatory-listener fires. Twelve. The listener is strikingly sparse, exactly as designed: it would rather say nothing than say something plausible-but-wrong.

But sparse-and-correct is not the same as useful, and usefulness is the number that matters. So I instrumented it — a telemetry sink that logs every whisper and, the load-bearing signal, whether the whispered map was actually pulled afterward. Did the agent take the tap on the shoulder, or ignore it?

And then I found out I'd been measuring nothing. The consume-detection hook — the one piece that closes the loop from "whisper emitted" to "whisper used" — had been silently unwired the whole time. Every whisper read as unused. A flat 0%. Whispers weren't failing; the wire that would have told me was cut. Which is the most on-brand failure this project could possibly have: I couldn't see that my own measurement was broken, so I couldn't go looking for the correction — the exact unknown-unknown shape the whole system exists to catch. I only caught it because the zero was too clean to be real.

That's fixed now. Real usage data starts accumulating from here. So this is where the post has to stop and wait for the data:

I built an instrument to answer "do whispers help," pointed it at the system, and learned the instrument itself was broken. Fixing that is a small proof of the thesis on its own — the failure was invisible until I built the surface that made it visible. But the verdict on whispers themselves is genuinely still out, and I'd rather ship that sentence than a triumphant one I can't yet back with data.

That loop — failure logged as knowledge, knowledge wired into a surface, surface gated by an eval harness built from the original failure, harness killing the seductive designs and blessing the boring structural one — is the whole flywheel argument in miniature. The model never got smarter. The context around it did. And the next agent that starts confidently riffing about my VPN will get one quiet line of correction offered before it builds the wrong thing — whether it takes that correction is the thing I'm now, finally, wired up to find out.

Capability is the model's job. Knowing when you're wrong is the harness's. Build the part that whispers.

It's Still Not the Model

2026-07-08T00:00:00-04:00

Two months ago I made a claim I couldn't measure. I argued that when a capable model fails on a task you actually needed done, the cause is almost never capability — it's one of two things: the model didn't have the right context, or the workflow wouldn't let it use what it had. Context failures and orchestration failures. I argued it in theory, with anecdotes. Now I've measured it.

So I ran it. Two small local models — qwen3.6:35b and gpt-oss:20b — that had struggled as engines under Claude Code, a big general-purpose harness, in my headless dispatch system completed 58–75% of a set of real coding tasks when I dropped them into a lean harness I built from scratch. The catch, which I'll get to: those fixture tasks are much smaller than dispatch-scale work, so this isn't a clean rematch. But the same weights that reached almost nothing under the big harness reached most of these under the lean one. Same models. Different machinery around them. The capability didn't move. What moved was whether anything could reach it.

That's an orchestration failure made visible. And the variable that made it visible is the one almost nobody isolates: the harness.

A harness you can hold still

I've been building a minimal agent harness in Rust — the loop that turns an LLM into an autonomous agent: prompt assembly, tool dispatch, model I/O, state, and the safety rails around all of it. The stated goal is to learn the design space by building one, with a concrete use case (another build engine for Agent GTD) keeping me honest. The unstated bonus, which I didn't fully appreciate until session four, is that a harness you build yourself is a harness you can hold still.

That's the property that makes it an instrument. The loop assembles one byte-stable system prompt and hands the model exactly six tools: read_file, list_files, edit_file, run_command, run_checks, and finish. Every model runs through identical machinery. The trick that makes "identical" true is an anti-corruption boundary drawn high: a ModelBackend trait where each adapter owns all the wire translation and hands the loop a normalized AssistantTurn. The loop never sees a provider-native shape. Anthropic's ID-tagged tool_use blocks and Ollama's ID-less tool calls come out the same door.

I got to watch that boundary pay off for real. The day I merged the Ollama adapter, an entirely different wire protocol — no tool-call IDs at all, so you match calls to results by name and order — drove the unchanged loop. One adapter even serves both localhost Ollama and the cloud endpoint, because they're wire-identical; only the auth header differs. When you can swap the model backend and change nothing else, the model becomes the only variable in the experiment. That's the whole point.

Why "58%" isn't a lie the model told me

Before any number in this post means anything, you have to trust the word "completed." Models hallucinate "done" confidently, and small local models do it more. So the harness never trusts the claim.

finish(done) is a claim. When the agent calls it, the harness runs the checks itself — it runs cargo test — and only if they pass does it construct a Done value. And here's the part Rust gives you for free: a Done is not constructible without a CheckReport or an explicit no-checks marker. The type carries its own evidence. A criterion can be marked Verified(Evidence), and only the harness ever writes that state — the agent literally cannot. A red check isn't a failed completion — the loop feeds the failure back and the model keeps going.

So when I report a pass, it's mechanically green, verified by the harness. When I report a failure, it's the loop bounding a model that couldn't converge — never a false "done." Across every verified trial in the metrics run, there were zero false dones. Every pass was cargo test green. Every failure was a model that couldn't converge.

(One of the build agents, unprompted, invented a private FinishClaim type the spec never named — a pre-verification view of the agent's assertion, separate from the verified Done — because the type system rewarded the distinction. I didn't ask for that. The "illegal states unrepresentable" ethos had propagated into the agents building the thing. More on that at the end.)

The matrix: same loop, four models

Here's the core evidence. Same loop, byte-identical prompts and tools, the same four fixture crates, k=3 trials each, max 12 iterations, every "done" verified by the harness running cargo test itself.

model	where	pass^k	mean iterations	notes
`claude-sonnet-4-6`	Anthropic API	11/12	5.00	1 StoppedWithoutFinish; an earlier run was 12/12 → combined 23/24
GLM-5.2 (756B)	Ollama cloud	12/12	5.00	found the intended fix every trial
`gpt-oss:20b` (think=high)	localhost Ollama	9/12	7.0–8.7	2× MaxIterations, 1× StoppedWithoutFinish
`qwen3.6:35b` (think unset)	localhost Ollama	7/12	8.3–11.3	all failures MaxIterations; ~2× GLM's iterations, ~3× tokens

Two wrinkles jump out of the data, and I'd rather name them than let you find them.

First, k=3 means real variance, even at the frontier. Sonnet landed 11/12 in this run but 12/12 in an earlier pre-metrics run — combined 23/24. Don't over-read a single-run pass^k difference; the noise floor is not zero.

Second, StoppedWithoutFinish — the model deciding it's done without calling finish — showed up at both ends of the capability spectrum, on sonnet and on gpt-oss:20b. That's a protocol-discipline failure — the model ignored the harness's contract for signaling completion — not a capability gap. The eval can measure it precisely, which is the point of having an eval that verifies mechanics instead of vibes.

The harness was the variable all along

Now the payoff, fenced with every caveat it needs to survive.

Under this purpose-built harness — six tools, a lean byte-stable prompt, mechanical verification — qwen3.6:35b and gpt-oss:20b completed 58–75% of the fixture tasks they'd failed to finish as engines under Claude Code. That is the definition of an orchestration failure from the first post: the capability was present; the workflow wouldn't let the model reach it. Change the workflow, and the capability shows up.

The caveats, stated plainly, because a claim this convenient deserves the scrutiny:

These fixtures are much smaller than dispatch-scale tasks. A fixture crate with a planted bug is not the same shape of work as a real GTD item, so this is not an apples-to-apples rematch — the local models didn't suddenly become viable dispatch engines. The think-level was uncontrolled between the two local models: gpt-oss:20b ran with think=high, qwen3.6:35b ran with think unset. So 20b outscoring 35b is suggestive, not a concluded capability ranking — I changed two things at once and won't pretend I didn't. And the whole thing is directional evidence for the harness-as-variable thesis, not a controlled harness-vs-harness study. I didn't run the big models through the small harness and the small models through both under a proper control matrix. That's a different experiment.

What I'll defend is the direction, and it's not subtle: the same weights that reach almost nothing under one harness reach most of these tasks under another. If capability were the binding constraint, that couldn't happen. It happened.

What the eval refused to prove

The most useful thing the eval did was refuse to cooperate with a story I wanted to tell.

I'd built three deliberately subtle bug fixtures to stress the frontier tier: a < vs <= boundary error in an interval merge, a missing recency-refresh in an LRU cache, a UTF-8 char-boundary panic in a text preview. Genuinely nasty bugs — the kind that survive a code review. Sonnet swept the suite 12/12, subtle bugs included.

That's the eval telling me my difficulty knob was the wrong knob — Sonnet isn't superhuman. A failing test does the bug localization for the agent — it points a bright arrow at the broken behavior. Subtlety of the bug barely matters once the test hands you the repro. The real difficulty lever is task shape: withhold the failing test, or hand the agent a prose bug report and make it find the repro itself. Bug subtlety saturates; task shape doesn't.

I'm reporting the eval's ceiling as a known limitation because that's what an instrument is for. It steers the roadmap. I don't hang it on a wall as a scoreboard. The same run surfaced the eval's own blind spot — failed trials are opaque, because there's no trajectory persistence, so a MaxIterations failure is just a shrug with no story. That opacity is now the data-driven argument for what I build next: durability and run-record wiring, so a failure explains itself. The eval found the hole in the eval. I'd rather that than a clean number I couldn't interrogate.

The factory that built the instrument

Here's the part that closes the loop back to the other posts. I didn't hand-write this harness. It was largely built by the orchestration practice those posts describe: 15-plus headless dispatch runs across the project, with an AI lead reviewing and merging every branch on the fully integrated tree — zero quality misses after a single typos miss in session two. The core of the v0.1.0 release landed as a three-wave overnight build, seven dispatches, roughly an hour of wall-clock while I was asleep.

The reason I trust these numbers is that the same verify-don't-trust discipline runs all the way down. The very first commit to this repo wasn't harness logic — it was the quality-gate scaffold: fmt, clippy at pedantic, nextest, coverage held at 95%, machete, deny, typos, gitleaks, enforced at commit and push. Gates before code, so the agents could run wild safely. Around 230–260 tests and ~98% coverage the whole way through. rustc itself is a gate — the borrow checker, exhaustiveness, no-null, Result-based errors give you for free what a pile of Python tooling has to buy.

And the releases are cut at capability boundaries, not engineering milestones. v0.1.0 was "the harness autonomously fixes a failing test in a real crate and verifies cargo test green itself." v0.2.0 was "the same loop, unchanged, completes the suite on non-Anthropic models, measured against verified checks." The test I apply is: what does the first CHANGELOG entry want to claim? If the answer is "the spine runs," it's not a release. If it's a capability, it is. That FinishClaim type from earlier is the tell: I never asked for it, but the illegal-states-unrepresentable ethos the factory runs on had started running in the agents too — they reached for the same distinction the harness's own design rewards. The instrument that measured orchestration was itself built by orchestration, down to the way its builders think. That's the argument itself, load-bearing.

Control for the harness

So here's the actionable version, and it's just the old diagnostic with data behind it now.

When a model fails on something you needed it to do, ask three questions in order. Did it genuinely lack the capability? Least likely — reach for it last. Did it lack the right context? Engineer the information layer. Did the workflow force it below its capacity? Engineer the workflow. Two of those three are fixable today, without waiting for a lab to ship anything.

The harness is the third one, and it's the variable almost every "the model can't do X" claim never isolates. People swap models, tweak prompts, adjust temperature — and hold constant the one thing that most determines whether the model's capability is reachable at all. Then they conclude the model can't. What they've measured is the harness.

I found this out by building one small enough to hold still. You don't have to build yours in Rust — you don't have to build a harness at all to get the lesson: before you conclude a model can't, control for the machine it's running inside.

And when you find the edge of that machine — the place where the tools stop and the capability is stranded just past them — don't route around it one more time. Build the missing piece. That's still the whole move.

Nobody Calls Finish

2026-07-11T00:00:00-04:00

I ran two small local models as build engines in my headless dispatch system — qwen3.6:35b and gpt-oss:20b, driving real GTD tasks under Claude Code — and neither one finished anything — not the hard tasks, and not the easy ones. They'd pick up an item, decide they needed to understand the codebase, and research it. Then research it more. The GTD task would fill with comments: researching codebase… researching codebase… researching codebase. They'd blow the context window, restart cold, and do it again, until the wall clock killed the run and the dispatch worker gave up. Hours of compute, zero merged commits. The work was, in some real sense, getting done inside those loops — and it never once said so.

That's a completed workout with a blank text box where the "done" should be. If you read the last post in this series, you know the image. This is that image at agent scale — and finish discipline turned out to be the last failure standing after everything else got solved, frontier and local alike: not the reasoning, the handoff.

Here's the sharper version of the same thing, and it's the reason I finally wrote this down. Earlier today my own Rust harness — the one I've been building across these posts, named Talos — failed to build a feature. Twice. Same task, same model, back to back. Then I handed the identical model weights to a different harness and it one-shot the whole thing clean. Same model, different machinery around it, and two failures became one success. That's the controlled experiment I've owed you since the last post — the same weights swapped between two harnesses, not a model that got smarter between attempts.

The control I admitted I ducked

In "It's Still Not the Model" I claimed the harness — not the model — is the variable almost nobody isolates, and then I said, plainly, that I hadn't run the clean control. Same weights, two harnesses, everything else held still. I ran directional evidence and called it directional. That debt has been sitting there. Today it got paid, not because I set out to pay it, but because the work forced it.

The task was the first item of the 0.4.0 bounded-autonomy wave: a finish-recovery protocol. Eighteen acceptance criteria across five files — a new RecoveryFacts schema type plus the loop-detection and nudge control flow that uses it. Real multi-file work, the shape of thing a dispatch engine actually has to do. I dispatched it to talos-glm: my young harness, driving glm-5.2:cloud on Ollama's cloud.

It failed. Then it failed again. The first run hit the iteration cap — MaxIterations at 24 — in about forty-eight seconds, never reaching a single verify cycle. The second spun all the way to fifty-six iterations, quit as StoppedWithoutFinish, and preserved no work in progress. Whatever it had built evaporated with the run.

So I pivoted. Same glm-5.2:cloud weights, but running inside Claude Code — a mature harness instead of my days-old one. It one-shot the entire eighteen-criterion item. Verified Done, all gates green, in about twenty-seven minutes. Commit 758cf4a on r7-research: 366 tests green, 97.72% coverage. Zero Anthropic spend, because GLM billed against Ollama cloud credits the whole way. I wrote the verdict straight into the commit message: after talos-glm failed it twice (MaxIterations@24, StoppedWithoutFinish@56): the gap was the harness, not the model.

There's an alibi that rules the model out cold. GLM scores 24/24 on my hardened eval ladder — tied with Sonnet on pass rate and ahead of it on efficiency. "The model wasn't capable enough" was never a credible reading of those two Talos failures. The weights that failed twice were the same weights that top my leaderboard.

And there's an irony that clinches it. Talos couldn't build finish-recovery precisely because Talos lacked finish-recovery. That second failure — fifty-six iterations, no work preserved — is a live demonstration of the exact two gaps the item exists to close: no nudge to force the completion, no preservation of good work when the model won't call it. The harness failed in the shape of the bug it was being asked to fix.

Two caveats before this gets oversold, because a result this convenient earns scrutiny. GLM is one model. This is a clean single-model control — same weights, two harnesses, opposite outcomes — not a full harness × model grid; I'm not claiming I mapped the space. And Talos's 24-iteration default was a blunt cap I'd tuned for tiny single-crate dogfood items; it blew out in forty-eight seconds on a five-file task before ever reaching verify. That cap is a harness design flaw, not a model limit or a safety feature, and I fixed it the plain way — bumped the default from 24 to 500 in d59b1d9. State the embarrassing part out loud; that's the deal.

The bottleneck is the last tool call

Name the bottleneck precisely, because it is not where people look.

The model does the work. The gates go green. The sealed holdout goes green. And then it never calls finish. It runs to the iteration cap and gets recorded as Failed. A real Done, mislabeled. Finish discipline was the universal residual failure across every one of the five models I measured before 0.4.0 — frontier and local alike, it was the last thing left standing after everything else got solved.

The tell that this is a harness gap and not a model gap: the identical failure shows up under claude -p dispatch too. When two unrelated model families fail the same way, that's a broken layer of the workflow they're both running inside, not an incapable model. This is the StoppedWithoutFinish I flagged in the last post as an unsolved "protocol-discipline failure" and left hanging at both ends of the capability spectrum. Here's the debt being paid.

The distinction that matters for the fix: the completed work is materialized correctly. The tree is right. The tests pass. Only the upward handoff is missing. It's a diligent junior who did the whole task and won't say the words "I'm done."

Force the handoff, never fake the claim

So the fix has one governing invariant, and it's the harness's oldest one: the harness never fabricates Done.

finish(done) is a claim. When the agent calls it, the harness runs the gate itself — cargo test against the acceptance criteria — and constructs a Done value only if it comes back green. Rust makes this structural: a Done is not constructible without its CheckReport evidence. The type carries its own proof, and only the harness ever writes the verified state. The agent literally cannot hand-author a green result. So whatever finish-recovery does, it is not allowed to invent completions.

What it does instead is force the handoff. Detect a green-and-static spin — the last run_checks came back green and the working tree hasn't changed for K iterations (K around 3) — and nudge the model: call finish(done) now, or give me a one-sentence status. Detection is deliberately high-precision. A model still actually working mutates the tree; a legitimately long task hasn't gone green yet. You only trip when work is demonstrably finished and demonstrably idle.

If the model still won't claim after N nudges (N around 2), the run terminates Failed{FinishDiscipline} — but it carries recovery facts out with it: gates_green_at_exit, tree_dirty, nudge_statuses. The worker reads those and preserves the WIP branch instead of deleting it. Because deleting a bad branch is cheap, and discarding hours of verified-green work on a signaling technicality is expensive. The claim moves up exactly one rung: if the model won't make it, the human lead gets an evidence-bearing package instead of a shrug. The harness never writes the Done the model didn't earn — it just refuses to throw away the work the model did.

The eval found the hole in the fix

Then I measured it, and this is the part that makes the post more than a victory lap.

I ran finish-recovery across the full six-model matrix — 144 trials — and it fired zero rescues. Not one. Zero FinishDiscipline terminals across every model. The fix fired zero times on the very suite I built to measure it. The null result is the finding. Finish-recovery catches models that spin — keep calling tools, green and tree-static for K iterations. It does not catch models that stop cold — emit a single no-tool-call turn and halt one iteration below the K-static threshold, before the nudge ever arms. Those two failure shapes look identical in the outcome column (StoppedWithoutFinish, work lost) and are completely different in the loop. My fix covered one of them.

Which one, and how many? That was undetermined in the matrix writeup — until a one-line instrument settled it. RunStats.gates_green_at_exit, mirroring the loop's last_gate_green at exit: seventy lines in engine.rs, seventy-four insertions across three files, commit bafb6ed. With it, I re-ran gpt-oss:20b — a fresh run, so its own StoppedWithoutFinish count — and classified its cold stops. Of fourteen StoppedWithoutFinish trials, six — about 43% — were post-green. Verified-green work, quit on and abandoned. The other eight stopped before the gate ever went green, which is honest non-convergence, nothing to rescue.

That ~43% scoped the next fix precisely: extend the nudge to also fire at the StoppedWithoutFinish terminal when the gate is green. Quantified upside — roughly double gpt-oss's pass rate on this run, six recoverable of twenty-four. I groomed it and dispatched it the same day. One caveat: gpt-oss:20b is high-variance across runs — six of twenty-four here, ten or eleven other times — so the robust signal is the post-green proportion, not any single count. And the stop-nudge extension is fresh: gate-green, built today, not yet battle-tested across a fleet. This is a scoped next step. I haven't shipped it yet.

The matrix

The hard evidence, on the 0.4.1 harness: eval config v5, every row: CODING_EVAL_K=3, max 24 iterations, eight fixtures — 24 trials per model, 144 total. Every "done" verified by the harness running the gate itself.

model	where	pass^k	holdout
`claude-opus-4-8`	Anthropic API	24/24	12/12
`claude-sonnet-4-6`	Anthropic API	24/24	12/12
`claude-haiku-4-5`	Anthropic API	24/24	12/12
`glm-5.2:cloud` (~756B)	Ollama cloud	24/24	12/12
`qwen3.6:35b`	localhost Ollama	24/24	12/12
`gpt-oss:20b`	localhost Ollama	10/24 (0.417)	8/12

Zero false dones across all 144 trials. That number is the choice/value boundary holding under load — because the harness only ever binds the value and never fabricates the claim, no run can report a Done it didn't verify. The one model the ladder still kicks is gpt-oss:20b; its v5 failures break down as ten StoppedWithoutFinish plus four MaxIterations.

Two things stand out. Efficiency is not capability: on the calc fixture, mean iterations run glm 4.00 < opus 7.33 < sonnet 9.00 < qwen 11.33 < haiku 15.33 — Haiku passes everything Opus does and takes more than twice the steps to get there. And the cap itself is load-bearing: Haiku's taskdeck run (the tier-4 write-your-own-tests fixture) averaged 19.0 iterations against the cap of 24 and burned 465k tokens. It would have failed outright at the old hardcoded cap of 12. The number in the config is a design decision that decides who passes.

The standing caveat: these fixtures are still much smaller than dispatch-scale GTD work. This is not the apples-to-apples rematch, and the small models did not become viable dispatch engines. Watch the gap between this table and the researching codebase… death spiral at the top of the post — same models, and the fixture is the reason one row reads 24/24 and the real dispatch read zero.

Finish is a Pick

All of this is one shape.

In "The Bounded-Choice Cascade" I named the mechanism under an orchestration fix: the reasoner never holds the pen on a value, only on a choice. Selection down, envelope up. I won't re-derive it — I want to extend it to the one boundary that post didn't touch: completion.

finish(done) is the executor's Pick. The model selects from a bounded disposition set — done, blocked, failed — and the harness never lets it emit the completion value. That's the Cascade verbatim, at the last rung: the pen is on a choice, not on "this is verified." The harness binds the value by running the gate itself; the disposition pick flows down into cargo test, and an evidence-bearing Done flows back up to the lead, who can merge or delete within it but can't re-litigate it. Selection down, envelope up — in a single call. The disposition-to-exit-code map is the concrete surface of that Pick: 0 verified Done, 10 Blocked, 20 Failed, 1 infra. Ratified back at 0.3.5, untouched by 0.4.0. No new exit code, because the shape didn't need one.

Now read the finish-discipline failure through that lens. It's the executor botching the envelope-up handoff. It materializes the whole thing — gates green, tree static, work done — and then just never hands it back. The completed workout, the blank text box. And finish-recovery closes it the Cascade way: it forces the handoff without ever making the pick for the model. If the model still won't claim, the claim moves one rung up to the human lead. The harness never fabricates a Done — which is exactly why zero-false-dones holds. Forcing the handoff and faking the claim are opposite moves, and the whole design is built to do the first and forbid the second.

And "the executor is a role, not a thing" stopped being a slogan today. Hold the model constant — GLM — swap the harness — Talos for Claude Code — and the outcome flips from two failures to a clean one-shot. The executor is a substitutable role, bounded by the spec and the harness around it. I asserted that last post. This post it's a measurement.

The factory that caught the trap

One more thing, because it's the reason I trust the fix and not just the story.

The stop-nudge extension in this very post was groomed by an adversarial critic workflow — draft, critic, synthesize — and that critic caught a trap CI could never have seen. The completion nudge has to be a fresh user message at the terminal, not an append onto the trailing message. Append it wrong and the nudge is silently dropped — and every mock test still passes, because the mocks don't model message boundaries. A green suite over a dead feature. The exact failure this whole post is about, one level up: the work looks done and never actually hands off. The critic flagged it before a line was written; then the fix went out to a cheaper model, claude-code-sonnet, to build. The instrument that measured the gap was a one-line instrument the loop's own tests pin.

0.4.0 shipped with 389 tests and about 97% coverage, gates before code, the way this repo has run since its first commit was the quality-gate scaffold rather than any harness logic. And claude-code-glm is now a proven zero-Anthropic-spend lane for complex, multi-file harness-core work — it shipped that 366-test, 97.72%-coverage finish-recovery change without touching the Anthropic account, because GLM runs on Ollama cloud credits. The cheap executor, doing real work, inside a harness that forces it to hand the work back.

Four posts, one variable

This is the fourth post on one idea. Name the stack. "It's Not the Model" named the disease: most AI failures are context or orchestration failures, not capability. "It's Still Not the Model" proved the harness is the uncontrolled variable — but admitted it never ran the clean control, and flagged StoppedWithoutFinish unsolved. "The Bounded-Choice Cascade" named the mechanism: pen on a choice not a value, selection down, envelope up, the executor as a role. This post ran the control the second one ducked, and solved the failure it flagged — the mechanism, landed on the completion boundary.

Read in order, it's one variable getting steadily isolated: named in theory, then shown to be uncontrolled, then given a mechanism, and now pinned by a control clean enough that the diagnostic is a measurement rather than a hedge about method. The remaining caveat isn't about how I held the variables still — it's just that I held one model still, and the grid is bigger than one row.

The honest coda, because the fix is not a free lunch. The harness forces the handoff and binds the value, but it's only ever as good as the finish contract on top and the gate underneath. A green gate on a weak spec still ships a mislabeled truth — claim-vs-verify is exactly as honest as what "verify" actually runs. The harness supplies the discipline. It does not supply the spec, and it can't verify what the gate doesn't check.

But the diagnostic is no longer a claim I'm asking you to take on faith. When a capable model stalls with the work done, suspect the harness before the model — I held the weights still and swapped the machine, and the machine was the answer.

So here's the one to keep in your pocket. When your executor goes quiet with the work already done, that's not a model that failed. That's a handoff the harness forgot to force.

The Bounded-Choice Cascade

2026-07-11T00:00:00-04:00

Grit Mile is an AI-powered run-training app I'm building as a side project. I started it because every training app I've tried opens with two or three weeks so easy they're a joke for me, then ramps so hard that by week five I can't hold on — what I want is a plan that responds to me the way a real coach would.

The first time Grit Mile's plan generator ran against live production Bedrock, it took eighteen minutes and thirty-one seconds to build a training plan. Twenty weeks, a hundred and forty-four workouts. Sixty-nine seconds elapsed before even week one appeared. Eighteen and a half minutes is not a latency budget. It's an outage with a progress spinner.

The architecture looked right on paper, which is what made it interesting. Deterministic orchestration in code, agentic reasoning at the leaf — exactly the shape I'd argue for. So my first reaction wasn't "the model is slow," it was worse: did we build plan-gen wrong? That question is expensive enough that I didn't want to answer it from a gut feeling, so I ran a four-lens latency audit before touching a line of code.

One detail from an earlier prototype comes back later, so set it here. A Grit Mile experiment had an AI "coach voice" narrate every workout in a fully pre-computed plan. On days the plan said hold steady, the voice layer produced an empty string. Nothing. Blank narration. Commit 051635d in the history is me decoupling move-proposal from the coach voice and fixing the empty-on-holds bug. It's the whole essay in one blank text box.

The villain is a repair loop

The audit found this: 57% of the entire wall-clock — 631 seconds across twenty-nine calls — was a repair loop. The micro tier would ask the LLM to propose the numbers for a week. Eleven code validators would check those numbers against the rubric. When they failed, the loop re-prompted the model to try again, and again, up to its budget. Twenty-nine of those re-invocations, more than ten minutes of clock, spent re-prompting the LLM back toward numbers the macro layer and the rubric formulas had already computed.

That's the one-line summary of the miss: deterministic code fighting the LLM over values the code already knows.

And the spend bought nothing. After all that repair traffic, the plan still shipped with twenty-seven soft-fail violations. The prompt fails validation 59% of the time — more than half the time the model proposes a week, the code rejects it — and only three of the twenty weeks validated on the first try. The non-LLM layer — all the orchestration, persistence, the actual code — summed to about six and a half seconds. 99.4% of the run was inside fifty strictly-serial Bedrock calls, most of them arguing.

The reflex fix is to swap the model. I tried it, because I wanted the number. Haiku 4.5 through the same pipeline was only 1.6× faster and shipped 2.3× the violations. A weaker model in a broken harness ships a worse plan faster, because the repair loop punishes it twice — once in latency and once in quality. Its softer instruction-following inflated the repair traffic, which is the exact thing eating the clock. The model was never the variable. This is the orchestration-failure bucket from It's Not the Model, made concrete enough to measure in seconds.

Invert it: the reasoner picks, the code binds

The fix is an inversion, and it's small enough to say in one breath.

Today's micro tier is "LLM proposes numbers, eleven validators reject, LLM re-proposes numbers." Invert it to "code composes the numbers, the LLM picks from a bounded menu, code binds the arithmetic."

The primitive I landed on I call the Pick. The LLM emits an archetype_id, some enum slot_choices, and a rationale — and never a number. It selects an archetype like a tempo-interval session from a catalog of named archetypes; it selects a distance from a range the code computed; a function called VdotResolve binds the actual paces, and code binds every rep, every duration, every weekly volume. The model chooses; the code does the math.

This collapses the repair loop at its source, and the reason is almost tautological once you see it. Everything the eleven hard validators police is, by definition, code-computable — that's why code can validate it. So if code composes the numeric scaffold in the first place, the scaffold is born compliant. It cannot breach the rules it was generated from. A wrong Pick becomes a cheap set-membership miss — "that archetype isn't in the catalog" — not a 1,900-token re-emission of an entire week that then has to survive eleven more checks. The thing you were spending 57% of your clock repairing can no longer occur.

One image carries the whole pattern: the reasoner only ever holds the pen on a choice, never on a value.

Two structural invariants fall out of that. Every handoff from a reasoner to the executor is a selection from a curated closed set — or a raw value the executor clamps to a supplied range, never one it repairs. And every handoff from the executor back to a reasoner is a hard envelope the next reasoner can only choose within, never violate. Selection down, envelope up. That pair is what kills the propose-reject-re-propose loop: there's no free value left anywhere for a validator to police.

Four rungs, one shape

The Pick wasn't a leaf trick. Once I had it, I saw the same shape at every altitude of the pipeline. Plan generation has four rungs, and they are all the same move: reason down to a small bounded set, let the LLM pick within it, let deterministic code execute the pick.

Altitude	The bounded choice the LLM makes	What code materializes
Intake	Converse with the athlete; pick values for a bounded set of "shell" knobs	The structured goal + constraints the rest of the pipeline runs on
Macro	Pick a plan-shape / ramp policy from a bounded set	The week-by-week volume ramp, clamped to ≤15% week-over-week
Micro	Pick a catalog archetype + a distance inside a code-computed envelope	Every rep, pace, and duration — `VdotResolve` binds the paces
Adapt	Pick from the bounded 9-move catalog when a workout comes in off-target	The concrete change to the rest of the plan

The bounds are real and deliberately small. The selection surface is about thirty-five named archetypes — biased small, to the 25–35 range, specifically because that's where LLM selection stays reliable. The adaptation catalog is nine moves, enumerated: load up minor, load down, load up major, swap quality to easy, add a rest day, repeat-or-extend the block, re-anchor pace zones, anchor zones up, re-plan. The macro ramp is +8% a week with a hard 15% week-over-week clamp the code owns. None of those are things I let the model invent. They're things I let the model choose among.

And correct-by-construction buys you more than a dead repair loop — it buys parallelism. The old micro tier was strictly serial because each week's proposal leaned on the prior week's committed actuals. Anchor each week to the macro targets instead, and that serial dependency dissolves; the weeks can fan out concurrently. Projected time-to-first-week drops from sixty-nine seconds to single digits, at which point the bottleneck isn't the model at all — it's how fast the infrastructure can deliver the event.

The tell that this was a real pattern and not me talking myself into one: a separate design effort, the workout-catalog debate, had independently crowned the same Pick primitive "the single most load-bearing mechanism in the whole debate." Two efforts, different rooms, converged on the identical move.

Why moving numbers to code isn't the parrot

There's an obvious objection here, and it's the important one. Isn't a fully deterministic plan with the LLM demoted to picking from menus just... a fancier deterministic plan? Haven't you built a spreadsheet with a chatbot stapled to it?

This is where the empty text box comes back. That prototype's blank narration on holds was the parrot exposed. The voice had nothing to say on a hold day because there was no judgment underneath it to narrate — the plan was fully pre-computed, the LLM was decoration, and decoration over a null decision renders as an empty string. Strava's "Athlete Intelligence" is the shipping version of this: generic comments on your run, valueless precisely because the AI had no hand in shaping the workout it's commenting on. A principle we've written down names the permanent lesson: the prose is never the product; the judgment is.

So the anti-parrot guarantee does not live in the prose layer. It lives in the code/LLM boundary. Moving the numbers into code is correct only because the LLM's judgment moves up — into archetype selection, distance choice, shaping the plan skeleton from a real conversation. Get that backward and you haven't fixed anything; you've rebuilt the valueless prototype at lower latency. A faster parrot is still a parrot.

Which is why the two wins are actually one fact. The rewrite is fast because it kills the repair loop, and it's non-parrot because the judgment moved up to genuine bounded picks instead of being deleted. You don't trade one for the other; the same structural move buys both.

The executor is a role, not a thing

The executor doesn't have to be code — and that's what makes this generalize past one running app.

The thing filling the envelope — the "executor" — doesn't have to be code. It can be a cheaper model. The exact same shape shows up in how I build software, and I've caught myself saying it as a one-liner: don't use Fable 5 to write code — use Fable to tell Opus to tell Sonnet to write code. The frontier model does the sparse, high-judgment reasoning. It narrows to a bounded spec — the groom. The cheaper model executes within that spec — the dispatch. Which, if you've read The Player-Coach Decade, you'll recognize as literally my own control-plane → groom → dispatch workflow. Bounded choice at the top, materialization below, recursing down.

Two substrates — deterministic code in Grit Mile, a cheaper LLM in the build pipeline — identical shape. That's how I know it's a shape and not a trick.

It also reframes what an orchestration failure is. It's symmetric. Under-powering a rung — handing an architecture call to a model too small to make it — is a failure. Over-powering a rung — burning a frontier model on boilerplate — is the same failure: a model working off its rung. I watch Amazon engineers fight over scarce Bedrock capacity, and a lot of the time the real diagnosis is: capacity misallocation — Fable-grade reasoning spent on Sonnet-grade work. The cascade fixes both directions at once, because its whole discipline is matching reasoning cost to judgment density at every rung — so nothing works above or below its level.

The pattern the last two posts were circling

I've written the perimeter of this idea twice without naming the center.

It's Not the Model named the disease: most AI failures aren't capability failures, they're context or orchestration failures. But "orchestration failure" stayed abstract — a bucket you couldn't act on. The Player-Coach Decade named the human's new job: orchestration, decomposition, inspection, running a stack of agents. But it stopped at the job, without the structure it produces.

The Bounded-Choice Cascade is the mechanism sitting under both. It's what an orchestration fix actually looks like when you draw it — selection down, envelope up, recursing through the altitudes. And it's what the player-coach is doing at the top rung: writing intent, and curating the closed sets everything below chooses from. The judgment moved up — into the picks and the set-design — while the factory materializes and recurses beneath it.

It's only as good as the sets

This is not a free lunch. The cascade is exactly as good as the curation of its bounded sets, and it fails in both directions. Make a set too small and you strangle judgment — the model produces something compliant and mediocre because the good answer wasn't in the menu you handed it. Make it too loose and you've reintroduced guess-mode and invited the repair loop right back in the door you just closed it through. There is no setting where the sets curate themselves.

But I'd argue that curation cost is the point, not the tax. Deciding it's about thirty-five archetypes and not two hundred, deciding the nine adaptation moves, pinning the +8%/15% ramp constants — that is the domain work. That's the coaching expertise, encoded. It's precisely where the moat lives, so it's effort spent exactly where you'd want to spend it. The cascade doesn't eliminate the hard thinking; it relocates it from per-request firefighting to up-front set design, which is a much better place to keep it.

And I want to be careful about the size of the claim. Two worked examples across two substrates is enough to name a shape — it is not a proven universal law, and I'm not going to dress it up as one. This is "here's a shape I keep seeing," not a theory of everything.

The move

The reusable instinct is a single question:

When you catch a reasoner and a validator fighting over a number — one proposing, the other rejecting, round after round — stop. That fight is the tell. Ask what the executor below could compose instead, and hand the reasoner a menu rather than a blank.

And at any rung, ask: is the model holding the pen on a value, or on a choice? If it's a value, you've got a repair loop or a parrot queued up somewhere downstream, whether you can see it yet or not.

I keep finding these in the same place: anywhere judgment and execution got collapsed into a single call, instead of interleaved down the altitudes. Pull them apart, and the eighteen-minute plan should collapse to single-digit seconds — because you finally stopped asking the model to hold a pen that was never its to hold.

The Deploy Is the Loop Now

2026-07-14T00:00:00-04:00

For a couple of days I steered Claude Code on Grit Mile's onboarding flow and watched it crawl. It couldn't seem to nail basic fixes, and my first instinct was the usual wrong one — the model isn't good enough at this yet. The model was fine. The problem was the loop I'd put it in: to find out whether any change worked, the agent had to deploy to AWS and read CloudWatch. I'd wired a cloud deploy into the middle of the build-test-iterate loop, and I'd never steered it toward a way to test locally.

Grit Mile is my AI running-coach side project — serverless AWS, a Lambda behind an API Gateway WebSocket, Aurora DSQL, Bedrock underneath. Onboarding is a conversation: the coach talks to a new athlete, gathers what it needs, and hands off to plan generation. It kept breaking in small, maddening ways — a field the coach should have captured but didn't, a question it re-asked instead of moving on.

Those bugs were slow to fix for an architectural reason. The function that runs a single intake turn — the coaching brain of the conversation — only existed inside the deployed chat-stream Lambda. There was no way to run it on a laptop. So every fix went: change the code, sam deploy, wait, click through the chat in a browser, watch it misbehave, scrape CloudWatch, guess, change again. Ten to twenty minutes a lap. The code change the agent was actually iterating toward took it maybe five.

The deploy didn't get slower. The build got faster.

Faster deploys have always meant tighter feedback loops. There was a time when "the cloud" was the fast option — you shipped to EC2 in minutes instead of racking a server in your own closet over weeks. The rule never changed; only the arithmetic did.

When a human wrote the feature, that human spent maybe five days on it. A 15-to-20-minute deploy dropped into that loop is a rounding error. It doesn't register as friction — against five days of typing, twenty minutes of CloudFormation feels like a tight loop. You'd never think to optimize it.

Now an agent writes the same feature in minutes. Frontend polish on a file or two lands in three to ten; a cross-stack change with tests, fifteen to twenty-five. Those are my lived numbers from months of working this way — measured, not benchmarked. Against a five-minute build, that same unchanged 20-minute deploy is now most of the loop. It dominates by three or four times.

The same deploy earns the opposite verdict now, because the build collapsed. Coding velocity was never the metric. Iteration velocity is — the cycle time from make a change to see the result to know the next step, and I've written before that it's the ratio that compounds. Agents collapsed the numerator to almost nothing. Whatever's left sitting in the loop is now the whole cost. If that thing is a cloud deploy, the cloud deploy is your dev loop — and you've handed most of your agent's speed advantage back to CloudFormation.

This is an orchestration failure, and the default agent walks right into it

I keep coming back to one diagnostic. When an agent fails at something it genuinely needed to do, the failure is almost never that the model can't. It's a context failure or an orchestration failure — the model lacks the knowledge in scope, or the workflow won't let it use a capability it plainly has.

This one is orchestration, cleanly. The coaching brain worked. The model could run an intake turn correctly. What was broken was the workflow around it: every diagnosis had to round-trip through a deploy and a log scrape, because the capability was trapped in a Lambda that only ran in the cloud. Capability fine; workflow throttling it.

This is what Claude Code does by default. Point it at a serverless AWS app and tell it to build, test, and iterate, and it will test the only way the architecture allows — deploy, click, read logs. It won't stop to build a local loop unless you steer it to. Default Claude Code, building on AWS, will make every wrong guess cost an afternoon. This isn't a trap I stumbled into once. It's the path of least resistance for any agent on this stack. Don't let yours take it.

The fix is a seam, not a rewrite

The move is ports and adapters — plain good engineering, nothing exotic. What's new is the cost math. The instinct is that the adapter, the local shim, is overhead: extra code you write to avoid the "real" thing. That was true when a human wrote the shim by hand. With an agent, the shim is cheap and the deploy is the expensive part. Once you internalize that inversion, building the local seam stops looking like a detour and starts looking like the obvious first move.

So I had Claude pull the coaching brain out of the Lambda. runIntakeTurnCore(input, deps) now lives in core and takes its dependencies as injected ports — a transport sink standing in for API Gateway's postToConnection, injected repos and clock and LLM client, a plan-gen trigger port in place of the direct Lambda invoke. Behind those ports, local runs get file-backed adapters, a replay Bedrock shim, and an in-memory sink that just collects what the coach would have streamed. A one-command CLI drives the real function on a laptop with zero AWS in the picture.

Production didn't fork. It injects the AWS adapters behind the same ports and calls the same core function. Shared logic, swapped adapters — that's the whole trick. The coaching brain has exactly one implementation, and both prod and the local loop run it.

If the local loop doesn't run the real code, it's theater

There's a real objection: a local harness that runs a fake proves nothing. You end up debugging a simulation of your app instead of your app, and the green result is a lie.

The seam has to be built against that explicitly. You shim the environment — repos, clock, LLM client, transport. You never fork the logic. There is no local-only path that decides what the coach says or when it finalizes. A guard-rail test asserts that both the deployed worker and the local path funnel through runIntakeTurnCore; if they ever diverge, the test breaks. If a local turn and a prod turn could disagree about what the coach would say, the harness is theater and a green local run tells you nothing.

The tell that the WebSocket scaffolding was pure transport and not brain is that it simply doesn't exist locally. API Gateway's WebSocket integration has a hard 29-second timeout, and a full intake turn runs closer to 40, so production needs an ack-fast self-invoke to dodge the timeout and a client keepalive ping to defeat the idle disconnect. On a laptop, an HTTP request that streams for 40 seconds is fine — no gateway timeout, no idle socket. That scaffolding was cloud plumbing, not coaching logic, and dropping it locally changed nothing about what the coach decided. That's how you know the seam is cut in the right place.

The first local drive caught what a green deployed suite missed

The moment the local loop worked, Claude used it — on its own, without me asking. I'd steered it to build the seam and stepped away. When I came back, local intake was running and Claude had already used it to isolate a bug that a fully-green, fully-deployed test suite had been hiding.

The suite was green because it tested against a stub model that emitted perfect tool calls. Onboarding captures each answer when the model emits a record_* tool call — but the real model would rather talk. Given the choice between a conversational sentence and a typed tool call, it picks the sentence almost every time, so the capture tools never fire. Six turns of real conversation, and the coach had captured only the fields pre-seeded from the signup form — every answer given by talking was dropped. The tests had validated a model that doesn't exist. The gap was invisible right up until a live local drive put the real model in the loop.

A deploy-and-CloudWatch hunt would have burned an afternoon on that, and it's the kind of bug you misattribute — blame the prompt, the tool schema, five other things — before you see its shape. The local loop surfaced it in one conversation with the raw model output sitting right there. That's the entire argument for the seam, and it paid off the first time the agent ran it.

Local-exercisability is a service-selection criterion now

Once you've felt this, it stops being a debugging tactic and becomes an architecture criterion. Local-exercisability belongs on the same shortlist as features and price when you pick a service. A service you can drive or cheaply shim on a laptop keeps the loop fast. A service that only exists in the cloud drags a deploy into every loop that touches it — and with agents doing the building, that's often the dominant cost, not a footnote.

EventBridge is the thing to watch for: an event bus you can't really stand up locally, so any path that fires through it wants a cloud round-trip to test. Grit Mile's local loop works precisely because it fakes that bus with an in-process one behind a port. Every time you choose a service like that, you're signing up to either build a faithful local shim or eat a deploy in your loop forever. Sometimes the service is worth it — but that cost belongs in the decision, not discovered later.

A WebSocket Lambda that only runs deployed — the exact thing I built — is debt. Skipping the seam front-loaded weeks of slow loops, until the agent and I went back and paid for it.

I work at AWS, on the team that owns the documentation, so I'll say this plainly instead of dancing around it: "can an agent exercise this on a laptop" is becoming a first-class question, and the platforms that answer it well have an edge agents feel immediately. Naming the gap is how it gets closed.

Build past the harness

The rule is blunt: if you have to deploy to a cloud provider to find out whether the thing works, you're wrong. Build the local loop. This is what building past the harness looks like in the small — you notice where the tools stop, and you construct the missing piece instead of routing around it one more time.

I promoted that straight into my global setup, so every agent I run now inherits it — never put a cloud deploy inside the build-test-iterate loop. Every code path should be fully exercisable locally. When the code leans on something the cloud provides, shim it: a local Postgres, an in-process event bus, a replay-cache LLM, a direct HTTP call where API Gateway would sit. Swap the adapter, keep the use-case.

The deploy was always in the loop, for any app that needs a deploy to prove it works. It just didn't matter when a human spent days writing the thing the deploy was gating. Now the human writes intent and the agent writes the code in minutes, and the deploy is the slowest thing left. The deploy didn't change; the build around it collapsed, so the deploy went from rounding error to the main event.

That's why the deploy is the loop now. Build the one that isn't.

Understanding Has an Altitude

2026-08-07T00:00:00-04:00

A five-item dispatch wave came back green. Every item merged, every gate passed, every acceptance criterion checked off. The feature was broken end to end.

No single item owned the seam between two of them. Each build agent had done exactly what its spec said, and the capability that lived between two specs belonged to nobody. Per-item review couldn't catch it — every reviewer was scoped to one item, and the gap was in the space between items. I found it by running an adversarial review workflow against the intent of the whole feature instead of the acceptance criteria of its parts. It came back does-not-meet, with two ship-blocking gaps that a hundred percent of the unit tests were happy to sit on top of.

Nothing at the diff level could have found that. The catch required holding the shape of the feature in mind — what it was supposed to do, end to end, for a user — and asking whether the landed code did that. Decomposition manufactures those gaps. Understanding at the level where the feature actually lives is the only thing that closes them.

That's the argument: you cannot steer an agent to build the right thing if you don't understand the system. What has changed — fast, and it's still changing — is the altitude at which that understanding has to live.

The debt already has a name

I'd been calling this understanding debt in my own notes before I went looking. The idea is well-trodden. Addy Osmani calls it comprehension debt: "the growing gap between the code your team has shipped and the code your team actually understands." Lars Janssen calls the same shape verification debt. SRLabs calls the throughput half of it the verification bottleneck and points out that it's human-limited by construction.

Osmani's framing resonates, because it's the distinction that makes the debt metaphor earn its keep: technical debt lives in the codebase, comprehension debt lives in the people. You can pay down technical debt with a refactor you delegate to an agent overnight. The other kind you pay down inside your own head, on your own clock, and there is nothing to dispatch it to.

Where I part company with the existing accounts is the remedy. The pieces I read converge on understand more of the code — read the diffs, audit the output, maintain your mental model. That advice doesn't survive contact with agent speed. Osmani gets at the arithmetic himself: a junior engineer can now generate code faster than a senior engineer can critically audit it. If the prescription is to close that gap by reading harder, the prescription is already lost.

The altitude keeps climbing

Last year I still had to understand the systems I was building at the level of interfaces, classes, and functions. Not usually the individual line — I wasn't hunting for off-by-ones — but I had to know what a class was for and what a function was supposed to return, because that's where the model's mistakes lived. Correcting the agent meant being able to say: that method has the wrong responsibility, that interface leaks its internals, that logic belongs on the other side of the boundary.

I don't do that anymore. I steer at architecture and business intent. Is this the right decomposition? Does this system boundary belong here? Is that what the user actually needs, or only what the ticket said? The models got good enough to own the level below me, so I moved up a floor.

The altitude has been climbing roughly a level a year. That's the part the debt conversation misses. Understanding didn't become less necessary — it relocated. Reading every line is now the wrong response, because it spends your scarcest resource at an altitude the agent already covers. The right response is to climb to the altitude where your understanding still binds the system, and be rigorous there.

Altitude is not the same as distance. A leader who can't read a system design and say what's wrong with it isn't steering from a high altitude; they've just stopped steering. I wrote about that gap in The Player-Coach Decade — the leaders getting multiplicative returns from these tools are the ones who never let their technical judgment atrophy. The altitude moves. The rigor doesn't get to.

Two different bills

The debt shows up as two costs. Separate them, because they have different fixes.

Understanding debt costs you steering. When an agent goes wrong, you correct it by knowing what right looks like at the level it went wrong. If your understanding sits below the level of the error, you'll fix the symptom and ship the same defect somewhere else. If it sits nowhere at all, you won't notice. That five-item wave is the clean case: everything was correct one level down, and wrong one level up, and the only instrument that could see it was a model of what the feature was supposed to do. This is a comprehension problem, and no amount of process fixes it.

Verification debt costs you throughput. This one is arithmetic, and the numbers are ugly. LinearB's 2026 benchmarks have developers using AI completing 21% more tasks and merging 98% more pull requests — with PR review time up 91%, and AI-generated PRs waiting 4.6x longer before a reviewer even picks them up. Sonar's survey found 96% of developers don't fully trust AI-generated code to be functionally correct, and only 48% always check it before committing. That's a queue, and the queue is made of humans.

Collapsing these two into "review is the bottleneck" is what leads teams to buy another review tool. The steering problem is not a queueing problem, and you cannot throw reviewers at the fact that nobody in the room understands the system.

Human review doesn't scale

Say the harder part out loud, because very few people will yet: human review is not the answer to the throughput bill either. Reviewers don't get faster, and adding more of them has never once closed the gap. If you're willing to move at the speed of human review, you've decided to leave the upside of every model release on the table. In a competitive market that's a choice with a price tag, and the price goes up each time the models improve.

What replaces review is automated evaluation — systems you build to prove to yourself that the software does what it claims. Your understanding of the system is what lets you steer an agent to build that eval. The argument closes on itself.

I built a photo uploader recently with optional privacy settings that strip location metadata before images go out to social media: always-strip, plus strip-by-geofence. That is a bad thing to get wrong. Nobody wants to find out the geofence leaked their home address after the post is public.

I never read that code line by line. I fed the agent real images from different cameras carrying different kinds of embedded location data, exported through different photo management tools — including the IPTC City / State / Country fields that Adobe Lightroom helpfully stuffs into the file where nobody thinks to look. Then I steered it to build integration tests that block promotion if any code path touching location privacy does the wrong thing.

Every judgment in that paragraph is an altitude judgment. Knowing that EXIF isn't the only place a coordinate hides. Knowing that "strip location" has to mean every code path, including the ones the ticket never mentioned. Knowing which failure is unacceptable rather than merely annoying. The agent wrote the tests, and it wrote them well. I could not have told it what to test without understanding the system and the desired behavior at the right altitude.

Critical thinking is the root

Anthropic published a randomized controlled trial in January: 52 junior engineers learning a Python library they hadn't used before. The group with AI access averaged 50% on the follow-up comprehension quiz. The hand-coding group averaged 67%. Roughly two letter grades. Debugging showed the steepest decline — which is precisely the skill you need in order to catch what the agent got wrong.

The finding underneath that headline is the one that matters. The gap tracked how the tool was used. Participants who used the model for delegation scored below 40%. Participants who used it for explanation and conceptual inquiry scored above 65% — the authors concluded that explanatory use "might not hinder, and could potentially aid" performance. Same tool. Same task. The difference was whether a person was thinking while it ran.

That matches my own experience, which is why I trust it. My learning has accelerated since I became a heavy AI user, and so has my thinking — but only because I'm deliberate about it. I treat these tools as a learning instrument that also happens to make me fast. Intent is the whole variable.

That's the mechanism. The debt accrues at the moment you stop having an opinion. Accepting a diff is delegation. Asking the agent why it chose that boundary, and having a view on the answer, is inquiry. Only one of those builds the model in your head that you'll need three weeks later, when the wave comes back green and broken.

What this looks like in practice

Make the agent argue before it builds. I groom complex work through a draft-critic-synthesize workflow rather than a single planning pass. The critics catch things CI structurally cannot — in one case, that a nudge had to be a fresh message rather than an appended one, where getting it wrong would have left a green test suite over a dead feature. The output is a better spec, and the byproduct is that I've been forced to have an opinion about every load-bearing decision before a line exists. Give the planner an adversary.

Interrogate, don't just accept. When an agent hands back a design, ask what it considered and rejected, and why. That answer is where your mental model gets refreshed for free, and it separates the 40% from the 65% in that study. "Looks good" costs you nothing today and quite a lot later.

Put the gates in before the code. Every repo I dispatch into gets its quality scaffold committed first, so the agents can run wild safely — mechanical verification they cannot fake, enforced at commit and at push. Simon Willison's framing is the one to hold onto here: a computer can never be held accountable, so that's your job. Automating the proof doesn't move the accountability anywhere. It stops you paying for it by hand. I've written about the harness as a stack and about investing in the flywheel — this is that argument pointed at comprehension.

Review against intent, not against acceptance criteria. Acceptance criteria are per-item by construction, which means they're blind to the seams between items. Somebody — or something — has to check the feature against what it was supposed to do. That check has to run at feature altitude or it isn't the check.

The part you can't dispatch

Every failure I care about in agentic work is a context failure or an orchestration failure, and both are fixable by building better machinery around a model whose capability was never the problem. Understanding debt sits upstream of both. You can't hand an agent context you don't have; the metadata field you've never heard of is the one that never makes it into the spec. You can't carve a system into clean items if you don't understand it at the altitude the seams live at, which is how five green items add up to a broken feature. And when either failure lands, it arrives as a confident, plausible answer that only your own understanding would flag. Understanding debt generates the failures and disables the detector in the same move.

You can engineer the model's context layer. There's nowhere to dispatch your own.

You can build a factory that catches seam gaps, forces handoffs, and verifies its own claims — I have, and it works. What the factory can't do is hold the model of the system that tells you the factory is building the right thing.

That model only exists in one place, and it decays if you stop feeding it.