This synthesized pattern separates building the work from checking the work. It maps
five jobs across several RossLabs tools: plan, execute, verify, review, and learn.
The short version
A reliable AI development loop separates five jobs. Plan the intended outcome. Execute the
scoped work. Verify facts that code can check. Review quality and fit with an independent
critic. Record useful lessons for the next run. Each step returns evidence to the next step,
and the work only counts as complete after the required checks pass.
The five steps
Start with the overview. The next section explains the checks, tools, and decisions inside
each step.
1Plan
Every build in the portfolio starts by naming the decision before making it — the goal, the files, the pass/fail criteria, and which design calls are being made. Build Loop's plan-verify runs a deterministic lint against that plan before any code moves. The RossLabs Agent Harness treats planning as a routing decision: which model is cheapest that still passes the task's oracle, and whether to invoke the harness at all. Interface Built Right's Design Director produces design-intent.json and concrete validation criteria before implementation starts.
Once a plan clears its check, execution runs local or dispatches to bounded parallel workers. Build Loop dispatches authorized parallel subagents with MECE file ownership. The RossLabs Agent Harness routes to the cheapest model that passes the task's oracle and runs it inside a sandbox with graceful fallback. Interface Built Right's Implement step applies Calm Precision and platform-specific routers to a plan that already named its constraints.
Run checks that return the same result for the same input. These checks run separately from the model that produced the work. They can validate schemas, tests, thresholds, permissions, or visual differences before the work advances.
Where a deterministic check can't reach — quality, coherence, whether a diff actually matches the plan's stated intent — an LLM judge steps in, but never ungated. Prompt & Model Benchmark Lab's judge is calibrated, can never set its own composite score, and gets flagged self-graded (confidence stripped) when it happens to be scoring one of its own candidate models. Build Loop splits judgment across two named Review sub-steps: Critic and Fact-Check.
The stage that keeps a loop from being a straight pass repeated forever. Build Loop's optional Learn phase detects recurring patterns across runs and auto-drafts experimental skills or agents with A/B tracking. Prompt & Model Benchmark Lab ingests real build-session signal as field observations, quarantined behind a hard wall so it can never move a calibrated score. The RossLabs Agent Harness's stated trajectory is self-hosting: once the core loop is reliable on a local model, the harness uses it to build its own later phases.
Open a step to see the checks, thresholds, and examples documented in the source projects.
Plan. A plan that names its own decisions, checked by a deterministic lint before code moves. Read moreShow less
Build Loop's plan has to name the goal, files, pass/fail criteria, commands, and design decisions before build work starts. plan-verify runs a deterministic lint — delete-with-callers, numeric-drift, route-change-evidence, package-state, missing-evidence — and plan-critic runs a softer critique on top of that. Plans crossing five or more synthesis dimensions auto-route to the strong model in one pass, a threshold set from testing where the fast model lost cross-decision context below it.
The RossLabs Agent Harness's planning step is really a routing decision made before any model runs: which model is cheapest that still passes the task's oracle, and whether to invoke the harness at all versus a direct or deterministic solve. Per-model and per-domain profiles inform that routing, so the model is treated as a swappable data row rather than a fixed choice made once.
Interface Built Right's /ibr:build orchestrator runs a Design Director step that produces design-intent.json and concrete validation criteria before implementation starts. The orchestrator — not the agent doing the work — is the one place that knows the active design system and the gates the result has to pass, so an agent can't skip the design step on the way to a passing test.
Execute. MECE-owned parallel subagents, a swappable model behind a sandbox, or a scoped implement step. Read moreShow less
Build Loop's Execute phase runs the work locally or dispatches authorized parallel subagents with MECE (mutually exclusive, collectively exhaustive) file ownership; the orchestrator integrates the final diff rather than letting workers merge their own pieces.
The RossLabs Agent Harness treats the model as a swappable data row — local via Ollama, MLX, or llama.cpp, or cloud via OpenAI, Groq, Fireworks, OpenRouter, Together, or DeepInfra — and routes to the cheapest model that passes each task's oracle. A Program-Aided code_exec lane and sandboxed execution (Docker → macOS Seatbelt → Linux Bubblewrap → explicit opt-in) mean it never runs unsandboxed silently; the fallback chain is itself part of the design.
Interface Built Right's Implement step applies Calm Precision, the web/iOS/macOS design routers, component patterns, and data-viz guidance as needed. Execution here is scoped by the plan and validation criteria set in the step before, not improvised as the build goes.
Verify. 6 of 7 garbage outputs rejected on one sweep, a five-check plan lint, a four-way pixel verdict — none of it graded by the model itself. Read moreShow less
On one documented OSS ingest sweep, the RossLabs Agent Harness gate rejected 6 of 7 invalid outputs that an ungated model call would have returned. The important boundary is not Rust itself. The policy runs as deterministic host code outside model control, so the model cannot waive or recompute its own acceptance rule.
Build Loop runs two separate gates. plan-verify's deterministic lint (delete-with-callers, numeric-drift, route-change-evidence, package-state, missing-evidence) blocks execution before it starts. A fixed seven-step Review chain — Critic → Validate → Optimize (opt-in) → Fact-Check → Simplify → Auto-Resolve → Report — with a single exit point gates merge, and routes back to Iterate on a failure, up to 5 repair-and-revalidate passes in classic mode and 25 in autonomous mode.
Interface Built Right runs a deterministic rule engine and a sensor layer of structured summaries, then classifies a Pixelmatch visual diff as MATCH, EXPECTED_CHANGE, UNEXPECTED_CHANGE, or LAYOUT_BROKEN. Pixelmatch's 0.1 value controls per-pixel color-difference sensitivity. It is not a whole-page mismatch allowance. The final verdict also depends on the amount and location of changed pixels.
Review. A composite score the judge isn't allowed to set, and a self-graded flag when it grades its own output. Read moreShow less
Prompt & Model Benchmark Lab runs deterministic graders where ground truth exists and an LLM-as-judge (rubric, temperature 0) where it doesn't, and every score carries a calibration confidence. The judge is gated by that calibration and can never set its own composite — the composite is computed in code from its per-dimension scores, not handed to the judge to total up. When the judge model happens to be one of the candidates being scored, the pipeline flags the cell self-graded and strips its confidence, because a model scoring its own work isn't evidence. A one-line Trust banner states plainly whether the evidence behind a score is objective, mixed, or judge-only-and-directional.
Build Loop splits judgment across two named sub-steps inside its Review chain: Critic, which evaluates the diff against the plan's stated design decisions, and Fact-Check, which verifies claims against the actual diff rather than the implementer's report of what it did.
The two projects that show the contrast sit outside the gate stage rather than inside it. Visual Test Suite left judgment to a human — accept a diff as the new baseline or flag it as a regression. Prompt Test Lab's original "judge" was keyword overlap and a hardcoded accuracy = 0.75, not a judge at all — what that cost, and what it changed, is the subject of the section below.
Learn. An optional phase that drafts its own skills, and a quarantine wall so unverified evidence can't move a score. Read moreShow less
Build Loop's optional sixth phase, Learn, detects recurring patterns across runs and auto-drafts experimental skills or agents with A/B tracking, so a lesson from one run becomes part of the next run's toolkit instead of staying in a transcript nobody rereads.
Prompt & Model Benchmark Lab's real build sessions across other projects generate their own model-fitness signal — which model shipped a chunk, whether it held. The lab ingests these as field observations, quarantined behind a hard wall: they can corroborate a score but can never blend into a calibrated score or move a recommendation's confidence. Observational evidence stays labeled observational, permanently — the wall is the point, not a temporary caveat.
The RossLabs Agent Harness states its own trajectory plainly: self-hosting. Once the core verification loop is reliable on a local model, the harness uses that loop to build its own later phases — the gate that verifies the work becomes the mechanism that extends the tool doing the verifying.
Examples across tools
This table maps the five steps to six tools. A dash means the project write-up does not
describe that step. It does not prove that the step is absent.
Verification adds work and latency. It creates value only when the check measures the claim
it is supposed to prove.
Prompt Test Lab's original "accuracy" scoring computed keyword overlap and a hardcoded
accuracy = 0.75 — not real grading. The gap surfaced during work on the
successor project, Prompt & Model Benchmark Lab, and the project's own page discloses
it rather than quietly patching it: "a benchmark you can't trust is worse than none; it
launders a guess as a number." The successor's entire design — deterministic graders, a
calibrated judge that can never set its own composite, a self-graded flag, a Trust banner —
reads as a direct response to that one failure.
Visual Test Suite is the inverse story — not a fake gate, but a gate that was the whole
product. It ran Playwright screenshots against Pixelmatch baselines for any URL and stopped
there: capture, diff, accept or flag. Its own write-up says plainly what happened next: "the
validation engine that used to be the product is now the verification layer underneath a
build orchestrator" in Interface Built Right. Comparison didn't go away — it became one step
inside a bigger loop instead of the whole product, with a Design Director, platform routers,
and a two-tier scan now built around it.
Read together: the pattern holds when the gate is deterministic and separate from the thing
being judged — a deterministic host check outside the model's control, a lint that runs before code moves, a
pixel-diff threshold nobody can talk their way past. It fails exactly where Prompt Test Lab
failed — when "grading" is dressed up in real-looking output but is actually a hardcoded
number, and nobody is checking the checker.