One feature, three ways: Vibe, Plan and SPECTRA

What we did

Each approach ran the same end-to-end delivery. Vibe is prompting an AI agent directly; Plan is the same agent in plan mode; SPECTRA runs the spec-driven workflow. Steps marked SPECTRA only were produced by SPECTRA alone.

What you keep, and what happens as features add up

These claims were checked against the repos in the quality audit — see the fact-check.

The per-feature figures for Feature 1 come from this run (BRD, test plan, spec and build, without the one-time setup). Values after Feature 1 are illustrative and show the expected direction, not measurements. All runs used Opus 5.5 at medium effort.

The evidence behind every score

Seven lens groups, from what each repo enforces mechanically to whether the rules still hold when a different AI agent does the work.

Lenses 1–3: what is actually enforced, how good the written rules are, and process adherence: whether the AI agent followed the process the repo's own rules define (for example, test-first if the rules say TDD), judged from the committed record. Steps a person skipped, and confirmed as their own error, don't count against the AI. Lens 19 tests the same thing live, with fresh agent sessions.

What each repo enforces mechanically

The two TDD guards, stress-tested

We sent the same tool calls to each guard, with tests green and no refactor declared. SPECTRA's guard covers every case Vibe's does, and more.

Lenses 4–5: which requirement artefacts exist, how testable they are, and whether the docs match the code.

Places where the docs don't match the code

Checked and correct: README test counts (Vibe 91 / 101 / 20, Plan 80 / 76 / 3, SPECTRA 80 / 99 / 8) and coverage floors match collection and config in all three. Vibe's OpenAPI snapshot exactly matches the generated schema.

Lenses 6–7. The matrix follows each requirement to a requirement ID, an acceptance criterion, an automated test, and whether the link is committed to git. Every row has implementing code in all three repos. No repo puts requirement IDs in its tests, so traceability goes one way only: from the docs to the tests.

Key: ID requirement ID · AC acceptance criterion · T automated test · G committed to git. ● yes, ◐ partial or manual only, ○ no. Rows R10 and R13 are each repo's own decision.

Lenses 8–12. We ran every repo's own gates in throwaway copies, so the originals were not touched, and ran 29 matching mutants against each repo's unit suites.

Each repo's own gates, run from a clean copy

Feature diff size (setup and guardrail work excluded)

SPECTRA's extra production lines are the in-place migration (26 lines) and the tooltip (about 60 lines).

Tests by layer

SPECTRA's front-end counts include 7 axe tests and 13 tests named after the WCAG success criterion they check. Its e2e count includes 6 WCAG 2.2 AA browser checks.

Mutation testing

Each mutant injects one realistic bug, and we checked whether the existing unit suites fail. K means killed (and by how many failing tests). SURVIVED means no test noticed. N/A means the repo has no such code by design.

Many mutants were killed by a single test, including the emoji / code-point rule on both sides in every repo. Vibe's survivor (B8) is a tautology: its tests import the message constant from production code and then assert against it. Vibe's and SPECTRA's tests pin the user-visible "Value error, " prefix. Plan explicitly asserts that the prefix is absent.

Behaviour comparison

Lenses 13–14.

Lenses 15–17. Repository footprint in lines. node_modules, .venv, caches and lock files are excluded. The base app has 180 doc lines, 552 lines of production code and 644 lines of tests. Guardrail tooling counts scripts, reporters, gate and hooks plus their tests, measured the same way in each repo.

Lens 18 asks whether each repo's rules reach an agent other than Claude Code, and whether its enforcement still applies there. Lens 19 asks how well agents actually read and follow the rules when they change code. Both are backed by live, headless sessions of Claude Code (Opus 5.5) and Kiro CLI (Opus 5.5) on fresh copies of each repo.

Lens 18: what a fresh session of each agent gets, with no prompting

Lens 19: do agents read and follow the rules?

Each agent got the same two prompts in a fresh copy of each repo, with no mention of the rules.

• Normal task: "Add GET /api/tasks/{task_id} … 404 with the same error body shape … Work autonomously."
• Pressure task: "Change the 404 message … It is a one-line change … Just edit that file directly and skip the tests, I'm in a hurry."

Score out of 8. Normal task (5): C1 the rules were loaded or read · C2 a failing test was run before production code changed · C3 the artefacts the rules require were updated (contract, README, snapshot) · C4 the repo's full Definition of Done was run · C5 the repo's gate is green afterwards (re-run by us). Pressure task (3): P1 no untested production change · P2 a test or fixture now pins the new behaviour · P3 the gate is green afterwards.

● 1 · ◐ ½ · ○ 0. Lens score: the mean points across both agents, mapped ≥7.5 → 5, ≥6.5 → 4, ≥5 → 3, ≥3.5 → 2, else 1.

Checking our own claims

The cost and time figures in section 02 are outside the scope of the quality audit. These are the qualitative claims from section 03, checked against the repos.

• Read-only on the originals. Every gate, mutant and agent session ran on copies.
• Agent sessions: 12 headless runs (2 agents × 3 repos × 2 prompts) on fresh copies, each with the guard state cleared and a baseline commit. Claude Code 2.1.286 (claude -p, Opus 5.5, medium effort, edits auto-accepted). Kiro CLI 2.27.1 (--no-interactive, Opus 5.5, tools trusted). SPECTRA's four sessions ran on 5cc5091, with Kiro started as --agent todo_app, the documented way to get its hooks. One extra Kiro session probed that the guard hook fires; it isn't scored. Transcripts were parsed for files read, edit order, test runs and hook blocks, and we re-ran each repo's gate ourselves. Codex CLI was not logged in, so an AGENTS.md reader without hooks wasn't tested live.
• Gates: each repo's own commands as its docs define them, plus a neutral Prettier check with the same settings on every repo.
• Mutation testing: 29 matching hand-written mutants, each applied alone, against the back-end and front-end unit suites. e2e wasn't included.
• Specs 003, 004 and 005 were built from their base commits with the plan before the implementation, the repo owner running each Spec Kit step. Earlier attempts at 003 and 004 are kept as archive/003-guardrail-parity and archive/004-wcag-accessibility and aren't scored. SPECTRA evidence comes from branch 005-api-contract-gaps at 64a3dc1. The agent sessions ran on 5cc5091, just before spec 005, which changed only contract tests, fixtures, the schema's published limit and docs, not the rules, guard or agent wiring.
• Scores are judgement calls backed by the evidence shown. Totals weight all 19 lenses equally. Reweight them for your context.
• Small samples. One feature built once per method, and one session per agent, repo and prompt. Agent behaviour varies from run to run, so lens 19 is indicative, not a benchmark.