Workflows · 2026-09-05 · 1,192 words · 5 min read

Spec first: how do you write a task a coding agent will finish?

A six-section spec template for coding agents, a worked CLI example, the published cost of underspecified prompts, and a linter to run before handing over.

specs prompting claude code codex verification

What you get here: a spec template with six sections that an agent can act on without asking, a worked example for a small command-line tool, the published numbers that show why bare problem statements cost resolution rate, and a small linter that refuses a spec before the agent ever sees it. The linter ran in this session on a good spec and a bad one, and the real output is below.

The gap between what people type and what benchmarks test

The coding-agent benchmarks that get quoted are built from GitHub issues that a maintainer already wrote up. The SWE-bench Verified dataset card describes its 500 tasks as "human-validated for quality", each with a problem statement, a gold patch, and named tests that must flip from failing to passing. That is a well-specified task by construction.

Real requests are not like that. The RealSWE paper (arXiv:2608.27831) built a taxonomy of what a request contains and how it is phrased, then measured real prompts against benchmark problems. Its abstract reports that "88% of real prompts but just 7% of benchmark problems" contain only a problem statement with minimal context, and that "87% of real prompts are casually written whereas 94% of benchmark problems are formal". When the authors rebuilt tasks as 381 families that share a patch but vary the information given, resolution dropped by 6.4 percentage points on average across seven models. The task did not change. The words did.

Vendor guidance says the same thing from the cost side. The Claude Code costs page states that "Vague requests like 'improve this codebase' trigger broad scanning", while a specific request "lets Claude work efficiently with minimal file reads". The best-practices page puts it as a before-and-after table: "add tests for foo.py" against "write a test for foo.py covering the edge case where the user is logged out. avoid mocks." The second one names the file, the scenario, and a constraint. That is a spec in one sentence.

So the cheapest lever you have is the text of the task. Everything below is about spending five minutes on that text.

The six sections

The best-practices page describes the specs that work best as "self-contained: they name the files and interfaces involved, state what is out of scope, and end with an end-to-end verification step that proves the feature works." I turned that sentence into a template with six headings. The agent reads all six; the linter checks all six.

## Goal
One paragraph. What exists when this is done, from the user's side.

## Non-goals
Things the agent must not do or touch. This section is never empty.

## Interfaces
Command lines, function signatures, file formats, exit codes.

## Files
The files to create or change. Anything not listed is out of bounds.

## Verification
The exact commands that prove it works, and what a pass looks like.

## Budget
Turns, minutes, dollars, or files. What to do when the budget runs out.

Two of these do most of the work. Non-goals is where you stop the agent from rewriting the argument parser you did not ask about. Verification is where you convert "it works" into an exit code, which matters because an agent stops when the work looks done, and without a check it can run, looking done is the only signal it has. The best-practices page says exactly this: "Without a check it can run, 'looks done' is the only signal available, and you become the verification loop."

A worked example

This is the spec for a small tool called tally, a CSV column summariser. It is the same spec the tutorial on building a CLI tool from a one-paragraph spec starts from.

## Goal
Build `tally`, a command-line tool that reads a CSV file and prints count, min, max,
mean, and median for one named numeric column. Non-numeric cells are skipped and
counted separately.

## Non-goals
No plotting. No multi-column mode. No reading from stdin. No third-party packages.

## Interfaces
`python tally.py <file.csv> --column <name>` prints one line per statistic as
`name: value`. Exit code 2 when the column does not exist.

## Files
tally.py (the tool), test_tally.py (unittest cases), a fixtures/ folder with two
small CSVs.

## Verification
`python -m unittest test_tally -v` passes with at least 6 tests, including the
missing-column case (exit code 2) and a file with blank cells.

## Budget
At most 20 agent turns and 3 files touched. Stop and report if the tests are still
red after 3 attempts.

Notice what the spec does not contain. It does not say how to compute a median or which module to import. Those are decisions the agent is good at. It does say what "skipped" means, what the exit code is, and which two edge cases must have tests. Those are decisions the agent will otherwise make silently, and you will find out during review.

How the spec gets written

For anything larger than tally, I do not write the spec first. I let the agent interview me. The best-practices page ships the prompt: "I want to build [brief description]. Interview me in detail using the AskUserQuestion tool. Ask about technical implementation, UI/UX, edge cases, concerns, and tradeoffs. Don't ask obvious questions, dig into the hard parts I might not have considered. Keep interviewing until we've covered everything, then write a complete spec to SPEC.md." The same page recommends starting a fresh session to execute it, so the implementation context holds the spec and nothing else.

Where the spec lives depends on the harness. For a one-off task I paste it as the prompt. For a repository, the standing rules go in the context file the harness reads on every session, and the per-task spec stays separate. The Codex documentation describes how its instruction files stack: "Codex concatenates files from the root down, joining them with blank lines. Files closer to your current directory override earlier guidance because they appear later in the combined prompt," with a default ceiling of 32 KiB set by project_doc_max_bytes. A spec does not belong in that file. It belongs in the task. There is a separate article on context files for the standing rules.

The linter

The shipped script reads a spec and applies the checks a reviewer would apply in the first thirty seconds: are the six sections present and non-empty, does Verification contain at least one runnable command in backticks, does that section say what a pass looks like, does Budget contain a number, and do vague verbs like "improve", "clean up", or "polish" appear anywhere without a measurable check within a few lines.

I ran it on the tally spec above and on a deliberately bad spec that reads "Improve the CSV tool so it is faster and cleaner. Clean up the argument parsing," with "Make sure it works" as its verification. This is the real output from this session, Python 3.13 on Windows:

SPEC.md: 0 error(s), 0 warning(s)
exit=0
----
ERROR   missing section: ## Non-goals
ERROR   missing section: ## Files
ERROR   Verification has no runnable command in backticks (e.g. `python -m unittest`)
WARNING vague verb 'Improve' near line 1 with no measurable check nearby
WARNING vague verb 'Clean up' near line 1 with no measurable check nearby
WARNING Budget has no number (turns, dollars, minutes, or files)
BAD.md: 3 error(s), 3 warning(s)
exit=1

The heuristics are crude on purpose. A regex cannot tell a good acceptance check from a bad one, and "refactor" is flagged as vague even when the next paragraph defines it precisely, because the point of the flag is to make you look, not to block you. Errors fail the exit code; warnings do not. Wire it into a pre-commit hook on SPEC.md, or run it by hand before you press enter.

python spec-first-prompting-coding-agent.py SPEC.md && claude -p "$(cat SPEC.md)" --permission-mode acceptEdits

What the spec does not fix

A spec does not make the agent read the codebase carefully, and it does not stop a wrong implementation that happens to pass the named tests. Those are review problems, covered by the diff checklist and by writing the failing test first. A spec also drifts: after the second correction you type into chat, the spec is out of date and the chat is the real spec. The best-practices page suggests that after two failed corrections you clear the session and write a better initial prompt with what you learned. In practice that means editing SPEC.md and starting over, which is faster than it sounds.

I have not measured the effect of this template on my own tasks with any rigour. The 6.4 point figure is the RealSWE authors' number on their benchmark with their models, and it is the only controlled measurement I could find of the words changing while the task stays fixed. The method article on time to green is where I intend to measure it for the ten tasks I actually run.