Open tooling, agent-run engineering, and the products built on both.

Prose Is Code Nobody Lints

  • #ai-governance
  • #agents
  • #open-source
  • #instruction-files
  • #tooling
  • #cost

Every rig that runs agents has a layer of prose inside it. Skill definitions, agent briefs, the governance file at the repo root, the memory index injected at session start. That prose decides what the agent does — and in most projects nothing checks it. No linter reads it, no test exercises it, no gate stops a contradiction from shipping. It is code, and it is the only code in the repo held to no standard at all. Two tools go open source together on exactly that observation.

The deterministic pass runs first

agent-audit treats the instruction layer as something a machine can inspect. Discovery walks the skill and agent files into a manifest, and a structural checker turns that manifest into a verdict per file using only what a script can know for certain:

  • frontmatter present, carrying a name and a description, with a name that matches the directory it lives in
  • trigger language — does the file ever say when to use it?
  • referenced files that do not exist, and sibling files the skill never mentions
  • declared external dependencies: URLs, environment variables, binaries invoked
  • a line-count complexity budget

Each finding carries a verdict — pass, warn, or fail — and an evidence line naming the check and the detail. No model is consulted for any of it, because none of it needs one.

Only then do the judgment passes run, in parallel, each its own agent process with its own context: one for security, one for quality, one asking whether a skill is still doing by prompt what a script should be doing by now. A final pass reads the whole set at once and reports what shows up only across files — overlapping triggers, dependency chains, conventions that drifted apart.

Every pass returns JSON against a declared schema, not prose for a human to re-parse. A pass that fails does not vanish. It lands in a coverage-gap section of the report, named, so a thin report and a clean report never look alike.

The files agents actually read

The same shape points at the instruction files themselves: governance files at the repo root, skill and command directories, memory indexes. Three passes run in parallel — semantic contradictions (severity, the conflicting directives as exact quotes, a suggested resolution), an effectiveness score per file across clarity, specificity, enforceability and coverage, and a context-budget pass that names what to cut and what cutting it saves.

When rigscore is installed, its mechanical instruction check runs first and rides along as a baseline. When it is not, the report prints skipped and the reason, rather than quietly grading on fewer inputs.

Fan-out you can read back

claude-batch-runner is the other half. Agent fan-out is normally a habit — a person deciding, per sitting, how many workers to run and what “good” looks like. Here it is a file:

 1{
 2  "worklist": "units.jsonl",
 3  "worker":   { "agent": "unit-worker", "tier": "<cheap tier>", "effort": "low" },
 4  "parallelism": 6,
 5  "schema":   { "type": "object", "required": ["finding", "confidence"] },
 6  "escalate": { "condition": "confidence < 0.8",
 7                "advisor": { "agent": "advisor", "tier": "<strong tier>", "effort": "high" } },
 8  "verify":   { "rubric": "rubrics/finding.md", "agent": "grader", "max_revisions": 1 },
 9  "deliver":  "report-only",
10  "report":   "reports/{date}-{campaign}.md"
11}

Tier names are whichever model tiers your agent CLI exposes. Loading that file is boundary validation, not a parse: an unknown field, a missing worklist, an unrecognized tier, a schema whose declared type is not a JSON type, a rubric path that does not resolve — each one raises an error naming the offending field, before a single agent starts. The worklist is one unit per line, a JSON value or a bare path. Workers run as parallel subprocesses at the declared width, and every result must satisfy the campaign schema.

The expensive model does not need to see most of the work

This is the part worth stealing even if you never run the tool.

Cheap workers do the sweep. The escalation rule is plain arithmetic over a field of the worker’s own output — a condition like confidence < 0.8 — and the unit that trips it, only that unit, is re-run by the strong-tier advisor. The advisor receives the cheap draft as data, inside a frame that marks it as material to analyze rather than instructions to obey.

The verify stage is separate and stricter. It loads a rubric of numbered criteria, and the grader must return a verdict plus concrete evidence for every one. A verdict with no evidence is rejected as malformed. A criterion the grader skipped is rejected. On a failure the worker is re-invoked with the failed criteria framed as data, for a bounded number of attempts; if it still fails, the unit ends FLAGGED. It is never auto-passed and never silently dropped. The grader cannot run on the cheap tier either — the loader refuses that combination outright instead of warning about it afterwards.

Then the bill. Every run ends with a cost table: calls and tokens per tier. Call counts are exact; token totals are read from each result envelope, and when some envelopes arrive without usage numbers the report says how many rather than printing a confident total built on partial data. Sweep cheap, escalate a handful, and the table shows you precisely that — which is the only way to know whether your tiering is real or a story you tell yourself.

What they refuse to do

  • The delivery mode that opens pull requests is a stub, and it raises rather than running. What it prints instead is the contract it is designed to satisfy: one worktree and branch per unit, a diff cap per pull request, the dispatcher rather than the worker running the suite before any commit. A stub that refuses is honest; a stub that half-runs is a bug with manners.
  • Neither tool reaches for an API key. Both shell out to the agent CLI already on your path and inherit whatever authentication that CLI already has.
  • Neither one grades itself green. A failed pass, an unmetered call, a skipped baseline — each is printed as an absence, because a report that hides its own gaps is worse than no report.

Where they are

Both repositories sit in the same organization as rigscore:

The through-line is the one this journal keeps landing on: behavioral rules are suggestions; structure is a constraint. A line in a governance file asking agents to be careful is a suggestion. A loader that refuses a campaign whose grader was pointed at the cheap tier is a constraint. Both tools are built almost entirely out of the second kind.

Configuration details reflect a production environment at time of writing. Implementation specifics vary based on tooling versions, platform updates, and organizational requirements. Validate approaches against current documentation before deployment.

← Back to Journal