THE FIRST PUBLIC PROOF POINT
We gave the same public issue and baseline to the same primary implementer and reviewer models. Selvage and a carefully constructed manual workflow both reached a verified, reviewed, built, documented, locally merged endpoint. The difference was who had to operate the system.
A single controlled engineering experiment—not an official SWE-bench result, not a claim of universal code superiority, and not evidence of lower total token use.
AT A GLANCE
Both routes passed. The difference is who ran the engineering process around the models.
| Engineering responsibility | Without Selvage Arm C | With Selvage Arm B |
|---|---|---|
| Final acceptance and review | 7/7 + approve | 7/7 + approve |
| Start from the public issue | Human builds protocol | slv run 1 |
| Run checks and route recovery | Human coordinates | Automatic |
| Launch fresh independent review | Human prompts | Automatic |
| Interpret verdict and choose next step | Human decides | Policy-routed |
| Capture evidence and usage | Human assembles | Automatic |
| Merge the approved local result | Human fast-forwards | Automatic |
means human-operated in Arm C, not impossible. The manual arm completed every step.
WHAT WE TESTED
The task added configurable think/eat timing and bounded shared byte accounting to a small Python Dining Philosophers project. Both arms started at commit 408fa95 and were judged by the same evaluator, frozen before either final run.
From that invocation, Selvage coordinated implementation, four verification/recovery cycles in one continued Sonnet session, a fresh Opus review, evidence capture, and a local auto-merge.
A human constructed and coordinated the implementation prompt, project verification, independent reviewer prompt, verdict interpretation, hidden evaluation, artifact aggregation, and local fast-forward merge.
WORKFLOW COMPARISON
The top lane is the path initiated by slv run 1. The lower lane shows the human coordination required to construct an endpoint-equivalent manual run.
PROVIDER-REPORTED COST
Provider-reported cost is the canonical measure here. Arm B spent more on iterative Sonnet implementation and recovery; Arm C spent far more on its long, manually prompted Opus review. Support-model calls in Arm C are grouped with the phase they supported.
Arm B did not use fewer total raw tokens: it also recorded 3,293,377 cache-read tokens versus 1,466,574 for Arm C. Its measurable gain was automated coordination, with much less expensive reviewer output, at essentially the same total provider cost.
ENDPOINT EVIDENCE
| Measure | Arm B · Selvage | Arm C · Manual parity |
|---|---|---|
| Frozen hidden evaluator | 7/7 passed | 7/7 passed |
| Final project suite | 13/13 passed | 19/19 passed |
| Package build | Passed | Passed |
| Documentation | Updated | Updated |
| Independent review | Fresh Opus · approve | Fresh Opus · approve |
| Provider-reported cost | $1.7371 | $1.7405 |
| Elapsed time | 9m 24s pipeline | 12m 21s wall time |
| Merge endpoint | Automatic local merge | Manual local fast-forward |
| Human orchestration | slv run 1 | Full manual protocol |
| Audit artifacts | Generated automatically | Assembled manually |
The project-suite counts are not a quality ranking: each implementation added a different number of regression tests. Reviewer finding counts are omitted because Arm C’s review prompt was deliberately more exhaustive, making raw counts non-comparable.
THE CLAIM BOUNDARY
A cheap patch is only cheap if you ignore the work required to ship it.
THE PUBLIC JOURNEY
This page is the beginning of a durable evidence trail. Future entries will add repeated runs, stronger controls, and efficiency measurements without rewriting what earlier milestones actually proved.
From a public issue to verified, independently reviewed, documented, built, locally merged code—initiated with slv run 1.
Measure AST/Tree-sitter context selection and context compaction against the same outcome bar.
Test verification deduplication and evidence reuse while preserving or strengthening coverage.
Repeat controlled comparisons across task shapes and repositories, publishing failures as well as wins.
Define scoring and acceptance criteria before execution and seek independent evaluation.
Planned items are research directions, not shipped product claims or release commitments.
REPRODUCIBILITY & DISCLOSURE
408fa9517c0ebee2f2e0b69slv run 14debfc65-e67c-46bd-8592-de4ed62c28c1The out-of-tree evaluator covered API compatibility, timing, wait reporting, shared byte accounting, partial final meals, repeated concurrent consumption, and deadlock freedom. The baseline passed 1 of 7 tests; both final arms passed 7 of 7.
Both experiments ended in a clean local merge. Arm B merged automatically; Arm C was fast-forwarded manually. Neither experiment result was pushed to the public repository.
An additional total_bytes plus default bytes_per_meal case was found after the frozen run. It is not counted among the seven preregistered acceptance checks.
Provider-reported usage, local reports, review artifacts, and evaluator output were used to prepare this page. Raw private run artifacts are not presented here as independently audited evidence.
MILESTONE 01 · ACHIEVED
Selvage reached autonomous issue-to-verified-review-to-merge execution. The next milestones will test how efficiently, repeatedly, and broadly it can do it.