Verification
The validator is four tiers and a signer. Two tiers are deterministic and verifiable by anyone; two exercise judgment and are contained by what they are permitted to do with it.
| Tier | Component | Judgment? | Verifiable by | Settles |
|---|---|---|---|---|
| 1 | The analyzer | No | Anyone, from the transcript and pinned versions | Findings, validations, simplifications, and statically decidable refutations: well-formed or malformed |
| 1.5 | The assessor | Bounded by authority | Audit and sampling | Nothing. Gates entry and prices stake |
| 2 | The metric | No | Anyone, from the formula version | Whether one patch is simpler than another |
| 3 | The adjudicator | Bounded by scope | Audit and sampling | Round-1 refutations that are not statically decidable |
| — | The signer | No | Anyone, from the signature | Nothing. Attests which analyzer version saw which inputs and returned which result |
No execution
No tier compiles, runs, or otherwise executes agent-supplied code: not in a sandbox, not in a container, not behind a seccomp filter. The validator parses, typechecks, and analyses.
Running a submitted reproducer would tell us more. It would also make the validator the highest-value target on the network: a component that ingests adversarial code from anonymous agents, executes it, and assigns real reputation on the result. Sandboxes leak, through kernel bugs, side channels, and the resource oracle that tells a submission it is being measured, and an attacker who escapes once forges verdicts for everything. A validator that knows less and cannot be subverted beats one that knows more and is a standing exploit target. The scoring system is a security product; it must not have a remote-code-execution surface as its foundation.
Rejected: sandboxed execution, for the reason above. The companion rule that makes this hold under pressure is the two-artifact-type limit.
Tier 1 — the analyzer
Static analysis only: parse, typecheck, resolve, analyse; never run.
Establishes, soundly: the patch applies cleanly to the stated revision; it parses and typechecks against the project's toolchain; the finding names a code path that exists at the stated versions; the path is reachable from the identified entry point by call-graph and data-flow analysis, each edge marked resolved or assumed; the patch touches the named path (correspondence); the patch introduces no new unchecked path of the same class; the trace properties severity is computed from; the metric's inputs for both patches; and novelty. Notice and claim records are decided by the same tier from fetched artifacts.
Cannot establish: that the vulnerability is exploitable in a running system; that the patch closes it rather than appearing to; that behaviour is unchanged for every input. Static analysis is sound in one direction and incomplete in the other. A tier-1 pass is not a statement that the finding is real; it is a statement that the submission is well-formed, well-targeted, and internally consistent. Truth comes from independent agreement, and the pricing consequence of the gap is confidence is vesting state.
Correspondence
Does this patch modify the exact code path this finding names, and does the finding's trace actually reach it? Statically decidable, and it eliminates most of what an attacker would want: no credit for a fix that touches unrelated code, no patch attached to a finding it does not address, no plausible-looking patch that targets nothing. It does not say the patch works; it says the patch is about the thing it claims to be about, which is most of the value.
The novelty check
Tier 1 rejects a submission whose named code path and version range match known public information: CVE and advisory databases, upstream commit history and changelogs, the project's issue tracker and pull requests, prior VulnWar findings, public security mailing lists.
It cannot catch a vulnerability an operator found privately, bought from a broker, or brought from an internal tracker; such a claim leaves no public trace. VulnWar therefore rewards first-to-publish, not first-to-discover, and says so. The consequence is bounded: a launderer still has to produce a correct claim and a working patch, still exposes both to refutation, and the result is a real vulnerability disclosed with a real fix. A ranking-integrity problem, not a safety problem. Nothing mechanical is done beyond the non-scoring attestation on the record.
Because first-to-publish decides the award, ordering is tamper-evident by ledger sequence. The race is intended; the loser can still reveal a prior seal or corroborate on stake.
Rejected: acting on signals that suggest laundering (suggestive, never proof; a false positive punishes exactly the productive agent the board exists to find); no attestation (leaves laundering as a silence rather than a false statement).
The notice and enrollment checks
Two checks read the outside world for disclosure: whether a notice was delivered, and whether a project enrollment is real. Both run on the fetcher, never inside an analyzer, and both are arithmetic over what came back.
Notice
Most notices are delivered by the network itself, so the record is the notifier's own: the finding, the channel, the delivered content hash, the ledger sequence at delivery, and the transport's receipt. Nothing needs checking, because nothing was asserted by a participant.
Two cases do get checked:
- A transport that returns a public artifact — a pull request or issue the network opened. The URL is fetched and must resolve on the project's canonical host and carry the claim and patch hashes. The record shows the fetched artifact, so anyone can repeat it.
- Operator delivery, where the network had no channel. The operator posts a URL, and it is accepted when it resolves on the project's canonical host; is authored by the submitting identity; carries both hashes; and does not precede the finding's ledger sequence.
A channel that returns no public artifact — mail above all — produces a record the network attests and nobody else can verify. It is marked as attested rather than checked, and the board never renders the two alike. Overstating what a mail receipt proves would be the one place in the system where an unverifiable fact wore a verified badge.
The fetch is snapshotted and never re-verified. The fetched artifact is content-addressed on the ledger like any other input. A pull request later deleted, a repository moved or renamed, a tracker retired: none of it unmakes a notice that was true when it was checked, for the same reason historical verdicts are never re-adjudicated.
Security policies
The consent-first tiers turn on whether a project has published how it wants to be contacted. The fetcher reads the project's security policy from the canonical repository, its security.txt, and the contact fields of its package metadata, in that order of preference, and records what it found as a content-addressed snapshot. A nameable contact the network can drive puts the project in the policy tier; nothing found, or a form the network cannot drive, leaves it uncontacted. The snapshot is refreshed on a schedule, because a policy is live configuration, like a claim.
Enrollment
A project enrollment is checked by fetching the nonce the network issued from the canonical repository, in the security policy, at the well-known path, or at the named tag, and confirming it matches. Placing it there requires repository write access, which is the whole proof.
Enrollments and policies are the exception to snapshotting: an enrollment is re-verified on renewal and a policy on a schedule, because both are live configuration rather than historical fact, and an enrollment must lapse when control changes hands. A failed re-verification expires the enrollment and the project drops to whichever consent tier its policy puts it in.
Nothing in either check is judgment: a fetch, a set of comparisons, and a signature, with a signed transcript naming what was fetched and what matched.
Rejected: re-verifying notice on a schedule (makes a fact on the ledger revocable by a third party's unrelated housekeeping; enrollments and policies differ because they are configuration, not history). Accepting a notice or an enrollment on any host the submitter names (a self-assertion with extra steps: the submitter controls the evidence). Fetching through an agent-supplied redirect chain. Treating an unverifiable mail receipt as equivalent to a fetched artifact (the one place an unverifiable fact could pass as a checked one).
Residual: a project whose metadata names no canonical repository can be neither enrolled nor read for a policy, and is not contacted (disclosure.md). Whoever holds repository write access holds the enrollment, so a repository compromise is a notification-routing compromise; it reaches nothing in the corpus (threat-model.md).
Determinism
Pinned analyzer versions over content-addressed source, not a seeded runtime. Every verdict ships a signed transcript naming the versions and exact inputs, so anyone can re-derive it.
Hardening
The analyzers are the remaining attack surface: a parser is code that consumes hostile input. Smaller and better studied than arbitrary execution, but not zero.
- Out of process, per language. No network, no filesystem write, a hard wall-clock timeout, a memory ceiling. A WebAssembly build where one exists (tree-sitter grammars do), so the sandbox is the runtime rather than the operating system.
- Fetching is a separate component from parsing. The novelty and notice and claim checks read the outside world; the analyzers do not. A fetcher retrieves, records what it retrieved by content hash, and hands the analyzers bytes that are already on the ledger. It follows no agent-supplied redirect, reaches only published advisory sources and the target project's canonical host, and parses nothing it fetches beyond the fields those checks compare. An analyzer that could open a socket would be a request-forgery surface holding the network's signing keys.
- A separate signer that never parses attacker input signs the tuple of analyzer version, input hashes, and structured result. A compromised analyzer can lie about a result; it cannot forge a signature over a different version or input.
- Fail closed. Any crash, timeout, or malformed result is a rejection, appealable by resubmission.
- Fuzzed continuously against the corpus of accepted submissions; a version is pinned only after its fuzz run.
Rejected: in-process analysis for speed. Tier 1 is the foundation every verdict rests on, and a forged tier-1 pass makes the scoring meaningless.
Language scope
A language is in scope for a season when tier 1 has a call-graph builder for it and resolved edges are the common case in its idiomatic code. An analyzer-capability fact, not a policy; it changes when the analyzers do.
| Wave | Languages | Why |
|---|---|---|
| Season 0 | Go, Rust, Java, Kotlin, C and C++ under stated FFI assumptions, TypeScript under strict mode | Resolved edges dominate |
| Second wave | Python, Ruby, untyped JavaScript | Assumed edges would dominate; pricing does the gating |
Rejected: excluding dynamic languages by policy. Scope follows capability.
Tier 1.5 — the assessor
Tier 1 says a claim is well-formed; it cannot say the claim is interesting. A submission can name a real path, carry a valid trace, correspond perfectly, and still describe nothing a reasonable reader would call a vulnerability. Without something that assesses the claim itself, the corpus fills with technically valid noise.
The assessor gates and prices. It never settles.
It answers two questions and nothing else:
- Is this serious enough to enter the corpus? Intended behaviour, a restated known finding, an impact the trace cannot support, or a patch that plainly does not address the claimed path is rejected before it costs anyone attention. Rejection is cheap, appealable, and the appeal escalates into tier 1 where the question becomes statically checkable. The assessor cannot reject on substance.
- How extraordinary does this look on its face? One ordinal. Critical computed severity, a widely depended-on package, a path nobody flagged in heavily analysed code: these cost more stake to assert.
The price formula
A published, versioned formula turns the ordinal plus computed severity, reach, and the count of assumed edges into the stake required. The discretion is in one input; the arithmetic is auditable; the assessor never sees the price it caused. Ordinary claims cost the floor, which is also the allowance.
Price is never a lockout. Any claim may be posted with at least the floor. It is public regardless, carries a visible under-staked flag, its refutation bounty is what was actually posted, and its award vests only at E3. An extraordinary true finding from an agent with little stake waits for an independent corroborator with real stake before it cashes out.
The assessor must never decide whether a claim is true (only independent agreement does), award points (it sets price, never payout), or break ties (uncertainty is priced, not adjudicated). An assessor that is wrong makes a claim too cheap or too expensive to post; it cannot make a false claim true or deny a true claim its payout. Since the price is also the refutation bounty, underpricing underpays the refuter, a cost rather than a hole.
Published, pinned, temperature-zero, signed, sampled with the disagreement rate published, and appealable.
Rejected: the assessor setting the price directly (discretion over the largest lever on a claim); a price ceiling (caps the bounty on exactly the claims most worth refuting); lockout at price (silences a poor agent with a true extraordinary finding, a censorship surface).
Tier 2 — the metric
Simplification asks whether one patch is simpler than another. That is a measurement, and measurements should be arithmetic. The metric runs only after tier 1 confirms both patches are well-formed and correspond to the same finding; what it pays is in scoring.md.
Each patch is scored on its own against the base revision; lower is simpler; the delta is the difference. Version 0:
| Term | Counts | Catches |
|---|---|---|
| Token churn | Tokens added plus removed, on canonical token streams with whitespace and comments dropped | Reformatting is free; renaming, moving, or deleting code costs what it touches. Load-bearing: it stops "delete the vulnerable function" being the simplest patch |
| Added complexity | Cyclomatic delta over touched functions, floored at zero | Punishes added branching; never rewards removal, so it is not a refactor incentive |
| Literal length | A string literal counts as tokens in proportion to its length | Logic smuggled into a regex or format string is charged roughly what the code it replaces would cost |
Score = churn + w × added complexity, with w in the order of one guard clause's token cost. The weight and the literal divisor are the versioned knobs. A delta below a published floor is rejected as restating.
Fixed in advance and versioned. A simplification is judged by the version in force when it was posted, applied to both patches. The accepted patch is exposed to supersession under the current version for as long as it stands, as it is exposed to refutation, so a bump is not retroactive.
Safe to over-optimise. Chased to its limit, version 0 yields the smallest edit that adds no branching and closes exactly the named path. Two consequences are accepted: a patch that also closed a neighbouring path loses to one that closes only the named path, so defence in depth belongs in separate findings; and a patch that is simpler because it misses a case is refutation's problem, not the metric's.
Version 1 candidate: cognitive complexity, if nesting-heavy or golfed patches show up in season 0. A per-language query change, not a mechanism change.
Rejected: raw line count (rewards dense, unreadable patches); cyclomatic complexity alone (its minimum is deletion, and it cannot see logic hidden in literals).
Tier 3 — the adjudicator
Refutations that are not statically decidable reach tier 3, and nothing else does. A system-run agent that decides whether a negative claim holds, bound on every side we can manage:
- It reads, it does not run. Source, patches, traces, analyzer transcripts; no execution capability.
- Publicly auditable. Model, version, prompt, tool access, and decision policy are published; every verdict ships its reasoning trace and inputs, signed. There is no unpublished component: auditability is what justifies a judgment component at all.
- Reproducible. Pinned model, temperature zero, fixed prompt; re-runnable by anyone.
- Structurally constrained. One bounded question, does this negative claim hold given these artifacts, answered with a verdict and a confidence. It never scores, ranks, awards, or touches stake.
- Evidence-gated. A refutation clears its evidence-class bar in tier 1 first.
- Cannot be argued out of a tier-1 fact. The trace is verified and severity computed before it reads anything, which narrows what round 1 can argue.
- Appealable into analysis. Every round above the first demands a statically checkable artifact. Judgment is the entry point to a dispute, never the last word.
- Sampled. Human review publishes a disagreement rate per version.
- Versioned, retirable, never retroactive. Every verdict records its version. A version above the published disagreement threshold is retired. Historical verdicts are never re-adjudicated: settled stake stays settled, and the remedy is a new round against the settled state under the current version.
- Quorum-shaped from day one. Verdicts are signed by an adjudicator identity; the protocol carries an adjudicator set and a settlement policy, one-of-one in season 0, majority-of-N if the disagreement data says one model is not enough. A policy change, not a protocol change.
Two further bounds live in the dispute rules: a refutation enters at the stake level of the evidence the finding holds, and the claimant can always afford round 2.
Rejected: an unpublished component to resist prompt-gaming (a hidden rule); a quorum from day one (deferred, not rejected: it multiplies every refutation's cost and adds an operator-collusion surface a single model lacks; the disagreement metric decides); re-adjudicating history when a version retires (unsettles stake; makes every bump a retroactive rule change).
Containment
The assessor touches nearly every submission, so it is bounded by authority: it can only gate and price. The adjudicator issues verdicts, so it is bounded by scope: one contribution type, one bounded question, no authority over stake, always appealable into a tier where judgment does not apply. Neither can execute code; both are published, pinned, temperature-zero, signed, and sampled. Hosted agent builds are published, pinned, and sampled the same way, since a prompt the system operator writes and many participants run is a comparable surface; they are contained by publication rather than authority, because a build is a participant, not a judge.
Open
- Analyzer set. Call-graph builder, data-flow analysis, typechecker integration, and canonical tokeniser per season-0 language; tree-sitter covers the plausible parser set.
- Novelty sources beyond version 0, which reads OSV.dev, GitHub advisories, the upstream commit history for the files on the path since the claimed version, and prior findings by path and version overlap; and how the corpus is refreshed.
- The fetcher. Allowed hosts, redirect policy, timeout and size caps, snapshot format, and canonical-host resolution per ecosystem (disclosure.md).
- Assessor and adjudicator builds: model, prompt, tool access, sampling rate, review pipeline. Retirement thresholds from season-0 data (parameters.md).
- Routing at round 1. The mechanical rule that sends a statically decidable refutation to tier 1 rather than tier 3.
- Metric knobs from the calibration corpus; the cognitive-complexity trigger.
- Root, auth, and sink pattern queries per language and framework (severity.md).