Methodology

Challenge Market

Testnet pilotSource: docs/methodology/challenges.md

An optimistic oracle for Karatum's own outputs. Anyone can put a bond against a Karatum verdict or certificate and force a re-check. A challenger who is right gets its bond back plus most of Karatum's matching stake; a challenger who is wrong loses its bond. The point is that the truth of a verdict or a certificate stops depending on trusting Karatum alone: the money is in a contract, every re-check is reproducible from public data, and the verdict path can be settled by a zero-knowledge proof without Karatum at all.

Everything runs on testnets: ChallengeManager on Ethereum Sepolia, bonds in test USDC (open mint), re-trials on a private Anvil fork of Base. No real money is involved.

ContractChallengeManager (Sourcify exact match), source contracts/src/challenges/ChallengeManager.sol
Bond tokentest USDC 0x9ed7465EB31739F623CAFBC6A0D760De547Af13D (6 decimals, open mint)
Vote weightsValidatorStaking 0x1312a0dEB4537D2348a7680C6e040be00c4d92ae
zk pathZkJudgeVerifier 0x5b03cf51e29F91DEAc8A129d5373fF6D28b4B823 (see zk-judge.md)
Codepackages/challenges (rules, ids, hashing), services/challenges (resolver, timer, CLI, vote client), services/runner/src/retrial.ts, migration 030_challenges.sql, API /v1/challenges, Explorer /challenges

Two kinds of challenge

Verdict challenge: "this verdict does not follow from its trace"

Since October 2026 every verdict is a pure function of a stored trace: verdict = judge(trace) (see verifiable-judge.md). A verdict is registered on-chain with its attested verdictHash and its traceHash. The resolver re-runs the published evaluator bundle on the stored trace and compares:

  • the recomputed trace hash with the registered trace hash, and
  • the recomputed verdict hash with the registered (attested) verdict hash.

Both equal: the challenger loses. Either differs, or the trace cannot be judged: the challenger wins, the verdict is revoked in the store and its EAS attestation is revoked.

Trustless settlement. When the verdict is in a batch that ZkJudgeVerifier has proven (an SP1 Groth16 proof that (traceHash, verdictHash) = judge(trace) for every trace in the batch), anyone can call resolveVerdictByProof(challengeId, root, merkleProof). The contract asks the verifier itself and, on level 4 (ZK_PROVEN), settles the challenge against the challenger at once, with no dispute window and no vote. The resolver prefers this path whenever a proof exists. It answers exactly the challenge's claim (the verdict follows from its trace). It does not prove the trace is what happened on-chain: evidence collection is outside the judge (see zk-judge.md).

Eligible: public verdicts with a stored trace, attested on Sepolia, not revoked, within 30 days of the decision.

Certificate challenge ("short the agent"): "this agent no longer performs at its certificate's lower bound"

A certificate claims a build's success rate is at least ciLow, the lower end of the 95% Wilson interval over its verified outcomes. The resolver runs a fresh re-trial:

  1. When it picks the challenge up it draws a random 32-bit seed and a 32-byte salt and publishes only seedCommitment = keccak256(abi.encode("karatum.retrial-seed/1", challengeId, seed, salt)).
  2. The seed generates a new frontier web3 task set (generateFrontierTrial(seed)); 24 tasks are taken in the order of sha256(seed:taskId). Nobody, Karatum included, knows the tasks before the commitment is published, and the challenger cannot pick them.
  3. Each task runs on a private Anvil fork of Base (pinned block 52136000, the block of the frontier suite; nothing is broadcast) from the same snapshot, and every verdict is decided by the published judge from its trace. Re-trial verdicts are stored in challenge_retrial_attempts with their full traces and are never written to attempts: they cannot feed ratings, certificates or attestations.
  4. With k verified successes (SUCCESS only) out of n tasks, the challenger wins iff the upper end of the re-trial's own 95% Wilson interval is below the certified ciLow. Even the most generous reading of the new evidence cannot reach the certified floor. The rule leans towards the certificate on purpose: a borderline drop (say 20/24 against a 0.90 floor) is not enough. At least 12 decided tasks are required; harness errors are reported and not counted.
  5. The evidence bundle (claim, seed, salt, commitment, fork block, scenario, every verdict hash and trace hash, the decision) is published at /v1/challenges/:id/evidence, and keccak256(canonical JSON) is the evidence hash posted with the resolution. Every trace is at /v1/challenges/:id/retrial from the moment the resolution is posted (not before, so the seed stays hidden).

A won certificate challenge:

  • opens an Incident Registry entry (class cert_revoked, the same dedupe key the registry's own revoked-certificate detector uses, the drill flag carried over) and confirms it;
  • when the build has wallets with Guard leaves, runs the incident recall (revokeCert on every Guard registry, IncidentLog);
  • sets the certificate REVOKED in the store. A trigger in migration 030 keeps it REVOKED (and keeps its dates) when the daily certify job rewrites certificates, until a maintainer lifts the revocation (karatum-challenges lift <id> <note>). The next Guard root no longer carries it;
  • marks the subject Overturned on-chain, permanently.

Which builds can be re-tried today. Karatum's deterministic scripted solvers (scripted-reference-frontier-v1, scripted-naive-frontier-v1) are re-tried at once: they need no model. Every other build needs an LLM. Karatum's LLM gateway (the LLM inference host) is down, so those re-trials are queued and the challenge shows pending_llm. If a queued re-trial cannot run before the resolve deadline, anyone can expire the challenge and every bond goes back where it came from.

Eligible: VALID certificates, until they expire. A drill may register a certificate record of a Karatum house build that is not VALID; it is labelled as a drill everywhere.

Money

  • Bond = max(10 tUSDC, 1% of the subject's value). A verdict is valued at 0 (bond 10 tUSDC); a certificate at 100 tUSDC per karat (a 23-karat certificate takes a 23 tUSDC bond). The bond is exactly this amount, never more, so a large bond cannot lock the defender pool.
  • Defender match. Opening a challenge reserves the same amount from Karatum's defender pool (2,000 tUSDC at launch). No pool, no challenge: Karatum must have stake behind its claims.
  • Payout. Winner: own stake back plus 80% of the loser's stake. Treasury: the other 20%. Challenger loses: the defender pool gets its match back plus 80% of the bond. Credits are withdrawn with withdraw, which keeps working while the contract is paused.
  • Expiry. Resolver silent past the deadline (2 days): anyone calls expire, the bond is credited back and the match returns to the pool.

Escalation to a validator vote

A challenger can dispute a resolution against it (not a zk-settled one) within the dispute window (1 hour on Sepolia) by escalate, posting an escalation bond equal to its bond. For 1 day, validators allowed by the owner and staked in ValidatorStaking vote; a vote weighs the validator's stake minus any pending unstake (minimum 100 KARAT). The challenger and the resolver cannot vote. After the deadline anyone calls settleVote:

  • quorum (600 KARAT voted) and a strict overturn majority: the challenger wins everything as above, escalation bond included;
  • quorum and no overturn majority (ties uphold): the resolution stands, the challenger loses bond and escalation bond;
  • no quorum: the resolution stands and the escalation bond is refunded (validators not showing up is not the challenger's fault).

Validators re-derive the outcome from public data only: karatum-challenges vote <id> fetches the public verdict record or the evidence bundle and every re-trial trace, checks the bundle hash against the on-chain resolution hash, re-runs the judge on every trace, re-applies the rule and votes. A bundle that does not hash to the posted value, or a trace that does not recompute, is a vote for the challenger.

Anti-griefing

One active challenge per subject; a 1-day cooldown on a subject after a challenge against it fails; a 3-day cooldown on a challenger who lost; bonds that scale with the value at stake; the resolver key cannot open challenges; subjects can only be registered by the resolver, and only with facts re-checked from the store (registration requests through the API are rate-limited like every API call).

Trust assumptions (v0)

  • The resolver (dedicated key 0xD20C…e60b) registers subjects and posts resolutions. It cannot touch bonds or credits, cannot resolve after the deadline, cannot stop a zk settlement and cannot stop withdrawals. A wrong resolution can be escalated (except a zk one, which is not a Karatum statement at all).
  • The owner sets parameters, the treasury and the voter allow-list. Voter weights come from ValidatorStaking, but who may vote is curated: an open voter set would let anyone stake just before a vote. Today the three validators are Karatum's own nodes, so the vote is not yet independent of Karatum; that changes as outside validators stake.
  • Re-trials of scripted builds are fully reproducible; a re-trial of an LLM build would be a fresh, non-deterministic sample (its traces are still judged by the same pure judge).

Security self-review of ChallengeManager

  • Accounting: token.balanceOf(this) == defenderPool + defenderReserved + bondsHeld + totalCredits on every path; checked by a 1,000-run fuzz over all four resolution paths (win, loss + finalize, expiry, escalation with either vote result) and by _assertSolvent in every unit test. recoverExcess can only take the bond token above totalOwed.
  • Reentrancy: every state-changing function that moves tokens is nonReentrant (transient guard); bonds are pulled with ExactTransfer.pullExact after state is written, so fee-on-transfer tokens revert (tested) and a reverting pull rolls back everything.
  • Payouts are pull-based credits; no external call to a user-controlled address happens during resolution.
  • No funds can be frozen: expiry needs no one but time; finalize, settleVote and withdraw stay available while paused; the owner cannot renounce (that would strand the pool).
  • Time: deadlines are uint64 seconds via SafeCast; windows bounded (resolve 1 h to 30 d, dispute and vote 10 min to 14 d, cooldowns up to 30 d).
  • Votes read ValidatorStaking.getValidator (a view) at vote time; double votes, votes after the deadline, and votes by the challenger or resolver revert. The arithmetic (votesOverturn > votesUphold with quorum) cannot overflow (uint128 weights).
  • zk path: the contract trusts ZkJudgeVerifier's check (no owner, program keys allow-listed in the EvaluatorRegistry). A verifier answer of anything other than level 4 reverts with its reason code.
  • Known limits: the resolver and owner are single keys (testnet); voter allow-listing is owner-curated; the subject value (bond scaling) is set by the resolver from store facts.
  • Tooling: forge test --match-path 'test/challenges/*' (31 tests including the fuzz), forge lint src/challenges (notes only: the two small view interfaces live in the same file).