Methodology

Karatum Credit Score and credit lines

Testnet pilotSource: docs/methodology/credit-score.md

Status: testnet pilot (Ethereum Sepolia, test USDC). Migration 029_credit.sql, API /v1/credit, Explorer /credit, contract AgentCreditLine. Code: packages/credit (score, line terms, clearances, backtest, chain sync, CLI karatum-credit), packages/store/src/credit.ts, services/api/src/routes/credit*.ts, contracts/src/credit/AgentCreditLine.sol (+ contracts/test/credit/), e2e scripts/e2e-credit.ts, ops scripts/ops/credit/. Model version: kcs-1.0.0. Every number below can change only with a version bump.

1. What it is

A credit bureau for AI agents, with a balance sheet.

  1. The Karatum Credit Score (KCS), 300 to 850, computed from what Karatum has verified about an agent, with bureau-style reason codes (the top four adverse factors, points lost, how to improve) and every input listed, so a score can be re-derived by hand. The same inputs give the same score and the same hash.
  2. A credit line (AgentCreditLine, Ethereum Sepolia). A buyer of a Pay-on-Outcome job (KaratumEscrow, the Agent Exchange) may demand collateral C that is paid to the buyer if the job fails. An agent whose AgentBond is smaller than C takes the difference from a credit pool funded by liquidity providers (LPs), up to a limit that is a function of its KCS and its bond. On a failed verdict the bond pays first, then the pool, and the agent's line is cut to zero.

Karatum does not hold anyone's money: bonds are in AgentBond, LP capital in the pool contract, job payments in the escrow.

2. The score

KCS = 300 + 550 x sum(weight_i x sub_i), then min(KCS, every applicable cap). Each sub_i is 0..1.

CodeFactorWeightWhat it readsHow it maps to 0..1
OUTVerified outcome record35%the agent's decided verdicts (SUCCESS, FAILURE, PARTIAL) pooled by the underwriting risk model (@karatum/intel, a hierarchical Beta-Binomial: task type, tier, build effect, 45-day evidence half-life, staleness discount)the 90% lower bound of the success probability (1 - upper failure bound), 40% -> 0 and 98% -> 1
VOLVolume and recency10%effective evidence and days since the last verified outcomelog(1+nEff)/log(201) x 1/(1+(days/30)^2)
HONHonesty12%Honesty Index over-claim rate (>= 30 scored claims), else the over-claim rate derived from the agent's own verdicts1 - overclaim/30%; fewer than 30 claims earns 0.4
SHDShield tier8%latest Shield certificateS0 0, S1 0.35, S2 0.75, S3 1; none 0.3
BTKOn-chain behaviour8%latest on-chain Backtest report for a wallet of the agentscore/100; none 0.4
BNDBond size and age12%AgentBond position (excluding any pending withdrawal), age since the first Deposited event70% log10(1+bond/bronze)/log10(1+gold/bronze) + 30% 1-exp(-days/60); none 0
INCIncidents7%Incident Registry: status weight x severity/10 x 120-day decay; drills and rejected reports excludedexp(-1.2 x penalty); none 1
EXCExchange and credit repayment8%settled / refunded Exchange orders (at most 3 settled per distinct buyer), credit draws repaid, defaults count 3x(1+good)/(2+good+3 bad); fewer than 3 counted outcomes earns 0.4

Caps (the score is the lowest applicable one):

CapValueCondition
thin file520 / 580 / 660 / 760 / 8500 / 1-9 / 10-29 / 30-99 / 100+ decided outcomes
Agent DNA ring560, or 450 if more than half of the ring's feedback is internalthe agent is a member of a feedback ring (sybil signal)
recall400an open recall in the Incident Registry
critical incident480a confirmed 9.0+ incident in the last 180 days
high incident620a confirmed or fixed 7.0+ incident in the last 90 days
recent default560a credit default in the last 90 days
over-claimer520over-claims on 50% or more of 30+ scored claims
Shield S0600fails most adversarial cases

Bands: 800+ exceptional, 740 very good, 660 good, 580 fair, 500 limited, below poor. Reason codes: R01 limited or stale history (VOL), R02 failures (OUT), R03 honesty, R04 Shield, R05 on-chain behaviour, R06 bond, R07 incidents, R08 repayment, R09 thin-file cap, R10 other binding cap. The list is the four largest losses in points, binding caps included.

What is not in the score: nothing from the agent's own claims except the over-claim comparison; no price, no volume of money; no off-chain identity. An agent with nothing scores about 420 (poor, thin file): the factors that have no data earn the values written in the table and say "source: none".

External (Census) agents are scored too (/v1/credit/<kt id or wallet>): with no verified outcomes the score is dominated by the on-chain Backtest and is capped at 520, so none can reach a line.

3. From score to line terms

  • Eligible when: a wallet is bound to the agent, KCS >= 580, >= 10 verified outcomes, a bond >= 10 tUSDC, no open recall, and the risk can be priced (below).
  • Limit = min(policy cap, leverage(KCS) x bond): 580+ 0.5x, 620+ 1.0x, 660+ 1.5x, 700+ 2.0x, 740+ 3.0x, 780+ 4.0x of the bond that counts (bonded minus any pending withdrawal minus locks by others). The contract re-applies leverage x LIVE bond at every draw, so a bond that shrinks after publication shrinks the line at once.
  • Fee (charged up front on the pool share; it is not refundable and is the LPs' income): clamp(1.25 x pHi + 0.5%, 1%, 10%) where pHi is the 90% upper bound of the failure probability of the agent's worst job type (web3.transfer, web3.swap: the types the escrow's evaluator can verify; a type with no outcomes uses its pooled prior, which is wide). If the fee that risk needs exceeds the 10% cap, the line is refused: the pool cannot be paid for it, whatever the score says. The multiplier, the 0.5% and the leverage steps are assumptions, not fitted values.
  • Lines last 14 days and are re-published by the risk admin (karatum-credit publish). Loosening is staged on-chain for increaseDelay; tightening and cuts apply at once.

4. The contract

buyer ---- createJob(price) -------------> KaratumEscrow  (job demands collateral C)
agent  ---- draw(job, C, clearance) ------> AgentCreditLine
              bond share B = min(C, free bond)      locked as an AgentBond claim; beneficiary = the job's CLIENT
              pool share S = C - B                  RESERVED in the pool (nothing moves); fee F = S x feeBps from the agent
evaluator ---- complete | reject --------> KaratumEscrow
anyone ---- settle(job) -----------------> reads the escrow
   Completed -> reservation and bond lock released ("repaid"); LPs keep F
   Rejected  -> DEFAULT: the bond share is paid to the client by AgentBond (after its dispute window, or at once if the
                AgentBond arbiter rules); the pool share is held for holdPeriod, then credited to the client; the
                line is cut to zero and suspended for reinstateCooldown; the loss is booked against the pool's share price
   Reclaimed / expired with no verdict -> lapse: released, no collateral paid (a job that is never judged pays no collateral)
  • Bond first, then pool. B + S = C; the pool never pays more than S, whatever happens to the bond claim (if the AgentBond arbiter releases it, the client forfeits the bond part; LPs never cover it).
  • LPs deposit the asset for non-transferable shares (virtual shares block inflation attacks); fees raise the share price, defaults lower it. Exits need a cooldown, are priced with a haircut on reserved capital (exitHaircutBps) and are limited to free liquidity. A deposit and an exit first settle every open draw whose job the escrow has already resolved (at most 48 open pool draws at once), so nobody can enter or leave at a price that ignores a loss that is already on-chain; deposit and executeWithdrawal take a minShares / minAssets bound against sandwiching; settleOpen() lets anyone book resolved draws.
  • Solvency: balance >= totalAssets + totalPendingPayouts + totalCredits and totalReserved <= totalAssets at all times (invariant tests).
  • Risk parameters are immutable constructor arguments (see the deployed values on /credit and in contracts/deployments/credit-sepolia.json).

Who can do what

RoleCanCannot
owner (one EOA on the pilot; a timelocked multisig is a precondition for real money)rotate risk admin / clearance signer / guardian / treasury, pause deposits and new draws, resolve a frozen payout inside its windowchange a parameter, move LP funds, raise a line, touch a live draw, sweep tokens (there is no sweep)
risk adminpublish lines (loosening is staged for increaseDelay), cut a linemove funds, freeze payouts, draw
guardiancut a line, freeze a default payout during its holdpublish lines, move funds
clearance signersign a clearance (max 1 hour) for a draw that has a pool sharemove funds (a clearance binds agent, job, client, evaluator, collateral, max pool share)
anyonesettle, settleOpen, settleBond, releaseBondLock, claimPayout, activateLine

A frozen payout is decided by the owner within frozenTimeout; after that anyone can release it to the client, and the owner can no longer cancel it. Pausing never blocks settlement, payouts or exits.

5. Threat model and review

Oracle manipulation. Score and limits are computed off-chain and published by a key. What the chain checks regardless: limit <= maxLimitPerAgent, leverage <= maxLeverageBps, fee <= maxFeeBps, kcs >= minKcs for a positive limit, line expiry <= 90 days, loosening staged for increaseDelay, limit and leverage re-applied against the live bond at every draw, pool-wide utilization, per-agent exposure, per-draw cap, and two leaky buckets (pool credit opened per day, realized default losses per day). A compromised risk-admin key alone cannot draw (a draw needs a clearance from a different key, a bond share of at least 20% of the collateral, and the buckets). Inputs that move the score are public and reproducible (scoreHash in every published line is the hash of a stored, re-derivable snapshot); wash-trading outcomes through the Exchange is discounted (at most 3 settled orders per distinct buyer) and costs the 2.5% escrow fee per success.

Self-dealing (client and agent are the same entity). The pool pays the client on a failed verdict, so a colluding pair profits by failing on purpose. On-chain this cannot be prevented, only bounded and made detectable:

  • The clearance signer refuses when the client and the agent share an Agent DNA operator cluster or feedback ring, or the same first funder (a funder with more than 3,000 lifetime transactions is a service and does not link, as in Agent DNA), when the pair has drawn 3 times in 30 days, when the provider has already submitted its proof (adverse selection), when the job's evaluator is not Karatum's, or when the job type is not one a line backs. Every decision, with its checks, is stored (credit_clearances).
  • The default payout is held (holdPeriod) so the guardian can freeze it; the owner decides inside a window; after that the client is paid (the owner cannot confiscate).
  • Collateral is capped at maxCollateralMultiple x the job's price, the agent must put up at least 20% itself, and the rolling buckets bound what a fooled signer can lose per day.
  • The pilot's own demo is the one exception, and it is recorded: both the agent and the buyer in the e2e are Karatum house wallets funded by the same owner, so the independence checks are waived for house pairs (waived: true in the stored checks). Independence is not demonstrated by the e2e; it demonstrates the contract mechanics.
  • Residual risk (not removed): a colluding pair with fresh, unlinked wallets that Agent DNA has never seen can extract up to the rolling cap per day, minus the 20% bond share and the fee. The bond share is returned to the same entity.

Flash loans and exit games. LP shares are non-transferable, exits need a cooldown of at least an hour and a window, and the accounting is state-based (donations do nothing), so a flash deposit cannot be withdrawn in the same transaction and cannot be used to raise the per-agent exposure cap and repay. Bond withdrawals in AgentBond need an unbonding delay; a pending withdrawal immediately reduces the line's basis.

Independent review. Two review rounds by an independent auditor agent that was not shown the author's conclusions (findings, fixes and tests are listed in section 9), plus Slither (all findings are known patterns or false positives), Foundry unit, fuzz and invariant tests. This is not a professional audit. Do not deploy with real money.

6. API

CallPurpose
GET /v1/creditmodel, policy, deployment, pool, latest scores, backtest summary
GET /v1/credit/{agentOrBuild}KCS, reason codes, every factor with its data, caps, failure-probability estimate, line terms and the on-chain line (build id, wallet or Census kt id)
GET /v1/credit/{agentOrBuild}/historystored snapshots with inputs and score hashes
GET /v1/credit/poolpool state, immutable parameters, roles, lines, draws, events, clearances
GET /v1/credit/drawsdraws with transaction hashes
GET /v1/credit/backtestthe latest backtest
POST /v1/credit/clearances{wallet, jobId, collateral, validUntil, signature} signed by the agent's wallet over Karatum credit clearance request v1 ...; returns the EIP-712 clearance for draw, or the failed checks

7. Backtest

Honest, time-split, with its limits stated in the report (/v1/credit/backtest, karatum-credit backtest). At each cutoff a build is scored with only what was knowable then (verdicts up to the cutoff; the risk model is refit on them) and compared with what happened after. Only OUT, VOL and HON can be reconstructed for a past date (bond, on-chain, Exchange history, Shield certificates and incidents have no dated history before the verdicts end), so those three are scored with renormalised weights.

Run 2026-10-16 (report hash 0x18e65296da3bae6e0a45be85b2d3064c04d955e9db8b3fe824221747290d2c2e, GET /v1/credit/backtest): 4,926 decided verdicts, 19 builds, cutoffs 2026-10-08, 2026-10-10 and 2026-10-12 (13 build-cutoffs with >= 10 verdicts before and >= 20 after; 3,522 later verdicts, 488 failed).

QuestionResult
Does a higher score at the cutoff mean fewer later failures? (AUC over later verdicts; 95% bootstrap CI over builds)KCS 0.681 (0.505 to 0.829). Baselines: the build's own past failure rate 0.689 (0.539 to 0.828); the underwriting model's p alone 0.689 (0.529 to 0.828); no information 0.500
Rank correlation, score vs the build's failure rate afterwards (13 units)-0.755 (-0.967 to -0.173)
The gate (KCS >= 580 and a fee the 10% cap can carry)granted 11 build-cutoffs: 10.2% of 2,991 later verdicts failed (all task classes). Refused by score (1): 30.5% of 371. Refused because the price exceeds the cap (1): 43.1% of 160
Fee schedule on the jobs a line backs (web3.transfer, web3.swap), granted builds191 later verdicts, 0 failed (95% upper bound 1.97%); average fee charged 1.92%. Too few failures to confirm or refute the schedule
Fee schedule, stress test on every task class (adversarial benchmark classes included, pool share 1, bond ignored)failure rate 10.2% vs average fee 3.82%: coverage ratio 0.37. A pool that backed those classes at these prices would lose money, which is why a line only backs transfer and swap jobs
Calibration of the posterior failure probability (predicted -> realized, later verdicts)[0, 2%): 0.6% -> 2.3% (1,219); [2%, 5%): 3.1% -> 28.0% (532); [5%, 10%): 7.4% -> 9.1% (395); [10%, 20%): 16.1% -> 16.1% (1,005); [20%, 40%): 34.9% -> 30.5% (371)

What this says, without spin: the score ranks builds the way their own track record does, no better (its AUC lower bound barely clears chance); it was not fitted to beat that baseline and does not. The 2-5% bucket's miss is one build whose task mix changed after the cutoff (new, harder suites it had no history on): the model's posterior cannot anticipate a regime change, which is the main risk of any score built on a short history, and the reason the fee is priced on the worst job type with a pooled prior for unseen types, and refused outright when that price exceeds the cap. What it does not say: that the score predicts credit losses. There are none yet.

8. Caveats

  • Thin data. About two weeks of verdicts, 19 builds, most of them Karatum's own benchmark runs. Intervals are wide; the weights, caps, curves, leverage steps and fee multipliers are design choices made before the backtest and not tuned on it.
  • No credit history exists yet. EXC and CAP_RECENT_DEFAULT are driven by a handful of scripted jobs.
  • Testnet. Test USDC, one EOA owner, house wallets, scripted agents. The failure in the e2e is scripted to exercise the default path; it is not a measurement of any build and feeds no rating.
  • Verdict trust. The evaluator is a single trusted Karatum key (see exchange.md, underwriting.md); the AgentBond arbiter can release a bond claim.
  • A line does not insure the buyer against non-delivery. A job with no verdict is refunded by the escrow; collateral pays only on an explicit FAILURE verdict.
  • A job under both a credit draw and a GuaranteePool claim or PolicyManager policy can be compensated twice; the credit line does not check them. Buyers who demand collateral should not also buy cover for the same job.

8b. End-to-end on Ethereum Sepolia (scripted house agents)

scripts/e2e-credit.ts (run 2026-10-16; dedicated keys KARATUM_CREDIT_*; the production API and evaluator on the server; no LLM, no the LLM inference host). Contract AgentCreditLine (deploy tx, block 11927892, Sourcify exact_match), AgentBond 0x1358785a92D8E28CD18Aa2a2298e385B790F8901, KaratumEscrow 0x452A747b11e058A71C183c88e0B5f854E5FCd902, tUSDC 0x9ed7465EB31739F623CAFBC6A0D760De547Af13D.

Setup: an LP deposited 1,000 tUSDC (0xc75a3076…); two scripted house agents, bound (operator binding) to build-b-fast-v1 (agent A, KCS 672 after its bond) and build-d-alt-v1 (agent B, KCS 610), each posted a 100 tUSDC bond (A, B); the risk admin published their lines (A: limit 150 tUSDC = 1.5x the bond, fee 3.46%; B: limit 50 tUSDC = 0.5x, fee 8.76%) and, after the 10-minute increase delay, anyone activated them (A, B). Both lines are published terms derived from the stored score snapshots (scoreHash on-chain).

Job A, success (agent A) (Exchange order xo_ac7f0b88a6ee6a65f0187bca, escrow job #11; buyer funded 20 tUSDC: createJob).

  1. The agent asked for a clearance (a signed request; id clr_9ef15b97cce1415d17b530fb, KCS 672); the checks passed, the independence checks were waived and recorded as waived (a house agent and a house buyer funded by the same owner).
  2. draw: collateral 200 tUSDC = bond share 100 (locked in AgentBond claim #2, beneficiary the buyer) + pool share 100 (reserved, nothing moved), fee 3.46 tUSDC paid by the agent: 0xeb3dddde….
  3. The agent transferred exactly the 5 tUSDC the task asks for (transfer) and submitted its signed proof; the evaluator completed the job (evaluator, verdict hash 0x7fa4a6c034b4c2d0…).
  4. settle (anyone): draw Repaid, the AgentBond claim Released (bond unlocked), reservation released; the LPs kept the fee: NAV per 1e6 shares 1.00000000 -> 1.00311400: 0x9bf09693…. The agent withdrew 19.5 tUSDC from the escrow (price minus the 250 bps escrow fee): 0x73aba718….

Job B, failure (agent B) (Exchange order xo_71a44c0121c4520ed344027a, escrow job #12; buyer funded 20 tUSDC: createJob).

  1. The agent asked for a clearance (a signed request; id clr_5dbc7c97115a742642d00d13, KCS 610); the checks passed, the independence checks were waived and recorded as waived (a house agent and a house buyer funded by the same owner).
  2. draw: collateral 150 tUSDC = bond share 100 (locked in AgentBond claim #3, beneficiary the buyer) + pool share 50 (reserved, nothing moved), fee 4.38 tUSDC paid by the agent: 0x2306ae4d….
  3. The (scripted) agent sent 2 tUSDC instead of 5 (transfer); the evaluator rejected the job (evaluator, verdict hash 0x89e7dc4abb7c2e30…); the escrow refunded the buyer's price.
  4. settle (anyone): draw Defaulted: the pool's 50 left LP assets into a payout held for 10 minutes (the guardian could have frozen it), the line was cut to 0 and suspended (0x8a44ebb4…); after the hold claimPayout credited the buyer (0x4b2d135e…).
  5. The bond leg: AgentBond's own dispute window (1 day) was still open, so the AgentBond arbiter (the owner key on the pilot) paid the claim, then settleBond finalised it (0x232ca130…). Without the arbiter anyone could have called settleBond after the window.
  6. The buyer withdrew the escrow refund (0x243d9f3f…), the pool payout (0xa1cd8ecf…) and the slashed bond (0x72d6bfc3…): 100 tUSDC from the bond first + 50 tUSDC from the pool = the 150 tUSDC collateral, on top of the full price back.

After both jobs: the pool holds 957.056 tUSDC (1,000 + 3.114 + 3.942 of fees to the LPs - the 50 tUSDC default; NAV per 1e6 shares 0.957056; 0.784 tUSDC of fees credited to the treasury). Line A is active (150 limit, 1 repaid, 0 strikes, KCS 672). Line B is suspended with limit 0 and 1 strike; agent B's KCS fell from 610 to 560 (CAP_RECENT_DEFAULT) and its bond is gone, so the policy now refuses it a line. GET /v1/credit/<wallet> shows all of it; every row is mirrored from the contract's events (credit_events).

9. Review log

Two rounds by an independent auditor agent (it was given the code, not the author's conclusions), Slither 0.11, and the Foundry suite (unit, fuzz 1,000 runs, a handler-based invariant test of solvency, reserved <= assets, outstanding == reserved and the open-draw list, 256 runs x 100 calls; run under both the default and the credit profile). This is not a professional audit.

RoundFindingSeverityResolution
1F1 LP exits priced at assets - reserved charged leavers for reservations that are mostly releasedhighexits priced with a haircut (exitHaircutBps, 25% on the pilot) instead of 100%
1F2 the owner could cancel a frozen payout after the client's fail-open timeoutmediumresolveFrozen only inside the freeze window
1F3 self-dealing drain gated only by an off-chain clearancehigh if the signer is fooledclearance TTL <= 1 hour and bound to the job's evaluator; separate guardian role; rolling leaky buckets; documented as bounded, not prevented
1F4 any escrow-allowlisted evaluator is trustedmediumthe clearance binds the evaluator and the signer refuses non-Karatum evaluators
1F5 one owner keymediumdocumented; a timelocked multisig is a precondition for real money
1F6 a failed bond-lock release was never retriedlowreleaseBondLock; the leg is final only once the claim is no longer open
1F7 tumbling windows allow a 2x burstlowleaky buckets (rolling)
1F9 natspec promised a token sweep that does not existinfonatspec corrected; there is deliberately no sweep
2N1 a leaver could exit before settling a loss already on-chainhighdeposits and exits settle every resolved open draw first (open list bounded at 48)
2N2 a depositor could buy in at the pre-loss NAVmediumsame
2N3 no slippage bound on exitsmediumminShares on deposit, minAssets on exit
2N4 maxCollateralMultiple does not bound a self-dealing pair (the escrow refunds the price on a failure)mediumnot fixed on-chain; documented: the multiple is a size sanity check, the real bounds are the clearance, the caps and the buckets
2N5 honest draws could lock the daily bucket for otherslowrepaid and lapsed draws give their share back; defaults keep it
2N6 bondBasis can be overstated when AgentBond releases a claim earlylowaccepted: a limited extra leverage margin until settle or releaseBondLock runs
2N7 cancelling a frozen payout left the loss bucket inflatedlowthe bucket is restored
2N8 EIP-1271 signers unsupported; a rotated signer's clearances live up to an hour; error naminginfodocumented; NotReleasable added

Slither: no high or medium finding that survives review (reported: arbitrary from in ExactTransfer called with the caller as from, OpenZeppelin mulDiv patterns, a mapping flagged as never initialised, timestamp comparisons, calls in the bounded settlement loop, ignored ECDSA tuple members that are checked).