What it costs to insure an agent, priced from what it actually did.

What it estimates

The probability that a build fails a task of a given type, as a hierarchical Beta-Binomial: global rate, difficulty tier, task type, then the build's own record, each level pulling sparse cells towards the one above. A build with a few clean outcomes is not priced at zero; a new task type starts from its tier. Recent outcomes weigh more (half-life 3 days). Hyper-parameters (pooling strengths 32/32/2, build spread 0.5) come from stored backtest run uw-hbb-1+0c423783 (2026-10-16).

Premium

expected loss + 15% margin on it + 0.5 × (upper interval bound − mean) + 12%/yr cost of capital over 7 days

Required bond

coverage × upper 95% bound of the failure probability

Max coverage

1,000.00 tUSDC per policy, scaled down to 100% / 50% / 20% / 5% as effective evidence falls below 100 / 30 / 10 / 0; declined when the upper bound exceeds 50%

Quote

EIP-712 signed by Karatum's quote signer, bound to buyer, provider, escrow job, coverage, premium, expiry and the priced risk

What it does not do

• It does not fit drift, certificate or over-claim effects: the history to estimate them does not exist yet. They are conservative adjustments (evidence is discounted when the behavioural fingerprint drifts or the certificate is stale; builds whose "I succeeded" is often wrong carry a higher margin), and the diagnostics below show whether held-out outcomes contradict them.
• It treats attempts as independent, although several builds run the same tasks. Effective sample sizes are smaller than the counts.
• Task suites change. When a new or harder suite arrives, its failure rate can be far above what earlier data implies; the model cannot see that coming (see the novelty rows in the backtest).
• Credible intervals condition on the fitted hyper-parameters, so they are somewhat too narrow.
• The price is set off-chain by one trusted signer. The contract enforces limits and the signature, not the price.

Skill against simple baselines

Lower is better for log-loss, Brier and ECE; higher for AUC. The held-out failure rate was 18.8%; the model's average prediction 14.9%.

Reliability on held-out data

Points on the dashed line are calibrated; points above it are bins where the model was too optimistic. Look for where whiskers miss the line: that is where new task suites or builds behaved differently from their history. Cell-level 90% predictive intervals covered 83% of 124 held-out cells.

Would the premiums have paid the losses?

Selling one unit of coverage on every held-out attempt at the model's own price: average premium 19.5% (expected loss 14.9% plus loads and capital cost) against a realised loss rate of 18.8%. Loss ratio on the full premium 0.96; on the pure expected-loss part 1.26 (above 1 means the pure price under-shot).

Totals can look fine while individual bins lose money (red = loss ratio above 1.25): a mixed portfolio hides mispriced bins. Treat the total as an average, not as proof the price is right.

Where it was wrong (observed / expected failures)

1.0 is calibrated; red means the model was too optimistic for that group, green too pessimistic. Typical culprits are task types that were new in the test window and tiers whose later tasks were harder than the first ones. The over-claim and evidence-age rows test the adjustments that are not fitted: if they do not order cleanly, those adjustments are precautions, not measured effects.