Incident Registry and Global Recall

Version incidents-v1 · severity scale KAS 1.0 · code: packages/incidents, services/incidents, services/api/src/routes/incidents.ts · contract: contracts/src/incidents/KaratumIncidentLog.sol · tables: migration 026_incidents.sql · pages: /incidents, /incidents/KAR-YYYY-NNNN

The Incident Registry is a public, CVE-style list of AI-agent failures. Every incident has a stable id (KAR-2026-0001), a class, a severity on a documented scale, evidence anyone can check, a status lifecycle, a disclosure timeline and the operator's response. A confirmed incident of severity high or critical triggers a recall: a coordinated set of on-chain and off-chain actions that make every consumer of Karatum's trust signals stop trusting the agent, measured in seconds from the maintainer's confirmation to the block that carries the first recall transaction.

Testnets only (Base Sepolia, Ethereum Sepolia). Nothing here moves funds. Where something below is a rehearsal rather than a real incident, it says so.

It does not replace the older incidents table (migration 008, ids inc_…, derived per organisation by services/enterprise). GET /v1/incidents/inc_… keeps answering exactly as before; KAR-… ids answer from the registry.

#1. The incident model

#2. Severity: KAS 1.0

KAS (Karatum Agent Severity) is CVSS v3.1 with the metrics redefined for autonomous agents. The arithmetic is exactly CVSS v3.1 (base and temporal equations, round-up to one decimal), so the numbers are comparable with what security teams already read. The test suite checks reference vectors from the FIRST.org specification (AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H = 9.8, the scope-changed maximum = 10.0, …).

Vector: KAS:1.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N/E:P

Bands (CVSS): 0.1 to 3.9 low, 4.0 to 6.9 medium, 7.0 to 8.9 high, 9.0 to 10.0 critical. A score of 0 is reported as low. High and critical trigger a recall.

Two floors a score cannot argue away: verified loss of real funds on a public mainnet that is still ongoing is critical (floor 9.0); verified loss that has stopped is at least high (floor 7.0). Forks and testnets never count as loss.

#Defaults per class (the starting point; the maintainer adjusts E and the rest to the evidence)

A confirmed fund_loss on a public mainnet is lifted to critical by the loss floor below, whatever the vector says. Two classes the defaults rank as medium (over_claim, scam_interaction) are never recalled automatically; a maintainer can still rescore them or override the threshold.

A public report gets the class default with E:U (or E:P when a reference resolves in our records). The severity is a starting point: a maintainer rescoring it writes a public timeline entry with the old and new vector and the reason.

#3. Status lifecycle

reported -> triaged -> confirmed -> recalled -> fixed
    |          |           \-------------------> fixed
    \-> rejected <-/            (rejected can be reopened to triaged)

• reported: private. Created by the form, by an operator, or by a heuristic detector.
• triaged: a maintainer (or a deterministic detector, see section 4) accepted that the report is substantiated enough to be public. Visibility and the disclosure clock are set here.
• confirmed: a maintainer verified the finding. Always a human decision; no detector confirms by itself. confirmed_at starts the recall clock.
• recalled: the recall ran and no step failed (a failed step leaves the incident confirmed and visible, and the command is simply run again; completed steps are not repeated).
• fixed: the operator fixed it and a maintainer verified it. Lifts the recall (section 6).
• rejected: not substantiated. Stays visible if it had been public.

Only the transitions above are allowed (enforced in code and tests). An operator can say "fixed" on the timeline; only a maintainer moves the status.

#4. Where incidents come from

Automatic detectors (packages/incidents/src/intake.ts, run by karatum-incidents run every 10 minutes for the deterministic ones). Idempotent: a finding has a dedupe key and opens at most one incident.

Deterministic detectors start triaged because their evidence is cryptographic or reproducible; heuristics start reported and wait for a maintainer. The heuristics run on demand (karatum-incidents scan --only drift,backtest), not on the timer.

Public report form (POST /v1/incidents/reports, page /incidents). Spam controls, none of which needs an account:

• evidence is mandatory (1 to 8 typed, format-checked references); URLs must be https without credentials; unknown fields are rejected; a honeypot field must stay empty;
• 5 reports per day per caller, 200 per day overall, counted before the body is parsed (garbage costs quota); callers are stored only as a salted hash;
• duplicates (same class, subject and evidence) are folded into the existing report as a private corroboration;
• a report is private: nothing becomes public, appears in the feed or triggers a webhook until a maintainer triages it. This is the main control: abuse of the form cannot publish anything about anyone;
• evidence that resolves in Karatum's records (a verdict id, a Shield attempt, a flight step, an attestation uid …) is marked verified and raises the exploit-maturity default.

Operator self-reports. A report filed with an API key that owns the named agent is an operator incident: disclosed immediately by the operator itself. The same key can answer any incident about its agents (POST /v1/incidents/:id/response: acknowledged, fix_claimed, dispute, statement); the answer lands on the public timeline.

#5. Disclosure policy

• 30-day coordinated disclosure for operators. When an undisclosed vulnerability in a third party's agent is reported, the operator is notified (karatum-incidents notify) and has 30 days from that notification (from triage when nobody could be notified) before the details are published. During the embargo the registry shows the class, severity, status and dates, and withholds the title, summary, subject and evidence; the signed feed and webhooks do not carry the incident at all.
• Immediate disclosure when any of these holds: verified active fund loss (users are still being hurt, so waiting helps no one); the affected build is run by Karatum itself; the operator filed it; or the evidence consists only of public records (on-chain transactions, attestations, signed flight records, our own published certificates and verdicts). A fact that is already public is not an embargoed vulnerability.
• A recall is never embargoed. Revoking a certificate on-chain is visible to everyone by definition, so a recalled incident is public at the moment of the recall. What stays embargoed is only what the maintainers have not yet had to reveal.
• An embargo ends at the 30-day mark (a sweep job publishes it, with a timeline entry) or earlier when the operator's fix has been verified.
• Reporter contact details are never published, never in the feed, never in a response.

#6. Global Recall

A recall is karatum-incidents recall <id> (or confirm <id> --recall). Preconditions: status confirmed, severity high or critical (a maintainer can override for a lower one). The steps run in parallel; each outcome is stored with its transaction, block number and block time the moment it is known; nothing is hidden when a step is skipped, the reason is recorded.

The Guard cannot be re-instated by the daily publication. guardInputs excludes every wallet and build of a recalled incident, so the next root does not carry the agent. Revocation in the registry only yields to a root published after it, and the publisher no longer publishes it. Fixing the incident re-includes the agent and forces a new root.

Fix and lift. fix <id> <note>: status fixed; KaratumIncidentLog.setStatus(fixed, wallets) lifts the recall of exactly the wallets this incident recalled (the list must name every wallet it still recalls, or the call reverts with RecallsRemaining); the Guard publisher publishes a new root on every chain (forced, even if the leaf set is unchanged) and proves watched agents; ERC-8004 feedback is revoked. Every step is on the incident page.

Several incidents, one wallet (contract rules since 2026-10-17). KaratumIncidentLog tracks recalls per (wallet, incident). A wallet named by several incidents stays recalled until every one of them is fixed or rejected, so a later incident, and a drill in particular, can neither overwrite nor clear an earlier real recall. isRecalled(wallet) returns the headline recall: the earliest still-active real recall, else the earliest active drill; activeRecalls(wallet) lists them all (at most 8 at a time). Status changes on-chain only move forward (reported → triaged → confirmed → rejected / fixed / recalled), except that a recalled incident moves to rejected or fixed, which lifts its recalls. Status recalled always has wallets: publish with wallets sets it, and setStatus(recalled) needs a non-empty wallet list and severity high or critical; any other status change takes no wallets. The owner cannot renounce ownership.

Idempotent and resumable. A failed step is recorded with its transaction hash when there is one; a re-run does not repeat completed steps and recognises a publication that was mined but not confirmed to the process (it finds the transaction in the logs instead of sending a second one).

#What is not one transaction

The deployed KaratumCertRegistry has a single attester and a per-certificate revokeCert, so the registry revocation and the incident announcement are two transactions per chain (sent in the same second, mined in the same block on Base Sepolia). A single transaction per chain would need a registry that lets a second contract revoke; that is a contract change this module does not make. The announcement itself, which is what contracts and wallets subscribe to, is one transaction for the incident and all its wallets.

#Measured latency

Confirmation time (confirmed_at, the server clock) to the block timestamp (chain clock, 1 s resolution) of the first and last recall transaction. Each number includes about a second of clock skew between the two clocks and cannot be below the block time (2 s on Base Sepolia, 12 s on Ethereum Sepolia). The incident page shows the same numbers computed from the stored actions.

KAR-2026-0003 is kept as it happened: two Ethereum Sepolia transactions ran out of gas because the recall had hard-coded gas limits (a revokeCert costs about 32k gas on Base Sepolia and about 131k on Ethereum Sepolia, which prices state creation about four times higher). The limits were removed and the incident resumed; KAR-2026-0004 is the clean run.

#7. Recall drills

A drill rehearses the mechanism on a real certificate and a real Safe without pretending anything was compromised. It is flagged on the incident, in the on-chain IncidentLog entry, in the feed (incident.drill; drills are left out of the feed unless ?drills=1, and never produce webhooks, so no consumer enforces on a rehearsal), and on every page.

services/incidents/src/drill.ts runs, against the Guard demo Safe on Base Sepolia and the house build build-a-careful-v1:

1. the Safe pays an allow-listed payee while the certificate is VALID (tx);
2. an operator self-report is opened, triaged, confirmed and recalled in the same process;
3. the same Safe, the same payee: the guard refuses (CertificateNotValid(REVOKED)), and the blocked transaction is on-chain;
4. the incident is fixed: IncidentLog lift, new Guard root, agent proven again;
5. the Safe pays again (tx).

Every transaction link is a public note on the incident's timeline. A drill never touches real EAS attestations or real ERC-8004 reputation. Two flags exercise those paths safely: --eas creates a throwaway attestation on its own schema and the recall revokes it with multiRevoke; --erc8004 publishes the incident feedback on the house build's identity and revokes it again at fix. KAR-2026-0005 ran both.

#8. The signed recall feed

GET /v1/incidents/feed?since=<cursor>&limit=<n> returns confirmed, recalled, fixed, rejected and published incidents (public ones only), oldest first; ?format=atom or Accept: application/atom+xml renders Atom 1.0. cursor in the answer is what to pass as since next time.

Each entry is an immutable snapshot (seq, event, the public incident, the recall summary with its transactions and measured latency) with a signature:

digest    = keccak256(utf8(canonicalJson(entry without "signature")))     // canonicalJson: keys sorted at every depth, no whitespace
signature = EIP-191 personal_sign of the 32-byte digest by the feed signer

The feed signer is returned in the answer (signer); it is an off-chain key that holds no funds (0xa7F04082E2e2dB1d015122525617701488794DeA today). A consumer verifies each entry itself, so a mirror or proxy cannot forge or alter a recall:

import { verifyEntry } from "@karatum/incidents"; // or: recoverMessageAddress({ message: { raw: digest }, signature })
const ok = await verifyEntry(entry, feedSigner);

In the Atom rendering each <entry> carries the same canonical JSON as its <content type="application/json"> and the digest, signature and signer as karatum:* elements, so it can be verified from the Atom document alone. Without a configured key the entries are served with signature: null (never forged).

#9. Webhooks

POST /v1/incidents/webhooks (API key) with { url, minSeverity?, events?, label? }. The secret is returned once. Every delivery is a POST carrying:

X-Karatum-Event      incident.recalled | incident.confirmed | incident.fixed | incident.rejected | incident.published
X-Karatum-Delivery   dlv_<hex>, stable across retries: de-duplicate on it
X-Karatum-Timestamp  unix seconds
X-Karatum-Signature  t=<unix>,v1=<hex HMAC-SHA256(secret, "<t>.<raw body>")>

Reject a delivery whose timestamp is more than 5 minutes old (verifyWebhook in @karatum/incidents does both checks). The body is { id, event, createdAt, entry } with the signed feed entry. A 2xx is success; anything else is retried after 1 min, 5 min, 30 min, 2 h, 6 h, 24 h (7 attempts, about 33 hours); a webhook failing 50 times is deactivated.

Destination safety: https only, port 443, no credentials, the host must be a public DNS name and every address it resolves to must be public (loopback, private, link-local, CGNAT, metadata, multicast and reserved ranges are refused), checked when the webhook is created and again before every delivery; redirects are never followed; 5 s timeout. DNS can change between the check and the connection; deliveries are signed payloads of public data, so the exposure is a blind request, not a leak. Maximum 5 active webhooks per key.

#10. On-chain: KaratumIncidentLog

Base Sepolia 0x9E4857c8cc0dB3AE5B0E26D038B3Bf5f5e9AE8e5 (block 48231068) · Ethereum Sepolia 0xF879fD5e560eA0E9c7d4D61DBf897Cfbe790Da18 (block 11933398) · owner (Ownable2Step) the testnet owner, reporter the dedicated incident key 0x6358B5e36988f5f48196796161FFD6876a9db508 · source verified on Sourcify.

event IncidentPublished(bytes32 indexed incidentId, string id, uint8 indexed severity, uint8 status, uint8 class_, bool drill, bytes32 evidenceHash, string uri);
event AgentRecalled(bytes32 indexed incidentId, address indexed agent, uint8 indexed severity, bytes32 subject);
event RecallLifted(bytes32 indexed incidentId, address indexed agent);
event IncidentStatusChanged(bytes32 indexed incidentId, uint8 indexed status);
function isRecalled(address agent) external view returns (bool recalled, bytes32 incidentId, uint64 since);

incidentId = keccak256(bytes("KAR-2026-0001")). A contract that wants to refuse a recalled agent directly reads isRecalled(agent) (one storage read); everything else subscribes to the events. Properties (10 Foundry tests including a fuzz): an id is published once; a recall needs severity high or critical and at most 32 wallets; only the reporter writes; a later incident's recall of a wallet replaces an earlier one and the earlier incident cannot lift it; fixed and rejected lift only the recalls the incident itself made. The log is an announcement channel, not enforcement: the Guard enforces through the registry.

#11. API

Maintainer actions (triage, rescore, confirm, recall, fix, notify) are not in the API: they run through karatum-incidents on the server, next to the database and the signing keys. The Explorer pages /incidents and /incidents/KAR-… show the registry and each incident.

#12. What this does not claim

• Confirmation is a human judgement. A recall is as good as the maintainers' decision to confirm; the evidence, the scoring vector and every status change are public so the decision can be checked, not so that it needs no judgement.
• A recall acts on what Karatum controls. It revokes Karatum's own signals (Guard certificates, attestations, reputation feedback) and announces the incident. It cannot stop an agent, and a consumer that ignores the Guard, the log and the feed is not protected.
• A certificate in the database is not rewritten. The K-rating certificate row keeps its computed status; the recall is an overlay (Guard exclusion, feed, log). A recomputed certificate of a recalled build is still excluded from new Guard roots.
• Severity is a model. KAS adopts the CVSS arithmetic, not CVSS's calibration against real exploit data; the class defaults are a documented starting point.
• Detectors find what our modules measure: Shield suites, signed flight records, fingerprints, Backtest flags, certificates. They say nothing about failures outside those.
• Everything is testnet. The Shield findings come from disposable forks; the drills use a house build and test ETH. Mainnet is not reachable from this service.