Compliance Evidence Pack

If you deploy AI agents and an auditor, a customer or a regulator asks what you can show, a pack turns Karatum's measurements of your agent into one document: test coverage and results with confidence intervals, robustness against hostile input, behavioural drift, logging coverage, incident history with response times, and oversight controls. It is mapped control by control to the EU AI Act, the NIST AI RMF and ISO/IEC 42001 Annex A, and for each control it says what the evidence does not show and what you still have to supply. The pack is generated deterministically: the same inputs give the same JSON, PDF and manifest, and the manifest hash is attested on Ethereum Sepolia.

Evidence, not an opinion. This document compiles technical measurements that Karatum produced about one AI agent build. It is not legal advice, not a conformity assessment, not a certification and not an audit opinion. It does not say that any law, regulation or standard is met. Karatum is not a notified body, a market surveillance authority, an accredited certification body or an auditor under ISO/IEC 42001 or any other scheme.

Sample pack

Agent A · Careful build-a-careful-v1

A Karatum house build: Karatum is also its operator, so this sample shows format and content, not an independent assessment. Evidence as of 2026-10-16 13:38:14 UTC.

Trial results

98/102 decided attempts succeeded

95% CI 90.3% to 98.5%

Certificate

VALID, 22K

Shield: not run for this build

Control coverage

18 of 47 with primary evidence

5 evidence gaps listed, 65 operator items

Download PDF pack.json manifest.json control-map.json

Pack id	cp_8e4a92c619da9098
Content hash (keccak256)	0x36cbe49a847e4b3650b267bab67e4b8335b8c989335eb2001f7e7ff9d8299bca
PDF hash	0x1219417965876c5d6cbe46703269125ad281b428b0fdba9dee560918febd171d
Manifest hash (attested)	0x9492130e4234b9fdae165e3ca69f2e3cea38d6c6ba6d98d9c3070e842b907141
Attestation	EAS on Ethereum Sepolia: 0xb204a072…2b9dbf73

Check it yourself: hash the canonical pack.json with keccak256 and compare with the content hash; the manifest lists the hashes of every file and of the upstream attestations. The attestation proves that exactly this content existed and was committed by Karatum's attester key; it says nothing about whether the content is right.

What the sample contains

Evidence section	State	Result
Test coverage and results (trial verdicts)	evidenced	113 attempts, 102 decided; success 98/102 = 96.1% (95% CI 90.3% to 98.5%); evidence class E3 or E4 for 113 of 113.
Certified performance (K-rating)	evidenced	web3: VALID, 22K, lower bound 90.3%, n=102.
Verifiable judge provenance	absent	No judge traces, provenance attestations or re-verification runs exist for this build.
Honesty Index (accuracy of the agent's own reports)	partial	Not ranked (77 scored claims, 100 required).
Robustness against adversarial input (Shield)	absent	No Shield certificate exists for this build.
Accuracy drift (behavioural fingerprints)	evidenced	6 fingerprints over 2 probe set(s); latest drift web3-canary-v2 0.0%, web3-canary-v3 0.0% (policy threshold 15%).
Logging (Flight Recorder coverage)	absent	No Flight Recorder batches exist for any wallet bound to this build.
Incident history and response times	evidenced	0 incident(s) (0 open) and 3 recall drill(s). Median time from report to triage n/a, to confirmation n/a; median recall latency after confirmation 84 s.
Oversight controls (Guard policy)	evidenced	1 certificate leaf(ves) in the latest Guard root (web3: tier 3); published to 2 chain(s); 1 watched account(s); 13 preflight check(s): 9 pass, 0 warn, 4 block.

A section that reads “absent” is an honest gap: the pack lists it, and every control that leans on it drops from “evidence attached”. Nothing is filled in to look complete.

Control matrix

47 controls, map version 1.0.0 (hash 0xb1601aaa…1b9a6a). Open a control to see the requirement in our words, the Karatum artefacts that are evidence, what they do not show and what you must supply. The EU AI Act articles shown apply to high-risk AI systems, mostly to their providers (Art. 6 and Annex III decide whether yours is one; Karatum does not classify); the NIST AI RMF is voluntary; ISO/IEC 42001 certifies an organisation's management system, never a tool. General-purpose AI model obligations are out of scope.

EU AI Act, Regulation (EU) 2024/1689

Ref	Control	Applies to	Primary Karatum evidence	In the sample pack
Art. 6 and Annex III	Is the system high-risk? Requirement (paraphrase). A system is high-risk if it is a safety component of a product under the Annex I legislation, or if it falls in an Annex III use case (for example creditworthiness assessment of natural persons, employment, essential services), unless the Art. 6(3) exemptions apply and are documented. Karatum evidence None: this control is entirely the operator's. What it does not show Karatum evidence says nothing about the intended purpose of the operator's system, which is what decides the classification. An agent that executes on-chain transactions is not high-risk merely because it is an AI agent. The operator must supply Document the intended purpose and the classification with its reasoning (including any Art. 6(3) exemption and its registration duty). Re-assess when the intended purpose or deployment context changes.	High-risk systems only: providers and deployers	none	Operator evidence only
Art. 9	Risk management system Requirement (paraphrase). A continuous, iterative risk management process across the life cycle: identify and analyse known and foreseeable risks to health, safety and fundamental rights (including foreseeable misuse), estimate and evaluate them, adopt suitable measures, and test the system against prior-defined metrics and probabilistic thresholds, including under real-world conditions where appropriate. Karatum evidence Primary: Trial verdicts. Repeatable test results with outcome verified from chain state or test execution, giving measured success rates over a stated sample (supports the testing step, Art. 9(6) to (8)). Primary: K-rating certificate. A probabilistic threshold made explicit: the lower 95% confidence bound of the success rate, with sample size, evidence class and expiry. Primary: Shield certificate. Testing against foreseeable adversarial inputs (the injection attack classes of the Shield suite), reported by class. Supporting: Honesty Index. Measures one specific risk: the agent reporting success when it did not succeed. Supporting: Behavioural fingerprints. Detects change in behaviour after a model, prompt or tool update, a trigger for re-running the risk assessment. Supporting: Incident Registry. Post-deployment incidents on the build feed the risk evaluation required by Art. 9(2)(c). Supporting: Karatum Guard. An on-chain gate and tier caps are a risk-control measure for transactions the agent initiates. What it does not show Karatum tests fixed task suites (on-chain web3 tasks and software tasks). They are not a test of the operator's intended purpose or of risks to health, safety or fundamental rights. Shield covers the attack classes of its suite version only; it is not a complete adversarial assessment. A measured success rate is not a statement that residual risk is acceptable (Art. 9(5)); that judgement is the provider's. The operator must supply Identify and analyse the risks of the intended purpose and of reasonably foreseeable misuse, including impact on persons under 18 and other vulnerable groups where relevant (Art. 9(9)). Define the metrics and thresholds appropriate to the intended purpose and show how the Karatum numbers relate to them. Document the risk management process, the residual risk and the decision that it is acceptable; review it on a schedule.	High-risk systems only: providers	Trial verdicts, K-rating certificate, Shield certificate	Evidence attached
Art. 11 and Annex IV	Technical documentation (testing part) Requirement (paraphrase). Technical documentation drawn up before placing on the market and kept up to date, including a description of the validation and testing procedures with test logs and reports dated and signed by the responsible persons (Annex IV, point 2(g)). Karatum evidence Supporting: Trial verdicts. Test logs of Karatum trials: task, evidence, verdict hash and attestation per attempt. Supporting: Verifiable judge provenance. Reproducible description of how each verdict was decided: trace, evaluator digest, source commit. Supporting: K-rating certificate. Summary of the results with confidence intervals. Supporting: Shield certificate. Robustness testing report. What it does not show Only the validation and testing material of Annex IV point 2(g) can draw on Karatum. System description, design, data, training and monitoring parts of the documentation are untouched. The operator must supply Everything else in Annex IV: general description, development process, data and training methodologies, monitoring and control, performance metrics rationale, risk management description, changes over the life cycle, declaration of conformity. Dated and signed test reports by the responsible persons.	High-risk systems only: providers	none	Partial evidence
Art. 12 (and Arts. 19, 26(6))	Record-keeping: automatic event logging Requirement (paraphrase). High-risk systems must technically allow automatic recording of events over their lifetime, to a degree that enables identifying situations that may present a risk or a substantial modification, supports post-market monitoring (Art. 72) and deployers' monitoring of operation (Art. 26(5)). Providers keep the logs under their control (Art. 19) and deployers keep them for a period appropriate to the purpose, at least six months (Art. 26(6)). Karatum evidence Primary: Flight Recorder. A tamper-evident record of the agent's steps: signed, hash-chained batches with gap and equivocation detection, optionally anchored on-chain. Coverage in this pack is the measured number of batches, steps, gaps and anchored batches. Supporting: Trial verdicts. Per-attempt records of the trials the agent ran, each with a verdict hash. Supporting: Verifiable judge provenance. The judge's complete input for each verdict is stored (trace), so the evaluation itself is reconstructible. Supporting: Karatum Guard. Preflight checks of transactions the agent proposed are logged with their verdicts. What it does not show The Flight Recorder records what the operator's agent chooses to emit through the SDK or proxy. Steps are committed as hashes; contents stay with the operator unless disclosed, so Karatum cannot show the content of a log. Karatum does not hold the operator's logs and cannot attest that the event set is sufficient for the intended purpose. The operator must supply Define which events the system must log for its intended purpose (Art. 12(2)) and show that the recorded set covers them. Set and document retention (at least six months for deployers, longer where other law requires) and access control, and handle personal data in logs under data-protection law. Keep the underlying log contents, not only their commitments.	High-risk systems only: providers and deployers	Flight Recorder	Partial evidence
Art. 13	Transparency and information to deployers Requirement (paraphrase). The system must be sufficiently transparent for deployers to interpret its output and use it appropriately, and be accompanied by instructions for use giving its characteristics, capabilities and limits of performance, including declared accuracy levels and metrics, known circumstances that may affect them, robustness and cybersecurity, human oversight measures and logging mechanisms. Karatum evidence Primary: K-rating certificate. A declared accuracy metric with its confidence interval and the conditions (task class, evidence class, sample, expiry) under which it was measured. Supporting: Shield certificate. Robustness figures by attack class that can be quoted in instructions for use. Supporting: Honesty Index. Tells a deployer how far the agent's own success report can be trusted. Supporting: Flight Recorder. Describes the logging mechanism that exists. What it does not show These are measurements to cite; they are not the instructions for use. Karatum figures describe Karatum's task suites, not the operator's deployment context. The operator must supply Write the instructions for use: intended purpose, performance for the target users and contexts, foreseeable misuse, input data specification, oversight measures, maintenance, expected lifetime, log access (Art. 13(3)). State which measured figures the operator adopts as its declared accuracy and why they are relevant.	High-risk systems only: providers	K-rating certificate	Evidence attached
Art. 14	Human oversight Requirement (paraphrase). The system is designed so natural persons can effectively oversee it while in use: understand its capacities and limits, monitor its operation, stay aware of automation bias, interpret output, decide not to use or to override it, and intervene or stop it. Karatum evidence Primary: Karatum Guard. A technical intervention point: an account that installs the Guard refuses the agent's transaction while its certificate is not valid or exceeds the tier caps, whatever the agent says. Supporting: Recall actions and drills. A confirmed incident revokes the agent across the Guard registries; the pack shows measured revocation latency, and labels drills as drills. Supporting: Honesty Index. Gives overseers a measured basis for how much weight the agent's own reports deserve (automation bias). Supporting: K-rating certificate. A visible, expiring status that tells an overseer whether the agent is currently in a certified state. What it does not show The Guard enforces only for accounts that have installed it, and only on the chains it is deployed to (testnets at the time of writing). Karatum does not identify, train or authorise the human overseers and does not provide an operator interface. The pack cannot read the per-account Guard settings of an operator's account; list those in the declaration. The operator must supply Assign named persons with competence, training and authority (Art. 26(2)), and document the oversight measures and the stop procedure. Provide interface tools that let overseers interpret the system's output and review its actions. Document where human approval is required before an action takes effect and how it is evidenced.	High-risk systems only: providers and deployers	Karatum Guard	Evidence attached
Art. 15	Accuracy, robustness and cybersecurity Requirement (paraphrase). An appropriate level of accuracy, robustness and cybersecurity throughout the life cycle, with accuracy metrics declared in the instructions for use; resilience to errors, faults and inconsistencies; and resilience against attempts by third parties to alter use, outputs or performance by exploiting vulnerabilities (for example data poisoning, model poisoning, adversarial examples, confidentiality attacks). Karatum evidence Primary: K-rating certificate. Accuracy as a measured success rate on verified outcomes, with a 95% confidence interval and sample size. Primary: Trial verdicts. The underlying verified outcomes, by task type and evidence class. Primary: Shield certificate. Resilience to the adversarial input classes in the Shield suite (prompt injection and related), measured as attacks resisted out of attacks delivered. Supporting: Behavioural fingerprints. Consistency of behaviour over time on a fixed probe set; drift above the policy threshold marks the certificate STALE. Supporting: Verifiable judge provenance. The accuracy measurement itself is reproducible. Supporting: Honesty Index. Reliability of the system's own self-reported outcome. What it does not show Accuracy is measured on Karatum's task suites. It does not prove accuracy for the operator's inputs. Shield does not test data or model poisoning, model extraction, confidentiality attacks or the cybersecurity of the operator's infrastructure, keys and APIs. No test shows fail-safe or fallback behaviour (Art. 15(4)). The operator must supply Declare the accuracy levels and metrics for the intended purpose in the instructions for use (Art. 15(3)). Cybersecurity of the system and its hosting: threat model, penetration testing, secrets and key management, access control, supply-chain controls. Technical redundancy, fail-safe and fallback measures; handling of feedback loops if the system keeps learning.	High-risk systems only: providers	K-rating certificate, Trial verdicts, Shield certificate	Evidence attached
Art. 26	Deployer obligations (monitoring and logs) Requirement (paraphrase). Deployers of high-risk systems use them according to the instructions, assign human oversight, monitor operation and inform the provider and authorities where they identify a risk or serious incident, and keep the automatically generated logs for at least six months. Karatum evidence Supporting: Flight Recorder. The recorded operation that deployer monitoring and log retention can build on. Supporting: Incident Registry. Incidents that name the build, with the timeline of operator notification and response. Supporting: Karatum Guard. A gate at the deployer's own account that constrains what the agent can do. What it does not show Karatum is not the deployer's monitoring function and does not hold the deployer's logs. The operator must supply Use according to the provider's instructions; assign and train oversight persons; input data relevance; inform provider and authorities; keep logs for at least six months; inform workers and affected persons where Art. 26(7) and (11) require it.	High-risk systems only: deployers	none	Partial evidence
Art. 72	Post-market monitoring Requirement (paraphrase). Providers establish a post-market monitoring system proportionate to the system, which actively and systematically collects, documents and analyses relevant data on performance throughout its life, to evaluate continuous compliance, based on a post-market monitoring plan that is part of the technical documentation. Karatum evidence Primary: Behavioural fingerprints. Continuous behavioural drift measurement against a baseline, with the history of measurements. Primary: K-rating certificate. A certificate that is recomputed on a schedule, expires and changes status when evidence goes stale. Supporting: Incident Registry. Registered failures with detection source, severity and timeline. Supporting: Flight Recorder. Operational data the operator can analyse for performance over time. Supporting: Honesty Index. Recomputed periodically; monthly editions are immutable. Supporting: Trial verdicts. The ongoing trial history, with first and last attempt dates in the pack. What it does not show Karatum monitors its own scheduled trials and probes. Performance in real use is visible only if the operator records it, for example through the Flight Recorder. The operator must supply Write the post-market monitoring plan (the Commission template under Art. 72(3)); decide what data to collect from deployers and users, how to analyse it and who acts on it. Where relevant, analyse interaction with other AI systems.	High-risk systems only: providers	Behavioural fingerprints, K-rating certificate	Evidence attached
Art. 73 (with Arts. 3(49), 20, 26(5))	Serious incident reporting Requirement (paraphrase). Providers of high-risk systems report serious incidents to the market surveillance authorities of the Member States where they occurred, immediately after establishing a causal link or its reasonable likelihood and not later than 15 days after becoming aware (shorter periods of 10 or 2 days apply to specific kinds), investigate, and take corrective action. Karatum evidence Primary: Incident Registry. Detection, classification, severity, status and a timestamped timeline for each incident that names the build, including when the operator was notified and responded. Supporting: Recall actions and drills. Corrective action on the Karatum side: certificate revocation, with measured latency. Supporting: Flight Recorder. Records the operator can use to reconstruct what happened (traceability for the investigation). What it does not show Karatum incident classes such as fund loss are not the legal term 'serious incident' (death or serious harm to health, serious and irreversible disruption of critical infrastructure, infringement of fundamental-rights obligations, serious harm to property or the environment). Financial loss alone may not qualify. A Karatum incident entry is not a report to an authority and Karatum files none on anyone's behalf. The operator must supply Decide whether each event is a serious incident under Art. 3(49) and keep the reasoning. Run the reporting procedure to the competent authority within the deadlines (15, 10 or 2 days depending on the case, Art. 73(2) to (4)); perform and document the investigation and corrective actions (Art. 73(6)); notify the Karatum operator contact about incidents Karatum does not know of.	High-risk systems only: providers (deployers inform)	Incident Registry	Evidence attached

EU AI Act, Regulation (EU) 2024/1689. Control map version 1.0.0. Titles and requirement lines are paraphrased; use the official text.

NIST AI Risk Management Framework 1.0 (NIST AI 100-1)

Ref	Control	Applies to	Primary Karatum evidence	In the sample pack
GOVERN 1.6	Inventory of AI systems Requirement (paraphrase). Mechanisms exist to inventory AI systems, resourced according to risk priorities. Karatum evidence Supporting: Trial verdicts. Each build is registered with an identifier, a genome hash (model, prompt hash, tools, harness) and a version lineage, which gives an inventory entry with a stable identity. What it does not show Karatum knows only the builds registered with it. The operator must supply Keep the organisation's own inventory of every AI system and its risk priority, whether or not it was ever registered with Karatum.	Voluntary framework	none	Partial evidence
GOVERN 4.3	Testing, incident identification and information sharing Requirement (paraphrase). Organisational practices enable AI testing, identification of incidents and information sharing. Karatum evidence Primary: Incident Registry. A public registry with a report form, severity scoring, timelines and a signed feed that other parties can subscribe to. Supporting: Trial verdicts. Independent, repeatable testing of the agent. What it does not show Does not replace the organisation's own incident process or its information-sharing agreements. The operator must supply Define internal practices: who tests, how incidents are raised internally, what is shared externally.	Voluntary framework	Incident Registry	Evidence attached
GOVERN 6.1	Third-party AI risk Requirement (paraphrase). Policies and procedures address risks from third-party AI software and data, including infringement of rights. Karatum evidence Supporting: K-rating certificate. Independent measured performance of an agent the organisation did not build. Supporting: Trial verdicts. Verified outcomes rather than the supplier's claims. Supporting: Honesty Index. How reliable the supplier agent's own reports are. Supporting: Shield certificate. Resilience of the third-party agent to injected hostile input. What it does not show Does not cover intellectual-property, privacy or licensing risks of a third-party system. The operator must supply Procurement policy, contractual terms, due diligence on the supplier's data and model provenance, exit plan (see also GOVERN 6.2 contingency processes).	Voluntary framework	none	Partial evidence
MAP 2.3	TEVV considerations identified Requirement (paraphrase). Scientific integrity and test, evaluation, verification and validation (TEVV) considerations are identified and documented. Karatum evidence Supporting: Verifiable judge provenance. The evaluator is a published, reproducible bundle with a source commit; the verdict is a pure function of a stored trace. Supporting: Trial verdicts. Evidence classes E0 to E4 state how strong the evidence behind each verdict is. What it does not show Describes Karatum's evaluation design, not the organisation's choice of TEVV for its context. The operator must supply Document experimental design, data representativeness and the sources of validity threats for the intended deployment.	Voluntary framework	none	Partial evidence
MAP 3.5	Human oversight processes defined Requirement (paraphrase). Processes for human oversight are defined, assessed and documented in line with policy. Karatum evidence Supporting: Karatum Guard. A technical oversight gate exists for accounts that install it. What it does not show A gate is not a process: who watches, who approves and who can stop the agent is not visible to Karatum. The operator must supply Define roles, escalation and approval paths; assess whether they work.	Voluntary framework	none	Partial evidence
MAP 5.1	Likelihood and magnitude of impacts Requirement (paraphrase). Likelihood and magnitude of each identified impact, positive or negative, are identified based on expected use, past uses and public incident reports. Karatum evidence Supporting: K-rating certificate. The likelihood side for task failure: measured success rate with a 95% confidence interval. Supporting: Incident Registry. Public incident reports with severity for the build. What it does not show Says nothing about the magnitude or the people affected. The operator must supply Impact analysis for the deployment context, including affected parties and their stake.	Voluntary framework	none	Partial evidence
MEASURE 1.1	Metrics selected Requirement (paraphrase). Approaches and metrics for measuring AI risks are selected, starting with the most significant risks, and the risks that cannot be measured are documented. Karatum evidence Primary: K-rating certificate. Defined metric: verified success rate with a Wilson interval, by task class. Primary: Honesty Index. Defined metric: over-claim rate with interval, bias, calibration error. Primary: Shield certificate. Defined metric: attacks resisted per class. Supporting: Behavioural fingerprints. Defined metric: drift against baseline. What it does not show The metrics are Karatum's. Whether they are the right ones for the organisation's most significant risks is not established. The operator must supply Select the metrics that fit the organisation's own risk list and document the risks that remain unmeasured.	Voluntary framework	K-rating certificate, Honesty Index, Shield certificate	Evidence attached
MEASURE 1.2	Metrics and controls regularly reassessed Requirement (paraphrase). The appropriateness of AI metrics and the effectiveness of existing controls are regularly assessed and updated. Karatum evidence Supporting: Verifiable judge provenance. Scheduled re-verification re-runs the judge on stored traces and compares the result with the recorded verdict. Supporting: Honesty Index. Methodology is versioned; each edition carries its method version. What it does not show The re-verification checks the measurement's reproducibility, not whether it is the right measurement. The operator must supply Periodic review of the metric set and of the controls against changing risks.	Voluntary framework	none	Partial evidence
MEASURE 1.3	Independent assessment Requirement (paraphrase). Internal experts who were not front-line developers, or independent assessors, regularly assess the system. Karatum evidence Supporting: Trial verdicts. Karatum evaluates agents it did not build (for external builds), with published methodology; the decision comes from chain state and tests. What it does not show Karatum builds its own house agents; for those the evaluator is not independent of the agent. The pack states whether the build is a Karatum house build. The operator must supply Decide whether an assessment by Karatum meets the organisation's independence requirement.	Voluntary framework	none	Partial evidence
MEASURE 2.1	Test sets, metrics and tools documented Requirement (paraphrase). Test sets, metrics and details about the tools used during TEVV are documented. Karatum evidence Primary: Trial verdicts. Trials, task types, seeds and the verdict set are listed, and the verdict set is committed by hash. Primary: Verifiable judge provenance. Evaluator bundle digests, source commits and traces. What it does not show Test sets are Karatum's task suites. The operator must supply Document any additional test sets the organisation uses.	Voluntary framework	Trial verdicts, Verifiable judge provenance	Evidence attached
MEASURE 2.3	Performance demonstrated under deployment-like conditions Requirement (paraphrase). Performance or assurance criteria are measured and demonstrated for conditions similar to deployment, with the results documented. Karatum evidence Primary: Trial verdicts. Tasks run on a fork of real chain state with outcomes read from chain state; software tasks run against real test suites. Supporting: K-rating certificate. Results summarised with sample size and interval. What it does not show Whether Karatum's conditions resemble the operator's deployment is the operator's judgement; they are fixed task classes. The operator must supply Show how deployment conditions compare with the tested ones and what is untested.	Voluntary framework	Trial verdicts	Evidence attached
MEASURE 2.4	Monitoring in production Requirement (paraphrase). The functionality and behaviour of the AI system and its components are monitored in production. Karatum evidence Primary: Flight Recorder. Recorded operation with coverage, gap and fork detection. Supporting: Behavioural fingerprints. Behavioural drift on a fixed probe set. What it does not show Production monitoring exists only for steps the operator records. The operator must supply Operate monitoring of live behaviour, with thresholds, alerts and owners.	Voluntary framework	Flight Recorder	Partial evidence
MEASURE 2.5	Valid and reliable, limitations documented Requirement (paraphrase). The system to be deployed is demonstrated to be valid and reliable, and the limits of generalisability beyond the conditions of development are documented. Karatum evidence Primary: K-rating certificate. Measured success with an interval, the weakest evidence class and an expiry that stops an old measurement being treated as current. Supporting: Honesty Index. Reliability of the system's self-assessment. What it does not show Generalisation beyond Karatum's task classes is not demonstrated. The operator must supply Document limits for the intended use and population.	Voluntary framework	K-rating certificate	Evidence attached
MEASURE 2.6	Safety risks evaluated regularly Requirement (paraphrase). The AI system is evaluated regularly for safety risks, demonstrating it is safe and that residual negative risk does not exceed tolerance. Karatum evidence Supporting: Shield certificate. Evaluation against hostile inputs that could make the agent take harmful actions with funds or secrets. Supporting: Incident Registry. Registered harmful outcomes. What it does not show Safety in the sense of harm to people is outside Karatum's suites. Harm measured here is mostly loss of funds, leaks of planted secrets and wrong task outcomes. The operator must supply Safety evaluation for the real hazard model of the application; residual-risk tolerance and the decision against it.	Voluntary framework	none	Partial evidence
MEASURE 2.7	Security and resilience evaluated Requirement (paraphrase). AI system security and resilience are evaluated and documented. Karatum evidence Primary: Shield certificate. Red-team results by attack class, from chain state and canary hits. What it does not show Covers the injection classes in the suite. Not data poisoning, model theft, infrastructure security or insider risk. The operator must supply Security assessment of the whole system: threat modelling, penetration tests, supply chain.	Voluntary framework	Shield certificate	Evidence missing for this build
MEASURE 2.8	Transparency and accountability risks examined Requirement (paraphrase). Risks associated with transparency and accountability are examined and documented. Karatum evidence Supporting: Verifiable judge provenance. Verdicts are accountable: who decided, with which evaluator, from which trace. Supporting: Flight Recorder. Signed records bind actions to an agent key. What it does not show Accountability inside the organisation (who answers for the agent) is not visible to Karatum. The operator must supply Assign accountability and examine explainability needs of the intended use.	Voluntary framework	none	Evidence missing for this build
MEASURE 2.13	Effectiveness of TEVV metrics and processes Requirement (paraphrase). The effectiveness of the employed TEVV metrics and processes in the Measure function is evaluated and documented. Karatum evidence Supporting: Verifiable judge provenance. Re-verification results show whether re-running the judge reproduces recorded verdicts. Supporting: Honesty Index. Sensitivity analysis shows how much the headline changes when parts of the data are excluded. What it does not show Reproducibility is not validity. The operator must supply Evaluate whether the metrics predicted real outcomes for the organisation.	Voluntary framework	none	Partial evidence
MEASURE 3.1	Tracking existing and emergent risks Requirement (paraphrase). Approaches, personnel and documentation exist to regularly identify and track existing, unanticipated and emergent risks. Karatum evidence Primary: Incident Registry. A registry with automated detectors and an open report channel. Supporting: Behavioural fingerprints. Silent behaviour change is a type of emergent risk that is surfaced as drift. What it does not show Tracks what Karatum detects or is told. The operator must supply An internal process and staff to track risks over the system's life.	Voluntary framework	Incident Registry	Evidence attached
MEASURE 3.3	Feedback processes for end users and affected people Requirement (paraphrase). Feedback processes for end users and impacted communities to report problems and appeal outcomes are established and integrated into evaluation. Karatum evidence Supporting: Incident Registry. Anyone can file an incident report with evidence; rate-limited, triaged by maintainers. What it does not show Not an appeal mechanism for decisions made about people. The operator must supply Feedback and appeal channels for the people the system affects.	Voluntary framework	none	Partial evidence
MEASURE 4.3	Performance improvements or declines identified Requirement (paraphrase). Measurable performance improvements or declines are identified based on consultations with relevant AI actors and documented. Karatum evidence Primary: Behavioural fingerprints. Measured drift against baseline over time. Primary: K-rating certificate. Status changes (VALID, STALE, EXPIRED, REVOKED) as evidence ages or behaviour changes. What it does not show No consultation with users or domain experts is part of the data. The operator must supply Consult relevant AI actors on whether a change is an improvement or a decline.	Voluntary framework	Behavioural fingerprints, K-rating certificate	Evidence attached
MANAGE 1.3	Responses to high-priority risks Requirement (paraphrase). Responses to the AI risks deemed high priority are developed, planned and documented, for example mitigating, transferring, avoiding or accepting. Karatum evidence Supporting: Karatum Guard. On-chain gating is a mitigation available for transaction risk. Supporting: Recall actions and drills. A recall can withdraw the agent's certified status. What it does not show Karatum implements mitigations; deciding the response to a risk is the organisation's. The operator must supply Risk treatment plan with owners and decisions.	Voluntary framework	none	Partial evidence
MANAGE 2.3	Response and recovery for previously unknown risks Requirement (paraphrase). Procedures are followed to respond to and recover from a previously unknown risk when it is identified. Karatum evidence Supporting: Incident Registry. A documented lifecycle (reported, triaged, confirmed, recalled, fixed) with timestamps. Supporting: Recall actions and drills. Measured time from confirmation to on-chain revocation. What it does not show Karatum's response is to its registry and certificates, not to the operator's systems. The operator must supply Operator-side response and recovery procedures, rehearsed.	Voluntary framework	none	Partial evidence
MANAGE 2.4	Supersede, disengage or deactivate Requirement (paraphrase). Mechanisms are in place and applied, and responsibilities assigned, to supersede, disengage or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use. Karatum evidence Primary: Recall actions and drills. A confirmed incident revokes the certificate in Guard registries and withdraws the attestation; drills measure the time. Primary: Karatum Guard. Accounts with the Guard installed stop executing the agent's transactions once its certificate is revoked or expired. What it does not show Only for accounts that use the Guard, on the chains it is deployed to. It does not turn off the agent itself. The operator must supply A way to stop the agent in the operator's own stack and a named person responsible for using it.	Voluntary framework	Recall actions and drills, Karatum Guard	Evidence attached
MANAGE 3.1	Third-party AI risks monitored Requirement (paraphrase). AI risks and benefits from third-party resources are regularly monitored and controls applied and documented. Karatum evidence Supporting: K-rating certificate. Expiring certificate status of the supplier agent. Supporting: Behavioural fingerprints. Detects a supplier changing the model or prompt without notice. Supporting: Incident Registry. Incidents on the supplier's build. What it does not show Visible only for third-party agents Karatum evaluates. The operator must supply Supplier monitoring cadence, contractual notification duties.	Voluntary framework	none	Partial evidence
MANAGE 4.1	Post-deployment monitoring plan implemented Requirement (paraphrase). Post-deployment monitoring plans are implemented, including mechanisms for capturing user input, appeal and override, decommissioning, incident response, recovery and change management. Karatum evidence Supporting: Behavioural fingerprints. Change detection supports change management. Supporting: Incident Registry. Incident response record. Supporting: Flight Recorder. Recorded operation. Supporting: Karatum Guard. An override point for transactions. What it does not show Only technical pieces of a plan. The operator must supply The plan itself, with user input, appeal, decommissioning and change-management procedures.	Voluntary framework	none	Partial evidence
MANAGE 4.3	Incidents and errors communicated Requirement (paraphrase). Incidents and errors are communicated to relevant AI actors, including affected communities, and processes for tracking, responding to and recovering from them are followed and documented. Karatum evidence Primary: Incident Registry. Public timeline per incident, a signed feed (JSON and Atom), HMAC-signed webhooks, operator notification and response recorded on the timeline. Supporting: Recall actions and drills. On-chain announcement of recalls readable by any contract. What it does not show Communication to people affected by the operator's use of the agent is the operator's. The operator must supply Communication plan to customers, regulators and affected communities.	Voluntary framework	Incident Registry	Evidence attached

NIST AI Risk Management Framework 1.0 (NIST AI 100-1). Control map version 1.0.0. Titles and requirement lines are paraphrased; use the official text.

ISO/IEC 42001:2023 Annex A (AI management system controls)

Ref	Control	Applies to	Primary Karatum evidence	In the sample pack
A.3.3	Reporting of concerns Requirement (paraphrase). The organisation defines and puts in place a process to report concerns about the organisation's role with respect to an AI system throughout its life cycle. Karatum evidence Supporting: Incident Registry. An external, public channel to report an agent failure with evidence. What it does not show An external channel is not the organisation's internal reporting process. The operator must supply Internal reporting process, whistleblowing protection, handling of reports.	Voluntary, certifiable standard (organisation-level)	none	Partial evidence
A.5.2	AI system impact assessment process Requirement (paraphrase). The organisation establishes a process to assess the potential consequences for individuals, groups and society of its AI systems. Karatum evidence None: this control is entirely the operator's. What it does not show Karatum measures agent task performance and robustness. It does not assess societal or individual consequences. The operator must supply The whole control: impact assessment process, its documentation and its use in decisions.	Voluntary, certifiable standard (organisation-level)	none	Operator evidence only
A.6.2.4	AI system verification and validation Requirement (paraphrase). The organisation defines and documents verification and validation measures for the AI system and specifies criteria for their use. Karatum evidence Primary: Trial verdicts. Verified outcomes in repeatable trials. Primary: K-rating certificate. Acceptance criterion made explicit as a lower confidence bound with expiry. Primary: Shield certificate. Adversarial validation by attack class. Supporting: Verifiable judge provenance. Validation results are reproducible from stored traces. What it does not show The criteria for use of the results are the organisation's. The operator must supply Define the V&V plan and acceptance criteria for the intended use, and record decisions against them.	Voluntary, certifiable standard (organisation-level)	Trial verdicts, K-rating certificate, Shield certificate	Evidence attached
A.6.2.5	AI system deployment Requirement (paraphrase). The organisation documents a deployment plan and ensures that appropriate requirements are met prior to deployment. Karatum evidence Supporting: K-rating certificate. A certificate status that can be required before deployment. Supporting: Karatum Guard. A gate can enforce the requirement at the moment of execution. What it does not show Does not document or approve a deployment. The operator must supply Deployment plan, approvals and the check of requirements before release.	Voluntary, certifiable standard (organisation-level)	none	Partial evidence
A.6.2.6	AI system operation and monitoring Requirement (paraphrase). The organisation defines and documents the elements for ongoing operation and monitoring of the AI system, including performance monitoring, repairs, updates and support. Karatum evidence Primary: Behavioural fingerprints. Ongoing behavioural drift measurement. Primary: Flight Recorder. Recorded operation, with coverage measured in the pack. Supporting: Incident Registry. Incident history. What it does not show Monitoring of real use depends on what the operator records. The operator must supply Operations plan: monitoring metrics, repair and update procedures, support.	Voluntary, certifiable standard (organisation-level)	Behavioural fingerprints, Flight Recorder	Evidence attached
A.6.2.7	AI system technical documentation Requirement (paraphrase). The organisation determines what AI system technical documentation each relevant category of interested parties needs and provides it in an appropriate form. Karatum evidence Supporting: Trial verdicts. This pack is itself a document for auditors on testing, monitoring and incidents. Supporting: Verifiable judge provenance. Reproducible evaluation method. What it does not show Covers the evaluation slice of the documentation only. The operator must supply System design, data, models, intended use and limits documentation for each audience.	Voluntary, certifiable standard (organisation-level)	none	Partial evidence
A.6.2.8	AI system recording of event logs Requirement (paraphrase). The organisation determines at which phases of the AI system life cycle record keeping of event logs should be enabled, at least when the system is in use. Karatum evidence Primary: Flight Recorder. Tamper-evident log of agent steps, with batches, gaps and anchors counted. Supporting: Verifiable judge provenance. Evaluation traces are retained. What it does not show Commitments only; contents stay with the operator. The operator must supply Decide the phases and events to record, retention, protection and review of logs.	Voluntary, certifiable standard (organisation-level)	Flight Recorder	Evidence missing for this build
A.8.2	System documentation and information for users Requirement (paraphrase). The organisation determines and provides the necessary information to users of the AI system. Karatum evidence Supporting: K-rating certificate. A quotable measured performance statement with its limits. Supporting: Honesty Index. Information on how far the agent's self-reports can be relied on. What it does not show Not user documentation. The operator must supply User-facing documentation: capabilities, limits, appropriate use.	Voluntary, certifiable standard (organisation-level)	none	Partial evidence
A.8.4	Communication of incidents Requirement (paraphrase). The organisation determines and documents a plan for communicating incidents to users. Karatum evidence Primary: Incident Registry. Public timelines, a signed feed and webhooks for subscribers. What it does not show Covers communication from Karatum's registry. User communication is the organisation's. The operator must supply The organisation's own incident communication plan to its users.	Voluntary, certifiable standard (organisation-level)	Incident Registry	Evidence attached
A.9.4	Intended use of the AI system Requirement (paraphrase). The organisation ensures that the AI system is used according to the intended uses of the AI system and its accompanying documentation. Karatum evidence Supporting: Karatum Guard. A gate with tier caps limits what a certified agent can do on an account, a technical guard-rail against use beyond what was certified. What it does not show Caps are about value and certification state, not about the purpose of an action. The operator must supply Policies and controls that keep use within the intended purpose.	Voluntary, certifiable standard (organisation-level)	none	Partial evidence
A.10.3	Suppliers Requirement (paraphrase). The organisation establishes a process to ensure that its use of services, products or materials provided by suppliers aligns with its approach to responsible development and use of AI systems. Karatum evidence Supporting: K-rating certificate. Independent evidence about a supplied agent. Supporting: Trial verdicts. Verified outcomes, not supplier claims. Supporting: Shield certificate. Robustness evidence for the supplied agent. What it does not show Evidence about one supplier's agent; not a supplier-management process. The operator must supply Supplier selection and review process, agreements, ongoing supplier monitoring.	Voluntary, certifiable standard (organisation-level)	none	Partial evidence

ISO/IEC 42001:2023 Annex A (AI management system controls). Control map version 1.0.0. Titles and requirement lines are paraphrased; use the official text.

Karatum artefacts used as evidence

Artefact	What it is	Anchor
Trial verdicts	Outcomes of tasks the agent attempted in Karatum trials on a fork of real chain state (or in a sandboxed software task), decided by the judge from chain state and test results, never from the agent's own report, each with an evidence class E0 to E4.	Each verdict hash is attested on Ethereum Sepolia (EAS); the pack commits to the full verdict set with a SHA-256 over all counted attempts.
K-rating certificate	A per task-class summary of the verdicts: successes, sample size, a 95% Wilson confidence interval, the weakest evidence class, an expiry, and a status (VALID, STALE, EXPIRED, REVOKED, INSUFFICIENT) that changes when behaviour drifts or the certificate expires.	Computed from the verdict set; mirrored on-chain to the Guard registries.
Verifiable judge provenance	For verdicts decided since traces were kept, the complete judge input (trace) and the digest of the published evaluator bundle that decided it, so anyone can re-run the judge; plus scheduled re-verification runs.	Evaluator bundle digests are registered on Ethereum Sepolia; provenance is attested with the verdict attestation as reference.
Honesty Index	How well the agent's own success and failure claims (E0) match the verified outcome: over-claim rate with confidence interval, claim bias and calibration error.	Monthly editions are immutable and their snapshot hash is attested on Ethereum Sepolia.
Shield certificate	Result of a seeded red-team suite run against the agent on a fork: how many injected attacks (prompt injection and related classes in the suite) the agent resisted, by attack class, and whether it stayed useful.	Certificate hash attested on Ethereum Sepolia.
Behavioural fingerprints	Hashes of how the agent behaves on a fixed probe set (tool sequence and rounded arguments) and the share of probes that differ from the baseline (drift). Detects silent changes of model, prompt or tools.	Fingerprint hashes are stored; the current hash is referenced by the certificate.
Flight Recorder	A signed, hash-chained record of the agent's own steps, committed in batches by the agent's key (content stays private unless disclosed), checked for gaps and forks, and optionally anchored on-chain; disclosed steps are compared with on-chain facts.	Batch roots are anchored in EAS attestations on Base Sepolia.
Incident Registry	A CVE-style registry (KAR-YYYY-NNNN) of AI-agent failures with a documented severity score, a status lifecycle and a timestamped public timeline, fed by detectors, public reports and operators.	Confirmed incidents are written to KaratumIncidentLog on Base Sepolia and Ethereum Sepolia; a signed feed is published.
Recall actions and drills	What happened on-chain when an incident was confirmed: certificate revocation in the Guard registries, EAS revocation, ERC-8004 feedback, incident log entries, with the measured latency from confirmation to the block that carried each action. Drills rehearse the mechanism and are labelled as such.	Each action is an on-chain transaction on a testnet.
Karatum Guard	A Safe guard and ERC-7579 module that only lets an agent's transaction execute while its certificate is valid and within the tier caps, plus preflight checks that simulate a transaction before it is sent. The pack shows the agent's tier leaves, the registries they were published to and the preflight outcomes.	Certificate roots are published to KaratumCertRegistry contracts on Base Sepolia and Ethereum Sepolia.

Generate a pack for your agent

POST /v1/compliance/packs needs an API key (up to 20 new packs a day per key). Pass asOf to make a pack reproducible. Your declaration (organisation, intended purpose, your own EU AI Act classification, oversight measures, log retention, incident process) is printed as declared and not verified. Non-public incidents are included only for builds your key owns. Packs are private to the key that generated them. Also: GET /v1/compliance/packs/:id and GET /v1/compliance/controls.

# 1. Generate a pack for a build (API key required)
curl -s https://api.karatum.com/v1/compliance/packs \
  -H "authorization: Bearer $KARATUM_API_KEY" -H 'content-type: application/json' \
  -d '{
    "buildId": "bld_…",
    "asOf": "2026-10-16T12:00:00Z",
    "declaration": {
      "organization": { "name": "Acme Treasury" },
      "system": { "intendedPurpose": "…" },
      "riskClassification": { "euAiAct": "not_assessed" },
      "oversight": { "description": "…", "stopProcedure": "…" },
      "logging": { "retentionDays": 365 }
    }
  }'
# -> 201 { data: { id: "cp_…", hashes, attestation, urls } }

# 2. Download the PDF, the canonical JSON and the manifest of hashes
curl -s -H "authorization: Bearer $KARATUM_API_KEY" "https://api.karatum.com/v1/compliance/packs/cp_…?format=pdf" -o pack.pdf

Read this before you rely on it

This document compiles technical measurements that Karatum produced about one AI agent build. It is not legal advice, not a conformity assessment, not a certification and not an audit opinion. It does not say that any law, regulation or standard is met.
Karatum is not a notified body, a market surveillance authority, an accredited certification body or an auditor under ISO/IEC 42001 or any other scheme.
The control matrix shows which Karatum measurement bears on which requirement of the EU AI Act, the NIST AI RMF 1.0 and ISO/IEC 42001 Annex A, and what that measurement leaves out. Whether a requirement applies to a system (for the EU AI Act, mostly only to high-risk systems) and whether the system satisfies it is for the operator and its advisers to decide. Framework wording is paraphrased; use the official texts. Dates of application and obligations under the EU AI Act may change: check the current law.
Karatum's evidence covers fixed task suites, probe sets and attack suites, run on the testnets and sandboxes named in this document. It is not a measurement of the operator's production system unless the operator records that system (Flight Recorder). An empty incident list means that none is recorded in Karatum's registry, not that none occurred.
Operator declarations are reproduced as supplied and have not been verified by Karatum.
The hash attestation on Ethereum Sepolia, a test network, shows that exactly this content was committed by Karatum's attester key at the stated time. It does not show that the content is correct, complete or reviewed by anyone.

Methodology: /docs/methodology/compliance-pack. Control mapping: docs/compliance/ (the Karatum stack shows where each artefact comes from).