Skip to content

Build

Accuracy & robustness: what AI regulations require

Testing, TEVV, red teaming, safe failure.

23 requirements · 11 regulations · 9 checks

Requirements by regulation

EUAI Act2

EX-009

Accuracy, robustness, cybersecurity Art. 15

EX-032

GPAI obligations with systemic risk Art. 55

UKUK principles1

UK-01

Safety, security and robustness Principle 1

EUDORA1

DORA-24

Operational resilience testing, TLPT for significant entities Art. 24-27

EUPLD1

PLD-07

Defectiveness: self-learning, cybersecurity, updates Art. 7

USNIST AI RMF11

GOVERN-4.1

Organizational policies and practices are in place to foster a critical thinking and safety-first mindset in the design, development, deployment, and uses of AI systems to minimize potential negative impacts. GOVERN

GOVERN-4.3

Organizational practices are in place to enable AI testing, identification of incidents, and information sharing. GOVERN

MAP-2.3

Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection (e.g., availability, representativeness, suitability), system trustworthiness, and construct validation. MAP

MEASURE-1.3

Internal experts who did not serve as front-line developers for the system and/or independent assessors are involved in regular assessments and updates. Domain experts, users, AI actors external to the team that developed or deployed the AI system, and affected communities are consulted in support of assessments as necessary per organizational risk tolerance. MEASURE

MEASURE-2.1

Test sets, metrics, and details about the tools used during TEVV are documented. MEASURE

MEASURE-2.3

AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment setting(s). Measures are documented. MEASURE

MEASURE-2.5

The AI system to be deployed is demonstrated to be valid and reliable. Limitations of the generalizability beyond the conditions under which the technology was developed are documented. MEASURE

MEASURE-2.6

The AI system is evaluated regularly for safety risks - as identified in the map function. The AI system to be deployed is demonstrated to be safe, its residual negative risk does not exceed the risk tolerance, and it can fail safely, particularly if made to operate beyond its knowledge limits. Safety metrics reflect system reliability and robustness, real-time monitoring, and response times for AI system failures. MEASURE

MEASURE-2.13

Effectiveness of the employed TEVV metrics and processes in the measure function are evaluated and documented. MEASURE

MEASURE-4.2

Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are informed by input from domain experts and relevant AI actors to validate whether the system is performing consistently as intended. Results are documented. MEASURE

MEASURE-4.3

Measurable performance improvements or declines based on consultations with relevant AI actors, including affected communities, and field data about context-relevant risks and trustworthiness characteristics are identified and documented. MEASURE

USUS federal1

USF-03

Minimum practices for high-impact AI: testing, impact assessment, oversight, monitoring, appeal M-25-21 §4

CANCanada1

CAN-02

Quality assurance, bias testing and human intervention by level DADM §6.3

BRPL 23382

BR-09

Applier documentation Art. 18 I.a, I.c, I.f

BR-13

Monitoring of results Art. 18 I.b

CNChina GenAI1

CN-02

Security assessment before public launch GenAI Art. 17

INTLISO 420011

ISO-A.6

Lifecycle: design, verification, deployment, operation, event logs A.6

INTLOECD AI1

OECD-1.4

Robustness, security and safety 1.4

Checks in this theme

Most shared first

Code	Check	Used by
VER-009-02	Accuracy and robustness verified and documented	AI Act PL 2338 China GenAI NIST AI RMF PLD UK principles
CHK-TEVV	TEVV plan, test sets, metrics and data considerations are documented	Canada ISO 42001 NIST AI RMF US federal
VER-AUTO-05	Accuracy monitoring in operation	AI Act PL 2338 NIST AI RMF
VER-009-F-04	Resilience to adversarial attacks tested	AI Act CRA DORA
CHK-SAFETY	Safety risks are evaluated and safe-failure demonstrated	NIST AI RMF OECD AI
VER-032-G-01	Model evaluation with standardised protocols and adversarial testing	AI Act
CHK-EFFECTIVE-CHALLENGE	Effective-challenge, red-team or three-lines-of-defense practices are in place	NIST AI RMF
CHK-TESTING-ENABLE	Organizational practices enable AI testing and identification of limitations	NIST AI RMF
CHK-INDEP-ASSESS	Independent or internal-expert assessment involves domain experts and affected communities	NIST AI RMF

Related themes

Data governance Technical documentation Logging & traceability Cybersecurity Environmental impact

Run these requirements across all your AI systems

TrustFlow inventories your systems, qualifies them under each regulation and collects evidence once for every referential.