AI Safety Assurance Claims in Europe

1. Meaning and Scope

AI safety assurance claims arise when an AI developer, deployer, auditor, testing laboratory, certification body, conformity-assessment body, consultant, or other responsible actor is alleged to have failed to adequately assess, test, validate, monitor, certify, or communicate the safety of an AI system, resulting in injury, property damage, financial loss, privacy harm, discrimination, or other legally recognised damage.

Safety assurance is broader than ordinary software testing. It can include:

safety testing;

model validation;

robustness testing;

reliability assessment;

cybersecurity assessment;

bias and discrimination testing;

adversarial testing;

human-oversight assessment;

risk assessment;

conformity assessment;

certification;

post-market monitoring;

incident investigation;

software-update verification;

documentation and record keeping.

Examples include:

an AI medical diagnostic system certified as safe despite dangerous false negatives;

an autonomous vehicle AI inadequately tested for unusual road conditions;

an industrial AI system incorrectly certified as safe;

an AI recruitment system certified without adequate discrimination testing;

an AI cybersecurity system that fails to detect foreseeable attacks;

an AI predictive-maintenance system whose safety risks were inadequately assessed.

There is currently no large body of European case law specifically titled “AI safety assurance claims.” Therefore, the most useful authorities come from product liability, medical devices, data protection, professional responsibility, automated decision-making and fundamental-rights jurisprudence.

2. European Legal Framework

A. EU AI Act

The EU AI Act creates an important regulatory framework for high-risk AI.

Relevant concepts include:

risk management;

data and data governance;

technical documentation;

record keeping;

transparency;

human oversight;

accuracy;

robustness;

cybersecurity;

conformity assessment;

post-market monitoring;

incident reporting.

Where an AI system is subject to conformity-assessment requirements, inadequate assurance can potentially produce regulatory as well as civil consequences.

3. Product Liability

AI safety assurance claims may also fall within European product-liability law.

A defective AI-enabled product can potentially involve:

defective software;

defective AI model;

defective sensor;

defective cybersecurity;

insufficient warnings;

inadequate instructions;

unsafe updates;

failure to address foreseeable risks.

The modern European product-liability framework is increasingly important because software and AI can form an integral component of products.

4. Professional Negligence

An independent safety assessor may potentially face liability where it:

failed to apply reasonable professional standards;

ignored known risks;

used inadequate testing methodology;

failed to identify foreseeable failure modes;

issued an unjustified safety opinion;

failed to perform contracted tests;

negligently certified an unsafe system.

The exact claim depends heavily upon national contract and tort law.

5. Data Protection and Fundamental Rights

Safety assurance can also involve:

GDPR;

Charter rights;

ECHR rights;

equality law.

For example, a safety assessment that ignores algorithmic discrimination may be inadequate where discrimination is itself a foreseeable legal and operational risk.

6. Important European Case Laws

Case 1: Boston Scientific Medizintechnik GmbH and Others

Court: CJEU
Joined Cases: C-503/13 and C-504/13
Year: 2015

Facts

The cases concerned implantable medical devices such as pacemakers and implantable cardioverter-defibrillators.

Certain devices were found to have a higher risk of failure than normally expected.

Decision

The CJEU recognised that a group of products may be regarded as defective where products belonging to that group present an abnormal risk of damage.

Principle

Safety expectations are assessed against the level of safety that persons are entitled to expect, including foreseeable risks associated with a category of products.

Relevance to AI Safety Assurance

This is an important analogy for AI systems.

Suppose an AI-enabled medical device passes a conventional certification process but later testing reveals that the entire model family has a systemic safety defect.

The existence of individual units that have not yet caused harm does not necessarily eliminate the underlying safety concern.

The case supports the proposition that systemic AI safety defects can be legally significant even before every individual system produces damage.

7. Case 2: Skov Æg v Bilka Lavprisvarehus

Court: CJEU
Case: C-402/03
Year: 2006

Facts

The case concerned European product liability and responsibility within a distribution chain.

Decision

The CJEU examined the relationship between producer responsibility and other participants in the supply chain.

Principle

European product-liability law establishes particular responsibility for producers while recognising circumstances in which other actors may also have legally relevant roles.

Relevance to AI Safety Assurance

AI safety often involves a complex supply chain:

developer → model provider → hardware manufacturer → integrator → deployer → certification body.

An AI accident may therefore require identifying:

who developed the AI;

who integrated it;

who tested it;

who certified it;

who deployed it;

who controlled safety updates.

Safety assurance cannot be analysed without identifying the relevant actor's legal role.

8. Case 3: O'Byrne v Sanofi Pasteur MSD Ltd

Court: CJEU
Case: C-127/04
Year: 2006

Facts

The case concerned liability for a defective pharmaceutical product and the identification of the relevant producer.

Decision

The CJEU examined the meaning of "producer" and responsibility within a complex commercial supply chain.

Principle

Identifying the legally responsible producer requires examination of the actual structure and role of the entities involved.

Relevance to AI Safety Assurance

AI safety claims can involve numerous actors:

foundation-model provider;

AI application developer;

hardware provider;

safety assessor;

conformity-assessment body;

cloud provider;

employer/deployer.

A claimant therefore needs to identify which actor had the relevant safety obligation and whether that obligation was breached.

9. Case 4: Donoghue v Stevenson

Court: House of Lords
Year: 1932

Facts

A consumer became ill after consuming a contaminated beverage.

Decision

The House of Lords recognised a general manufacturer's duty of care toward consumers.

Principle

Manufacturers must take reasonable care to avoid foreseeable harm to persons who may reasonably be affected by their products.

Relevance to AI Safety Assurance

Although predating AI and EU product-liability legislation, the principle remains foundational to negligence analysis.

An AI developer or safety assessor may face analogous questions:

Was harm foreseeable?

Was reasonable testing performed?

Were known safety risks addressed?

Were appropriate warnings provided?

Was the system released despite unresolved defects?

10. Case 5: Henderson v Merrett Syndicates Ltd

Court: House of Lords
Year: 1995

Facts

The case concerned professional responsibility and assumption of responsibility.

Decision

The court recognised that professional responsibility can arise from the relationship between a professional service provider and persons relying upon that service.

Principle

Professional expertise combined with an assumption of responsibility can create duties concerning the quality of professional services.

Relevance to AI Safety Assurance

This is particularly important for:

AI auditors;

safety consultants;

conformity assessors;

cybersecurity testers;

validation specialists.

If an organisation specifically holds itself out as providing competent AI safety assurance, the question may arise whether it assumed responsibility to perform that work with reasonable professional competence.

11. Case 6: Caparo Industries plc v Dickman

Court: House of Lords
Year: 1990

Facts

The case concerned reliance on professional information and whether a duty of care existed toward a particular claimant.

Decision

The court developed the familiar framework involving:

foreseeability;

proximity;

whether imposing a duty would be fair, just and reasonable.

Relevance to AI Safety Assurance

This becomes important when a third party claims that it relied on a safety assessment.

For example:

AI safety assessor → certification → hospital purchases system → patient suffers harm.

The legal question may be whether the assessor owed a duty to the patient, hospital, purchaser, deployer or another third party.

The answer depends upon the applicable national law and the relationship between the parties.

12. Case 7: SCHUFA Holding AG v Verbraucherzentrale Bundesverband

Court: CJEU
Case: C-634/21
Year: 2023

Facts

SCHUFA generated automated credit scores used in decision-making.

Decision

The CJEU held that automated scoring can constitute automated decision-making when the score effectively determines a consequential decision.

Principle

AI outputs that materially determine decisions cannot necessarily be treated as merely advisory.

Relevance to Safety Assurance

An AI safety assessor cannot necessarily describe a highly consequential AI system as "human controlled" simply because a person formally signs off the final decision.

A meaningful safety-assurance process requires consideration of:

what the AI actually does;

what risks it creates;

how humans interact with it;

whether human oversight is genuine.

13. Case 8: Dun & Bradstreet Austria GmbH

Court: CJEU
Case: C-203/22
Year: 2025

Principle

The CJEU addressed meaningful information concerning automated decision-making and the logic involved.

Relevance to Safety Assurance

Safety assurance depends upon understanding how an AI system operates sufficiently to identify relevant risks.

If an AI provider refuses meaningful information on the ground that:

"The algorithm is proprietary,"

that may create serious governance problems.

A safety assessor may need information about:

model architecture;

input variables;

decision logic;

limitations;

error rates;

training data;

validation methodology.

The precise information required depends upon the legal and contractual context.

14. Case 9: Wirtschaftsakademie Schleswig-Holstein

Court: CJEU
Case: C-210/16
Year: 2018

Principle

Entities may share responsibility for personal-data processing even when technological operations are carried out by another platform.

Relevance to AI Assurance

An organisation cannot automatically outsource safety responsibility simply by purchasing AI from a vendor.

For example:

Hospital → AI vendor → cloud provider → external testing laboratory.

Each actor may have different responsibilities.

The hospital may remain responsible for determining whether the system is suitable for its intended use, while the vendor may have responsibility for model design and the testing laboratory for the quality of its assurance work.

15. Case 10: Fashion ID GmbH & Co. KG v Verbraucherzentrale NRW

Court: CJEU
Case: C-40/17
Year: 2019

Principle

Integration of third-party technology can generate legal responsibility for the entity incorporating that technology into its operations.

Relevance

An AI deployer cannot necessarily say:

"The external AI provider was responsible for everything."

The deployer must assess:

intended purpose;

suitability;

data;

configuration;

security;

foreseeable risks;

monitoring.

This is particularly important where AI is used in high-risk environments.

16. Case 11: Österreichische Post AG

Court: CJEU
Case: C-300/21
Year: 2023

Principle

A claimant seeking GDPR compensation must distinguish:

infringement;

damage;

causal connection.

Relevance to AI Safety Assurance

A failed assurance process does not automatically establish every category of damages.

For example:

inadequate AI safety audit → AI incident → financial loss.

The claimant still needs to connect the assurance failure to the legally recoverable damage under the applicable cause of action.

17. Case 12: Schrems II

Court: CJEU
Case: C-311/18
Year: 2020

Principle

Organisations must evaluate whether data-transfer mechanisms actually provide adequate protection rather than relying merely on formal contractual structures.

Relevance to AI Safety Assurance

This provides an important substance-over-form principle.

An AI organisation should not assume that compliance paperwork alone establishes safety.

A safety assessment must consider actual operational risks:

cybersecurity;

data leakage;

foreign access;

model extraction;

adversarial attacks;

insecure interfaces.

Formal documentation without substantive protection may be inadequate.

18. What Constitutes a Defective Safety Assurance Process?

A safety assurance process may be challenged where it:

1. Uses inadequate testing

Testing only normal conditions while ignoring foreseeable abnormal situations.

2. Ignores known failure modes

Known risks are not investigated.

3. Uses inappropriate datasets

The test population does not resemble actual deployment conditions.

4. Fails to test edge cases

The system is tested only on common scenarios.

5. Ignores distribution shift

The system is certified using historical data but deployed in substantially different conditions.

6. Fails to test human interaction

The assessment examines the model but not how humans actually use its output.

7. Ignores cybersecurity

An AI may perform accurately under normal conditions but fail catastrophically following adversarial manipulation.

8. Fails to validate updates

A software/model update may change system behaviour without renewed safety testing.

19. AI Safety Assurance and Human Oversight

Human oversight is not simply:

"A human is somewhere in the process."

Effective oversight should permit the responsible person to:

understand system limitations;

detect errors;

interpret warnings;

override the system;

suspend operation;

investigate unusual outputs;

prevent unsafe deployment.

A safety assurance claim can therefore arise where an organisation creates the appearance of human oversight without giving the human operator meaningful control.

20. AI Safety Assurance and Bias

Safety is broader than physical safety.

An AI can be unsafe because it produces systematically discriminatory outcomes.

Examples:

medical AI under-diagnosing a demographic group;

recruitment AI systematically excluding disabled candidates;

facial recognition producing disproportionately inaccurate results;

credit AI systematically disadvantaging particular populations.

Therefore, an adequate safety-assurance process may need to consider:

accuracy;

robustness;

discrimination;

accessibility;

foreseeable social harms.

21. Liability of the Safety Assessor

A safety assessor may potentially face liability where it:

negligently performs contracted testing;

gives an unjustified safety opinion;

ignores material evidence;

uses incompetent methodology;

fails to identify foreseeable defects;

falsely represents the scope of testing;

fails to communicate material limitations.

The precise legal basis may be:

contract;

professional negligence;

tort;

statutory liability;

regulatory sanctions.

22. Liability of the AI Developer

The developer may face claims for:

defective design;

inadequate testing;

insufficient warnings;

known model limitations;

unsafe training methodology;

cybersecurity vulnerabilities;

inadequate updates;

failure to monitor post-deployment incidents.

The existence of third-party certification does not automatically eliminate developer responsibility.

23. Liability of the Deployer

The deployer may be liable where it:

uses AI outside its intended purpose;

ignores warnings;

fails to conduct appropriate risk assessment;

deploys the system in an unsuitable environment;

fails to train operators;

ignores incidents;

fails to maintain safeguards.

For example, an AI model may be safe for one clinical population but unsafe when deployed to a materially different population.

24. Causation

AI safety claims can involve multiple causal stages:

Design defect

↓

Inadequate safety testing

↓

False assurance/certification

↓

Deployment

↓

AI failure

↓

Human reliance

↓

Accident

↓

Damage

Courts may need to determine which failures materially contributed to the harm.

Potential intervening causes include:

misuse;

operator negligence;

unforeseeable environmental conditions;

cyberattack;

third-party modification;

failure to follow instructions.

25. Evidence

Important evidence includes:

Technical records

source/model documentation;

testing protocols;

validation results;

benchmark results;

failure rates;

error logs;

model versions;

system updates.

Assurance records

audit reports;

certification documents;

conformity assessments;

risk assessments;

test plans;

laboratory records;

corrective-action reports.

Governance records

AI policies;

risk registers;

DPIAs;

incident reports;

internal approvals;

board/management decisions.

Operational evidence

system logs;

user reports;

warnings;

maintenance records;

human overrides;

deployment conditions.

26. Defences

A defendant may argue:

No duty

The claimant was not within the scope of the defendant's responsibility.

Reasonable professional standard

Testing complied with accepted standards.

No defect

The AI met applicable safety requirements.

Unforeseeable misuse

The system was used outside its intended purpose.

Intervening cause

Another party caused the damage.

Warning

Users were adequately informed of limitations.

No causation

The alleged assurance failure did not cause the injury.

No recoverable damage

The claimant cannot establish legally compensable loss.

27. Remedies

Depending on the legal basis, remedies may include:

Compensation

For:

personal injury;

property damage;

financial loss;

other legally recognised damage.

Injunction

Preventing continued deployment of unsafe AI.

Corrective measures

software update;

model retraining;

additional testing;

safety restrictions;

additional human oversight.

Regulatory remedies

administrative penalties;

withdrawal from deployment;

corrective orders;

conformity-assessment consequences.

Governance remedies

independent audit;

new risk assessment;

incident investigation;

enhanced monitoring.

28. Comparative Case Table

29. Practical Legal Test for an AI Safety Assurance Claim

A court or regulator can examine the following sequence:

1. Identify the AI system

What does it do and in what environment?

2. Identify the safety risk

What harm could reasonably result from failure?

3. Identify the responsible actor

Developer, deployer, auditor, certifier, manufacturer or another entity?

4. Identify the assurance obligation

Contractual, statutory, regulatory, professional or tortious?

5. Examine the methodology

Were appropriate tests performed?

6. Examine foreseeable risks

Were known and reasonably foreseeable failure modes assessed?

7. Examine the data

Were validation and testing datasets representative?

8. Examine human oversight

Could humans detect and correct unsafe outputs?

9. Examine cybersecurity

Was malicious manipulation reasonably considered?

10. Examine post-deployment monitoring

Were incidents and model drift monitored?

11. Establish breach

Did the assurance provider/developer/deployer fall below the applicable standard?

12. Establish causation

Did the assurance failure materially contribute to the damage?

13. Establish damage

What legally recoverable harm occurred?

14. Assess defences

Was there misuse, intervening conduct, unforeseeable circumstances or adequate warning?

15. Determine remedy

Compensation, injunction, corrective testing, withdrawal, retraining, audit or regulatory action.

30. Key Distinction: Safety Assurance Is Not a Guarantee

An important legal principle is that certification or assurance does not necessarily mean that an AI system is guaranteed never to fail.

The relevant question is generally whether the responsible actor:

complied with its legal obligations;

used reasonable professional methods;

identified foreseeable risks;

performed adequate testing;

communicated material limitations;

maintained appropriate safeguards.

Thus:

AI failure ≠ automatically negligent certification.

But equally:

Certification ≠ automatic immunity from liability.

The court must examine the actual assurance process.

31. Overall Legal Position

AI safety assurance claims in Europe are likely to become increasingly important as AI moves into:

healthcare;

autonomous transport;

industrial systems;

financial services;

employment;

critical infrastructure;

public administration;

consumer products.

The strongest existing legal principles come from the interaction of EU product liability, the EU AI Act, professional negligence, GDPR and fundamental-rights jurisprudence.

The most useful cases are Boston Scientific, Skov Æg, O'Byrne, Donoghue, Henderson, Caparo, SCHUFA, Dun & Bradstreet, Wirtschaftsakademie, Fashion ID, Österreichische Post and Schrems II.

The central legal question can ultimately be expressed as:

Was the AI system subjected to the level of safety testing, validation, monitoring, transparency and professional care that the applicable law and circumstances reasonably required, and did any failure in that assurance process cause legally compensable harm?

Where the answer is yes, liability may potentially extend beyond the AI developer to deployers, manufacturers, professional assessors, testing laboratories, conformity-assessment bodies or other actors, depending upon their respective legal duties and the causal connection between their conduct and the resulting harm.