Decommissioning criteria for unsafe models

Introduction

Decommissioning criteria for unsafe models refer to the legal, technical, and regulatory standards used to determine when a scientific, computational, artificial intelligence, biological, or technological model should be withdrawn from operation because it creates unacceptable risks.

Models are increasingly used in biotechnology, healthcare, environmental monitoring, drug discovery, genetic research, and automated decision-making. Although these models can improve accuracy and efficiency, they may produce unreliable predictions, discriminatory outcomes, incorrect scientific conclusions, or unsafe recommendations.

For example, an AI-based medical model may generate inaccurate diagnostic predictions because of biased training data or insufficient validation. Similarly, a computational biological model may produce unreliable research conclusions that could affect public health or environmental safety.

Decommissioning provides a mechanism for preventing further harm by suspending, withdrawing, replacing, or permanently retiring unsafe models. It also establishes accountability for organisations responsible for developing, deploying, and maintaining these systems.

Legal framework governing unsafe model decommissioning

India does not currently have one comprehensive statute specifically establishing decommissioning criteria for every category of unsafe computational or scientific model. However, several existing legal frameworks provide relevant regulatory principles.

1. Information Technology Act, 2000

The Act provides a legal framework for electronic records, cybersecurity, and certain forms of unauthorised access and misuse of computer systems.

Sections 43 and 66 address specified unauthorised acts involving computer resources. These provisions may become relevant when unsafe models result from unlawful access, manipulation, or interference with computer systems.

2. Digital Personal Data Protection Act, 2023

The Act establishes obligations concerning the processing and protection of digital personal data, subject to its applicable commencement provisions and rules.

Where an unsafe model processes personal information, organisations must consider applicable requirements concerning lawful processing and security safeguards.

3. Consumer Protection Act, 2019

The Act addresses defective goods, deficient services, unfair trade practices, and misleading advertisements. Where an AI-based or technological service causes legally recognised consumer harm, relevant provisions may become applicable.

4. Environment (Protection) Act, 1986

Where scientific or computational models are used in environmental management, unsafe decisions may create environmental risks. The Act provides a framework for environmental protection and regulatory intervention.

5. Medical and biotechnology regulations

Models used in medical diagnosis, clinical research, or biotechnology applications may also be subject to applicable requirements under the Drugs and Cosmetics Act, 1940, the New Drugs and Clinical Trials Rules, 2019, and relevant medical device regulations.

Meaning and importance of decommissioning

Decommissioning means formally withdrawing a model from active use when it no longer satisfies applicable safety, reliability, legal, or ethical requirements.

It may involve temporary suspension, restricted deployment, replacement with a validated model, or permanent retirement.

For example, a hospital may withdraw an AI diagnostic model after discovering that its predictions are unreliable for a particular patient population. Continuing to use the model without corrective action could create risks to patient safety.

The major objectives include:

Public safety: Preventing harm caused by unreliable model outputs.

Scientific reliability: Ensuring that research decisions are based on validated methods.

Legal accountability: Establishing responsibility for unsafe deployment.

Privacy protection: Preventing continued misuse of sensitive personal information.

Environmental protection: Avoiding harmful decisions based on inaccurate predictive models.

Institutional transparency: Documenting why a model was withdrawn and what corrective action followed.

Major criteria for decommissioning unsafe models

1. Accuracy and reliability failures

A model should be reviewed when repeated testing demonstrates that its performance falls below established safety or reliability requirements.

2. Algorithmic bias

If a model produces systematically discriminatory or unfair outcomes, its continued deployment may require suspension or retirement until the problem is properly addressed.

3. Data integrity problems

Models trained on inaccurate, incomplete, manipulated, or unauthorised datasets may produce unreliable results. Serious data integrity failures can justify decommissioning.

4. Cybersecurity vulnerabilities

A model may become unsafe when its infrastructure is exposed to serious security vulnerabilities, unauthorised modifications, or malicious interference.

5. Regulatory non-compliance

Models that fail to satisfy applicable statutory, licensing, approval, or safety requirements may need to be withdrawn.

6. Unacceptable health or environmental risks

Models producing recommendations that create serious risks to patients, communities, or ecosystems should undergo immediate safety review.

7. Lack of explainability and auditability

Where a model is used in a high-risk environment, an inability to investigate its decisions or verify its performance may create serious governance concerns.

Legal procedure for decommissioning

A responsible decommissioning process should follow a documented and transparent procedure.

Risk identification: Identify the safety problem through audits, complaints, performance monitoring, or regulatory investigations.

Technical assessment: Evaluate the severity, frequency, and possible consequences of model failures.

Temporary suspension: Restrict or suspend deployment where continued operation creates unacceptable risks.

Regulatory notification: Inform relevant authorities where required by applicable law.

Stakeholder communication: Notify affected users, institutions, and other relevant parties.

Data preservation: Preserve necessary records, audit logs, and evidence for investigation and legal compliance.

Permanent withdrawal: Retire the model when its risks cannot be adequately controlled.

Post-decommissioning review: Examine the causes of failure and establish safeguards to prevent recurrence.

Important case laws

Direct Indian judicial decisions specifically concerning the decommissioning of unsafe AI or computational models remain limited. However, the following cases establish principles relevant to safety, scientific reliability, accountability, and regulatory intervention.

1. A.P. Pollution Control Board v. Prof. M.V. Nayudu (1999)

The Supreme Court emphasised the importance of scientific expertise in environmental decision-making. It recognised that complex technical questions require reliable scientific assessment.

The judgment is relevant because unsafe environmental or biological models should not continue operating merely on the basis of unsupported technical claims. Their reliability must be evaluated through appropriate scientific scrutiny.

2. Vellore Citizens' Welfare Forum v. Union of India (1996)

The Supreme Court recognised the precautionary principle as an important part of Indian environmental law.

The principle supports preventive action where an activity creates a credible risk of serious environmental harm, even when complete scientific certainty is unavailable. By analogy, it supports withdrawing models that create unacceptable environmental risks.

3. K.S. Puttaswamy v. Union of India (2017)

The Supreme Court recognised privacy as a fundamental right under Article 21 and established that interference with privacy must satisfy legality, legitimate purpose, and proportionality.

This judgment is relevant when unsafe models process sensitive personal information. Where a model creates serious privacy risks, organisations may need to suspend or redesign its processing activities.

4. Jacob Mathew v. State of Punjab (2005)

The Supreme Court examined the standards governing medical negligence and professional responsibility. It discussed the importance of professional competence and appropriate standards of medical care.

The judgment is relevant by analogy when medical institutions rely on unsafe computational models. Technology should not replace professional responsibility, and institutions must ensure that clinical decisions satisfy appropriate standards of care.

5. Shreya Singhal v. Union of India (2015)

The Supreme Court examined provisions of the Information Technology Act and struck down Section 66A. It emphasised constitutional protections concerning freedom of speech and expression in the digital environment.

Although the case did not concern model decommissioning, it demonstrates that government regulation of digital technologies must remain within constitutional limits.

6. General Electric Co. v. Joiner (1997)

The United States Supreme Court held that courts may exclude expert opinions where there is an excessive analytical gap between the underlying scientific evidence and the conclusions drawn.

This principle is relevant to computational models because systems producing unsupported scientific conclusions may require rejection, reassessment, or withdrawal from evidentiary use.

Challenges in implementing decommissioning standards

Several difficulties arise when organisations attempt to withdraw unsafe models.

Unclear responsibility: Developers, operators, and service providers may dispute responsibility.

Commercial interests: Companies may resist withdrawing profitable models.

Technical dependence: Institutions may depend heavily on existing systems.

Evidence preservation: Removing a model without preserving relevant records may affect investigations.

Regulatory uncertainty: Different sectors may follow different safety requirements.

Replacement difficulties: A suitable validated alternative may not be immediately available.

Cross-border operations: Models deployed internationally may be subject to different legal requirements.

Safeguards and regulatory recommendations

Organisations should establish written model lifecycle management policies covering development, validation, deployment, monitoring, suspension, and retirement.

Independent audits should be conducted for high-risk models, particularly those used in healthcare, biotechnology, environmental management, and public administration.

Decommissioning decisions should be supported by documented evidence, clear risk assessments, and appropriate regulatory review. Organisations should also preserve relevant technical records and communicate significant safety concerns to affected stakeholders.

A model should not be withdrawn solely because of commercial inconvenience or unsupported allegations. Decisions must be evidence-based, proportionate, and consistent with applicable legal requirements.

Conclusion

Decommissioning criteria for unsafe models are essential for responsible technological and scientific governance. They ensure that unreliable, discriminatory, insecure, or harmful systems do not continue operating without appropriate safeguards. Indian constitutional, environmental, medical, and information technology laws provide important principles for regulating unsafe models, although specific lifecycle and retirement standards require further development. Clear accountability, independent validation, risk-based suspension, and documented withdrawal procedures can strengthen public safety and institutional responsibility.