Decommissioning criteria for unsafe models

Introduction

Decommissioning criteria for unsafe models refer to the legal, technical, and regulatory standards used to withdraw, disable, restrict, or permanently discontinue artificial intelligence models, biological simulation systems, predictive research models, and other computational tools that create unacceptable risks.

In biotechnology and scientific research, computational models are used to predict genetic interactions, simulate biological processes, analyse clinical trial information, and identify potential environmental risks. However, an unsafe model may generate inaccurate predictions, produce discriminatory results, expose confidential biological information, or contribute to harmful scientific decisions.

Decommissioning establishes a formal procedure for removing such models from operational use while preserving necessary research records, audit trails, and regulatory evidence. Its primary objectives are public safety, scientific reliability, accountability, and prevention of further harm.

Legal framework governing decommissioning in India

1. Information Technology Act, 2000

The Information Technology Act provides a legal framework for electronic records, cybersecurity, and certain forms of unauthorised access to computer systems.

Section 43 addresses specified unauthorised activities involving computer systems, while Section 66 provides criminal consequences where relevant activities are committed dishonestly or fraudulently.

Where an unsafe computational model creates cybersecurity vulnerabilities or enables unauthorised access to sensitive biological information, applicable provisions may become relevant.

2. Digital Personal Data Protection Act, 2023

The DPDP Act establishes obligations concerning lawful processing and protection of digital personal data.

Section 8 requires Data Fiduciaries to comply with applicable obligations and implement reasonable security safeguards. If an AI model processes personal or sensitive contextual information, organisations must examine the applicable legal basis, purpose limitations, and security requirements.

Decommissioning may become necessary where continued operation creates unacceptable risks to personal information or prevents compliance with applicable legal obligations.

The Act's commencement notifications and applicable rules must be considered when assessing specific obligations.

3. Environment (Protection) Act, 1986

The Environment (Protection) Act empowers the Central Government to take measures to prevent environmental harm and regulate activities involving environmental risks.

If a computational model is used to control an environmental or biotechnology intervention, inaccurate predictions may contribute to unsafe decisions. Regulatory authorities may require corrective measures, additional assessments, or discontinuation of activities under the applicable statutory framework.

4. Rules, 1989 governing genetically engineered organisms

The Rules for the Manufacture, Use, Import, Export and Storage of Hazardous Microorganisms, Genetically Engineered Organisms or Cells, 1989 establish regulatory oversight for specified biotechnology activities.

The rules provide a biosafety framework involving institutional and governmental regulatory committees. Where computational models support research involving genetically engineered organisms, their outputs should not replace mandatory biosafety assessments or regulatory approvals.

Major criteria for decommissioning unsafe models

1. Scientific unreliability

A model may require withdrawal when independent validation demonstrates that its predictions are consistently inaccurate or scientifically unreliable. This is particularly important in drug discovery, genetic research, and clinical decision-making.

2. Biological and environmental risks

Models used to evaluate genetically engineered organisms may produce misleading safety assessments. If their outputs could contribute to ecological damage or unsafe biological interventions, continued use should be reconsidered.

3. Privacy and data protection violations

A model should be restricted or withdrawn when it unlawfully exposes identifiable genetic information, medical records, or confidential research datasets.

4. Cybersecurity vulnerabilities

Models containing unresolved security weaknesses may expose research databases, laboratory systems, or institutional networks to unauthorised access.

5. Discriminatory or misleading outputs

AI systems used in healthcare and biotechnology may produce biased predictions because of incomplete or unrepresentative training datasets. Serious and uncorrectable bias may justify suspension or decommissioning.

6. Regulatory non-compliance

A model may need to be withdrawn when its operation violates applicable statutory requirements, regulatory approvals, contractual restrictions, or institutional biosafety policies.

Decommissioning procedure

A responsible decommissioning framework should involve the following stages:

Risk identification: Identify scientific, environmental, privacy, cybersecurity, and operational risks.

Independent assessment: Conduct technical and regulatory evaluation to determine the severity of the problem.

Temporary suspension: Restrict access or suspend operational use while the risks are investigated.

Regulatory notification: Inform the relevant authority where required by applicable law, approval conditions, or contractual obligations.

Data preservation: Preserve necessary research records, audit logs, validation reports, and regulatory evidence.

Permanent withdrawal: Disable the model, remove operational access, revoke associated credentials, and prevent unauthorised reuse.

Verification: Conduct an independent review to confirm that the model is no longer being used in the prohibited operational environment.

Post-decommissioning review: Document the reasons for withdrawal and establish corrective measures for future systems.

Decommissioning does not necessarily require destroying every research record. Organisations may need to preserve documentation for audits, litigation, regulatory investigations, and scientific reproducibility.

Relevant case laws

There are limited Indian judicial decisions directly addressing decommissioning of unsafe AI or biological models. However, the following judgments provide relevant legal principles.

1. Vellore Citizens Welfare Forum v. Union of India (1996)

The Supreme Court recognised the precautionary principle as an essential component of Indian environmental law. The judgment supports preventive action where scientific uncertainty creates a possibility of serious environmental harm. This principle is relevant when unsafe predictive models influence ecological or biotechnology interventions.

2. A.P. Pollution Control Board v. Prof. M.V. Nayudu (1999)

The Supreme Court emphasised the importance of scientific expertise and precaution in environmental decision-making. Its reasoning supports independent validation and reassessment of computational models used for environmental risk prediction.

3. Justice K.S. Puttaswamy v. Union of India (2017)

The Supreme Court recognised privacy as a fundamental right under Article 21. This judgment is relevant when computational models process identifiable genetic, medical, or other personal information. Models creating serious privacy risks may require restriction or redesign.

4. Shreya Singhal v. Union of India (2015)

The Supreme Court struck down Section 66A of the Information Technology Act because it violated freedom of speech and expression. Although it was not an AI safety case, the judgment demonstrates that restrictions on digital technologies must have a valid legal basis and comply with constitutional requirements.

These judgments provide analogous environmental, privacy, and constitutional principles rather than direct rulings on AI model decommissioning.

Challenges and compliance requirements

Decommissioning unsafe models creates several practical and legal challenges:

Determining the appropriate threshold of unacceptable risk.

Distinguishing temporary technical errors from fundamental model failures.

Preserving research records while preventing unsafe reuse.

Identifying responsibility among developers, researchers, and deploying institutions.

Managing financial losses resulting from model withdrawal.

Ensuring that discontinued models are not secretly reintroduced into operational systems.

Institutions should establish written model governance policies, independent safety audits, validation standards, incident-reporting procedures, and clear responsibility for approving suspension or withdrawal.

Conclusion

Decommissioning criteria for unsafe models are essential for maintaining scientific reliability, environmental protection, data security, and institutional accountability. Indian legislation, including the Information Technology Act, 2000, Digital Personal Data Protection Act, 2023, and Environment (Protection) Act, 1986, provides relevant legal foundations. Although India does not currently have a single comprehensive statute specifically governing the decommissioning of unsafe AI or biological models, existing environmental, privacy, and cybersecurity principles support preventive regulatory action. A transparent framework involving independent assessment, temporary suspension, evidence preservation, permanent withdrawal, and post-decommissioning verification can reduce technological and scientific risks.