Conversational AI Claims

1. Meaning

Conversational AI claims are legal claims arising from the design, operation, output, use, or commercial deployment of AI systems that communicate with people through natural-language conversations.

Examples include:

• AI chatbots giving false or harmful information;
• AI assistants making misleading statements;
• chatbot discrimination or biased recommendations;
• unauthorized collection or use of conversation data;
• AI impersonation or voice cloning;
• unauthorized use of copyrighted material;
• automated customer-service decisions;
• financial or medical guidance supplied by AI;
• contractual disputes involving chatbot representations;
• deceptive AI-generated advertisements;
• reputational harm caused by AI-generated statements; and
• platform liability for AI-generated content.

Important qualification: Indian law does not presently recognize “conversational AI claims” as one independent statutory cause of action. Such claims generally have to be fitted into existing causes of action involving contract, consumer protection, negligence, privacy, copyright, defamation, intermediary liability, product/service liability and constitutional rights.

The recent Supreme Court decision in Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. (2026 INSC 668) is particularly significant because it directly recognizes the problem of AI hallucination and insists upon human control and verification when AI is used in adjudication. 

2. What Can Give Rise to a Conversational AI Claim?

A claim may arise from several different forms of AI conduct.

A. False or hallucinated information

A chatbot may confidently provide information that is factually incorrect.

For example:

A consumer asks an AI assistant about a financial product and receives incorrect information that causes financial loss.

Possible claims may involve:

• negligence;
• deficiency in service;
• misrepresentation;
• consumer protection;
• breach of contract.

B. AI-generated defamatory statements

An AI system may generate a false statement concerning an identifiable person.

Potential issues include:

• defamation;
• publication;
• attribution;
• foreseeability;
• responsibility of the developer or deployer;
• intermediary liability.

The difficult question is:

Who legally published the AI-generated statement?

Indian law has not yet developed a comprehensive AI-specific answer.

C. Privacy and conversation-data claims

Conversational AI systems can process:

• names;
• telephone numbers;
• addresses;
• financial information;
• employment information;
• health information;
• preferences;
• search history;
• voice recordings;
• biometric information;
• inferred characteristics.

The constitutional foundation for informational privacy is particularly important after K.S. Puttaswamy.

D. Consumer claims

A consumer may complain that an AI service:

• failed to perform as promised;
• supplied materially false information;
• caused financial loss;
• concealed important limitations;
• made misleading representations;
• improperly denied a service;
• generated defective outputs.

The Consumer Protection Act, 2019 can become relevant where the AI system forms part of a consumer-facing product or service.

E. Copyright claims

Conversational AI may:

• reproduce copyrighted text;
• summarize protected material;
• generate substantially similar content;
• use copyrighted material in training;
• provide copyrighted material to users.

These issues raise questions under the Copyright Act, 1957 concerning reproduction, communication, adaptation, fair dealing and authorization.

3. Core Legal Issues

The major legal questions in conversational AI claims are:

1. Who is legally responsible for the AI output?
2. Was the output reasonably foreseeable?
3. Was the user warned about limitations?
4. Was there human supervision?
5. Did the AI provider owe a duty of care?
6. Was personal data lawfully processed?
7. Was the AI output defamatory?
8. Did the output infringe copyright?
9. Did the service contract allocate responsibility?
10. Was the AI system marketed deceptively?
11. Did the consumer reasonably rely upon the output?
12. Did that reliance cause legally recoverable damage?

4. Conversational AI and Contract Law

A chatbot can become relevant to contract law in two different ways.

First: AI as the contracting interface

A consumer may enter into a transaction through a chatbot.

For example:

“Your subscription has been cancelled and you will receive a refund.”

If the consumer reasonably relies upon that representation, a dispute may arise concerning whether the statement binds the company.

Second: AI output as representation

An AI assistant may make representations concerning:

• price;
• warranty;
• refund;
• delivery;
• eligibility;
• contractual rights;
• service availability.

The key issue becomes whether the AI was authorized to make the representation.

A company may attempt to argue that:

“The chatbot's response was generated automatically and does not constitute a contractual promise.”

The enforceability of that defence will depend upon the contract, the chatbot's role, authorization, representations made to the user and applicable consumer law.

5. Conversational AI and Consumer Protection

The Consumer Protection Act, 2019 may become relevant where an AI service is supplied to consumers.

Potential claims may involve:

Deficiency in service

If the AI-enabled service fails to perform the promised service.

Unfair trade practice

Where AI is used to make misleading representations or deceptive commercial claims.

Product liability

Where AI is integrated into a product and the statutory requirements for product liability are satisfied.

Misleading advertisements

Where businesses use AI-generated conversational representations to advertise goods or services deceptively.

Unfair contracts

Where standard AI-service terms impose excessively one-sided obligations or exclusions.

6. Conversational AI and Privacy

Privacy is one of the most important areas.

A conversational AI system may continuously process the content of conversations and create inferences from them.

The Supreme Court in K.S. Puttaswamy (Retd.) v. Union of India recognized privacy as a fundamental right and specifically recognized informational and decisional dimensions of privacy.

Therefore, AI-related claims may concern:

• collection;
• storage;
• profiling;
• secondary use;
• disclosure;
• retention;
• deletion;
• security;
• automated inference.

7. Important Case Laws

1. Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd.

2026 INSC 668

This is currently the most directly relevant Indian Supreme Court authority concerning AI hallucination.

The NCLT and NCLAT relied upon purported judicial authorities that were subsequently found to be nonexistent, fabricated or incorrectly attributed and generated through AI.

The Supreme Court set aside the decisions.

It declared a zero-tolerance approach toward production, citation or reliance upon AI-generated fake precedents without verification. The Court also emphasized maintaining human control over adjudication.

Importance for conversational AI claims

This case establishes an important principle:

AI-generated information cannot automatically be treated as reliable simply because it is presented confidently or appears authoritative.

It is particularly relevant to claims involving:

• hallucinated AI outputs;
• professional reliance on AI;
• verification duties;
• human oversight;
• AI accountability.

The case does not, however, create a general civil cause of action against every AI provider for every hallucination.

8. Justice K.S. Puttaswamy (Retd.) v. Union of India

(2017) 10 SCC 1

A nine-judge Constitution Bench recognized privacy as a fundamental right under the Constitution.

The judgment recognized privacy in dimensions including:

• bodily privacy;
• informational privacy;
• decisional autonomy;
• personal liberty and dignity.

The Court also held that privacy is not absolute and restrictions must satisfy constitutional requirements.

Importance for conversational AI

This case provides the constitutional foundation for claims concerning:

• collection of private conversations;
• AI profiling;
• processing of sensitive information;
• surveillance through conversational systems;
• unauthorized disclosure;
• informational autonomy.

It is therefore a foundational authority whenever a conversational AI system processes personal information.

9. Karmanya Singh Sareen v. Union of India

Delhi High Court, 23 September 2016

This case concerned WhatsApp's proposed sharing of user information with Facebook.

The petitioners challenged the proposed change in privacy practices and sought protection against sharing and exploitation of user data.

The Delhi High Court noted the contractual nature of the relationship between users and the platform and issued directions concerning deletion and non-sharing of existing data for users who deleted their accounts.

Importance for conversational AI

The case is highly relevant by analogy to AI chatbot services because conversational AI also operates through:

• terms of service;
• privacy policies;
• collection of user information;
• processing of communications;
• potential sharing with affiliated entities.

It demonstrates how contractual terms and privacy expectations can intersect in technology-platform disputes.

10. Shreya Singhal v. Union of India

(2015) 5 SCC 1

The Supreme Court considered intermediary liability and online speech under the Information Technology Act.

Section 66A was struck down, while the Court considered the constitutional framework applicable to online expression and intermediary regulation.

Importance for conversational AI

Conversational AI creates difficult intermediary questions.

For example:

A user asks a chatbot to generate defamatory or unlawful content.

Questions include:

• Is the AI provider merely an intermediary?
• Is it an active creator?
• Does Section 79 safe harbour apply?
• Did the provider have sufficient control over the generation?
• Does the provider's moderation system affect liability?

Shreya Singhal provides an important constitutional and intermediary-liability framework, although it predates generative AI.

11. MySpace Inc. v. Super Cassettes Industries Ltd.

2016 SCC OnLine Del 6382

The Delhi High Court considered intermediary liability concerning copyrighted material uploaded by users to an online platform.

The Court considered the interaction between:

• Section 79 of the Information Technology Act;
• Section 51 of the Copyright Act;
• intermediary knowledge;
• identification of infringing material;
• takedown obligations.

Importance for conversational AI

The reasoning is relevant where an AI platform:

• receives user prompts;
• generates content;
• hosts user conversations;
• distributes AI-generated material;
• processes copyrighted material.

It provides an important starting point for deciding whether an AI service should be treated as a passive intermediary or an active participant in creating or distributing content.

12. Indian Performing Right Society Ltd. v. Eastern India Motion Pictures Association

(1977) 2 SCC 820

The Supreme Court considered copyright in musical works incorporated into cinematographic films.

The decision is important for distinguishing different copyright interests and explaining how rights may be exploited commercially.

Importance for conversational AI

Conversational AI may involve several layers of intellectual-property rights:

• training data;
• literary works;
• musical works;
• databases;
• generated outputs;
• user-created material.

The case therefore helps demonstrate why ownership of one component does not automatically establish ownership of every resulting commercial use.

13. R.G. Anand v. Deluxe Films

(1978) 4 SCC 118

The Supreme Court established the famous distinction between idea and expression in copyright law.

Copyright protects expression rather than ideas themselves.

Importance for conversational AI

This principle becomes relevant when an AI system generates text that resembles existing works.

The question is not simply:

“Did the AI discuss the same idea?”

but:

“Did the AI reproduce protected expression to an extent prohibited by copyright law?”

This makes R.G. Anand an important foundational case for AI-generated text disputes.

14. ANI Media Pvt. Ltd. v. OpenAI

The Delhi High Court litigation involving ANI Media Pvt. Ltd. and OpenAI is one of the most important emerging Indian disputes concerning generative AI and copyright.

ANI alleged unauthorized use of its news content in connection with AI development and also raised concerns concerning outputs attributed to ANI.

The litigation illustrates the emerging questions concerning:

• AI training;
• copyrighted datasets;
• reproduction;
• fair dealing;
• AI-generated outputs;
• attribution;
• hallucinated information;
• jurisdiction;
• remedies.

The 2026 proceedings are particularly significant because Indian courts are now directly confronting the relationship between copyright law and generative AI rather than merely applying traditional online-content principles.

Importance

This litigation demonstrates that conversational AI disputes can extend beyond the output and concern the entire AI lifecycle, including training data and commercial deployment.

15. Conversational AI and Negligence

A negligence-based claim may require consideration of:

Duty of care

Did the developer or service provider owe a duty to the user?

Breach

Was the AI system designed or deployed without reasonable safeguards?

Causation

Did the AI output actually cause the loss?

Foreseeability

Was the harm reasonably foreseeable?

Damage

Did the claimant suffer legally recognizable loss?

For example:

An AI financial assistant gives an incorrect investment instruction, the user relies upon it, and suffers a financial loss.

The claimant would potentially need to establish more than merely:

“The AI was wrong.”

The legal question is whether the provider's conduct amounted to a legally actionable breach and whether that breach caused the damage.

16. AI Hallucination Claims

An AI hallucination occurs when an AI system generates information that appears plausible but is factually false.

Examples:

• invented case laws;
• fabricated citations;
• nonexistent persons;
• false medical information;
• fictional financial regulations;
• fabricated news;
• false statements about a business.

Pooja Ramesh Singh is particularly important because the Supreme Court expressly confronted the consequences of hallucinated legal material and required human control and verification in judicial use.

However, the case should not be overstated.

It does not mean:

Every ordinary chatbot hallucination automatically creates civil liability.

Liability will still depend upon the nature of the service, the user's reliance, warnings, foreseeability, causation, contractual terms and applicable statute.

17. Human-in-the-Loop Principle

One emerging principle is human oversight.

A conversational AI system may be low-risk when used for:

• brainstorming;
• entertainment;
• ordinary conversation.

But greater human oversight may be necessary where AI is used for:

• legal decisions;
• medical recommendations;
• financial decisions;
• employment decisions;
• insurance;
• credit;
• education;
• public administration.

The Supreme Court's 2026 decision expressly emphasized human control in adjudication while acknowledging the legitimate use of AI as an aid.

18. AI Impersonation and Voice-Cloning Claims

Conversational AI can imitate a person's:

• voice;
• manner of speaking;
• identity;
• conversational style.

This may create claims involving:

• privacy;
• passing off;
• personality/publicity rights;
• defamation;
• fraud;
• consumer deception;
• contractual liability.

The legal challenge is particularly serious where the recipient reasonably believes that the AI-generated communication came from a real person.

Indian law is still developing a unified doctrine dealing specifically with AI voice cloning.

19. AI and Defamation

A conversational AI may generate:

“Person X committed fraud.”

If that statement is false and concerns an identifiable person, potential defamation issues arise.

Important questions include:

1. Who generated the statement?
2. Who supplied the prompt?
3. Who published it?
4. Was it displayed only to the user?
5. Was it subsequently shared publicly?
6. Did the provider know that the system could generate such statements?
7. Was reasonable correction available?

Traditional defamation principles therefore have to be adapted to an AI environment.

20. AI and Consumer Deception

A business may deploy a chatbot that tells consumers:

• “This product is government approved.”
• “You are guaranteed a refund.”
• “This medicine is completely safe.”
• “This investment cannot lose money.”

If such representations are false, consumer-protection consequences may arise.

The fact that the statement was generated by AI does not necessarily eliminate the business's responsibility for deploying the system commercially.

A company generally cannot automatically avoid responsibility simply by saying:

“The chatbot said it, not us.”

The precise result depends upon authorization, contractual arrangements, system design and applicable law.

21. Defences Available to AI Providers

Possible defences include:

1. Lack of causation

The claimant cannot show that the AI output caused the loss.

2. No reasonable reliance

The service clearly warned users that outputs could be inaccurate.

3. User misuse

The user employed the system in an unforeseeable manner.

4. Contractual limitation

The terms of service may allocate certain risks, subject to mandatory law and consumer-protection restrictions.

5. Intermediary protection

Where legally applicable, the provider may invoke intermediary protections.

6. Fair dealing

Copyright claims may face a fair-dealing defence depending upon the precise use.

7. Lack of substantial similarity

An AI output may not reproduce protected expression.

22. Evidence in Conversational AI Claims

Evidence can be especially important because AI outputs may change over time.

A claimant should preserve:

• complete conversation logs;
• screenshots;
• prompts;
• timestamps;
• system messages where available;
• model/version information;
• terms of service;
• privacy policy;
• warnings shown to the user;
• API records;
• payment records;
• evidence of reliance;
• evidence of resulting loss.

This is particularly important because the same prompt may not necessarily produce the same output later.

23. Remedies

Depending upon the cause of action, remedies may include:

Civil remedies

• damages;
• injunction;
• declaration;
• restitution;
• compensation;
• account of profits;
• correction or withdrawal;
• deletion of unlawfully processed data.

Consumer remedies

• refund;
• compensation;
• replacement;
• discontinuation of unfair practices;
• corrective measures.

Privacy remedies

• deletion;
• restriction or cessation of processing where legally available;
• compensation or other statutory remedies.

Copyright remedies

• injunction;
• damages;
• account of profits;
• delivery-up/destruction of infringing material where appropriate.

Defamation remedies

• damages;
• injunction in appropriate cases;
• correction/retraction where legally available.

24. Key Principles

The developing law of conversational AI claims can be summarized as follows:

1. AI output is not automatically legally reliable.
2. AI hallucination can create serious legal consequences in high-stakes contexts.
3. Human oversight becomes particularly important in consequential decision-making.
4. Privacy law applies to personal information processed through conversational systems.
5. Contractual terms can determine allocation of AI-related risk.
6. Consumer law can apply when AI is supplied as part of a consumer service.
7. Copyright disputes can concern both training inputs and generated outputs.
8. Intermediary liability principles may become relevant, but generative AI raises questions beyond traditional passive hosting.
9. A claimant normally needs to establish causation and legally recognizable damage.
10. Merely proving that an AI answer was incorrect does not automatically establish liability.
11. The identity of the legally responsible actor—developer, deployer, business, user or platform—is often the central question.
12. Pooja Ramesh Singh demonstrates the judiciary's insistence that AI assistance must not replace verification and human control in judicial decision-making. 

Conclusion

Conversational AI claims represent an emerging category of technology-related civil and regulatory disputes rather than a separate, fully developed cause of action under Indian law. Existing doctrines—particularly contract, consumer protection, negligence, privacy, copyright, defamation and intermediary liability—provide the principal legal tools.

The most important recent development is Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. (2026 INSC 668), where the Supreme Court set aside adjudicatory decisions tainted by AI-generated fake or hallucinated authorities and adopted a zero-tolerance approach to their unverified use. The judgment simultaneously recognized legitimate AI assistance while insisting upon human control in adjudication.

Thus, the emerging principle is not that AI itself is legally liable, but that the humans and organizations designing, deploying, controlling, relying upon and commercially benefiting from conversational AI may incur liability under existing legal doctrines when their conduct satisfies the relevant elements of a recognized claim.