Ai Surveillance Misuse And Criminal Liability
Key Cases / Legal Developments in China on “AI Surveillance Misuse” (Face‑Swap / Deep Synthesis)
Case 1: Beijing Internet Court — First “AI Face‑Swap” (换脸) Software Infringement Cases
In June 2024, the Beijing Internet Court adjudicated two of the city’s first “AI‑换脸” software infringement lawsuits. The plaintiffs were two well-known short‑video creators (models/influencers) who claimed that a face‑swap app had used their video content to create face‑swap “templates” without their permission.
Claims: The plaintiffs argued that their personal information rights (especially their facial data and dynamic video) were violated because the defendant used their video footage, made templates via AI, and then offered those templates in the app for paid use by others. They also claimed a violation of their portrait (肖像) rights.
Court’s Ruling:
The court determined that the face‑swap app operator had indeed used the plaintiffs’ video footage to make “face‑swap templates.”
However, it held that there was no violation of the plaintiffs’ portrait (肖像) rights, because after the face swap, the identifiable facial features were replaced, and the template video no longer corresponded one-to-one to the original person in a way that the general public could reliably recognize.
The court did decide, though, that the operator had infringed the plaintiffs’ personal information rights (because the operator processed their facial video data without consent).
It also noted that a significant part of the defendant’s profit came from the labor / creative effort of the plaintiffs (their original video, facial expressions, lighting, wardrobe, etc.), and the defendant was “freeloading” on that input.
Remedy: The app operator was ordered to apologize and pay compensation for the plaintiffs’ personal‑information-related losses (economic and mental distress).
Significance: This case is vital — it sets a precedent in Chinese civil law for handling AI face‑swap templates. The court drew a distinction between portrait (肖像) rights and personal information rights. Even if the swapped face is not “clearly you,” the processing of your video data for AI synthesis still implicates personal-data protection.
Case 2: Intellectual‑Property / Personal-Info Guidance by Supreme People’s Court
The Supreme People’s Court (SPC) has issued clearer judicial guidance on “new‑type AI” cases (in 2025) under its intellectual‑property and personal-rights protection framework.
According to these guidelines:
Use of “AI face‑synthesis” (换脸) or “AI face‑cloning” technologies that process facial features (e.g., face recognition, feature extraction) may infringe both portrait rights and personal‑information rights, depending on how the face data is used.
For example, if a face-swap app uses a person’s video or facial data without permission to generate synthetic content, the user or developer may be liable for personal-data infringement even if the synthetic face is not clearly “recognizable” as the original person.
The court also emphasized the balance between protecting individuals’ rights and allowing reasonable use of AI technologies, so that legal protection does not completely stifle innovation — but misuse must be restrained.
Case 3: Civil Liability — Voice‑Cloning / AI Virtual Character Case (Shanghai Court)
In a Shanghai court (judgment made public via court-case summary), a user sued an AI‑company for unauthorized use of her voice in an “AI companion” app. The app allowed users to create AI characters (“virtual companion”) whose voice, avatar behavior, and responses were trained / generated using data submitted by users. The plaintiff claimed that her voice recording (uploaded earlier) was used without adequate consent, and was integrated into the AI character’s voice model.
Legal Issues:
Whether use of a real person’s voice to build an AI‑driven persona / virtual character without express permission constitutes an infringement of that person’s voice-right (voice identity, personal data).
Whether this use also constitutes misuse of her personal data (voice as biometric data).
Court Ruling:
The court found that the company’s virtual character system, by relying on the plaintiff’s voice data, had indeed misused her personal voice data without properly protecting her right to control that biometric “voice identity.”
It recognized a civil right to voice identity (or voice right) in the AI context.
The court awarded the plaintiff monetary compensation for the violation of her voice-right / personal data rights.
Significance: This case shows that Chinese civil courts are extending personal‑rights protection into the domain of AI voice synthesis, not just face synthesis. It confirms that voice data is a form of personal (sensitive) information deserving legal protection when AI is used.
Case 4: Hangzhou Internet Court — “AI Face‑Swap” App Infringement
In a case heard by the Hangzhou Internet Court, a short‑video artist (model) sued the developers of a “face‑swap” mobile app. She claimed that the app, via deep‑synthesis technology, took her video images (her face in certain styled videos) and allowed its users to “swap” their own faces into her video templates.
Court Findings:
The court agreed that the app operator used the plaintiff’s video (including her face, posture, stylized clothing, makeup) to build face‑swap templates.
The court held that such “information‑technology-based fabrication” (i.e., face synthesis) may violate 肖像权 (portrait rights) because the tool produces a “fake but realistic” version of her image in a form of her own motion / posture.
Remedy: The court ordered the app developer to issue an apology to the plaintiff and pay 5000 yuan in compensation.
Significance: This illustrates early Chinese judicial willingness to treat face-synthesis apps as infringing on portrait rights, especially when the original video creator’s content is used to derive the face‑swap templates.
Case 5: Beijing Internet Court — Large Volume of AI Face‑Swap / Personal Info Protection Cases (Statistics + Trends)
In a 2024 judicial summary, the Beijing Internet Court reported that between October 2023 and October 2024, it handled 113 personal‑information protection cases, of which “AI face‑swap” (AI 换脸) type cases formed a growing and noticeable portion.
Court Observations / Legal Reasoning:
Many of these cases involve the unauthorized collection, use, or publication of facial data, especially in synthesized video templates.
The court recognized that AI face-synthesis implicates deep personal information rights, due to the richness of facial-feature data (key-point extraction, feature fusion, expression mapping).
The court pointed out that in many “face‑swap” app cases, even if the resulting “swapped face” is not unmistakably identical to the original person, there may still be a violation of data / information rights because the original face data (video) was used without consent.
Significance: This is not a single litigated case but a judicial summary / trend report — but it is important because it shows systemic recognition by a specialized court (Beijing Internet Court) that “AI face-swap” technologies pose real, novel risks to personal data protection, and they are actively handling such cases.
Analysis: Key Legal and Liability Themes
Civil Liability More Advanced Than Criminal
Most publicly reported and court‑decided cases are civil law cases (information-rights, portrait‑rights, personal‑data rights), not criminal prosecutions.
This reflects how much of AI face‑swap misuse is being channeled into data protection and personality‑rights litigation, rather than “AI fraud” in a criminal context.
Distinction Between Portrait (肖像) and Information (Personal Data) Rights
Courts are carefully distinguishing when face-swapped content violates portrait rights vs when it violates personal data rights. Even when a swapped face is not clearly identifiable as a person, the use of facial data can still infringe data protection rights.
This distinction helps strike a balance: not all use of face-synthesis will be treated as “portrait identity theft,” but misuse of face data still can be unlawful.
Liability of Face‑Swap App Providers
Developers / operators of “换脸” (face‑swap) apps are being held liable, especially when they profit from templates derived from real people’s videos.
The “labor input” (original video production) of content creators is being recognized in valuation of harm — when someone uses your video to make a face-swap template, they are “free-riding” on your creative work / effort.
Emergence of Voice‑Right Protection
Beyond face data, courts are also protecting voice data when used in AI-driven virtual characters: voice cloning without consent is actionable.
This increases the scope of personal rights in AI-generated content.
Judicial & Regulatory Response to AI
The Supreme People’s Court is clarifying rules in the AI era, particularly around how personal‑rights protection applies to AI-generated content.
Specialized courts (like Internet Courts) are developing new jurisprudence for “AI personal data misuse,” which is critical for future litigation and regulation.
Limitations & Gaps
Few Public Criminal Judgments: There is a noticeable lack of widely published criminal court judgments in China specifically charging deepfake / face‑swap fraud as a crime (e.g., fraud + AI). Most cases reported involve civil claims.
Emerging Technology, Evolving Law: As AI technologies evolve, courts are still building legal doctrine. Some judgments are first‑instance; appeal outcomes may shape future law.
Data on Sentences / Penalties: In many civil cases, compensation amounts are modest (e.g., a few thousand yuan), which may not strongly deter large-scale misuse.
Enforcement vs Awareness: Even though courts are prosecuting civil misuse, there is still a gap in public awareness about how face-swap or voice‑clone apps might infringe your rights.
Conclusion:
While criminal prosecutions for AI surveillance misuse (deepfake fraud) in China are not yet broadly visible in published judgments, civil litigation is already active and shaping the law.
Face‑swap apps are being held accountable for infringing on personal information rights, and courts are issuing compensation and cease‑use orders.
Chinese courts are also extending personal-rights protection to voice data in AI contexts.
Judicial and regulatory bodies are recognizing that AI face/voice synthesis raises novel legal challenges, and are adapting existing personal-information and personality-rights frameworks to address them.
RELATED Blog
comments