1. What Makes Banking AI "High Risk" Ethical?

Under EU Artificial Intelligence Act - Regulation 2024/1689 - applicable from 2 Aug 2026, Article 6 + Annex III:

All AI used by banks for:

• Credit scoring / Loan approval / Creditworthiness assessment
• Risk assessment and pricing of life and health insurance
• Transaction monitoring for AML / Fraud

Is classified as High-Risk AI. Means:

Banks must ensure AI systems comply with privacy laws but remain transparent enough for decisions to be explained and challenged. Rather than relying solely on new AI-specific laws, banks should recognise that many existing legal obligations already apply.

UK Report Jan 20 2026:

Published on 20 January 2026, report warns regulators are taking too cautious wait-and-see approach despite more than three-quarters of UK financial services firms now use AI in core operations. UK financial law already assumes firms remain responsible for tools they use. AI is not treated as independent decision-maker. FCA expects firms to take responsibility for outcomes regardless of...

2. Six Ethical Obligations - With Regulatory Source

Obligation 1: Fairness / Non-Discrimination - No Disparate Impact

• US: Equal Credit Opportunity Act - ECOA prohibits discrimination on race, color, national origin, sex, marital status, age, receipt of public assistance. Algorithmic underwriting models that rely on ZIP code, education, cash-flow behaviors may inadvertently create impermissible disparities. CFPB has made it clear financial institutions remain liable.
• India: RBI Draft Guidance on Regulatory Principles for Model Risk Management 2025 + Article 14 Constitution + Fair Lending.

Obligation 2: Transparency & Explainability - Right to Explanation

• EU AI Act Art 13, 86 - Borrower has right to explanation if loan denied by AI - Meaningful information about logic, parameters.
• RBI: Model must be explainable, not black box.

Obligation 3: Accountability & Human Oversight

• EU AI Act Art 14 - Human oversight mandatory - Human must be able to override AI.
• Cannot delegate to AI vendor - Bank remains liable.

Obligation 4: Privacy & Consent

• DPDP Act India takes full effect 13 May 2027 + GDPR + CCPA
• Bank must ensure prospect of customer data being used for AI is disclosed in contractual arrangements with each new customer and express consent obtained

Obligation 5: Model Risk Governance

• RBI: Enterprise-wide model governance framework - Independent validation, bias testing, drift monitoring, audit trail.
• EU: Conformity assessment, CE marking for high-risk AI.

Obligation 6: No Hallucination / Professional Duty

• Supreme Court of India in Pooja Ramesh Singh vs J&K Bank Ltd - Civil Appeal No. 11950 of 2025 - 2 July 2026 - Justice Pamidighantam Sri Narasimha - Established zero-tolerance policy toward hallucinated case law and set guidelines for responsible use of AI in legal profession. Before citing judgment advocate has professional duty to verify authenticity from reliable legal sources.
• Applied to banks: If bank uses GenAI for legal notice, recovery letter with fake case law, bank liable.

3. Case Laws - Where Banks Paid Price

A. Disparate Impact - Origin Case

Griggs vs Duke Power Company, US Supreme Court 1971 - 401 US 424

• First used term disparate impact.
• Court ruled under Title VII, even facially neutral practice that disproportionately harms protected group is illegal unless justified by business necessity.
• Applied to AI: Even if AI model does NOT use gender/race, if it uses ZIP code that correlates with race and denies loans disproportionately, it is disparate impact discrimination. Bank must prove model necessary and no less discriminatory alternative.

This is foundation for all AI lending cases.

B. Apple Card / Goldman Sachs Gender Discrimination - 2019-2021

Scott and Scott LLP - When AI Creates Adverse Impacts Based Upon Gender

Facts: David Heinemeier Hansson tweeted Apple Card gave him 20x credit limit than his wife despite joint taxes, same address. Steve Wozniak same. NYDFS investigated Goldman Sachs, issuer of Apple Card.

Goldman response:

Goldman Sachs responded indicating it is careful not to discriminate and does not even know gender of applications it processes.

Goldman has rejected notion that gender bias learned by AI solution and stated likely cause is credit history differences

Result: NYDFS 2021 - No intentional discrimination found, but ordered Goldman to improve model explainability. Lesson: "We don't collect gender" is NOT defence. If AI learns proxy for gender via credit history, bank liable. US law prohibits collection and use of gender-related data in non-mortgage credit models - So you cannot even test bias properly - Catch-22.

C. Earnest Operations LLC - Massachusetts AG $2.5 Million Settlement - 10 July 2025 - Most Important AI Bias Case

On July 10 2025 Massachusetts Attorney General Office announced $2.5 million settlement with loan provider Earnest Operations LLC to resolve allegations of unlawful discrimination in student loan decisions.

Practices included use of artificial intelligence models and manual underwriting that AGO alleged led to disparate impact harm to Black, Hispanic, and non-citizen applicants and borrowers. Used cohort default rate CDR, a data point...

Facts: Earnest's AI model used educational institution's Cohort Default Rate + other education data as variable. CDR highly correlated with race - HBCUs have higher CDR. Model denied Black, Hispanic applicants at 2x rate.

Settlement:

• $2.5m + must remove CDR, modify AI, conduct bias audit, report to AG for 3 years
• First US case where AG explicitly called AI model itself discriminatory under state fair lending + UDAP
• Applies to all banks using alternative data - e.g., college, employment, social media.

D. Bunq vs De Nederlandsche Bank - Dutch Administrative High Court - 2023 - AI in AML Allowed

Decision by Dutch Administrative High Court to allow bank to use artificial intelligence in transaction monitoring processes. Court said central bank was wrong to ban Bunq from using AI in screening clients. DNB had accused Bunq of failing to comply with AML laws that require banks to determine risk profiles

Facts: Neo-bank Bunq used AI for transaction monitoring instead of rule-based. DNB said not compliant with AMLD.

Court Held: Use of AI is permitted if bank can explain logic, ensure human oversight, and prove effectiveness. Bank not required to use traditional rules. First EU case endorsing AI in AML - Provided ethical obligations met - Explainability + Accountability.

E. Turner vs Nuance Communications, Inc. - 735 F Supp 3d 1169 N.D. Cal 2024 and Gladstone vs Amazon Web Services 739 F Supp 3d 846 W.D. Wash 2024

Plaintiffs claimed AI use by banks violated California Invasion of Privacy Act. Sued AI service providers under California Penal Code 631(a) prohibits tapping phone calls without consent

Facts: Banks used AI voice analytics (Nuance, AWS) to analyse customer service calls without consent.

Held: Bank using AI vendor to listen/analyse calls = wiretap if no consent. Bank cannot hide behind vendor. Privacy breach.

India Equivalent: Under DPDP Act 2023, bank using AI chatbot that records calls without consent = breach.

F. HSBC Beats Bias Claims - Employment Tribunal UK - Aug 2026 - AI-Generated Advice Rejected

HSBC Beats Bias Claims From Worker Who Relied On AI - Employment tribunal struck out staff member's race and disability discrimination claims against HSBC, finding arguments based in part on AI-generated advice showed...

Facts: Employee used ChatGPT to draft discrimination claim against HSBC.

Court: AI-generated legal argument without verification is not evidence. Similar to Pooja Ramesh Singh vs J&K Bank - Court imposed costs.

Bank Learning: If bank's HR uses AI to screen CVs - Amazon case 2018 - Amazon scrapped AI recruiting tool after discovering tool biased against women because trained on years hiring data dominated by male applicants. If bank uses similar tool, liable.

4. Consequences of Breach of Ethical AI Obligation

1. Regulatory: EU AI Act fines up to 7% global turnover or EUR 35m, US CFPB fine, RBI penalty
2. Loan Unenforceable: If discriminatory AI loan, borrower can claim loan void under ECOA, Fair Housing Act
3. Class Action: Turner, Gladstone style privacy class action
4. Reputational: Apple Card, DWS, Vanguard cases
5. Director Personal Liability: Under Sec 172 CA 2006 UK and Sec 166 Companies Act India - Duty to act with care includes ensuring AI not biased

5. Compliance Checklist for Bank Board

1. Bias Audit Before Deployment: Test AI on protected groups - Use 4/5th rule, disparate impact ratio
2. Model Risk Management per RBI draft: Independent validation team, not data science team
3. Human in Loop: No fully automated rejection - Human must review if AI denies
4. Explainability: Provide adverse action notice - e.g., "Declined due to low cash flow stability, not credit score alone" - CFPB requirement
5. Consent & Disclosure: Update privacy notice - "We use AI for credit decisions, fraud monitoring"
6. Vendor Liability: Contract with AI vendor must have EU-only data processing, strict deletion protocols, indemnity if AI service fails - European Bank Case Study OpenAI advisory
7. No Over-Reliance on AI Legal Advice: Verify case law - as per Pooja Ramesh Singh judgment