LEGAL LIABILITY OF AI-AUTONOMOUS SYSTEMS IN HEALHTCARE & MEDICAL NEGLIGENCE : Author: Nikita Patidar
The convergence of advanced machine learning algorithms, deep neural networks and autonomous surgical robotics into healthcare is fundamentally redefining clinical medicine. However, when autonomous clinical tools misdiagnose pathologies or cause physical injury, traditional legal frameworks encounter severe conceptual hurdles. In India, medical liability rests on human fault doctrines, principally the Bolam Test under common law, deficiency of service under the Consumer Protection Act, 2019 and criminal negligence under Section 106 of the Bharatiya Nyaya Sanhita, 2023. This paper provides an exhaustive research analysis on how AI autonomy disrupts these legal foundations
ARTICLE
ABSTRACT
The convergence of advanced machine learning algorithms, deep neural networks and autonomous surgical robotics into healthcare is fundamentally redefining clinical medicine. However, when autonomous clinical tools misdiagnose pathologies or cause physical injury, traditional legal frameworks encounter severe conceptual hurdles. In India, medical liability rests on human fault doctrines, principally the Bolam Test under common law, deficiency of service under the Consumer Protection Act, 2019 and criminal negligence under Section 106 of the Bharatiya Nyaya Sanhita, 2023. This paper provides an exhaustive research analysis on how AI autonomy disrupts these legal foundations. It critically evaluates the 'black box' evidentiary barrier under the Bharatiya Sakshya Adhiniyam, 2023, establishes a tripartite liability matrix across clinicians, hospitals and software developers, conducts a comparative regulatory review with the EU AI Act and US FDA rules and proposes a comprehensive model legislative framework featuring mandatory algorithmic audits and a compulsory No-Fault Compensation Insurance Scheme.
Keywords: Artificial Intelligence, Medical Negligence, Bolam Test, BNS Section 106, Product Liability, Black Box Problem, Bharatiya Sakshya Adhiniyam, No-Fault Compensation, SaMD.
1. INTRODUCTION
Medical decision-making is rapidly transitioning from human-centered clinical intuition to data-driven algorithmic execution. In contemporary Indian healthcare, Artificial Intelligence (AI) systems are deployed across diverse medical domains: from deep learning tools detecting diabetic retinopathy and oncological lesions in rural diagnostic centers, to autonomous surgical robotics performing high-precision laparoscopic procedures in metropolitan tertiary hospitals.
Despite their immense therapeutic potential, AI medical devices are legally fundamentally distinct from conventional medical instruments. Traditional diagnostic equipment (such as X-ray systems, electrocardiograms, or scalpels) functions strictly as passive mechanical instruments under direct human control. In contrast, Machine Learning algorithms particularly deep neural networks utilizing continuous learning possess dynamic autonomy. They process vast clinical datasets, identify intricate sub-visual patterns and generate clinical recommendations without disclosing the exact mathematical weightage behind their logic. This autonomous, opaque operational model creates an acute legal challenge: when an autonomous AI system causes patient injury or death, traditional jurisprudence struggles to assign legal accountability, as existing tort, consumer and criminal laws are built entirely upon the premise of human agency and culpable fault.
2. INDIAN STATUTORY FRAMEWORK & LANDMARK CASE LAW
In India, medical liability is governed across three distinct legal avenues. Applying these traditional regimes to AI-induced medical harm reveals severe enforcement gaps:
A. Civil Liability under Tort Law & Common Law
Under Indian civil jurisprudence, a plaintiff claiming damages for medical negligence must establish three indispensable conditions laid down in Laxman Balkrishna Joshi v Trimbak Bapu Godbole (AIR 1969 SC 128):
· Duty of Care: The existence of a legally recognized duty owed by the medical practitioner to the patient.
· Breach of Duty: A breach of that duty by failing to attain the standard of a reasonably competent practitioner.
· Direct Causation: Consequential physical or financial damage directly caused by the breach.
In cases involving autonomous AI, proving direct causation (causa causans) becomes exceedingly difficult due to multi-party interactions between software code, network infrastructure, hardware maintenance and human oversight.
B. Consumer Protection Act, 2019
Under Section 2(42) of the Consumer Protection Act (CPA), 2019, medical services are classified as a "service," enabling aggrieved patients to file claims before Consumer Disputes Redressal Commissions for "deficiency in service" (Section 2(11)). Crucially, Chapter VI of the CPA 2019 introduced statutory Product Liability (Sections 82–87), holding product manufacturers, service providers and sellers strictly liable for harm caused by defective products. This statutory framework offers a vital legal avenue for holding AI software developers strictly liable without requiring proof of clinical fault.
C. Criminal Liability under Bharatiya Nyaya Sanhita, 2023
Criminal liability for medical failure was historically prosecuted under Section 304A of the Indian Penal Code (IPC), now replaced by Section 106 of the Bharatiya Nyaya Sanhita (BNS), 2023. In the landmark decision of Jacob Mathew v State of Punjab ((2005) 6 SCC 1), the Supreme Court established that to fasten criminal liability upon a doctor, the prosecution must prove gross negligence or recklessness, exceeding mere lack of care or error of judgment. Attempting to prosecute a physician under Section 106 BNS for an algorithmic failure of complex software introduces an unwarranted risk of criminalizing clinical practice.
3. INAPPLICABILITY OF THE BOLAM TEST TO AUTONOMOUS AI
The standard of care in medical jurisprudence is determined by the historic Bolam Test, Bolam v Friern Hospital Management Committee [1957] 1 WLR 582), affirmed in India by Jacob Mathew (2005). Under Bolam, a doctor is not negligent if they act in accordance with a practice accepted as proper by a responsible body of medical professionals. The Bolam Test breaks down entirely when applied to autonomous AI across three core legal pillars:
· Duty of Care: Established legally through the doctor-patient relationship and statutory registration under the National Medical Commission (NMC) Act, 2019. AI algorithms lack legal personhood and cannot owe a direct common-law duty of care to patients.
· Standard of Skill & Care: Human doctors are evaluated against peers in their specialty. For AI, it is completely unresolved whether performance should be benchmarked against a human generalist, a specialist, an error-free gold standard or functional software specifications.
· Causation & Transparency (The 'Black Box' Barrier): Human doctors can explain their clinical reasoning under cross-examination. Deep learning neural networks operate as opaque 'black boxes', masking the exact mathematical pathways used to reach a diagnosis and obscuring legal causation.
4. TRIPARTITE LIABILITY ALLOCATION MATRIX
To prevent liability voids, legal responsibility for autonomous AI medical injuries must be allocated across three primary entities:
1. Clinician / Attending Physician Liability
The doctor acts as a "Learned Intermediary." Clinician liability splits into two distinct legal traps:
· Automation Bias (Over-Reliance): Occurs when a doctor blindly accepts an AI diagnosis without independent clinical verification. If an AI misdiagnoses an active tumor as benign and the doctor skips standard diagnostic checks, the clinician is directly negligent for breach of standard care.
· Turn-Away Risk (Under-Reliance): Occurs when a doctor unreasonably overrides a correct AI diagnostic warning, causing patient harm. As AI diagnostic accuracy exceeds human averages, overriding verified AI findings may eventually be legally recognized as substandard care.
2. Hospital & Healthcare Institutional Liability
Hospitals owe a non-delegable duty of care (Paschim Banga Khet Mazdoor Samity v State of West Bengal (1996)). Institutional liability arises from procurement negligence (deploying uncertified AI), failure to maintain hardware/software updates, or failing to properly train medical staff on algorithmic limitations.
3. AI Developer & Software Manufacturer Liability
Under Chapter VI of the Consumer Protection Act, 2019, software vendors face strict product liability. Developers are strictly liable if medical software fails due to defective training data, algorithmic bias, or coding defects, without needing to prove medical negligence.
6. EVIDENTIARY CHALLENGES UNDER BHARATIYA SAKSHYA ADHINIYAM (BSA), 2023
Proving liability in AI cases presents severe evidentiary hurdles under the Bharatiya Sakshya Adhiniyam (BSA), 2023:
· Electronic Evidence Admissibility (Section 63 BSA): Proving the authenticity and untampered state of AI decision logs, system memory and real-time inputs under Section 63 BSA requires strict technical audit trails.
· Expert Opinion (Section 45 BSA): Standard medical experts lack computer science training to testify on algorithmic code, while software engineers lack clinical expertise, creating an expert witness impasse in court.
7. REGULATORY ANALYSIS & COMPARATIVE INTERNATIONAL LAW
India regulates AI as Software as a Medical Device (SaMD) under the Medical Devices Rules, 2017 (CDSCO). However, CDSCO focuses purely on pre-market safety, leaving post-market civil liability unaddressed. Internationally:
· European Union (EU AI Act): Categorizes medical AI as "High-Risk," imposing mandatory algorithmic transparency, strict dataset quality rules, continuous human oversight and strict vendor liability under the updated EU Product Liability Directive.
· United States (FDA Framework): Implements Good Machine Learning Practice (GMLP) and 'Predetermined Change Control Plans' (PCCP) for adaptive AI, relying on state product liability torts to compensate injured patients.
8. PROPOSED STATUTORY REFORM & MODEL POLICY FRAMEWORK
To resolve these legal challenges, Parliament should enact specialized AI Healthcare Liability Guidelines comprising four core pillars:
1. Statutory Strict Product Liability for AI Vendors: Codify strict vendor liability for software bugs, dataset biases and system failures under the Consumer Protection Act, 2019.
2. Mandatory Algorithmic Transparency & Auditability: Require all medical SaMD deployed in India to maintain tamper-proof audit trails for legal admissibility under BSA 2023.
3. Clear Judicial Distinction for Clinicians: Protect doctors from criminal prosecution under Section 106 BNS when operating approved AI in good faith, restricting liability to gross clinical recklessness.
4. Compulsory No-Fault Compensation Insurance Fund: Establish a statutory insurance fund financed by AI vendors and healthcare providers to compensate patients rapidly without protracted litigation.
9. CONCLUSION
As artificial intelligence transitions from an advisory tool to an autonomous decision-maker in healthcare, India's medical jurisprudence must evolve. Relying on legacy doctrines like the Bolam Test or applying criminal sanctions under Section 106 BNS to clinicians operating complex software stalls medical innovation while leaving patients unprotected. By adopting a modern framework combining strict developer product liability, clinician accountability for automation bias and a compulsory No-Fault Compensation Scheme, India can foster healthcare innovation while safeguarding patient rights.
REFERENCES
· Bolam v Friem Hospital Management Committee [1957] 1 WLR 582.
· Laxman Balkrishna Joshi v Trimbak Bapu Godbole AIR 1969 SC 128
· Paschim Banga Khet Mazdoor Samity v State of West Bengal (1996) 4 SCC 37.
· Jacob Mathew v State of Punjab (2005) 6 SCC 1.
· Consumer Protection Act, 2019
· National Medical Commission Act, 2019