The rapid integration of artificial intelligence into the clinical environment has outpaced the development of comprehensive legal and ethical frameworks, creating a "regulatory vacuum" that healthcare providers and policymakers are now scrambling to fill. In the latest installment of "The Business of Health with Chip Kahn," Dr. Michelle Mello, a professor of law at Stanford Law School and a professor of health policy at the Stanford University School of Medicine, joined host Chip Kahn to dissect the complex intersections of technology, accountability, and patient safety. As AI transitions from a speculative tool to a mainstay of diagnostic and administrative workflows, the central question shifts from whether the technology works to who is responsible when it fails.
Dr. Mello, who co-leads the Healthcare Ethical Assessment Lab for AI (HEAL-AI) at Stanford, emphasized that the "rules of the road" for medical AI are currently being written in real-time, often by the institutions deploying them rather than centralized federal authorities. This episode, the 13th in a specialized series focused on AI’s impact on the healthcare business, highlights the urgent need for a shift from reactive litigation to proactive governance.
The Governance Gap: Who Sets the Standard?
In the current landscape, the Food and Drug Administration (FDA) remains the primary federal gatekeeper for medical AI, specifically through its "Software as a Medical Device" (SaMD) framework. However, Dr. Mello pointed out that the FDA’s oversight is largely concentrated on the pre-market phase—evaluating whether a tool is safe and effective before it reaches the hospital. Once an AI tool is integrated into a health system, the oversight becomes fragmented.
The challenge lies in the "black box" nature of many deep-learning algorithms. Unlike traditional medical devices, such as a pacemaker or an X-ray machine, AI systems can evolve through continuous learning or produce outputs that are not easily traceable to a specific logic chain. This creates a significant hurdle for hospital administrators and clinicians who must decide which tools are trustworthy. Dr. Mello’s work with HEAL-AI serves as a blueprint for institutional governance, where multidisciplinary teams—including ethicists, lawyers, clinicians, and data scientists—subject proposed AI tools to rigorous "ethical stress tests" before they are deployed in patient care.
Accountability and the Liability Paradox
One of the most contentious topics discussed by Kahn and Mello is the evolution of medical malpractice in the age of AI. Traditionally, the legal doctrine of "corporate negligence" or "vicarious liability" has held hospitals and doctors responsible for the quality of care. However, AI introduces a third party into the clinical relationship: the software developer.
If an AI diagnostic tool fails to identify a tumor, or if an administrative algorithm inadvertently deprioritizes a patient based on biased data, the legal path to restitution is murky. Dr. Mello noted that current tort law is ill-equipped to handle cases where a physician follows an AI’s recommendation that turns out to be incorrect. Does the physician bear the brunt of the liability for "failing to exercise independent judgment," or does the liability shift to the developer for a "product defect"?
Supporting data suggests that legal fears are a primary barrier to AI adoption. According to a 2024 survey of healthcare executives, nearly 60% cited "legal and liability concerns" as a top three inhibitor to deploying generative AI in clinical settings. Dr. Mello argued that without clear statutory guidance, we may see a wave of "defensive medicine" where clinicians either over-rely on AI to avoid personal blame or ignore AI insights altogether to minimize risk, both of which could compromise patient outcomes.
A Chronology of Medical AI Regulation and Development
The journey toward the current state of AI in medicine has moved through several distinct phases over the last decade:
- 2016–2019: The Diagnostic Era. Early AI applications focused primarily on image recognition, such as identifying diabetic retinopathy or skin cancer. During this period, the FDA began streamlining its approval process for AI-enabled devices.
- 2020–2022: Pandemic Acceleration. The COVID-19 pandemic forced a rapid digital transformation. AI was utilized for resource allocation, predictive modeling of infection surges, and triage. This period exposed the risks of algorithmic bias, as several tools were found to disadvantage minority populations due to skewed training data.
- 2023–2025: The Generative Shift. The emergence of Large Language Models (LLMs) moved AI from back-office diagnostics to front-end patient interaction and clinical documentation. This era saw the introduction of the White House Executive Order on Safe, Secure, and Trustworthy AI, which tasked the Department of Health and Human Services (HHS) with creating a task force to monitor AI safety.
- 2026 and Beyond: The Governance Mandate. As highlighted in the podcast, the focus has shifted toward institutional accountability. Organizations like HEAL-AI represent the new standard, where the focus is on "lifecycle monitoring"—ensuring that an AI tool remains accurate and unbiased long after its initial implementation.
Algorithmic Bias and Social Justice
A significant portion of the conversation centered on the ethical imperative to address bias. AI models are trained on historical data, and if that data reflects existing societal inequities—such as differences in healthcare access or historical underrepresentation of certain demographics in clinical trials—the AI will likely codify and amplify those biases.
Dr. Mello highlighted that "neutral" algorithms can still produce discriminatory results. For instance, an algorithm designed to predict which patients need intensive care management might use "past healthcare spending" as a proxy for "health needs." Because lower-income and minority populations often have lower historical spending due to lack of access, the AI may incorrectly conclude they are "healthier" and deny them necessary resources.
The HEAL-AI approach advocates for "algorithmic impact assessments," which require developers and hospitals to prove that their tools do not disproportionately harm marginalized groups. This proactive stance is seen as essential for maintaining public trust in the healthcare system.
Supporting Data: The Economic and Clinical Scale of AI
The scale of AI integration discussed by Kahn and Mello is reflected in recent industry statistics. The global market for AI in healthcare is projected to grow from roughly $20 billion in 2023 to over $180 billion by 2030, representing a compound annual growth rate (CAGR) of 37%.
On the clinical side, the FDA has authorized over 700 AI and machine learning-enabled medical devices as of mid-2024. The majority of these are in the field of radiology (approximately 75%), followed by cardiology and hematology. However, the rise of "generative AI" for clinical note-taking and patient communication is the fastest-growing sub-sector, with some health systems reporting a 50% reduction in "pajama time" (the time doctors spend on administrative tasks after hours) following the implementation of AI scribes.
Professional Responses and Institutional Responsibility
The medical community’s response to AI has been a mix of cautious optimism and rigorous skepticism. The American Medical Association (AMA) has released a set of principles for "Augmented Intelligence," emphasizing that AI should be a tool to enhance human intelligence rather than replace it. The AMA’s stance mirrors Dr. Mello’s advocacy for "human-in-the-loop" systems, where the final clinical decision always rests with a licensed professional.
Furthermore, the American College of Radiology (ACR) has established its own "Data Science Institute" to create standardized "use cases" for AI, ensuring that developers are building tools that address actual clinical needs rather than just pursuing profitable niches.
Dr. Mello suggested that the next phase of official response should include a "no-fault" compensation fund for AI-related injuries, similar to the National Vaccine Injury Compensation Program. Such a system would ensure that patients are compensated quickly for errors caused by "black box" algorithms without the need for decade-long litigation against individual doctors or cash-strapped hospitals.
Broader Implications: The Future of the Patient-Provider Relationship
The episode concludes with a look at the long-term implications for the patient-provider relationship. Chip Kahn and Dr. Mello discussed whether the "standard of care"—the legal benchmark for medical competence—will eventually change to require the use of AI. If an AI tool is shown to be significantly more accurate than a human doctor in a specific task, could a doctor be sued for not using it?
This shift would represent a fundamental change in the nature of medical expertise. As AI takes over more of the "technical" work of medicine, the role of the physician may evolve back toward its roots in the "healing arts"—focusing on communication, empathy, and complex ethical decision-making that machines cannot replicate.
However, this transition requires a level of transparency that is currently lacking. Dr. Mello argued that patients have a right to know when AI is being used in their care. Transparency is not just about disclosure; it is about "explainability"—the ability of a clinician to explain to a patient why an AI suggested a particular course of action.
Conclusion: A Call for Collaborative Oversight
The insights shared by Dr. Michelle Mello on "The Business of Health" underscore a critical reality: the technology of the future cannot be governed by the laws of the past. As Chip Kahn noted throughout the series, the business of healthcare is increasingly becoming the business of data.
To ensure that AI serves the best interests of patients, a collaborative effort is required between the legal system, the tech industry, and the medical profession. The work being done at Stanford’s HEAL-AI serves as a vital laboratory for these efforts, testing the theories of governance in the high-stakes environment of a working hospital. As AI continues its race into everyday medicine, the success of the technology will ultimately be measured not by its processing speed, but by the strength of the ethical and legal guardrails that keep it on track.
