The rapid integration of artificial intelligence into clinical environments has outpaced the development of comprehensive regulatory frameworks, leaving healthcare providers, legal experts, and patients in a state of transition. In the latest installment of the AI Series on The Business of Health, host Chip Kahn sat down with Dr. Michelle Mello, a professor of law and health policy at Stanford University, to dissect the ethical, legal, and operational challenges accompanying this technological shift. As AI tools move from experimental prototypes to essential diagnostic and administrative assistants, the central question shifts from whether AI should be used to how it can be governed to ensure safety and equity.
Dr. Mello, who co-leads the Healthcare Ethical Assessment Lab for AI (HEAL-AI) at Stanford, brings a unique dual perspective as both a legal scholar and a health policy expert. Her work focuses on the "rules of the road" for medical technology—rules that are currently being written in real-time as hospitals across the country deploy algorithms to predict patient deterioration, automate radiology readings, and manage complex billing cycles. The conversation highlighted a critical gap in the current healthcare landscape: the disparity between the speed of algorithmic innovation and the deliberate, often slow pace of legal and ethical standard-setting.
The Governance Gap in Clinical AI
The primary concern addressed by Kahn and Mello is the lack of a centralized, federal blueprint for AI oversight in the clinical setting. While the U.S. Food and Drug Administration (FDA) has cleared hundreds of AI-enabled medical devices, its purview is largely limited to the pre-market phase. Once a tool is deployed within a hospital system, the responsibility for its ongoing performance and ethical impact falls almost entirely on the individual institution.
Dr. Mello emphasized that many healthcare organizations are currently "flying blind," lacking the internal infrastructure to audit algorithms for bias or drift. Drift occurs when an AI model’s accuracy degrades over time because the clinical environment or patient population changes. Without rigorous post-market surveillance, a tool that was effective at launch could eventually provide inaccurate recommendations, leading to potential patient harm.
To address this, Stanford’s HEAL-AI serves as a model for institutional governance. The lab conducts multidisciplinary ethical assessments of AI tools before they are integrated into Stanford Health Care facilities. This process involves evaluating not just the technical accuracy of the tool, but its potential to exacerbate existing health disparities. For instance, if an algorithm used to prioritize patients for kidney transplants was trained on a dataset that reflects historic biases in access to care, the tool might inadvertently perpetuate those same inequities.
Liability and the Question of Accountability
One of the most complex issues discussed in the episode is medical liability. In traditional medicine, the "captain of the ship" doctrine often places the ultimate responsibility on the physician. However, as AI systems become more autonomous, the line of accountability blurs. If an AI system fails to flag a malignant tumor on an imaging scan, or if it suggests a treatment plan that results in an adverse reaction, where does the blame lie?
Mello noted that the legal system is still grappling with how to treat AI: is it a tool, like a scalpel, or is it more akin to a consulting physician? If the law treats AI as a product, then the manufacturer may be held liable under product liability statutes. However, if the AI is viewed as a service or a component of professional practice, the burden may remain with the physician for "negligent use" of the technology. This creates a significant "liability vacuum" that could leave patients without recourse and doctors feeling exposed.
Furthermore, there is the "black box" problem. Many deep-learning algorithms are so complex that their decision-making processes are not transparent even to their creators. Mello argued that for AI to be truly integrated into the legal framework of medicine, there must be a move toward "explainable AI," where the rationale behind a clinical recommendation can be scrutinized and defended in a court of law.
The Chronology of AI Regulation in Healthcare
The discussion between Kahn and Mello takes place against a backdrop of evolving policy. Over the last several years, the trajectory of AI regulation has moved from broad ethical principles to specific mandates:
- 2023: The White House issued an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, which directed the Department of Health and Human Services (HHS) to establish a safety program to capture reports of AI-related clinical errors.
- 2024: The FDA transitioned toward a "Total Product Lifecycle" approach, encouraging manufacturers to submit plans for how they would update their algorithms post-market without needing entirely new clearances for every minor adjustment.
- 2025: Several states, led by California and New York, introduced legislation requiring healthcare providers to disclose to patients when AI is being used to make significant medical decisions.
- 2026: The current landscape sees a push for "algorithmic impact assessments," similar to environmental impact reports, which would require hospitals to prove that an AI tool is safe for their specific demographic before implementation.
This timeline suggests a shift toward decentralization, where the burden of proof is increasingly placed on the "deployers" (hospitals) rather than just the "developers" (tech companies).
Supporting Data: The Scale of the AI Incursion
The necessity of the "rules of the road" discussed by Mello is underscored by the sheer volume of AI adoption. According to recent industry data:
- FDA Authorizations: As of mid-2026, the FDA has authorized over 950 AI and machine learning-enabled medical devices, a nearly 400% increase from five years prior. The vast majority of these are in the field of radiology.
- Market Growth: The global market for AI in healthcare is projected to exceed $150 billion by 2030, driven largely by the administrative need to reduce physician burnout and the clinical need for personalized medicine.
- Physician Sentiment: A 2025 survey of American Medical Association (AMA) members found that while 65% of physicians see the potential for AI to improve diagnostic accuracy, 80% remain "highly concerned" about liability and the erosion of the patient-physician relationship.
These figures illustrate a tension that Chip Kahn highlighted throughout the series: the "business of health" is currently a race to gain efficiency through technology, but the "practice of health" remains a human-centric endeavor that requires trust and transparency.
Broader Implications and the Future of Patient Care
The insights shared by Dr. Mello suggest that the future of AI in medicine will not be determined by the sophistication of the code, but by the robustness of the social and legal contracts we build around it. If the public perceives AI as a "black box" that prioritizes hospital profits over patient safety, the backlash could stifle innovation for a generation.
One of the broader implications discussed is the potential for "automation bias," where clinicians become so reliant on algorithmic suggestions that they lose their own diagnostic sharpness or fail to override a machine when it is clearly wrong. Mello argued that the next phase of medical education must include "algorithmic literacy," teaching future doctors not just how to use these tools, but how to skeptically evaluate them.
Moreover, the conversation touched upon the global perspective. As the United States develops its own internal standards, it must also contend with international frameworks like the European Union’s AI Act, which classifies healthcare AI as "high risk" and imposes strict transparency and data quality requirements. U.S. hospitals and tech firms operating globally will need to navigate a patchwork of regulations that could complicate the scaling of AI solutions.
Conclusion: Setting the Standard
As Episode 13 of the AI Series concludes, the message from Chip Kahn and Dr. Michelle Mello is clear: the integration of AI into healthcare is an inevitability, but its success is not. The "rules of the road" must be grounded in the fundamental bioethical principles of beneficence, non-maleficence, autonomy, and justice.
Dr. Mello’s work at Stanford and her dialogue with Kahn serve as a call to action for healthcare leaders to move beyond the "wow factor" of AI and begin the hard work of institutional governance. By establishing clear lines of accountability, auditing for bias, and maintaining transparency with patients, the healthcare industry can ensure that AI serves as a tool for healing rather than a source of new risks. The business of health, in this new era, is as much about ethics and law as it is about medicine and technology.
