Experts favor a new regulatory framework for clinical AI models
Congress should enact a specific regulatory framework for artificial intelligence (AI) models developed and used in academic medical centers, Scott Gottlieb argued in an opinion piece published in JAMA Health Forum. Such systems should not be regulated as commercial medical devices or left unregulated, according to Gottlieb, a former Commissioner of Food and Drugs, now at the American Enterprise Institute. Three experts who spoke with Focus said they support the plan.
While a significant number of academic medical centers are using electronic health records (EHRs) to build and train clinical AI models, testing and vetting is inconsistent, Gottlieb wrote. Just 61% of hospitals that use predictive models tested their accuracy with local data, while 44% assessed their bias, according to a national study.
Some academic centers argue that an AI model trained, hosted, and used within the institution to care for its own patients should be considered a clinical process rather than a commercially available medical device and therefore not subject to FDA oversight. These tools are seen as falling under professional self-regulation and institutional governance rather than federal device regulation, Gottlieb noted.
Yet Gottlieb argued that this has led to a "two-tier" market: one for academic centers, which can rapidly develop and modify AI tools with little regulatory burden, and the other for commercial medical device developers, who must submit such systems for FDA review.
“An academic center can deploy a high-value algorithm in-house and revise it as often as it likes. A company offering the same tool to other hospitals must finance FDA review and take on continuing device revision obligations,” Gottlieb wrote.
Gottlieb pointed out that laboratory-developed tests (LDTs) started in a similar fashion and sat “outside the FDA’s purview…even if the law gave the agency authority to regulate them as medical devices. The FDA allowed laboratories to design, validate, and run an assay in-house, often to fulfill specialized requirements or needs too small or urgent to interest a commercial manufacturer.”
The laboratory itself is governed by the Clinical Laboratory Improvement Amendments (CLIA), administered by the Centers for Medicare & Medicaid Services (CMS). Under this system, CMS enforces requirements for LDT performance and quality systems.
In the opinion piece, Gottlieb proposed a framework in which these AI models would be subject to CLIA-like oversight under FDA jurisdiction. FDA would certify certain health systems to develop and deploy certain categories of AI. Such certification would be based on prespecified requirements for an algorithm’s intended use, validation methods, data training sets, and postmarket surveillance. The institution would be required to regularly report to the FDA on the algorithm’s performance.
Experts in the field supported the proposal. “I'd say it's a serious proposal. I appreciate how it is aiming to balance flexibility and innovation with an assurance of quality,” said Joshua Sharfstein, former FDA commissioner, now at Johns Hopkins Bloomberg School of Public Health.
Brian Richard Jackson, professor of pathology at the University of Maryland School of Medicine and director of clinical pathology at the University of Maryland Medical Center, said, “I strongly agree that Congress should create a regulatory pathway for AI that roughly mirrors the CLIA approach to lab oversight.”
Jackson said he made the same argument in a study published last year in the Journal of the American Medical Informatics Association, in which he proposed adapting the existing federal CLIA framework to clinical tests that use AI algorithms.
Jackson added, “Gottlieb mentions some of the unresolved issues in LDT regulation. He doesn't seem to propose any specific solution for this in the AI world. But I think he's correct that the most important thing right now is to get the basic framework set up, and that this will require an act of Congress.”
Mark Sendak, a co-author of the 2025 paper along with Jackson, is a physician and the co-founder and CEO of Vega Health, and he said that he largely supports the paper's conclusions, stating that clinical AI should be regulated through a combination of centralized oversight and local hospital governance. Previously, Sendak was affiliated with the Duke University Health Innovation program.
“The part of his argument that is true is that we have this collection of validated assets that sit in these academic medical centers that don’t get commercialized, and we need to diffuse those innovations across settings, and we do need rigorous local evaluation and monitoring,” according to Sendak.
