Stakeholders disagree over FDA’s regulatory approach to generative AI

While the MedTech lobby group AdvaMed argues that the US Food and Drug Administration (FDA) already has sufficient premarket and postmarket capabilities to regulate generative artificial intelligence (GenAI) products, the American College of Radiology (ACR) says the agency needs to update its pathways to accommodate the new technologies.

Last November, FDA's (FDA) Digital Health Advisory Committee and other stakeholders met to discuss how the agency should oversee GenAI products. After the meeting several stakeholders wrote to the agency with their recommendations, including AdvaMed’s Medical Imaging Division. The group wrote to FDA about its takeaways from the meeting, in which it reiterated the need to take a risk-based approach and avoid potential duplicative work when overseeing GenAI. (RELATED: Generative AI: FDA adcomm makes recommendations on postmarket performance of medical devices, Regulatory Focus 22 November 2024)

“It is important we continue to acknowledge that the specific use cases of generative AI in medical imaging devices are still emerging, and that use cases for GenAI in medical devices and their associated risks should be well understood before new regulatory approaches are considered,” said the group. “It is also imperative to maintain flexibility in regulatory approaches to account for the ongoing evolution of Generative AI technologies.”

During the meeting, Jessica Paulsen, associate director for digital health in the Office of Product Evaluation and Quality (OPEQ) at the Center for Devices and Radiological Health (CDRH), said FDA has long promoted a total product lifecycle (TPLC) approach for regulating medical devices, which has become more relevant for products that are able to iterate and change very quickly.

In that vein, AdvaMed argued that FDA’s current regulatory framework is already capable of addressing many of the principles proposed for regulating GenAI during the meeting. AdvaMed pointed out that the TPLC approach allows regulators to oversee products that require premarket validation and postmarket surveillance. It added that a risk-based perspective will ensure innovation in the sector by focusing on the intended use and ensuring the balance of the product benefit against risk, rather than focusing on specific technologies.

“The FDA’s current framework is likely sufficiently robust to manage the unique considerations of generative AI in medical devices,” said AdvaMed. “By maintaining a focus on risk-based regulation and leveraging existing pathways, the FDA can continue to support innovation while ensuring the safety and effectiveness of these devices.”

“Additional authorities or regulations targeting GenAI-enabled devices without first understanding if there are any gaps in the existing framework are unnecessary and could hinder progress,” the group added. “Instead, we encourage the FDA to refine existing approaches with flexibility to accommodate device-specific considerations, foster collaboration among stakeholders, and ensure patient safety remains the top priority.”

During the meeting, Paulsen also noted that FDA already considers the intended use and technological characteristics of a device in its risk-based regulatory framework.

“All medical devices are held to strict standards to ensure the safety and effectiveness of the device for its intended use and to monitor device performance across the total product lifecycle,” said AdvaMed. “Importantly, under the current FDA regulatory framework, manufacturers need to demonstrate mechanisms to control or mitigate output variability in the premarket submission in order to demonstrate the safety and effectiveness of the device.”

“Addressing this variability is critical to maintaining patient safety and upholding the FDA's high standards, which is achievable with the FDA’s existing safety and effectiveness requirements,” the group added.

Labeling has been a major topic of discussion for AI-based products and was brought up during the November advisory committee meeting. AdvaMed cautioned against using a one-size-fits all model when creating GenAI labeling requirements because such labels may need to be more nuanced or require substantial contextual interpretation, depending on the product.

AdvaMed also raised concerns about using third-party quality assurance labs to oversee GenAI products, which may duplicate much of FDA’s work without adding substantial benefit. The group argued that such labs might lack the contextual understanding to evaluate AI products properly, which could lead to inaccurate evaluations.

AdvaMed proposed several measures to improve regulatory oversight of GenAI products, including targeted education on the technology for academic, clinical, industry, and legislative stakeholders; prioritizing FDA’s risk-based approach by focusing on the product's intended use and benefit-risk profile; and relying on already existing regulatory pathways.

While AdvaMed argued that current FDA regulations are sufficient for overseeing GenAI products, ACR said that the traditional premarket evaluation approach needs to be significantly adapted due to the novel regulatory oversight challenges that such products create. The healthcare provider group noted that most medical AI products on the market are not considered medical devices and, therefore, do not require FDA oversight. However, the group said adapting the premarket evaluation approach is critical to determining whether a device falls under the agency's purview.

ACR also said FDA's risk management approaches/controls should be stratified based on the device's clinical impact, the qualifications of its user population, and users' ability to intervene and mitigate risks before they affect their care. The group said GenAI technology requires enhanced postmarket oversight strategies as part of the agency’s TPLC approach.

“Although expanded statutory authorities may be necessary, FDA should explore implementation specific considerations to ensure safety and effectiveness,” said ACR. “These implementation considerations should include site-specific validation requirements to define monitoring cadence, performance comparisons across difference sites, and integration with existing [quality assurance (QA)] programs at those sites.”

ACR noted that its Data Science Institute (DSI), which participated in the November meeting, has expertise in implementing and monitoring AI technologies in radiology settings, and can help advise on the development and oversight of GenAI products. It recommended regulators leverage existing QA infrastructures, such as its registries and monitoring programs, when developing regulatory approaches to GenAI products.

ACR also asked FDA to publish a draft guidance on foundation model evaluation and monitoring. More specifically, the group said the guidance should include risk-informed tiers for foundation model oversight and noted that base foundation models will need appropriate documentation and transparency, and their intended uses will need performance testing with proper validation and monitoring.

“There should be a standard FDA framework for clinical validation that includes minimum requirements for training data diversity, standardized testing protocols across different clinical scenarios, and performance benchmarks for specific clinical tasks,” said ACR. “FDA should also consider healthcare equity-related requirements, including mandatory testing across diverse patient populations, performance monitoring in different healthcare settings, and other bias identification/mitigation approaches.”

Public Docket

Stakeholders disagree over FDA’s regulatory approach to generative AI

Related topics

Recommended content

Welcome to the new RAPS Digital Experience