FDA seeks input on evaluating AI-enabled medical devices
The US Food and Drug Administration’s (FDA) Center for Devices and Radiological Health (CDRH) is seeking input on methods for assessing and evaluating AI-enabled medical devices in a newly released discussion paper. This includes strategies for identifying and addressing performance drift, which involves detecting changes in both input and output.“AI, including GenAI, presents opportunities to improve patient outcomes, advance public health, and accelerate medical innovation. At the same time, these technologies introduce new considerations as it relates to assuring the maintained safety and effectiveness of AI-enabled medical devices across the total product life cycle, and particularly with respect to assessing their performance, safety, and reliability after deployment in real-world settings,” states the paper.
The request follows a public meeting of FDA’s Digital Health Advisory Committee in November 2024, where discussions focused on effective real-world strategies to ensure that AI-enabled medical devices remain safe and effective post-deployment. (RELATED: Generative AI: FDA adcomm makes recommendations on postmarket performance of medical devices, Regulatory Focus 22 November 2024)
Since the FDA approved its first AI-based medical device in 1995, the agency has authorized over 1,000 AI-driven medical devices. Additionally, there has been a ten-fold increase in submissions for AI-based devices since 2020, according to FDA authors who shared these insights in a special communication published in JAMA last year. (RELATED: FDA officials outline need for oversight of AI in healthcare, biomedicine, Regulatory Focus 17 October 2024)
The paper states that the performance of an AI system can be influenced by various factors, including changes in clinical practices, patient demographics, data inputs, and healthcare infrastructure. These variations are often referred to as data drift, which can result in performance degradation, bias, or diminished reliability, as noted in the paper. Additionally, user behavior and the integration of the system into workflows can also impact its performance.
Many AI-enabled medical devices are currently assessed mainly through retrospective testing or static benchmarks, said the paper. While these methods can help establish a basic understanding of a device's performance, they do not effectively predict how these devices will behave in dynamic, real-world environments. There is a growing recognition of the importance of ongoing and systematic performance monitoring to ensure the safe and effective use of AI by observing how these systems function during clinical deployment.
The agency is seeking comments in six areas: performance metrics, real-world evaluation methods and infrastructure, postmarket data sources and quality management, monitoring triggers and response protocols, human-AI interaction, and user experience.
Stakeholders are invited to provide feedback on the metrics or performance indicators they utilize to assess the safety, effectiveness, and reliability of AI-enabled medical devices, along with how these metrics are defined and weighted.
Members of the public are also asked to discuss the tools and methodologies they use for monitoring the performance of AI-enabled medical devices after deployment.
Stakeholders are also asked to identify the triggers that necessitate additional assessments and more intensive evaluations. Another question pertains to how clinical usage patterns and user interactions affect the performance of AI-enabled medical devices over time.
The deadline for submitting comments is 1 December 2025. Comments should be submitted here.
Discussion paper
×
Welcome to the new RAPS Digital Experience
We have completed our migration to a new platform and are pleased to introduce the updated site.
What to expect: If you have an existing login, please RESET YOUR PASSWORD before signing in. After you log in for the first time, you will be prompted to confirm your profile preferences, which will be used to personalize content.
We encourage you to explore the new website and visit your updated My RAPS page. If you need assistance, please review our FAQ page.
We welcome your feedback. Please let us know how we can continue to improve your experience.