FDA asked to clarify AI pilot objectives for early-phase clinical trials
Industry and academic stakeholders have asked the US Food and Drug Administration (FDA) to clarify its objectives and plans for a proposed pilot program to evaluate how artificial intelligence (AI) technologies can help improve early-phase clinical trials. They also said the agency should develop a governance structure with specific metrics and goals and consider aligning its thinking with other regulators.
FDA issued a request for information (RFI) in April, asking stakeholders for input on the proposed pilot program. The agency highlighted that it wants to explore how AI technologies can accelerate innovation and speed product development through the clinical trial process.
"Early-phase clinical trials represent a critical bottleneck in drug development, often characterized by high uncertainty, limited patient populations, and inefficient decision- making processes," said FDA. "This pilot program aims to explore how advances in AI and data science can improve trial efficiency, enhance safety monitoring, facilitate dose selection decisions, and enable more informed early go/no-go decisions (e.g., a regulatory decision as to whether a Phase 1 study may proceed) while maintaining FDA's rigorous scientific and regulatory standards and promoting trustworthy AI systems.
"The pilot program will be guided by principles aligned with the National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF)," the agency added.
PhRMA
Since announcing the RFI, almost 240 entities have submitted their feedback to FDA on how AI technologies can help streamline early-phase clinical trials, including the Pharmaceutical Research and Manufacturers of America (PhRMA). The drug industry group was optimistic about AI's potential to accelerate innovation and said the pilot could realize that promise.
PhRMA asked FDA to ensure that the pilot incorporates the agency's risk-based approaches to using AI in regulatory decision-making and that AIs are evaluated based on their purpose. The group also said the pilot should focus on common AI use cases in drug development.
"PhRMA acknowledges that AI is being used in an array of different contexts spanning the entire drug development timeline," said PhRMA. "To 'maximize learnings' from the pilot, we encourage FDA to apply the pilot trials across different therapeutic areas, with priority on broadly applicable examples that can generate near-term learnings.
"In terms of assessing 'maturity,' we encourage FDA to consider whether the AI has been previously assessed for the proposed context of use, and whether the use case will advance regulatory decision-making," the group added.
PhRMA noted that while third-party AI tools can help the medical industry develop innovative approaches, they can pose certain regulatory challenges. To address such challenges, the group asked regulators to use the pilot as an opportunity to explore mechanisms that may help sponsors choose the right tools.
Furthermore, PhRMA said the pilot should have a solid governance framework, and FDA should be clear about how it will protect confidential commercial and trade secret information to ensure participant confidence and willingness to engage. The group also said the agency should use what it has learned from past pilots to develop the new pilot and clarify whether and how it may use AI in the pilot.
"There have been some public statements from FDA suggesting that the Agency itself may utilize AI as part of the pilot program," said PhRMA. "Should this be the case, it is important for FDA to be clear and transparent upfront about how and when the Agency will be utilizing AI.
"Importantly, FDA should apply symmetric expectations to its own AI use that it expects of regulated industry; that is, FDA AI tools as part of the pilot should themselves be under documented credibility assessment and change control, as appropriate, and their output should be subject to human review before influencing a regulatory decision," the group added.
Additionally, PhRMA asked FDA to clarify how Real Time Clinical Trials (RTCT) may be part of the agency's thinking and separately engage with stakeholders on issues related to RTCT. The group also asked the agency to hold pre-meetings with pilot participants and confirm that participation wouldn't delay or replace their standard regulatory meetings with the agency.
BIO
The Biotechnology Innovation Organization (BIO) commented that FDA's efforts to use novel technologies, such as AI, are an important step toward improving data integration in clinical development. The group said the proposed pilot should be based on a safety-oriented framework with an emphasis on early-phase clinical trial settings.
BIO said it wasn't clear whether the agency's primary objective in proposing the pilot program was to advance real-time clinical trial capabilities or, more broadly, to promote the use of AI tools. As such, it asked the agency to better articulate the objectives of the RFI and consider more stakeholder engagement. The group also said the pilot should focus on use cases aligned with the agency's existing regulatory authority and decision-making role.
"While AI-based approaches may represent one potential pathway, BIO encourages the Agency to consider broader engagement with industry on potential means of real-time data sharing and application of any alternate approaches that may support achievement of the program’s core objective," said BIO. "Potential applications for AI in clinical trials, on the other hand, are much broader in scope than the pilot program proposed by FDA, and warrants a separate focused engagement with industry, patient groups, and academic researchers.
"FDA might consider a public workshop to facilitate an open and collaborative discussion about the merits of different AI and data sharing approaches," the group added.
BIO asked FDA to clarify its definition of real-time data access and review, and furthermore, clarify whether such access refers to raw electronic health record data, uncleaned electronic data capture information, or sponsor-released datasets following source data verification, source data review, and query resolution. The group also highlighted the need for greater transparency about the AI approaches the agency will consider for the pilot program.
"At present, there is limited clarity regarding the underlying models, intended use cases, and decision-making frameworks that FDA expects to evaluate," said BIO. "This includes how sponsors document these decisions in regulatory submissions, whether and when sponsors should disclose AI use in study documents such as protocols and informed consent forms, whether AI-generated recommendations related to dosing or safety would trigger protocol amendments, and the scope of documentation needed to demonstrate alignment with the NIST AI Risk Management Framework.
"Additional detail will be important to ensure that participating sponsors can appropriately assess the suitability, interpretability, and reliability of AI-enabled methods," the group added. "Sponsor concerns regarding algorithmic opacity, insufficiently defined performance metrics, and unclear success criteria may otherwise limit confidence in the pilot and impede meaningful participation."
Similar to PhRMA, BIO also asked FDA to lay out clear governance and risk management expectations, including how the agency intends to distinguish between AI used as human-in-the-loop decision support and autonomous decision-making, how reproducibility and auditability should be maintained, and what protections would apply to proprietary data incorporated into the AI framework. The group said the agency should clearly state its expectations for AI governance, oversight, and accountability, and added that pilot sponsors need clarity regarding documentation requirements, validation expectations, and lifecycle management.
BIO added that FDA should consider global regulatory alignment as the project moves forward and develop a targeted evaluation plan for the real-time clinical trial pilot based on specific metrics. The group also said the agency should stick to its earlier statement that the pilot is not expected to require submission of patient-level data and would instead use predefined safety and efficacy signals when working with sponsors.
Duke-Margolis
The Duke-Margolis Institute for Health Policy asked FDA to provide more details on its highest-priority goals for the pilot. The health policy group said it is unclear what regulatory gap the agency is trying to close with its proposed pilot and wants more clarity to ensure the pilot is fit-for-purpose and generates the right regulatory decision-making information.
Duke-Margolis raised concerns that the proposed three- to six-month timeline for the pilot seems relatively short for the kinds of information regulators are hoping to generate and poses a significant challenge to achieving meaningful outcomes. It noted that during an industry day event, FDA suggested that sponsors could consider adapting ongoing projects for the pilot to address the short timeline. However, the group said that, in the short term, potential participants would benefit from guidance on which components of their current projects the agency considers most adaptable and could fit into the pilot.
Despite the potential for AI to speed clinical trials, Duke-Margolis cautioned that not every decision benefits from speed, and there needs to be clarity on how using AI can provide benefits when compared to potential risks. The group also said the agency should use several core, consistent metrics to evaluate AI-enabled approaches in the pilot, such as trial efficiency, transition time, the quality and timeliness of decision-making, and the validity of the AI system.
"FDA’s AI-Enabled Optimization of Early-Phase Clinical Trials Pilot Program places the agency in a unique position to engage sponsors in a pre-competitive environment (i.e., via public workshops, meetings, etc.), whereas knowledge and lessons learned gained during the pilot can be shared," said Duke-Margolis. "In addition, FDA could share benefit-risk evaluation frameworks that the agency might update or develop during or in response to pilot program experiences."
