Commenters seek improvements in FDA's computational modeling draft guidance

Regulatory NewsRegulatory News | 31 March 2022 |  By 

Stakeholders who commented on draft guidance from the US Food and Drug Administration (FDA) for assessing credibility of computational modeling and simulation (CM&S) generally praised the agency’s attempt at creating a risk-based framework in this area, but also suggested several ways it could improve the framework in the context of medical device regulatory submissions.
The draft guidance, issued in January 2022, is a nine-step generalized framework based on the American Society of Mechanical Engineers (ASME) V&V 40 Assessing Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices standard, which is recognized by FDA and applies to a variety of models and simulations used in premarket approval (PMA) applications, humanitarian device exemptions (HDE), investigational device exemptions (IDE), 510(k) premarket notifications and de novo requests. The guidance also applies to CM&S methods in “in silico” clinical trials that use a virtual cohort, or for device testing (RELATED: Computational modeling and simulation: FDA outlines framework for assessing credibility, Regulatory Focus 04 January 2022).
“The purpose of this guidance document is to provide a general framework for assessing CM&S credibility in medical device regulatory submissions that incorporates both traditional V&V evidence and/or other types of supporting data. This guidance document is applicable to physics-based, mechanistic, or other first principles-based models, such as models commonly used in electromagnetics, optics, fluid dynamics, heat and mass transfer, solid mechanics, acoustics, and ultrasonics, as well as mechanistic models of physiological processes,” FDA wrote in the draft guidance. “This guidance is not intended to apply to statistical or data-driven models such as machine learning or artificial intelligence.”
Comments from stakeholders
Industry stakeholders agreed that the guidance is needed. In their comments, the Advanced Medical Technology Association (AdvaMed) told FDA the guidance is “an important step toward highlighting the importance of rules and physics-based modeling and simulation” as evidence in regulatory submissions, but noted CM&S should be considered based on context, rather than model type. There also may be situations where crossover occurs with multiple models, and FDA should take these cases into consideration, AdvaMed said.
“To this end, we also recommend you consider developing guidance around use of multiple CM&S models within a single device submission or system submission and ensuring that application of CM&S and model credibility works as a whole,” they wrote.
According to AdvaMed, what FDA is creating is guidance for CM&S modeling combined with empirical data such as calibration test data and validation test data, rather than CM&S in isolation. “[T]his guidance goes beyond the credibility of CM&S and is instead describing a framework for addressing the totality of evidence that includes modeling and simulation to address a question of interest,” they wrote. “The combination of a mechanistic model with supporting evidence from empirical testing is more robust than empirical testing alone and addresses the limitations of ‘correlation is not causality’ from purely data-driven, empirical forms of evidence.”
Comments from stakeholders in two working groups within the International Society of Pharmacometrics Quantitative Systems Pharmacology Special Interest Group characterized the draft guidance as a “reasonable attempt” to create a framework around CM&S credibility in medical device submissions “that balances risk and supportive evidence for simulation-supported submissions, while flexibly allowing non-traditional forms of modeling evidence as supportive factors.”
While FDA uses several examples of quantitative systems pharmacology (QSP) in the guidance, the working groups questioned whether the framework could be applied to QSP models and their use cases. “A substantial cause for concern for extension to QSP is that when we think of models used for medical devices, they often may be steeped in basic physics whereas QSP models may also incorporate some pathways in a more abstract or top-down manner,” they wrote.
The working groups said while some but not all QSP models could follow the framework, they recommended a separate guidance be created for that purpose. “This approach will facilitate submissions weighing risk and other appropriate assessment considerations for QSP models and their typical applications, and guidance of appropriate supporting analysis,” they wrote.
International scientific consortium SIMCor made more specific suggestions, recommending FDA add a real-world data category to the list of evidence categories within the guidance as well as revising the category definitions to focus on data type and accuracy. They also recommended FDA standardize language surrounding applicability and credibility assessment to better align with the AMSE V&V40 standard.
FDA’s definition of in silico trials should include assessments of safety in addition to performance, according to SIMCor, which requested more clarity on how FDA sees in silico trials intersecting with real-world clinical trials. “[W]e welcome the initiative of drafting the current document,” they wrote. “We believe the in silico community and the quality CM&S results in regulatory submission will benefit from such guidance after revisions.”
The In Silico World (ISW) consortium concurred with SIMCor’s suggestion to include safety and efficacy testing as well as the need to better adhere to the AMSE V&V40 standard when considering the applicability of credibility activities. Better clarity is needed in areas where the draft guidance and AMSE V&V40 diverge in the case of controlled experiments and validation, they said.


© 2023 Regulatory Affairs Professionals Society.

Discover more of what matters to you