CDRH Outlines Vision for Good Machine Learning Practices
Posted 28 May 2019 | By
Ahead of the deadline for feedback on the US Food and Drug Administration’s (FDA) discussion paper on artificial intelligence (AI) and machine learning (ML), FDA’s medical officer for digital health Matthew Diamond detailed the vision for good ML practices (GMLP) at the AdvaMed Digital MedTech Conference.
GMLP “would be analogous to good manufacturing practices (GMP)” in applying a quality system approach, though it would be “specifically focused on the unique challenges for machine learning,” Diamond said during an FDA fireside chat on grappling with AI regulation last week. GMLP “do not necessarily” exist as of today “but it would make everyone’s life easier if they did,” said Diamond.
He highlighted some of the unique aspects to machine learning technologies—the way that data needs to be curated and separated as well as the way that statistics for certain problems need to be informed.
Diamond explained that the newly proposed concept of GMLP is not only intended to leverage GMP but is also aligned with the agency’s good clinical practices (GCP) and good laboratory practices (GLP). In practice, GMLP would mirror GMP, GCP and GLP in terms of compliance with codified quality standards.
“Part of the goal is if someone comes to us with a submission and says, ‘We are conforming to these codified standards of good ML practices,’ it would allow the review process to go more smoothly for” both FDA and the submitter, Diamond said.
“There is a lot of synergy” in PreCert and the AI/ML discussion paper, said Diamond. “The two approaches share a common goal, which is to align the regulatory process with the natural lifecycle of software and to facilitate movement of these technologies in a safe manner to the public as timely as possible.” PreCert has been in the pilot phase since 2018 and CDRH invited
more volunteers last week.
With the PreCert program for digital health technologies still under development, Diamond discussed whether adaptive algorithms should be approved or cleared by FDA. “There is nothing in the regulation that would prevent us from clearing or approving adaptive algorithms,” he said, while arguing “a lot of the value for a program like PreCert is that it will allow companies to make modifications to their software in a more streamlined manner.” The concept of GMLP is intended to share this value-add also.
GMLP is among the concepts of the AI/ML discussion paper
from the last month for which CDRH is hoping to receive detailed feedback. CDRH’s overall goal with proposing the AI/ML discussion paper “was to ask questions to the community,” Diamond stressed. He highlighted three “fundamental components,” starting with the emphasis on GMLPs followed by a robust premarket review that includes a plan for adapting ML and transparency, including more information about real-world performance.
According to Diamond, the AI/ML discussion paper intentionally “incorporated generalizability” and did not define “in too much detail” what the concept of GMLP meant. As such, the development GMLP largely hinges on the input CDRH receives from submitted comments. Diamond posed the targeted questions of “what are these practices, how can we codify them and encourage their universal adoption?” The comment period for the discussion paper on AI/ML is set to close next Monday.