Experts: Specialized AI has great potential for regulatory, needs SME oversight

BARCELONA – Experts said that specialized artificial intelligence (AI) tools can be invaluable in helping companies address regulatory needs for medical devices, while saving time and money, during a panel discussion at the RAPS AI in Regulatory Conference. However, they noted significant limitations of such tools, which ultimately rely on subject matter experts being in control of the AI.

Understanding the limitations

Sander Denissen, CTO of Guideways B.V., discussed the limitations of large language models (LLMs) when used to develop clinical evaluation plans. He also outlined how generalized AI agents have been used and noted that some users have argued they can make lots of mistakes, especially if the tool relies on unverified assumptions.

"The humans should do what they do well: Ethics, strategy, and verifying that the system actually does what it is supposed to do and tell the system step by step what it should do," said Denissen. "What do the agents do well? They're very good at processing large amounts of data and doing searches.

"You need a dedicated agent to help you determine your device classification, dedicated agents to do literature search, dedicated agents to do the unit search, etc., and then finally, you have a fully step-by-step verified, traceable solution that you can generate your set from," he added.

Denissen said the process of having a human verify the AI's work may take hours to days, but it is essential to avoid errors. He added that the AI agents should be designed to keep sensitive documents safe and to ensure that every rule is read and reviewed by a human before the LLM is structured. Finally, he said the system must be fully curated, meaning all device data must be independently verified and approved before use.

‘A real game changer’

Liz Munro, president of Toronto-based Liz Munro Consulting, said she uses Guideways AI, a specialized AI platform for medical devices and digital health, in her work, which primarily involves consulting with early-stage medtech companies. She noted that most of the companies she works with are still at the concept or prototype stage and are trying to determine a regulatory strategy for their products.

Munro presented two case studies where she used specialized AI tools. She noted that before using such tools, she spent most of her time doing research for her clients but didn't have a lot of time left over to synthesize that research into a foundational strategy. However, the situation has flipped, and she said she can now prompt an AI tool with questions without spending hours compiling research herself. Instead, she said the AI can conduct highly accurate research, and she can spend a more of time with her clients on which strategy they should consider.

"It's been a real game changer," said Munro. "I just feel like I can give them so much more in a short period of time than I previously would have been able to."

One area where specialized AI can be especially helpful, according to Munro, is helping a company determine what claims it wants to make about its product based on external factors such as the time and money available to develop it. She said companies may often want to make certain claims about their product, but they may not have the time or resources to make those claims, and instead may decide to make a more limited claim about the product so it can get to market sooner.

Despite the potential benefits of specialized AI, Munro also reminded the audience about its limitations. She noted that its judgment is still very much in the gray zone, and it is crucial to have a human in the loop to make the final decisions. For example, she noted that the AI may suggest a device classification, but when dealing with regulators such as the US Food and Drug Administration (FDA), there is rarely a single right answer regarding which classification the product falls under.

Munro also said that the AI may be limited to publicly available information, which a human may have more context on, and it may base its recommendations on de-linked precedent information because it doesn't understand how regulators may treat different devices in the same disease space.

Despite the limitations, Munro said specialized AI can be useful in developing pre-market strategy by providing clarity on the regulatory landscape and competitive precedent, identifying key risks, and addressing questions about safety and effectiveness. Similarly, it can be valuable in developing design controls and postmarket strategy, and in informing clinical study operations.

"There's a huge potential for these tools in actual clinical study operations, going from a study synopsis and having specialized tools, drafting complete compliant protocols, monitoring, and source data verification that's currently done manually and takes hours and hours and hours of CRO time," said Munro. "There are ways to do it much more efficiently, and then analysis and report generation that in the future are going to be done by these types of agents."

SMEs in control

Christine Teich, director of quality assurance and regulatory affairs at Disruptive Novotech in Sweden, discussed how her company has used agentic AI for regulatory purposes. She said that, initially, her company adopted AI for quality management, but now also uses it to draft regulatory records, create technical documentation, and assemble product submissions. However, she said her company found that the real potential of agentic AI lay in conducting gap analysis.

"You throw in whatever record you want, and you can check it against your own QMS, you can check it against standards, regulations, guidance documents," said Teich. "It comes with every clause that you are missing...that's our superpower, and it's so much faster than if you do it by hand."

Teich said the technology has certain limitations, such as relying on outdated information, lacking nuance, basing its outputs on black-and-white findings, failing to iterate, and requiring experienced users in quality assurance and regulatory affairs to ensure effective functioning.

"The tool is only as powerful as the person that is using it," she said. "You still have to be aware. You still have to use your brain and cannot blindly trust it. You have to steer it sometimes in the right direction."

Despite that, Teich noted that it has been effective at conducting gap analyses, providing source-backed findings, and producing faster first drafts of documents. She said the AI can also provide faster answers to regulatory questions and lower consultant costs. To effectively use the AI tool and bring it under control, she said it should be brought into the QMS.

Teich also said the use of AI should begin with the user's needs, and they shouldn't be afraid to validate the tool. Finally, she reiterated the importance of keeping humans in the loop.

"Please keep experienced people in control because the subject matter expert that is using the tool makes it powerful," said Teich.

RAPS AI in Regulatory Conference