Experts call for streamlined EU AI framework, urge greater literacy

BARCELONA – Experts called for a more streamlined framework for artificial intelligence (AI) for healthcare products in Europe and urged regulatory professionals to increase their AI literacy, while cautioning against overreliance on the technology, during a panel discussion at the RAPS AI in Regulatory Conference.

Regulatory framework

Leander Vranken, manager for AI and digital health at MedTech Europe, said he wants to see the immediate implementation of a workable AI regulatory framework in Europe because the technology is present and evolving, noting the ongoing effort to revise the Medical Device Regulation (MDR) and In Vitro Diagnostics Regulation (IVDR).

"We need a regulatory framework that bolsters innovation, enhances patient access, and creates legal certainty," said Vranken. "Please finalize the revision of the MDR and IVDR as quickly as possible."

Vranken said that one of the main concerns his members have raised is that their AI products will have to navigate parallel regulatory frameworks under legislation such as the AI Act, MDR, and IVDR. He said topics such as risk management and oversight are addressed in multiple pieces of legislation, leading to duplicative work, increased administrative burdens, and potentially higher costs and longer timelines.

"We believe as Medtech Europe that the primary vehicle, the primary framework, for AI-enabled medical devices to be regulated should be the MDR and the IVDR," said Vranken. He noted, however, that the European Commission has already recommended this.

"We really believe that the high-risk AI requirements in the AI Act need to be moved to the MDR and IVDR," he added.

Sebastian Fischer, regulatory strategy principal at TÜV SÜD, argued that concerns about parallel regulatory frameworks are unsubstantiated because the AI Act was designed from the outset to ensure that all procedures were integrated. As an example, he noted that Article 43 of the AI Act, addressing conformity assessments, integrates the requirements of the MDR and the AI Act, meaning there's only one TD required.

Erik Vollebregt, an attorney at Axon Lawyers, wasn't as confident that there would be seamless integration of the AI Act with the conformity assessment requirements of the MDR and IVDR. He said there are still major unresolved problems that could have been addressed in the Digital Omnibus and may be addressed in the upcoming revisions to the MDR and IVDR.

"The law is always behind the facts, but the good part is that the law can also fix it," said Vollebregt. "There are quite a lot of rumors going around in industry saying this is not possible. It completely is possible because that's the nice part about the law. It's magic.

"But the problem is that there's always a political component involved in lawmaking," he added. "The problem is now: do we... consider that it is important enough to have the MDR and the IVDR in a frictionless evaluation of integrated AI?"

Vranken concurred, noting that the EU's Irish presidency has shown a willingness to make the necessary changes and highlighting political will from other European countries to streamline the process.

AI literacy

During the discussion, Fischer cautioned that AI shouldn't make people's lives too easy and said professionals should think critically about the technology. He said that overreliance on AI could potentially make people complacent and lead them to eventually forget how to do certain tasks.

"It's not about creating a lot of content and becoming faster and faster without really thinking about what you're doing because AI right now is creating a lot of bad or mediocre content which doesn't really add much value," said Fischer. "The challenge is not to become too lazy and sometimes make it hard for yourself on purpose.

"We still have to continue to think for ourselves and to critically ask whether what the AI, what the [large language model (LLM)] is giving us makes any sense; that's the risk," he added. "Whatever you do, think about proper [quality assurance (QA) and quality control (QC)]."

Nebojsa Serafimovic, head of the clinical investigations unit at the Austrian Agency for Health and Food Safety (AGES), echoed similar sentiments and said the impetus is on everyone involved to learn how to use AI properly, check the results produced by AI, check how the results were determined, and understand which AI to use given the task.

"We don't need to become AI experts, all of us, but we all need to learn how to handle the hammer," he said. "Those who don't will stay behind."

Vollebregt, highlighted that while AI can process large amounts of data faster, it can also create unnecessary extra work if the technology isn't suited to the task. Despite the advancements in AI, he noted that the law is still lagging and urged greater AI literacy.

"People in the regulatory affairs department can sometimes lack AI literacy, which means that they have difficulty judging what the AI is doing, whether that's really what they want, or whether that's the right result," he added.

Fit-for-purpose tools

Vollebregt bemoaned that, in his experience, management sometimes doesn't understand that AI isn't a one-size-fits-all solution and said those in the regulatory affairs and legal departments need specialized tools that aren’t prone to hallucinating. He emphasized the need for management to be educated in different AI tools and why regulatory affairs and legal professionals need specialized AI.

"Especially for regulatory affairs people, the focus should be on improvement of quality," said Vollebregt. "This is why, as a company, you should be very careful to deploy unspecialized large language models because they will do exactly what we don't want: generate large chunks of information that may actually be wrong.

"As management, you should understand that AI is not a productivity tool when it comes to regulatory affairs," he added. "Yes, of course, you are able to process more Information efficiently, but the primary goal should always be to improve quality."

Over the past few years, Fischer said he's noticed an increasing use of LLMs in medical device technical documentation (TD) TÜV SÜD has received. He also noted that companies have increasingly used methodologies such as quantitative techniques and specific algorithms to analyze data and imaging. He also noted that TÜV SÜD uses an in-house AI system called IDAS (Intelligent Document Analysis Solution) to help their experts assess TDs.

Regarding AI use, Serafimovic said AGES has been using REGULUS, a closed AI tool not connected to the Internet, developed by the Swedish Medical Products Agency (Läkemedelsverket), and noted that they are receiving a large volume of AI-generated documents, making it very hard for them to respond. 

"I personally do use AI as a supportive function for the work at the competent authority," he said. "But it's not like the AI is doing any assessment. It's just supportive."

Serafimovic emphasized that other competent authorities in Europe are further ahead in their AI implementation because they have more resources and expertise. However, he noted that AGES treats medical device applications with AI as software and uses existing standards to assess them as software devices that include AI.

While Vranken noted that the European medtech industry supports EU proposals to allow predetermined change control plans (PCCP), which have been adopted by the US Food and Drug Administration (FDA), Fischer was skeptical of their effectiveness. He noted that, in addition to adopting PCCP in the EU, there is also an International Medical Device Regulators Forum technical document on using the tool to allow certain predetermined changes to a software product without submitting a new application.

"I think that PCCPs can be very useful in the context of adaptive AI systems, but then the question is: How many adaptive AI systems are there going to be? Probably not all that many," said Fischer. "Overall, I think PCCP sounds like a great concept for general medical devices, but I don't believe it will be used all that much in Europe."

He argued that the risk-based system in Europe already allows certain changes without the need to submit a new application, provided the manufacturer documents them.

A key concern highlighted by Fischer was that similar to the bottleneck of applications caused by the lack of notified bodies when the MDR and IVDR went into effect, there's an imminent risk that there will be a similar bottleneck of applications due to a lack of qualified notified bodies when the AI Act goes into effect.

"Designation of notified bodies has not started yet and is legally still not possible," said Fischer. "We are preparing as good as we can, but essentially we have to wait until there are simple things like competency requirements for an awful lot of personnel who are not available.

"We have competent personnel, but we still might have to increase or hire different people," he added. However, he also noted that the bottleneck will likely not be as severe as what the industry saw with the MDR and IVDR, because it will affect only a subset of AI products.

RAPS AI in Regulatory Conference