Convergence: Experts offer advice on leveraging AI in regulatory

LONG BEACH, CA – As the world grapples with the proliferation of artificial intelligence (AI) tools, including generative AI, experts urged regulatory professionals not to lose sight of the human element. While such tools can be incredibly valuable, they also require human oversight and interpretation to realize their full potential.

Experts who spoke at the opening plenary session discussed how AI can be leveraged across industries, the promise of the technology, and the challenges in realizing those possibilities. The panelists included Ryan Kurtzman, technology partnerships officer for the City of Long Beach; Bradley Thompson, an attorney at Epstein Becker Green; Michelle Jump, CEO of MedSec; and Jay Mashburn, director of regulatory policy at Recursion Pharmaceuticals. The panel was moderated by Megha Iyer, director of global strategic regulatory affairs at Thermo Fisher Scientific.

Thompson noted that he uses AI to build databases from publicly available US Food and Drug Administration (FDA) documents, including 510(k) summaries. He also said that he has created an AI algorithm that can predict whether FDA will require a clinical trial for a medical device mased on its intended use with a high level of accuracy.

"A lot of what I do is what would be referred to as natural language processing because, as a lawyer, I focus on words quite a bit, and there's so much information locked in unstructured texts, like within [medical device reports] MDRs, the narrative description of what went wrong with the device," said Thompson. "These days, there are millions of MDRs, and there is no feasible way for humans to read those."

"Creating a natural language processing algorithm to extract meaning, trends, and other information out of large unstructured texts is another example [of AI's potential]," he added.

Thompson said that lawyers often use AI for legal research, and that some tools are capable of writing legal memos. Such tools can save lawyers up to a quarter of the time they would spend writing memos themselves, he said, adding that AI-written memos must still be checked for accuracy.

“A lot of these algorithms might have 95% accuracy, which means you have to disagree with them at least 1 out of 20 times if you want to be right,” said Thompson. “You have to take your logic and apply it to [the AI].”

Jump echoed Thompson’s sentiment and noted that regulatory affairs professionals are often inundated with information from the documents they have to process. She added that, as someone who is neurodivergent, AI tools, such as ChatGPT, have very helpful in starting document analysis before then creating more detailed documents with her interpretations.

“Regulatory professionals are constantly charged with writing documents and editing documents,” said Jump. “All of those things can be significantly alleviated and supported by things like ChatGPT and other kinds of generative AI.”

Jump also said that it is important to be skeptical about the results people get from AI, and in her case, she tends to ask the algorithm about the sources it used to get answers because she has found a lot of the information comes from marketing sources rather than government regulatory sources.

"In the regulatory field it is extremely important that we remain reliable, knowledgeable, and the content we generate is trustworthy," she added. “[AI] does not replace human intelligence.”

Mashburn said he uses AI in his policy intelligence work and regulatory operations for routine administrative tasks and preparing regulatory submissions. He noted that he's been somewhat successful in creating first drafts of submission documents, though some tools are better than others, and a lot of work still needs to be done in the space. He also uses the technology to automate and expedite some of his work.

Ultimately, Mashburn said stakeholders should focus on four areas to ensure there are good guardrails around AI. These include data governance to ensure proper storage, sharing, and use of the data; ethical considerations to protect patients and address potential biases; regulatory clarity and compliance to ensure everyone is on the same page on specific terms and is legally compliant; and ensuring the technology is overseen by humans.

Mashburn noted that the vast amount of data being generated today necessitates the use of AI. “It’s paramount that all of us reframe our thinking [to use more AI] because it's only going to get worse," he said, adding that AI is only powerful with the right human inputs, because it doesn’t understand nuance, context, or ethics.

“AI technology is not replacing your job,” said Mashburn. “You shouldn't rely on it and get lazy.”

“It should sharpen your regulatory experience because you're the ones that are going to have to pull out that context, that nuance, that contextualization,” he added.

Kurtzman said Long Beach has used AI to serve its residents for a long time, but the novelty of generative AI is changing how the city operates and does business. On a personal level, he uses it for tasks such as writing emails and analyzing qualitative survey data, but the city also uses the technology in areas such as traffic control and customer service. Ultimately, he said they focus on using a risk-based approach to have consistent, proactive, and secure policies and procedures to ensure that the AI they use conforms to good governance.

"We're in the early stages of this right now, and we've published a guidance for our over 6,000 city staff, which essentially provides guardrails that instruct them on how to use it and we're also consistently trying to talk to our staff to learn, ‘Does this work for you,’" said Kurtzman. “At this stage the technology is outpacing how fast we’re teaching people about the technology, so I think governance and education are really important components of AI and generative AI.”

RAPS Convergence 2024

Convergence: Experts offer advice on leveraging AI in regulatory

Related topics

Recommended content

Welcome to the new RAPS Digital Experience