Evaluating Next-Generation Sequencing: From Design Concept to Performance Standards

Discussions are underway on how the US Food and Drug Administration (FDA) plans to develop new ways of regulating next-generation sequencing (NGS) diagnostics.

Kicking off a two-day public workshop today, FDA Commissioner nominee Robert Califf said that NGS tests hold great promise for patients, but still present "tremendous uncertainty." The issue, he said, is determining "how accurate [and] how useful can we make the predictions that come from these tests."

Califf also indicated that any regulation governing NGS "needs to adapt to the advances in technology."

Background

FDA has been working to develop a regulatory framework for NGS diagnostics since 2014 as a component of the Obama Administration's Precision Medicine Initiative (PMI).

In December 2014, the agency issued a discussion paper, Optimizing FDA's Regulatory Oversight of Next Generation Sequencing Diagnostic Tests, in which the agency proposed "implementing analytical standards that would ensure that NGS tests produce accurate and reliable results" and "developing ways to use well-curated databases of genetic variants to guide clinical interpretation of NGS test results."

Then in September 2015, FDA released a new discussion paper, Developing Analytical Standards for NGS Testing, which presented possible approaches to ensure the validity of NGS-based tests: design concept standards (where developers validate their own tests based on design principles and performance needs), performance standards (where developers would have to conduct studies to hit specific performance criteria) or a hybrid approach combining the two.

Standards Based Approaches

From FDA's perspective, the more flexible design concept approach "would rely on the established ability of the developer to implement well-described principles of test design and validation, and thus consistently generate tests that meet user needs, with performance characteristics that allow correct clinical interpretation."

On the other end of the spectrum, FDA says it could implement a performance standards approach that would define "individual performance evaluation studies that must be conducted and specific performance criteria that must be met" in order to validate NGS diagnostics.

And although David Litwack, a staff fellow at FDA's Office of In Vitro Diagnostics and Radiological Health, cautioned that the proposals in the discussion paper are "just preliminary ideas or approaches," he said the agency hopes to get input to help "develop and implement standards to assure the quality of testing [and to] develop open-sourced tools to help test developers meet those standards."

Panelists at the meeting seemed to endorse a combination of the two approaches.

Zivana Tezak, associate director for science and technology at the Office of In Vitro Diagnostics and Radiological Health, expressed surprise to hear endorsements for the combination approach after early feedback indicated greater support for a design concept approach.

Nomenclature

Throughout the day panelists expressed concerns that many factors related to NGS are not standardized or commonly defined.

"There's a need to resolve issues in nomenclature … we need to ensure all of this is flexible enough to accommodate modifications today as well as in the future," moderator Adam Berger, senior staff fellow for personalized medicine at FDA said.

One panelist, Deanna Church, senior director of genomics and content at Personalis expressed the need for "improved benchmarks to make it easier to compare across labs."

Panelists also agreed that there needs to be standards for defining the coverage NGS tests have of parts of the genome. Narayanan Veeeraraghavan of the Human Genome Sequencing Center raised the question: "Depending on who is doing confirmation and the type of confirmation, how would that [coverage] effect your metrics?"

Others expressed concerns that the internal validation done by NGS developers needs to be made public in order to better communicate their tests' strengths and weaknesses, and to provide assurance of their validity.

Next Steps

FDA is accepting comments on the two public workshops and the ideas presented in its discussion papers until 25 November 2015.

The agency is also planning to host two additional public workshops on NGS in February and March 2016.

FDA