Industry calls for flexibility on RWD data sources, validation

Industry leaders want to see more flexibility and clarity from the US Food and Drug Administration (FDA) as it seeks to finalize guidance on the use real-world data (RWD) from electronic health records (EHRs) and medical claims to support regulatory decision-making.
 
The comments were in response to the FDA’s draft guidance “Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products,” which was issued in September 2021 (RELATED: FDA issues draft guidance on RWD sourced from EHRs, claims data, Regulatory Focus 29 September 2021). The comment period closed on 24 January 2022. 
 
The guidance focuses on the relevance of data sources, study validation, and data quality related to electronic health records and medical claims data to support regulatory decision-making. While most commenters praised the agency for taking a transparent and detailed approach, they asked for more clarity around metrics of data quality, study validation variables, and the timeline and framework for industry-FDA interactions.
 
Data quality, sources, validation
 
The Pharmaceutical Research and Manufacturers of America (PhRMA) called on the FDA to work with drugmakers and other groups to develop a framework around quantitative and qualitative metrics of data quality that could become an appendix to the final guidance to support regulatory submissions.
 
“We believe that more specific guidance is needed to help enable sponsors and data providers to generate consistent assessment reports for data quality that will meet the Agency’s needs,” PhRMA commented.
 
PhRMA also called on FDA to exercise flexibility around validation. They suggested the variables of importance could be pre-specified for validation and sponsors could submit those plans to FDA in the protocol for discussion before the study begins. They also called on FDA to take a “risk-based approach” to validation of confounders and covariates.
 
Amgen also commented that the guidance could more clearly outline ways to seek agreement with FDA on key variables that require validation. “We agree that validation is an important aspect in any study design, but there will be circumstances when validation may not be always needed or useful (e.g., when validation studies have been previously conducted in the same data source),” the company commented.
 
Flatiron Health, a provider of RWD, urged a risk-based approach to study-specific verification and validation. “It is currently challenging for a retrospective observational study using RWD to generate the same level of evidence as a clinical trial or a prospective observational study with intentional data capture. The Final Guidance would benefit from clarification of this point,” the company said.
 
It also recommended that FDA consider developing data-quality guidelines or convene a series of workshops prior to issuing the final guidance to help stakeholders define factors related to data quality and to align expectations across the industry.
 
Aetion, a health technology company that specializes in generation of real-world evidence (RWE), also called for more specifics from the agency. In cases where some documentation about the data source is not available, Aetion said the FDA could provide recommendations for how sponsors and analytics providers should evaluate the level of available information when deciding whether to use a commercially available dataset in a study. They also asked the FDA to be more explicit about its willingness to consider a “clearly rationalized tradeoff-based approach to the overall validating of a given study,” recognizing that the best approach will vary depending on the context of a specific study.
 
Aetion also recommended that FDA run demonstration projects to more fully explore the use of unstructured data and to determine standards for its use.
 
The RWE Alliance, a coalition of RWD and analytics organizations, encouraged the FDA to “acknowledge explicitly” in the final guidance that the agency will consider the specific circumstances of an RWE study, including the clinical and regulatory contexts, when evaluating the suitability of using data from a particular source for a specific regulatory purpose.
 
The RWE Alliance also asked for more discussion about how to use RWD curated from more than one source, and for more information on the distinctions between recommendations for EHR and medical claims data.
 
IQVIA, a health care analytics technology company, asked for greater clarity on several points including, clear instructions on how to submit protocols and statistical analysis plans, what information is needed to demonstrate “historical experience and use of the selected data source for research purposes,” and more detail on considerations for data linkage, including patient attrition and privacy concerns.
 
HIMSS Electronic Health Record Association (EHRA) recommended that FDA provide guidance to address the need for transparency about data that is escaped or excluded from use as part of data cleaning, normalization, or harmonization processes, since this can impact confidence in study findings. They also called for guidance from the agency about how to manage patient consent around use of identified patient data.
 
The Biotechnology Innovation Organization (BIO) also called on the FDA to offer regulatory flexibility. Specifically, BIO said the draft guidance focused on studies aimed at determining causal inference, but RWD could be used to fill critical evidence gaps in benefit-risk and safety assessments. “For example, this would include an approach tailored for aggregated data and data used to establish natural history, standard of care, or to add context to the results of an RCT,” BIO wrote.
 
Agency interactions, meetings
 
BIO and PhRMA both urged the FDA to streamline the FDA-sponsor communications related to RWE development. The type of protocol and statistical analysis plan submission process outlined in the draft guidance could take several months to years to develop per research project, BIO noted.
 
Janssen Research & Development commented that the company is concerned that requiring sponsors to submit all essential elements of study design, analysis, conduct, the protocol/statistical analysis plan, and/or final study reports prior to submitting a meeting request may inadvertently constrain innovative RWD approaches and lead to delays in evidence generation. The company encouraged FDA to consider revisions to the guidance that allow for a series of focused meetings between FDA and sponsors that allow for review and feedback at various milestones.
 
Flatiron recommended that the FDA consider establishing mechanisms for RWD/E providers to meet with the agency to discuss topics on RWE platform development that are unrelated to a specific sponsor’s study. Currently, generators of RWD/E have no mechanism for directly engaging with FDA on technical or methodological considerations outside the scope of a sponsor’s program, Flatiron noted. “This lack of direct engagement hampers development of RWD/E platforms, including data management and monitoring practices, and establishment of new technical approaches needed to support sponsors utilizing this guidance to design an observational study with EHR and/or claims-derived RWE,” Flatiron commented.
 
Public comments on draft guidance