The Case for Quality, a public-private partnership between the US Food and Drug Administration (FDA) and the Medical Device Innovation Consortium (MDIC), has released a new report on the feasibility and effectiveness of using analytics to measure medical device quality in healthcare decision-making.
In order to measure these factors, a Case for Quality working group ran a pilot focusing on knee and defibrillator implants at seven hospitals to determine whether data, both public and private, could be used to measure device quality.
Background: The Case for Quality
FDA launched the Case for Quality initiative in 2011 after a report by the agency found that companies with a greater focus on quality tended to see fewer complaints and less frequent internal investigations, oftentimes while spending less on quality-related issues.
According to FDA Center for Devices and Radiological Health (CDRH) Director Jeff Shuren, these insights, coupled with other enforcement-related inefficiencies called for a paradigm shift around quality.
"The number of inspections with official action indicated outcomes has remained high over a sustained time-period, with the same issues occurring frequently year after year. Essentially, we're just playing a game of whack-a-mole," Shuren said, adding that oftentimes "one device manufacturer can meet FDA requirements and still make a poor quality device, whereas a second manufacturer may not comply with all FDA requirements and yet make a high quality device."
Shuren says this isn't cause to throw out the agency's quality system regulation (QSR), but emphasizes that the QSR alone is not enough to ensure device quality.
Instead, Shuren says the goal for the Case for Quality is to identify best practices for quality within industry and for FDA to adapt its standards to foster an environment that promotes quality.
Analytics for Device Quality
According to the report, while healthcare providers use data to make decisions about which devices to use, inconsistencies and gaps in that data can undermine the integrity of those decisions.
Specifically, the report says that healthcare providers often must make decisions about which devices to use while lacking "unbiased, relevant, consistent and available data," and in lieu of defined quality dimensions or analytical methods for measuring quality.
Currently, the groups that make such decisions, referred to as value analysis committees (VACs), say they struggle to differentiate devices based on quality due to limited or untimely access to data.
In order to address these issues, the Case for Quality Product Quality Outcomes Analytics (PQOA) team launched a pilot to look at whether data from public and private sources could be used to rank devices across seven relevant domains: safety, effectiveness, reliability, usability, compatibility, patient experience and availability.
However, despite utilizing sources such as PubMed Central, Clinicaltrials.gov, FDA's Manufacturer and User Facility Device Experience (MAUDE) database, and online patient forums, the pilot was unable to find data sources to measure device availability.
For the remaining six domains, the pilot ran analyses of the data to calculate key performance indicators for each area, which were then given rankings of gold, silver or bronze based on how the company performed in each area compared to the average.
These rankings were then used to populate dashboards comparing quality performance by data source, manufacturer and product.
While the feedback on the dashboards was positive, there was strong concern among participants over the potential for bias in the data used to generate the dashboards.
Going forward, the report says that three issues must be addressed:
- "Third-party adoption and development of product quality outcomes analytics across the seven quality domains
- Creating demand for, and broad-based acceptance and utilization of, the quality criteria across provider stakeholders
- Development of formal feedback mechanisms to manufacturers based on the outcome of analytics across the seven quality domains"
"This is the first step toward applying a standardized model of data and techniques to device performance. More work remains ahead to mitigate data bias, ensure accurate interpretation and use of the data and independent management of it," said Joanna Engelke, senior vice president of global quality and regulatory at Boston Scientific and PQOA working group member.
Press Release, Report