The Learning Portal will be under maintenance Saturday, 8 August between 6 AM and 6 PM EST. Portal functionality will be unavailable during this window.
We apologize for any inconvenience caused during this time.

RAPS is closely monitoring developments in the Coronavirus (COVID-19) outbreak. See our public safety page for the latest updates.

Regulatory Focus™ > News Articles > 2020 > 7 > A military-civilian perspective on real-world evidence to support regulatory decision making

A military-civilian perspective on real-world evidence to support regulatory decision making

Posted 29 July 2020 | By Todd E. Rasmussen, MD, and Brian J. Young, MBA 

A military-civilian perspective on real-world evidence to support regulatory decision making

This article summarizes the framework promoting the collection and analysis of real-world data (RWD) in the healthcare system. The authors emphasize how the US Military Health System (MHS) used RWD during the Iraq and Afghanistan Wars to achieve historic rates of survival and describe a new era of collaboration between the US Department of Defense (DoD) and the US Food and Drug Administration (FDA). The article reviews the FDA evidentiary standards for medical product approval and gives examples of how RWE can help meet those standards.
During the Afghanistan and Iraq Wars there was an imperative for the MHS to develop medical products to save the lives and improve the recovery of injured service members.1 This effort, coupled with an inability to perform typical clinical research, such as randomized controlled trials (RCT), during the wars, required the DoD to maximize the use of clinical data from its trauma registry to advance care.2,3 This approach, referred to as “focused empiricism,” used available data to refine treatments and develop new products until higher levels of evidence could be developed.1-4
The military’s use of RWD to achieve historic rates of survival was noted by the National Academies of Sciences, Engineering and Medicine as a lesson for civilian health systems, federal research programs, and industry.4 In an era of expanding sources of health data, which can be used to develop new approaches to care, there is a need for the private and public sectors to recognize and use a new framework that guides RWD.5 This article reviews the topic of RWD, outlines its strengths and limitations, and gives examples of how it can be used within a spectrum of evidentiary standards for regulatory decision-making. The article also describes the DoD medical research program and how it now works within the context of a new Public Law 115-92 to coordinate with the FDA on the delivery of military-relevant medical products.
Definition of real-world data and real-world evidence
Growth in use of RWD in healthcare has accelerated with the use of electronic health records (EHRs) and advances in computing, health care apps, and wearable sensors that enable the collection of real-time physiologic data.5 The 21st Century Cures Act charged the FDA with evaluating the use of real-world evidence (RWE), including its potential to support the approval of medical products.6 FDA responded by issuing guidance and providing a framework for an RWE Program.7-10 FDA defines RWD as that relating to patient health status and/or the delivery of care that is routinely collected from a variety of sources during the management of patients.5,6 RWE is derived from the analysis of RWD and provides confirmation of the potential benefits, limitations, and risks of a particular approach to care or medical product.
In addition to EHRs, the use of registries that apply to certain medical conditions and injury patterns has expanded as a way of conducting continuous quality improvement (CQI) in these areas of care. The DoD Trauma Registry (DoDTR), as well as registries supported by the American College of Surgeons, the Society for Vascular Surgery, and the American Association for the Surgery of Trauma (AAST), are among the sources of RWD relevant to the military as it optimizes care and develops products for wartime injury.11-14 These databanks rely on electronic and/or manual abstraction to populate clinical data elements common in the routine course of patient care and not just as endpoints for hypothesis-driven research.
Real-world evidence versus controlled trials
RWD is used as part of CQI programs in certain areas of care as a source for academic endeavors and as a way to optimize claims and billing within healthcare systems.5 Sources of RWD can be examined with a variety of methodologies, ranging from chart reviews and observational studies, to cohort analyses, video reviews, and even pragmatic trials. Evidence derived from RWD may also be used for regulatory purposes, including approval of new products and expanding the indications or tracking the performance of products already on the market (Table 1). FDA describes two domains of RWE that should be considered with its use. These include the setting in which the evidence is generated and the methodologic approach used to conduct the surveillance or study.5


Sources of RWD, such as clinical registries and national databases, often have multiple stakeholders, including professional organizations, the federal government, health systems, and guilds of providers with varying degrees of interest or competition. This factor, along with limitations inherent to the retrospective analysis of data of inexact quality and provenance, add complexity to using RWD. As such, a great deal of planning should go into designing RWE studies, starting with understanding the stakeholders, the origins and completeness of the data, and the best methodologic approach. Although studies of RWD can yield results that can answer original hypotheses, ill-conceived studies are prone to generate unreliable or misleading findings.15
Unlike RWD analyses, prospective clinical studies, such as RCTs, often involve just one stakeholder and have a single purpose and study design. RCTs are hypothesis driven and have inclusion criteria, endpoints, and statistical strategies that account for confounders and bias and are specific to the effectiveness question. Although RCTs are a standard for attaining level I and II evidence, they have drawbacks owing to their complexity, expense, and the time required to plan and complete them. RCTs require millions of dollars and take years to finish. In addition, their strict design can limit the generalizability of findings to real clinical practice.5 The duration of some RCTs may also make their results impractical for evaluating products amidst rapid technological change. In these situations, the development of next-generation products may outpace RCT completion and reporting, thereby lessening the study’s relevance and its human and financial capital.
Not an ‘all-or-nothing’ approach
Generating and analyzing evidence to evaluate medical products is not an all-or-nothing approach. RCTs will remain a standard in this effort, especially because trials are structured in a pragmatic fashion using adaptive and Bayesian methodology.16 However, in an era of increasing amounts of RWD, properly developed RWE tailored to the characteristics of a specific product or disease condition can inform the design of RCTs and supplement the information gathered from completed trials. Registry data from clinical settings, for which it is uniquely difficult to perform an RCT (e.g., cardiac arrest and hemorrhagic shock), help define the scope of the problem and identify patient groups that may be amenable to randomization and/or controlled study. Registry data pertaining to a product that has received regulatory approval can also serve as postmarket surveillance to provide a more comprehensive picture of the benefit-risk profile.
Biotechnology companies have used RWE to support innovation and commercialization efforts, including examples of doing so in coordination with FDA and the DoD. A recent public-private partnership, partially funded by FDA, called the National Evaluation System for Health Technology Coordinating Center (NESTcc) is an example of a forward-leaning initiative whose mission is to “catalyze timely, reliable, and cost-effective development of RWE to enhance regulatory and clinical decision-making.”17 However, lack of access to and understanding of the sources and types of RWD can hamper the effectiveness of even the most concerted efforts. Moreover, those working with the DoD may not understand the military’s sources of RWD or its new cooperative position with FDA. Understanding the breadth of this RWE ecosystem is important for effectively navigating medical product development.
FDA evidentiary standards
There are many types of regulatory decisions spanning the total product lifecycle that necessitate different levels of evidence (Table 1). These include regulatory approval, as well as postmarket compliance decisions. Whereas the evidentiary standard for human drug and biological products approvals is “substantial evidence” of effectiveness based on adequate and well-controlled investigations, the evidentiary standards and decision-making criteria for medical device marketing approval varies according to the risk-based classification of the device.
Devices are grouped into Class I (low risk), Class II (moderate risk), and Class III (high risk) categories, based on their associated risk and the level of regulation necessary to ensure safety and effectiveness pursuant to the requirements in Section 513(a) of the Federal Food, Drug and Cosmetic Act (FD&C Act). This risk-based classification has a direct bearing on the quantity and quality of evidence FDA requires to support marketing approval.
Class I
Class I device types comprise about 50% of all medical devices in the US and generally can be marketed in the US without prior FDA review.
Class II
Class II device types comprise about 43% of all medical devices in the US. Class II devices most often require FDA 510(k) clearance to be marketed in the US, pursuant to requirements in 21 CFR 807.100(b). FDA 510(k) decisions are based on a finding of “substantial equivalence” to a valid, legally marketed, predicate device. Substantial equivalence means that the device is as safe and effective as the predicate, as determined by its intended use and technological characteristics. The FDA requests clinical data for about 10% of 510(k)s.18 “In many cases, the data necessary to support a 510(k) involves a relatively small number of patients and may involve a simpler study design than is necessary to support a premarket approval application.”19
De novo
This process provides a pathway to Class I or Class II classification for medical devices for which general controls or general and special controls provide a reasonable assurance of safety and effectiveness, but for which there is no legally marketed predicate device. De novo classification decisions are based on the same evidence requirements as Class III premarket approval (PMA) devices (i.e., reasonable assurance of safety and effectiveness).
Class III
Devices in this class comprise a small fraction of all medical devices in the US, for example, only 64 original PMAs were approved in 2017.20 Class III (high risk) devices are subject to PMA and require scientific evidence to provide a “reasonable assurance of safety and effectiveness” in accordance with the rules set forth in 21 CFR 860.7. Meeting this standard often requires a large RCT with hundreds of patients.
FDA’s evidentiary standard for premarket review of devices is “valid scientific evidence,” a standard established by Congress in 1976 that still sets the benchmark for evidence to support premarket submissions.20 The definition of “valid scientific evidence” in 21 CFR 860.7 includes “evidence from well-controlled investigations, partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience with a marketed device,” and specifically excludes “isolated case reports, random experience, reports lacking details to permit scientific evaluation, and unsubstantiated opinions.”
Emergency use authorizations
In contrast, the evidentiary standard for emergency use authorizations (EUA) in Section 564 of the FD&C Act is a reason to believe, based on the totality of evidence, that a product “may be effective” in diagnosing, treating, or preventing an agent that is the subject of an emergency declaration (e.g., COVID-19). The “may be effective” standard provides a lower level of evidence than the “effectiveness” standard FDA uses for product approvals. Regulations about treatment investigational new drugs (IND) and investigational device exemptions (IDE) also use the words “may be effective.” A request for a treatment IND for a drug or biologic intended to treat a life-threatening condition may be granted when, among other things, there is evidence it may be effective for its intended use in its intended population (21 CFR 312.320(a)(3)(ii)).
Humanitarian use devices
Humanitarian use devices, intended to treat small numbers of patients without available alternative therapies, likewise have a lower level of evidence burden. A device that with humanitarian use device designation is eligible for a humanitarian device exemption if, among other criteria, evidence is provided that it will not expose patients to an unreasonable or significant risk of illness or injury, and the probable benefit from use of the device outweighs the risk from its use. A review of recent humanitarian device exemption decisions indicates that most studies are relatively small, involving about 30 patients.
As noted, evidentiary standards range from a demonstration the product is safe and effective on one extreme, to a demonstration a product will not expose patients to unreasonable or significant risk of illness or injury (see Table 2).


Accordingly, matching trial design to the benefit-risk profile of a product and FDA evidentiary standard is imperative. In some circumstances, RWE can have advantages and may detect conditions not exposed in traditional trial designs. The key to using RWE, as with any trial design, is to use appropriate rigor in matching the study methodology to the benefit-risk profile of the device, study hypothesis, and data needs. A poorly designed trial using RWE is of little value, whereas a carefully constructed trial can provide insight to guide regulatory and clinical decision-making. Examples of RWE methodological approaches are shown in Table 3.


An analysis of real-world evidence in recent PMA and de novo decisions
Class III products subject to PMA and de novo devices rely on the same “reasonable assurance of safety and effectiveness” standard.20 As such, they fall within the upper limits of the evidentiary standards spectrum for medical devices. To test this assumption, the authors performed an analysis of the last 10 original approved PMA’s for which a Summary of Safety and Effectiveness is available, and the last 10 de novo classification orders for which a Decision Summary is available. As shown in Table 4, this exercise confirmed that RCTs and other traditional trial designs were overwhelmingly the most common means by which evidence was generated to support class III original PMA and de novo approvals. It stands to reason that original PMA’s would most often involve RCT’s and other traditional study designs, whereas PMA supplements and 510(k)s would be more amenable to alternative approaches.


A new era of DoD and FDA coordination
The DoD is engaged in a range of FDA-regulated product development through its medical research program.21 The MHS research program is charged with delivering new drugs, devices, and technologies to address priority areas of casualty care, as well as disease conditions that affect its roughly 10 million beneficiaries. The DoD accomplishes this mission with a requirements-driven program that invests in the various research and development stages of different medical products from basic research, to preclinical translational study, to clinical and regulatory trials.21 To be effective at delivering these innovations into clinical practice, the MHS must work in a coordinated manner with civilian academia, industry, and FDA.
The structure governing product development efforts between the MHS and FDA changed in December 2017, with the signing of Public Law (PL) 115-92.22 The law expands the scope of a secretary of defense designation of a military emergency and, thereby, the EUA authority as retained by FDA under the law for scenarios beyond chemical, biological, radiological, or nuclear (CBRN) threats. Prior to the law, the MHS was often unable to justify an EUA request for urgently needed, investigational treatments for battlefield scenarios where there was no link to a CBRN threat. The expanded scope of the EUA accommodates products aimed at saving the lives, limbs, and eyesight of US service members injured from common sources of trauma, such as explosions and gunshot wounds.
PL 115-92 also allows the DoD to ask FDA to expedite the development of a product, or products, if it can be shown to “reasonably likely” prevent, diagnose, treat, or mitigate specific and imminently life-threatening consequences of an attack on US forces.22 PL 115-92 also prompts FDA to offer flexibility to accelerate consideration of military priorities through use of one or more of its expedited programs (e.g., fast track, breakthrough therapy or device, accelerated approval, or priority review). Finally, the law directs the DoD and FDA to engage in enhanced collaboration, including statutorily directed meetings “to discuss the development status of regenerative medicine therapies, blood and vaccine products and projects that are of highest priority to the DoD.”22
PL 115-92, resulting from lessons learned during the Iraq and Afghanistan wars, has already had an impact. Six months after the law was signed, FDA granted the military EUA to enable emergency use of pathogen-reduced, leukocyte-depleted, freeze-dried plasma manufactured by the French military. An initial work plan, signed by the MHS and FDA, outlined steps that FDA will take in response to PL 11-92. Currently, there are a number of drugs, devices, and biologics being considered under the new law. Although RWE is not mentioned specifically in PL 115-92, its use in the context of efficient regulatory decision-making for military products is implied in the spirit of the law.
Good versus poor RWE study designs
FDA’s openness to RWE does not signify a relaxation of requirements for showing safety and effectiveness. The height of the evidence bar for study quality depends on how the data will be used in regulatory decision-making. For example, registry data may be sufficient to support postmarket surveillance in a 522 order, but not to support premarket approval of a new use.
While the quality of study design is subjective, good RWE studies must yield relevant and reliable data. FDA defines considerations for relevance and reliability in its RWE guidance.8 Relevant and reliable study designs typically have the following characteristics: early collaboration among stakeholders; identification of sources of data; a defined hypothesis; a well-design protocol and statistical plan; identification and control of bias in source data and methods; and compliance with all regulatory requirements.
Failure to obtain stakeholder (i.e., the FDA) buy-in before initiating the study is perhaps the single biggest mistake made in RWE trials. If the RWE study is intended to support regulatory approval, then the approving authority should be consulted before the study begins. Failure to do so could lead to delays, increased costs, and, in some cases, a study that must be redesigned and/or repeated. FDA, and most regulatory authorities, have standardized processes for obtaining presubmission feedback (e.g., FDA Q-Submission Program).
Categories for use of real-world evidence
Postmarket surveillance of approved products
The use of RWE to monitor the safety of approved products is a common approach. For example, the Netflix documentary, “The Bleeding Edge,” portrayed weaknesses in the system for approving devices for hip implants, implantable birth control, transvaginal mesh, and surgical robotic systems. The film portrayed the lack of premarket clinical data and postmarket follow-up as being at the heart of the reported problems. Notably, three of the products are permanent implants, which poses challenges in trial design owing to the length of surveillance needed to detect problems that may arise years after the procedure. It was alleged in the documentary that there had been no thorough postmarket study of these implants. One strategy to mitigate this risk is to continue to study or surveille patients who were enrolled in the original RCT(s) conducted to meet the evidentiary standards, assuming a premarket clinical study was required for approval.  Although attaining long-term data on a product is essential, investment in postmarket study is difficult to sustain, and outcomes data are diminished as patients are lost to follow-up.
As another example, in 2019, the FDA released a letter to health care providers in which it expressed concern relating to paclitaxel-coated balloons and paclitaxel-eluting stents.23 The devices are used to treat limb ischemia resulting from peripheral arterial disease (PAD) and had received approval on the basis of evidence from RCTs in the preceding years. In December 2018, a meta-analysis identified a long-term mortality signal associated with the use of these products.24 Through a series of actions, FDA alerted providers, patients, and the public of this finding and took steps to address it, including an initiative referred to as Registry Assessment of Peripheral Arterial Devices, or RAPID.25,26 RAPID includes experts from academia, industry, and the clinical community and aims to improve the RWE ecosystem around PAD devices.26
RCTs may ultimately be needed to confirm or refute any relationship between paclitaxel-eluting devices and mortality, but such an approach would take years to complete. In the near-term, sources of RWD, such as patient registries and information on patients who were part of the original RCTs, can be useful. The RAPID initiative and efforts, such as NESTcc, that are aimed at improving the various forms of RWD, are helping to provide the best current data possible to guide physicians and patients until more conclusive evidence can be attained.
Expanded indications for approved products
In 2017, FDA approved a new procedure for transcatheter aortic valve replacement (TAVR). The manufacturer gained initial market approval from FDA in 2011, after which it established a registry to collect data from more than 100,000 TAVR cases. Among these were 600 physician-led, off-label uses of the new TAVR procedure. Based on RWE derived from the data in this product registry, FDA subsequently approved this new procedure without requiring a traditional RCT. FDA’s acceptance of RWE in this case expedited approval of the new procedure. Although the US was the 42nd country to approve the original device, it was the first to approve the new procedure.27-29
Successful RWE was also used to expand indications for a new hemorrhage control and resuscitation device innovated by the US military and referred to as resuscitative endovascular balloon occlusion (REBOA).30 The manufacturer obtained the original 510(k) clearance in 2015 based on bench and animal testing demonstrating substantial equivalence to a predicate device. No clinical data was required for the original clearance. Postmarket RWD on the use of this device in the civilian trauma setting was collected using the AAST Prospective Aortic Occlusion for Resuscitation in Trauma and Acute Care Surgery registry and data on its use in deployed settings gathered in the DoDTR.30 Evidence from these sources of RWD supported another 510(k) in 2017 to modify product labeling to allow its use without medical imaging (x-ray), thereby enabling its use in emergency and resource-limited settings.
Peer-reviewed, published RWE on the REBOA device was later used to support another 510(k) to amend the labeling to remove a contraindication for pregnancy so that the device could be readily used to control emergent postpartem hemorrhage. RWD from patient records and videotaped procedures was later coupled with novel design features aimed at reducing unwanted effects to support clearance of a new partial REBOA catheter in 2020.31 Real-world evidence was used three times in five years to expand the labeling and support design changes for this device. Pragmatic use of RWE in this case was welcomed by the DoD as way to hasten delivery of a high-priority bleeding control device to the deployed setting, but at a lower cost to the government.
Historical controls for regulatory studies
 For certain medical products intended to treat conditions not amenable to prospective RCT, existing sources of RWD, such as registries, may provide information on comparable sets of patients. In these situations, RWE may be used as a control in a “single arm” clinical study. This approach reduces the burden on the sponsor by decreasing the number of patients and the cost and time of the study. The RWE control may come from a natural history study on the disease progression or a condition or injury receiving the current standard of care. RWE controls may also come from a meta-analysis of an approved treatment.
Natural history studies (retrospective or prospective) are often useful for rare diseases.32 By way of example, in 2017 FDA approved Brineura (cerliponase alfa) following a single-arm study that used historical controls. Brineura was the first approved treatment to slow loss of walking ability in patients with late infantile neuronal ceroid lipofuscinosis type 2 (CLN2), rare disorder that most affected children do not survive past their teens. The clinical trial establishing Brineura’s efficacy was a nonrandomized, single-arm, dose-escalation study in 22 patients. The “control,” or comparator, in the study consisted of 42 untreated children with CLN2 from a natural history cohort. Patients treated with Brineura suffered fewer declines in walking ability compared with the untreated patients in the natural history cohort.27,33,34
Historical controls from sources of RWD are also used to support device studies in furtherance of FDA marketing approval. One recent example is the de novo classification order for the Bluegrass Vascular Technologies’ Surfacer Inside-Out Access Catheter. The single-arm pivotal US IDE study of the Surfacer system showed that reported complications for the system were lower than historical data for standard-of-care approaches to central venous access reported in the literature.35
Expanded access program
Experience gained through the emergency use of an investigational products program is another mechanism through which safety and efficacy data can be gained. In these cases, physicians apply to FDA for a single-use IND or IDE to treat a patient with a medical product outside of a clinical trial when no comparable or satisfactory alternatives are available. Because the product is often used in an extreme clinical setting, RWD from these cases can supplement information being gathered from more controlled regulatory trials. Obtaining expanded access approval requires coordination between the physician, his or her Institutional Review Board, and the manufacturer, who must provide authorization for product use. If a number of expanded access cases are performed during the conduct of separate regulatory trials, RWD from these experiences could add to the evidence (good or bad) pertaining to a given medical product, although this is not common.
Compelled by the urgency of the wars, the MHS has used focused empiricism to develop new approaches for casualty care. In doing so, it implemented RWD and evidence into a system of performance improvement and product development to achieve historic rates of survival. At the same time, there was an exponential increase in the number of RWD sources in civilian and federal healthcare. Evidence derived from these sources has had the potential to improve the quality of care and the development of new medical products for military and civilian patients.
Potential notwithstanding, RWE is not a panacea, and work must be done to improve its sources and the analytics by which it is derived. Investments should be made with an emphasis on collecting relevant and more complete data. As the fidelity of data improves, so should the analytical approaches. Understanding the strengths and limitations of RWD, as well as ways in which it can be applied to meet FDA evidentiary standards, will optimize investigative methods. Finally, advances should be made in harmonizing sources of data on similar patient populations among disparate or even competitive stakeholders.
There is now an ecosystem supporting the use of RWE that is underpinned by FDA guidance and initiatives and empowered by private-public partnerships and Public Law 115-92. Now, more than ever, it is important for civilian health systems, federal programs, and industry to recognize and invest in this ecosystem to improve efficiency of medical product development and advance patient care.
AAST, American Association for the Surgery of Trauma; CLN2, ceroid lipofuscinosis type 2; CQI, continuous quality improvement; DoD, [US] Department of Defense; DoDTR, DoD Trauma Registry; EHR, electronic health record; EUA, emergency use authorizations; FDA, [US] Food and Drug Administration; FD&C, Federal Food, Drug, and Cosmetic [Act]; IDE, investigational device exemptions; IND, treatment investigational new drugs; MDR, medical device reports; MHS, Military Health System; NESTcc, National Evaluation System for Health Technology Coordinating Center; PAD, peripheral arterial disease; PMA, premarket approval; RAPID, Registry Assessment of Peripheral Arterial Devices; RCT, randomized controlled trial; REBOA, resuscitative endovascular balloon occlusion of the aorta; RWD, real-world data; RWE, real-world evidence; TAVR, transcatheter aortic valve replacement.
  1. Rasmussen TE, Kellermann AL. Wartime lessons – shaping a national trauma action plan. N Engl J Med. 2016;375(17):1612-5.
  2. Howard JT, Kotwal RS, Stern CA et al. Use of combat casualty care data to assess the US military trauma system during the Afghanistan and Iraq conflicts, 2001-2017. JAMA Surg.
  4. Bailey JA, Morrison JJ, Rasmussen TE. Military trauma system in Afghanistan: Lessons for civil systems? Curr Opin Crit Care. 2013;19(6):569-77.
  5. National Academies of Sciences, Engineering, and Medicine. A national trauma care system: Integrating military and civilian trauma systems to achieve zero preventable deaths after injury. Washington, DC: National Academies Press, 2016.
  6. Sherman RE, Anderson SA, Dal Pan GJ et al. Real-world evidence – what is it and what can it tell us? New Engl J Med. 2016; 375(23):2293-97.
  7. 21st Century Cures Act. H.R. 34, 114th Congress. 2016. Accessed 24 July 2020.
  8. Food and Drug Administration. Framework for FDA’s Real-World Evidence Program. 2018. Dated December 2018. Accessed 30 June 2020
  9. Food and Drug Administration. Use of real-world evidence to support regulatory decision-making for medical devices. Guidance for industry and Food & Drug Administration staff. Dated August 2017. Accessed 30 June 2020.
  10. Food and Drug Administration. Real-world evidence. Last updated 23 March 2020. Accessed 30 June 2020.
  11. Food and Drug Administration. Submitting documents using real-world data and real-world evidence to FDA for drugs and biologics. Guidance for industry. Last updated 23 March 2020. Accessed 24 July 2020.
  12. American College of Surgeons. ACS Trauma Quality Improvement Program (TQIP). Accessed 24 July 2020.
  13. American College of Surgeons. ACS National Surgical Quality Improvement Program (NSQIP). Accessed 24 July 2020.
  14. Society for Vascular Surgery. SVS Vascular Quality Initiative (VQI). Accessed 24 July 2020.
  15. American Association for the Surgery of Trauma. Multi-institutional studies. Accessed 24 July 2020.
  16. Cannon JW, Rasmussen TE. Addressing limitations in case-control study of patients undergoing resuscitative endovascular balloon occlusion of the aorta. JAMA Surg. 2019;154(12):1166-7.
  17. Food and Drug Administration. Adaptive Designs for Device Clinical Studies. Guidance for Industry and FDA Staff. Dated July 2016. Accessed 5 July 2020.
  18. NEST Coordinating Center website. Accessed 5 July 2020.
  19. Food and Drug Administration. The 510(k) program: Evaluating substantial equivalence in premarket notifications [510(k)]. Guidance for industry and Food and Drug Administration staff. 28 July 2014. July 2014. Accessed 3 July 2020.
  20. Food and Drug Administration. The 510(k) program: Evaluating substantial equivalence in premarket notifications [510(k)]. Draft Guidance for industry and Food and Drug Administration staff. Dated December 2011.  Accessed 28 July 2020.
  21. Food and Drug Administration. Medical device safety action plan: protecting patients, promoting public health. Accessed 4 July 2020.
  22. Rasmussen TE, Kellermann AL, Rauch T. A primer on the Military Health system's approach to medical research and development. Acad Med. 11 February 2020 [Epub ahead of print]. [Behind paywall]
  23. Kelly JJ. Public Law 115-92: A new era of collaboration between DoD and FDA. FDAL Updates. August/September 2018. Accessed 24 July 2020.
  24. Food and Drug Administration. Treatment of peripheral arterial disease with paclitaxel-coated balloons and paclitaxel-eluting stents potentially associated with increased mortality [Letter to health care providers]. Dated 17 January 2019. Accessed 24 July 2020.
  25. Katsanos K, Spillopoulos S, Kitrou P, Krokidis M, Karnabatidis D. Risk of death following application of paclitaxel-coated balloons and stents in the femoropopliteal artery of the leg: A systematic review and meta-analysis of randomized controlled trails. J Am Heart Assoc. 2018;18;7(24):e011245.
  26. Food and Drug Administration. August 7, 2019 Update: Treatment of peripheral arterial disease with paclitaxel-coated balloons and paclitaxel-eluting stents potentially associated with increased mortality. Accessed 24 July 2020.
  27. Medical Device Epidemiology Network. Registry Assessment of Peripheral Arterial Devices (RAPID) phase III. Face to face meeting executive summary March 20, 2019. FDA headquarters. Accessed 24 July 2020.
  28. Belson NA. FDA’s historical use of ‘real-world evidence.’ 2018. Accessed 3 July 2020.
  29. Shuren J, Zuckerman B. How creative FDA regulation led to first-in-the-world approval of a cutting-edge heart valve. FDA Voice. Posted 14 June 2017. Accessed 24 July 2020.
  30. Mezher M. FDA used real-world evidence in heart valve approval. Regulatory Focus. Posted 14 June 2017. Accessed 24 July 2020.
  31. Rasmussen TE, Eliason JL. Military-civilian partnership in device innovation: Development, commercialization and application of Resuscitative Endovascular Balloon Occlusion of the Aorta (REBOA). J Trauma Acute Care Surg. 2017;84(4):732-5.
  32. Russo RM, Franklin CJ, Davidson AJ et al. A new, pressure-regulated balloon catheter for partial resuscitative endovascular balloon occlusion of the aorta (REBOA). J Trauma Acute Care Surg. 27 April 2020. Online ahead of print.
  33. Califf RM, Sugarman J. Exploring the ethical and regulatory issues in pragmatic clinical trials. Clin Trials. 2015;12:436-41.
  34. Food and Drug Administration. Rare diseases: Natural history studies for drug development. Guidance for industry. Draft guidance. Dated March 2019. Accessed 4 July 2020.
  35. Food and Drug Administration. FDA approves first treatment for a form of Batten disease. Released 27 April 2017, updated 28 March 2020. Accessed 4 July 2020.
  36. BlueGrass Vascular Technology website. Accessed 4 July 2020.
About the authors
Todd E. Rasmussen, MD, is a colonel in the United States Air Force; professor of surgery, and associate dean for research at the nation’s military medical school at the Uniformed Services University and a vascular surgeon at Walter Reed National Military Medical Center in Bethesda, Maryland. He has led a number of Department of Defense medical research programs and has served on medical device panels conducted by the FDA. Dr. Rasmussen received his undergraduate degree in pharmacy and premedical studies at the University of Kansas in Lawrence, Kansas, and his medical degree from Mayo Medical School in Rochester, Minnesota. He can be contacted at
Brian J. Young, MBA, is a regulatory affairs professional with more than 30 years diversified industry experience in regulatory, quality, and clinical affairs. He currently splits his time between Prytime Medical Devices Inc, where he serves as senior vice president, regulatory and quality affairs, and at Health Policy Associates, where he serves as vice president, regulatory and clinical affairs. Young received his undergraduate degree in medical technology at the University of Iowa, in Iowa City, and his MBA at Duke University’s Fuqua School of Business in Durham, North Carolina. He can be contacted
Acknowledgment Heather Astley, vice president, Health Policy Associates, evaluated the use of RWE in recent original PMA approvals and de novo classification orders in support of this publication.
Disclaimers The opinions and assertions expressed herein are those of the author(s) and do not necessarily reflect the official policy or position of the Uniformed Services University or the Department of Defense. Mention of a specific medical device, drug or biologic in this article does not endorse or renounce the product.
Disclosures Todd E. Rasmussen, MD, a colonel in the United States Air Force, has no financial conflicts of interest to disclose in relation to the writing of this article. Brian J. Young has no financial conflicts of interest to disclose in relation to the writing of this article.
Citation Rasmussen TE, Young BJ. A military-civilian perspective on real-world evidence to support regulatory decision making. Regulatory Focus. July 2020. Regulatory Affairs Professional Society.

Regulatory Focus newsletters

All the biggest regulatory news and happenings.