“When conducting a risk analysis manufacturers are expected to identify possible hazards associated with the design in both normal and fault conditions. The risks associated with the hazards, including those resulting from user error, should be calculated in both normal and fault conditions. If any risk is judged unacceptable, it should be reduced to acceptable levels by appropriate means.”
The above statement from the US Food and Drug Administration (FDA) appears in the comments section (the preamble) that precedes the Quality System Regulation published as a final rule for Current Good Manufacturing Practices for medical devices.1 This statement is more broadly worded than Section 820.30(g), where risk analysis is included in the design validation section. The regulation itself merely states that design validation shall include software and risk analysis, where appropriate. Moreover, there is no mention of risk analysis, risk assessment, risk management or risk control in the definitions section or anywhere else in the Quality System Regulation. Thus, device manufacturers may conclude that they have performed a risk analysis without considering all that is required by the preamble or included in the guidance document issued on 11 March 1997.2
Fortunately, most manufacturers in the US understand that they can rely on ISO 14971:2007, an international standard written to specifically address all processes for managing risks and to develop and maintain a risk management system.3 In addition, there is a Global Harmonization Task Force (GHTF) document that describes general principles of a risk management system.4 FDA actively participated in the development of ISO 14971 and endorsed its use. FDA also led the effort to provide the GHTF guidance document on risk management.5 The latter refers to the standard when defining risk management as the systematic application of management policies, procedures and practices to the tasks of analyzing, evaluating and controlling risk.
In essence, risk management is an approach or process designed to minimize risks.
FDA uses risk-based approaches to prioritize and focus on various activities concerning the oversight of GMP requirements for human drug, biological and veterinary drug product quality. The agency also provided a broad framework for incorporating human factors engineering approaches into a risk management process in a guidance document issued in 2000.6 It is evident that risk management has become the underlying theme or bedrock of FDA regulation in the 21st century.
Much of the work related to managing the risks of FDA-approved products was begun by former FDA Commissioner Dr. Jane Tenney. She established a task force to assess risk management practices within the overall healthcare system. The task force reviewed the agency’s premarketing risk assessment and approval processes to determine if serious adverse effects were occurring at a higher rate than in the past. Then the task force evaluated postmarket surveillance and risk assessment programs to ascertain their effectiveness. Finally, it analyzed all of FDA’s risk management activities to evaluate their role in the overall system for managing medical product risks. The agency recognized that medical products are developed and used within a complex system involving the following key participants: manufacturers who develop and test products and submit applications for their approval to FDA; the agency, which has an extensive premarketing review and approval process and uses a series of postmarketing surveillance programs to gather data and assess risks; other participants in the healthcare delivery system, including practitioners; and patients, who rely on the ability of this complex system to provide them with needed interventions while protecting them from injury. The final report of the task force was issued in 1999.7
While the agency’s pre- and postmarketing review programs were deemed to be efficient, the task force recommended increased emphasis on the quality assurance of the premarketing review programs and expansion of the postmarketing programs.
Considering the above responsibilities faced by manufacturers, it is extremely important for regulatory professionals to become conversant with all facets of risk management and understand its provenance. This article reviews the early history of risk calculation and probability, and describes a few of the significant terms. The references cited provide an excellent resource for additional information.
History
“Risk” derives from the early Italian risicare, which means “to dare.” Thus, risk is a choice rather than a fate, and the actions people dare to take are what the story of risk is all about.5 The word itself was first used in the 17th century, and it rapidly became limited to cases in which the outcome would be unfavorable.8
In simple statistical terms, risk can be thought of as expected loss—the probability of some adverse event, multiplied by some measure of the severity of that loss.9 Thus, risks have two major characteristics: adverse consequences and uncertainty.10 Risk connotes both the probability and nature of unfavorable outcomes. Probability was first used to express an objective characteristic of an event, something external that is discoverable by repeated experiments. Discussions relative to this type of probability can be drawn from games of chance and, indeed, cards and dice games played a major role in risk management. The history is fascinating in that the concept of risk management began with the mathematical observations of Omar Khayyam (poet and author of the Rubaiyat), who lived from about 1050 to 1130. His observations were the basis for the concepts later developed by the 17th century French mathematician Blaise Pascal (1623–1662), one of the fathers of the theory of choice, chance and probability.8 It was the correspondence between Pascal and Pierre de Fermat (1601–1665), starting in 1654, that first described a method for predicting mathematical futures.11 Fermat was a lawyer and jurist with a keen and profound interest in mathematics. Even though the letters written by de Fermat and Pascal were addressing a game where one player wins in fewer than the maximum number of rounds, they had a much broader implication. The letters laid the foundations of probability and thus influence decisions each of us makes each day.
It should be mentioned that calculating probabilities is an integral part of many of the techniques used for the analysis of hazards and risks, including failure mode effect analysis (FMEA) and fault tree analysis (FTA). In FMEA, the company must identify all potential failure modes, and estimate severity, occurrence and detection. Once failure modes have been enumerated, a fault tree can aid in estimating the likelihood of any given mode.
Pascal and de Fermat were the first individuals to explain how to predict the future by calculating, often with extraordinary precision, the numerical likelihood of the occurrence of a particular event. They were not aware of an earlier work written by Girolamo Cardano (1501–1576), the most famous physician from the 16th century. His treatise on gambling, entitled Liber de Ludo Aleae (Book on Games of Chance), appears to have been the first serious effort to develop the statistical principles of probability. The word “probability”, however, does not appear in the book. For anyone interested in the derivation of words, the Latin root of probability is a combination of probare, which means to test, to prove or to approve, and ilis, which means able to be. Interestingly, Cardano’s book was not available during his lifetime, but was found among his manuscripts when he died. It was first published in 1663, and by then impressive progress in the theory of probability had been made by others who were unaware of Cardano’s seminal efforts.11
In 1657, just three years after the de Fermat-Pascal correspondence, Dutchman Christiaan Huygens provided the first account of what is recognized as modern probability theory. He is generally regarded as the leading scientist of his day. His 16-page paper “De ratiociniitis in ludo aleae” (“On Reckoning at Games of Chance”) became the standard text in probability theory for the next 50 years. The paper established the basic rules for computing probabilities. Huygens acknowledged that his work was built on the breakthrough made by Pascal and de Fermat, but he went well beyond the two Frenchmen in recognizing the potential to apply the methods of probability theory outside the gaming rooms.11
The works of Cardano, Pascal, de Fermat and Huygens were augmented by other famous individuals in the late 17th and early 18th centuries. John Graunt (1620–1674), William Petty (1623–1687) and Edmund Halley (the famous astronomer, 1656-1742) each applied the concept of probability to the analysis of raw data.8 Graunt was the innovator of sampling theory and his efforts laid the foundation for the science of statistics.12 His analysis today is known as statistical inference—inferring a global estimate for a sample of data. It was Graunt who suggested key theoretical concepts needed to make decisions under conditions of uncertainty. William Petty helped Graunt with his work on population statistics. Halley carried the analysis even further while analyzing life expectancies and annuities.8
Other mathematicians contributed to the development of probability theory over the intervening years. The Bernoulli family (eight members in all) distinguished themselves, and three made major contributions. Jakob (1654–1705), derived the law of large numbers, which states that the relative frequency of an event will more accurately predict the likelihood of its occurrence. Some years later, Jakob Bernoulli’s ideas were taken up by Abraham de Moivre (1667–1754), a French mathematician. De Moivre showed how a collection of random observations would distribute themselves around their average value (the normal distribution). His measure allows users to judge whether a set of observations is sufficiently representative of the entire population.11
Today, we can use probability to measure the likelihood of being correct when predicting one particular event. This ingenious and powerful mathematical formula was developed by an obscure eighteenth century minister in England, Thomas Bayes (1702–1761). Like Girolamo Cardano, Bayes published no original mathematics during his life. When he died, he bequeathed his papers to a friend. One document, entitled “Essay towards solving a problem in the doctrine of chances,” outlined a radically new way to approach and compute probabilities. Bayes’ method, however, was largely ignored by statisticians. This all changed in the 1970s because of the availability of powerful computers and the ability to perform iterative processes.11 FDA finally recognized the value of Bayesian statistics and has issued guidance for its use in clinical trials.13
Final Thoughts
Many of us are innumerate, or not mathematically inclined, a fact that should make us even more appreciative of pioneering individuals who were. These scientists and lay people from earlier centuries helped us develop the theory of risk management–a concept that influences all of the processes in our daily and professional lives. Their seminal work involved a process to effectively identify, analyze and control risks. This process incorporated in the term “risk management” is the primary consideration for all of us involved in healthcare delivery.
References
-
Federal Register Vol. 61, No. 195, Monday, 7 October 1996.
-
Design Control Guidance for Medical Device Manufacturers. 11 March 1997.
-
ISO 14971 Medical Devices—Application of risk management to medical devices. Second Edition 2007-03-01.
-
GHTF Final Document—Implementation of risk management principles and activities within a Quality Management System. 20 May 2005.
-
Kimmelman E. “The challenges of integrating risk management into a compliant quality management system.” RAJ Devices, Jul/Aug 2006:207–214.
-
Medical Device Use—Safety: Incorporating Human Factors Engineering into Risk Management. US Food and Drug Administration, Center for Devices and Radiological Health, Division of Device User Programs and Systems Analysis, Office of Health and Industry Programs.18 July 2000.
-
Managing the Risks from Medical Product Use—Report to the Commissioner from the Task Force on Risk Management, Food and Drug Administration, May 1999.
-
Bernstein PL. Against the Gods—The Remarkable Story of Risk. John Wiley and Sons, New York, 1996.
-
Wilson R and Crouch EAC. Risk-Benefit Analysis. Harvard University Press, Cambridge, MA, 2001.
-
Hadden SG. Read the Label—Reducing Risk by Providing Information. Westview Press, London, 1986.
-
Devlin K. The Unfinished Game. Basic Books, New York, 2008.
-
David FN. Games, Gods, and Gambling. Hafner Publishing Co, New York, 1962.
-
Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials, US Food and Drug Administration, Center for Devices and Radiological Health, Division of Biostatistics, draft released for comment, 23 May 2006.
Author
Max Sherman is president of Sherman Consulting Services Inc., in Warsaw, IN. He can be reached via email at
maxsherman@kconline.com.