Software development

This article applies to software development and validation and discusses the security and usability engineering aspects of software. It uses the term “software” for software as a medical device (SaMD) and medical device software (MDSW), including MDSW that is embedded on, drives, or influences the use of a hardware medical device, software accessories, and software components. Keywords ‒ agile, development, incremental, software, static, waterfall

 

Standards and guidance

Manufacturers should take standards and regulatory guidance into consideration when implementing software development processes. This section introduces the most important standards and guidance documents.
 
The internationally accepted framework for lifecycle processes for medical device software is IEC 62304.^1,2 This standard defines the processes, activities, and tasks used in the development and maintenance of medical device software. This article will detail the parts of IEC 62304 that apply to software development.
 
IEC 62304 defines software development lifecycle activities, except for design validation of the finished device, i.e., the process for confirming software specifications conform to user needs and intended uses. Design validation is covered by IEC 60601-1^3,4 for the software part of medical electrical equipment, or IEC 82304-1⁵ for software-only products.
 
IEC 62304 defines processes, not development techniques. The sequence of development process steps described by IEC 62304 appears to suggest a waterfall model be used, i.e., a once-through strategy, often represented in the classical V-model for sequential development. Nevertheless, the standard does not prescribe any specific software development methodology, approach, principle, or practice. The standard for quality management systems, ISO 13485,^6,7 and the US Food and Drug Administration (FDA) guidance on design controls⁸ appears to suggest the design process should be completed in the following sequence: planning, design input, design output, design review, verification, validation, and transfer. Still, these documents are not intended to prescribe a specific chronological order for these activities.
 
Historically, (software) development departments felt burdened by the waterfall approach in writing plans and requirements specifications, documented to the last comma and point, before starting with the real work: coding. In contrast to the waterfall approach, with an agile or iterative development approach, system development and delivery are performed in small increments.
 
Agile development provides useful functions much earlier in the project, generates early feedback from strategic customers and users, allows developers to improve the functionality already on the market, and informs corrections to the original specification. Agile’s principal purpose is to overcome the problem of discovering, at the end of a (large) development project, that the developed system does not meet the customers’ real (and often, new) requirements. New systems will inevitably not only require, but also induce, changes to working practices unforeseen at the start of a project. That is why specifications are unlikely to be either complete or correct at the start of a project. Adhering to a rigid waterfall model can result in disaster early in the project. Changes to requirements will likely be needed throughout the project; regardless, it is unlikely the finished system will be able to meet its users’ needs. If a system is expected to be large and lengthy, and its requirements cannot be fully defined with confidence at the start, big-bang delivery using the once-through waterfall model may not be such a good idea.
 
Agile development practices allow anomalies to be discovered early in the development stage, so the specification and development of later increments can be refined through user feedback. Manufacturers can use the agile approach even for the development of nonsoftware products. Obviously, agile versus waterfall has a serious impact on the implementation of software development processes in a quality management system; this article discusses both methodologies.
 
The Technical Information Report (TIR) AAMI TIR45⁹ for agile software development is widely used and accepted and is supported by FDA.¹⁰ AAMI TIR45 provides recommendations for complying with international standards and FDA guidance documents when using agile practices to develop medical device software. This article uses some of definitions in AAMI TIR45 and discusses some of its development concepts.
 
Software user interface design is clearly part of software development. User interface specification and formative and summative evaluations are an essential element of usability engineering. IEC 62366¹¹ defines processes related to safety for the usability engineering of a medical device. This article discusses the interaction between software development processes and usability engineering and how the usability engineering processes can fit into the waterfall or agile methodology.
 
IEC 62304 does not cover the medical device’s validation and final release; however, validation of user needs is often part of software development.¹² Furthermore, IEC 60601-1¹³ and IEC 82304-1 both require validation for programmable electrical medical systems (software). ISO 13485¹⁴ and FDA guidance on design controls also require design and development validation.¹⁵ IEC 82304-1, although not harmonized in the EU, but recognized in the US,¹⁶ can be considered the state-of-the-art standard for software validation. It is important to note that the scope of IEC 82304-1 applies to software-only products. This article provides details on the software validation requirements of IEC 82304-1.
 
The European Medical Devices Regulation (EU MDR)¹⁷ includes several general safety and performance requirements (GSPR) related to security. For instance, GSPR 17.2 requires that “Medical Device Software shall be developed according to researcher at the National eHealth Living Lab (Leiden University Medical Center) if state of the art, including information security.” The Medical Device Coordination Group (MDCG) released guidance on security for medical devices, MDCG 2019-16.¹⁸ Notified bodies (NB) must use MDCG guidance documents in their product assessments. Therefore, it is highly recommended (needed) that manufacturers also follow the MDCG guidance. FDA issued guidance on management of cybersecurity in medical devices¹⁹ and a guidance on cybersecurity for networked medical devices containing off-the-shelf (OTS) software.²⁰ IEC 60601-1 presents requirements for devices incorporated into an IT-network. MDCG and FDA guidance as well as IEC 60601-1 do not address how to include security into software development.
 
IEC/TR 60601-4-5²¹ provides detailed technical specifications for security features and capabilities of medical devices used in medical IT-networks,²² which includes medical electrical equipment²³ and medical device software,²⁴ even though these products are not in the scope of the IEC 60601 series. The TR defines specifications for security levels based on existing documents for IT security, like IEC 62443-4-2²⁵ and IEC TR 80001-2-2.²⁶
 
IEC 81001-5-1,^27,28 addresses the product lifecycle activities from a security-specific perspective of connected medical devices or health software and is modeled after IEC 62304. This article uses edition 1.0²⁹ of IEC 81001-5-1 to address security-specific aspects during software development. The scope of IEC 80001-1⁶⁷ adds general requirement in the application of risk management for the accountable organization for the use and maintenance of medical devices in a medical IT-network..
 
Several definitions are used for medical device software. In the EU, MDCG 2019-11,³⁰ defines two types of software:

• Medical Device Software (MDSW)
• Software driving or influencing the use of a medical device

 
The International Medical Device Regulators forum (IMDRF) has defined SaMD. This article makes no distinction between SaMD, MDSW, or software driving or influencing the use of a medical device, and the term “software” refers to any type of software.

Definitions

Evolutionary. The evolutionary strategy acknowledges that the user needs are not fully understood, and all requirements cannot be defined upfront. In this strategy, customer needs and system requirements are partially defined upfront, and refined in each subsequent build. [IEC 62304, paragraph B.1.1]

 
Emergence/emergent. In agile contexts, a concept usually ascribed to a system that cannot (or is not desirable to) be defined fully in advance of doing development work, and where the actual conduct of work informs and refines both the definition of the system and the work process itself, providing both better system and work results. [AAMI TIR45, definition 3.8]
 
Incremental. The incremental strategy determines customer needs and defines the system requirements, then performs the rest of the development in a sequence of builds. The first build incorporates part of the planned capabilities, the next build adds more capabilities, and so on, until the system is complete. [IEC 62304, paragraph B.1.1]
 
Iteration. In lifecycle terms, iterative development usually means repeating a lifecycle model or a significant portion of the lifecycle model multiple times in the course of a single project, rather than a single execution of a selected lifecycle. [AAMI TIR45, definition 3.15]
 
Usability test. Method for exploring or evaluating a user interface with intended users within a specified intended use environment. [IEC 62366-1, definition 3.19]
 
Validation. Confirmation by examination and provision of objective evidence that the requirements for a specific intended use can be consistently fulfilled. [21CFR820.3(z)]. Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled [ISO 9000:2015, definition 3.8.13]
 
Verification. Confirmation through provision of objective evidence that specified requirements have been fulfilled. [ISO 14971,³¹ definition 3.31]
 
Software item. Any identifiable part of a computer program. [IEC 62304, definition 3.25]
 
Software of unknown provenance (SOUP). Software item that is already developed and generally available and that has not been developed for the purpose of being incorporated into the medical device (also known as off-the-shelf software) or software previously developed for which adequate records of the development processes are not available. [IEC 62304, definition 3.29]
 
Software system. Integrated collection of software items organized to accomplish a specific function or set of functions. [IEC 62304, definition 3.27]
 
Software unit. Software item that is not subdivided into other items. [IEC 62304 definition 3.28]
 
Waterfall. Once-through strategy [lifecycle model] of performing the development process a single time. Simplistically: determine customer needs, define requirements, design the system, implement the system, test, fix and deliver. [IEC 62304, paragraph B.1.1]
 
Software development processes
Introduction to waterfall and agile

IEC 62304 defines the following software development process steps:³²
 

• Software development planning
• Software requirements analysis
• Software architectural design
• Software detailed design
• Software unit implementation and verification
• Software integration and integration testing
• Software system testing
• Software release

 
IEC 62304 does not require a particular software development lifecycle model (waterfall, agile, or other). Nevertheless, there are dependencies between the processes listed above, because one process generates inputs for another process. To comply with the standard, the dependencies must be kept consistent. Figure 1 visualizes the waterfall model by plotting the software development process dependencies.
 

 
User needs are at the top of the waterfall, as also indicated in the section below on stage-gated development. With an agile strategy, this can be different, as various agile strategies are possible. Let us consider two of the most common approaches:
 

• The evolutionary strategy is acknowledging that the user needs are not fully understood and that not all requirements can be defined up front. In this strategy, customer needs and system requirements are partially defined up front and are then refined in each succeeding software build. After a preliminary requirements analysis and an initial high-level design activity, the agile project starts the build cycle. In this strategy, receiving customer feedback is essential. Customer feedback is incorporated in the next build, user needs can be added or changed, and software requirements are refined. This cycle can be repeated until the project budget is spent, the number of planned builds is completed, or the customer is satisfied.
• The incremental strategy determines customer needs and defines the system requirements, with the rest of the development conducted in a sequence of builds. The incremental strategy can be adapted to the manufacturer’s release strategy:

• Staged delivery: In this software development lifecycle, an architectural design is developed, based on the user needs and software requirements. Detailed design, coding, testing, and delivery of the incremental build is executed in (predefined) stages. The release of the software is targeted after completion of all stages.
• Design to schedule: Unlike a staged delivery, in a design-to-schedule-approach, each incremental build is based on the priority of that build. The release of the software can, for example, depend on budget spent or time to market.

 
Medical devices (often) have a specific medical purpose, a (carefully) developed benefit-risk profile, and clinical claims. The manufacturer must conduct a clinical evaluation to adequately prove the benefit-risk and clinical claims for the finished device. The clinical evaluation shows the dependency between user needs, intended purpose, benefit-risk, and clinical claims. A change in one of these characteristics can have a severe impact on another. The manufacturer must bear in mind the relation between user needs, intended purpose, benefit-risk, and claims against the available clinical data, for example, if marketing makes an (additional clinical) claim, additional clinical data will need to be generated.
 
From a regulatory perspective, the evolutionary strategy appears inefficient. a conformity assessment or premarket submission is required before a manufacturer can place a medical device on the market. An evolutionary strategy that develops when the product has an ever-changing intended use caused by additional user needs and subsequent increase in software requirements is impractical.
 
When the software is embedded on dedicated hardware using a specific controller, the incremental strategy is preferred over the evolutionary one. Even though the software’s functionality might be built upon hardware abstraction layer(s), the hardware will not change.
 
Therefore, this article does not consider the evolutionary agile approach and focuses instead on incremental agile development in (predefined) staged deliveries as indicated above.
 
User needs
ISO 13485, FDA’s design control guidance, and IEC 62304 do not identify the specifications of user needs (also called use specifications or customer needs) as a specific stage in the design and development process. Sometimes, this stage is indicated by Phase 0; in the FDA guidance on design control, user needs specifications are placed at the top level of the waterfall design process.
 
All product design should start with a definition of user needs. In documenting user needs, manufacturers should think about their product idea. User needs describe how the product is going to be used. An appropriate definition makes it easier to define the intended use describing the clinical issue(s) the product addresses. ISO 14971 also indicates using the use specification³³ as an input to determine the intended use.
 
The following is a list of questions to help manufacturers better understand user needs that address the intended use:
 

• What do you want it to do?
• Who is going to use it?
• When will it be used?
• What is the intended patient population, for example, age groups (adults, children, adolescent, elderly), gender (male, female, nonbinary), or disease state (life-threatening, moderate in progression, minor chronic illnesses)
• What important features and attributes should be considered?
• How will the user and patient interact with the device? What type of (clinical) procedures will the device be used for?
• What is the clinical workflow?
• What type of environment will the device be used in (e.g., home, hospital, intensive care unit)?
• Is the device used only once or repeatedly?
• With what other products will the device interact and interface?

 
The manufacturer shall document and keep up to date all user needs, including those that address the intended use.
 
Software development planning
Agile is a programmer’s technique that is flexible in managing change. Changes can be large; sometimes, the change can be cosmetic or not essential to the design. A design change creates the risk of the design deviating from the original specification and increasing the developers’ workload, resulting in a project with time and budget over runs. Applying agile requires strategic planning for change and keeping the project and design within set boundaries.
 
Applying agile as a development technique can have a significant impact on the design controls, in particular configuration control. With agile, several versions may need maintenance or may be under development simultaneously. At the same time, one version might be under development, while a second version in the field shows a safety issue that needs immediate resolution, and a third that is ready for release but also needs the bugfix that is related to the safety issue.
 
Version and revision control require adequate tools and procedures to keep each version traceable and accessible, including the tools, libraries, and other items that are needed to guarantee the exact reproduction of a specific version. With agile version and revision control, tools are essential, and the manufacturer must ensure its development staff is well trained in using these tools.
 
Agile’s high level of flexibility and ability to work with multiple versions can impose a severe burden on design controls. Maintaining updated, reviewed, approved, and signed design documents aligned with each version can easily become a highly inefficient process, particularly with medical device software, where requirements related to safety and performance are critical.
 
IEC 62304 defines the software lifecycle planning requirements but lacks guidance on certain aspects vital to security and data protection. The manufacturer can choose to adopt a secure coding standard that adds specific security aspects, such as:
 

• Avoidance of potentially exploitable implementation constructs (e.g., implementation design patterns that can have security weaknesses)
• General secure coding practices (e.g., memory management)
• Error handling and logging
• Input validation
• Output encoding

 
The software development (and maintenance and configuration) lifecycle processes can be supported with, for example, a version and revision control system that manages secure change controls and audit logging.
 
As defined by IEC 62366-1, usability engineering does not require drafting a usability engineering plan. Still, it suggests tailoring the usability engineering effort to the size and complexity of the user interface or the severity of the harm associated with the use of the medical device. The manufacturer can integrate the usability engineering planning aspects and the choice of methods and tools to perform usability engineering in the software development plan(s).
 
Requirements management
IEC 62304 defines software requirements analysis as a process step³⁴ where the software requirements are derived from the system requirement. System requirements are what is needed to address the intended use and user needs. For software-only devices, the system and software requirements are identical. IEC 62304 provides the following enumeration on requirements:
 

• Functional and capability requirements
• Software system inputs and outputs
• Interfaces between the software system and other systems
• Software-driven alarms, warnings, and operator messages
• Security requirements
• User interface requirements implemented by software
• Data definition and database requirements
• Installation and acceptance requirements
• Requirements related to methods of operation and maintenance
• Requirements related to IT-network aspects
• User maintenance requirements
• Regulatory requirements

 
The following are some other types of requirements not explicitly addressed above:
 

• Repeatability, reliability, and performance requirements
• Risk control requirements and requirements for functional safety and single fault condition
• [Minimum] requirement for supported platforms
• Interoperability requirements
• User documentation requirements

 
A requirement is a specification of what is needed to meet a certain (user) need. IEC 62304³⁵ and ISO 13485³⁶ require traceability between system requirements, software requirements (including risk control measures implemented in software), and software system testing. To this end, requirements are uniquely identifiable.
 
Requirements are the starting point for software design and are the basis for performing the subsequent design tasks. Therefore, development of complete, nonambiguous, and nonconflicting requirements is of paramount importance in software development.
Requirements are best managed by a cross-functional team. It is best practice to define the responsible persons and roles for each requirement. Requirements can be attributed by owners (marketing, lead developer, product specialist, etc.) and roles (implement, test, approve)
 
During requirements reviews, each team member is responsible for executing their perspective role in the sequence of required activities: requirement feasibility, implementation, and testing to deliver a commercialized product with functionality that is sufficient to meet user needs.
 
Requirements are managed through a formal system. After they are reviewed, requirements must be approved and are subject to change management. A changing requirement must be reviewed again, the impact on past and future activities must be determined, and the project adapted accordingly.
 
Architectural design
IEC 62304 defines software architectural design as a process step.³⁷ During this process activity, software requirements are transformed into an architecture. The architecture describes the software’s structure and identifies the software items. The software architecture document describes the interfaces (the relations in the architectural diagram) between the software items and the hardware and software components external to the medical device software. The architecture can serve as a tool to scope and set the boundaries of the medical device software when the software is integrated with a larger set of software or hardware components, either medical or nonmedical.
 
Software of unknown provenance (SOUP) is defined as a software item; therefore, SOUP items need be represented in the architecture. For each SOUP item, the functional and performance requirements must be defined, and the requirements for system hardware and software supporting its operation must be specified.
 
Configuration management
IEC 62304 defines a software configuration management process,³⁸ although the standard is rather vague on defining the configuration items. The keyword associated with configuration management is traceability. Several sources require traceability, including EU MDR,³⁹ ISO 13485,⁴⁰ and IEC 62304.⁴¹ Traceability is required between system requirements (including risk control measures implemented in software), software requirements, and software system testing. This regulatory objective is clearly related to safety and performance. In case of any specific adverse event, serious adverse event, or device deficiency⁴² due to a software failure or anomaly, the specific software version must be made operational for tracing the issue that caused the event or deficiency. The following is a list of configuration items that could be used:
 

• Software build tools and setting
• SOUP items
• Operating systems developed and tested on
• Test tools used

 
The planning scheme for configuration management is included or referenced in the software development plan.
 
Design reviews
In Figure 1, the software development process steps in IEC 62304, according to the waterfall approach, are shown. Not indicated in the figure are the required design reviews. The following list indicates the design reviews required by the standard:
 

• Verify software requirements.⁴³ As indicated earlier, the management of software requirements is a formal process, including change control and impact assessment.
• Verify software architecture.⁴⁴ Typically, the software architecture is verified by conducting a traceability analysis to the software requirements, interfaces, and the SOUP items.
• Verify detailed design.⁴⁵ Generally, the detailed design can be verified by showing a traceability analysis of architecture to software detailed design.
• Verify software unit.⁴⁶ Typically, software unit verification is performed by automated unit testing tools.
• Verify software integration.⁴⁷ Software integration verifies that the software units have been integrated into software items and/or the software system. See also the usability engineering section below. Often, such verifications are the output of software development tools.

 
The first three verifications depend on the software requirements having been defined. Clearly, the challenge with agile is managing the frequent and interdependent requirement changes, the related documentation overhead, and the need for managing test scripts and reports.
 
Some manufacturers may be able to apply an automated process for software unit and software integration verifications. In contrast, when such verification includes code reviews through manual code inspection and code analysis, agile’s overhead can be huge.
 
Safety and agile
Risk management
While the use of emergent design approaches is valuable, it poses significant challenges when applied to safety-critical systems like medical devices. According to EU MDR, GSPR 4, medical device manufacturers must reduce risks as far as possible without adversely affecting the benefit-risk ratio. Considering that medical device safety is essential, how can a manufacturer design the software’s architecture in a fail-safe manner if the requirements are not (yet) fully understood, and if the design is emerging and evolving across multiple releases? Furthermore, how can a safety-critical system be relied upon if it is brought into operation in stages? Can you rely on half of a system to reduce the risk as low as possible? The agile development method is a programmer’s method but does not provide a holistic approach that supports safety.
 
There can be benefits and constraints to using agile versus once-through software development. It may be more practical not to include specific steps or activities in the increment development and instead define them up front to serve as an input for the increments. Let us consider how certain software development aspects relate to safety and security risk management.
 
User needs. The intended use is a result of an analysis of the use specification (user needs). The risk management process starts with an analysis of, but is not limited to:
 

• The intended use and reasonably foreseeable misuse
• Identifying hazards and hazardous situations associated with a medical device
• Estimating and evaluating the associated risks

 
As argued earlier, to be practical, user needs must be static (as far as possible) regardless of which software development methodology is followed, waterfall or agile.
 
Software requirements. Functional and safety software requirements should also be static. A manufacturer must trace software requirements to system requirements. System requirements are, in turn, derived from user needs. Also, risk control measures can include software requirements. Risk identification must be performed at a sufficiently detailed level to be effective. Ideally, to be practical, system and software requirements are static. Otherwise, the manufacturer needs to repeat the risk assessment whenever a requirement changes. An agile (or waterfall) software development methodology would therefore require static (to a large extent) software requirements. Alternatively, if keeping requirements static is not an option, triggers could be built into the requirements management tool to repeat risk management activities when a requirement changes due to agile development.
 
Security requirements. Also, security-related needs, such as those required by IEC/TR IEC 60601-4-5, need to be addressed. Their controls and security capabilities found in IEC/TR 80001-2-2 and IEC 62443-2-4 need to be implemented and known early in software development. An agile (or waterfall) software development methodology would require static (to a large extent) software requirements. Alternatively, if keeping them static is not an option, triggers could be built into the requirements tool to repeat risk management activities when a specification changes due to agile development.
 
Software architecture. IEC 62304 advises the manufacturer to decompose the software system into segregated software items (modules, classes, objects). Subsequently, the manufacturer assigns each software item its own software safety class (A, B, or C) according to a decision tree provided by the standard.
 
The software safety class inheritance runs from the software items at the bottom of the tree to the software system at the top (Figure 2). The manufacturer must specify the segregation and provide test evidence to prove the segregation is effective. The higher the software safety class, the more process rigor the manufacturer must follow during product development. Manufacturers that assign a class to each segregated software item may reduce the administrative burden of IEC 62304 compliance. That is because software items with software safety Class A must fulfill only a limited set of requirements to comply with the standard. In contrast, software items with software safety Class C must comply with the entire standard.
 

 
ISO 14971 considers inherently safe design and manufacturing to be the highest priority risk control option. Inherently safe design requires an architecture that serves safe operation. Only after finalizing the software architecture and determining how the software items relate to safety can software development continue. Also, this is the phase in which software items can be classified definitively according to their relation to safety.
 
When looking at the software architecture from a security perspective, the following strategies can be applied to architecture development:
 

• Use of proven software components (software item) with security capabilities
• Attack surface reduction of software item
• Reduce accessibility of interfaces of software items
• Implement trusted boundaries in software items

 
These architectural decisions can be part of the software architecture and are associated with the threat model analysis, i.e., (security) risk management.
 
The discussion above shows that applying an agile software development method that includes incremental development of the architectural software design is not practical, i.e., the software safety classification prescribed by IEC 62304 is not practical for agile software development. The software item classification defines the basis for the process rigor of subsequent software development. Any change in the architecture might invalidate the software safety classifications and could trigger rework and up-classification of software items.
 
SOUP. A SOUP item is a software item^48,49 identifiable in the software architecture. SOUP items also are addressed in the risk analysis and assigned a software safety classification. For similar reasons as described above, a change of SOUP or its role in the architecture might invalidate its software safety class and trigger rework.
 
In summary, functional safety, security requirements, the software architecture, and SOUP items are closely related and associated with risk management. These items are inputs for the subsequent waterfall or agile development process. Applying a strategy for emergent requirements does not pose a regulatory issue. Still, it might be inefficient from a practical point of view because it forces the manufacturer to reassess risks and re-design the architecture. The continuous re-evaluation inherent to agile results in an inefficient process but is not a regulatory constraint. It just requires more change control, and more change control requires more resources.
 
Gated development and agile
Many manufacturers use stage-gated development, historically due to ISO 13485 and FDA guidance on design control.⁵⁰ In these documents, the design process is presented as a linear process or waterfall model. As a result, procedures for (software) design and development were also written in the same linear fashion. Both ISO 13485 and FDA guidance define a similar set of process steps:
 

• Design and development planning
• Design and development inputs
• Design and development outputs
• Design and development review
• Design and development verification
• Design and development validation
• Design and development transfer

 
At appropriate stages in the process, reviews of design and development process steps are conducted. Design reviews are formal documented reviews⁵¹ of a design stage, often held to bring the design process to the next stage, i.e., the development stage under review should sufficiently meet predefined requirements and criteria to pass the review gate and proceed to the next stage.
 
IEC 62304, like ISO 13485 and FDA guidance on design controls, defines a development process that is presented in a linear fashion, from planning to release in a consecutive sequence of process steps. In Figure 1, the software development lifecycle in IEC 62304 is presented as a waterfall model. Figure 3 presents a similar model for the agile incremental software development lifecycle, taking into account the risk management discussion provided earlier, using predefined requirements and architecture, and where the software is developed in predefined increments.
 

 
The manufacturer must assure the software is in compliance with IEC 62304 and comply with a quality management system, typically ISO 13485; however, the manufacturer is not bound to follow the exact sequence of process steps presented in those documents. Manufacturers may apply design and development models as appropriate, as long as the deliverables required for compliance with those standards are generated.
 
The software development process steps defined in IEC 62304 are listed above. In Figure 3, the agile incremental software development lifecycle is shown. In Figure 4, the IEC 62304 software development process steps are mapped into agile increments, as presented in Figure 3.
 
These or similar figures provide a means to define software development procedures that are designed to generate the deliverables needed for compliance with IEC 62304. Obviously, a similar strategy can be applied to map software development process steps onto the ISO 13485 design and development process, using the agile or waterfall method.
 

 
Usability engineering
Medical device software with a user interface is subject to the requirements of IEC 62366‑1 on the application of usability engineering for medical devices. Like IEC 63204, this is a process standard, i.e., it defines the process, activities, and tasks required for usability engineering. IEC 62366‑1 includes requirements for the analysis, specification, development, and evaluation of the usability of medical devices; more specifically, it requires the following activities:
 

• Prepare use specifications
• Identify user interface characteristics related to safety and potential use errors
• Identify known or foreseeable hazards and hazardous situations
• Identify and describe hazard-related use scenarios
• Select the hazard-related use scenarios for summative evaluation
• Establish user interface specification
• Establish user interface evaluation plan
• Perform user interface design, implementation, and formative evaluation
• Perform summative evaluation of the usability of the user interface

 
The last three usability engineering process steps of the nine listed above are briefly explained below to show how these process steps can be integrated in the incremental software development lifecycle, as described in the previous section:
 

• Establish user interface specification.⁵² The user interface specification includes testable technical requirements relevant to the user interface. The requirements can include risk control measures. Typically, such requirements are technical specifications, such as display color, character size, or placement of the controls. Also, information for the user related to installation, safe use, maintenance, decommissioning and disposal, and specific training requirements are defined.
• Establish user interface evaluation plan.⁵³ The plan documents the objective and identifies the method of any planned formative evaluations and summative evaluations. User interface evaluation may be performed in a variety of locations, for example, in a laboratory setting, in a simulated use environment, or in the actual use environment.

• Perform user interface design, implementation, and formative evaluation.⁵⁴ During this step, the user interface is designed and implemented. Also, any needed documentation and training, as defined in the user interface specification, is developed. An evaluation during this formation step is conducted with the intent to explore user interface design strengths, weaknesses, and unanticipated use errors.
• Perform summative evaluation of the usability of the user interface.⁵⁵ At the end of the user interface development, when a final or production equivalent is available, the user interface is evaluated with the intent to obtain objective evidence that the user interface can be used safely. During the evaluation, data of evaluations on previous models may be used, if the manufacturer justifies why the assessed characteristic is identical in view of the test goal.

 
Figure 5 indicates integration and system testing. These steps are defined in IEC 62304 and are clarified here for understanding the logic in the figure:
 

• Software integration and integration testing.⁵⁶ Software transfers data across the systems’ internal function interfaces, i.e., input and output of functions. Integration testing is the phase in software testing in which individual software modules (software items and software units) are combined and tested as a group. Integration testing is conducted to evaluate the compliance of a system or component with specified functional requirements after unit testing system testing. Integration testing takes as its input modules that have been unit tested, groups them in larger aggregates, applies tests defined in an integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing.
• Software system testing.⁵⁷ The objective of software system testing is to verify that the software’s requirements have all been implemented and that the software system functions and performs according to its specification.

 

 
In Figure 5, the usability engineering process steps 6 to 9 are now integrated into the incremental software development lifecycle as shown in Figure 3. The manufacturer can include user interface requirements in the requirements list for each development increment.
 
During each increment, the software detailed design and the user interface design are integrated. As in Figure 5, the formative evaluations are performed immediately after the finishing of an incremental release. Developers may find it useful to focus their efforts on the formative evaluation early on. The manufacturer can select the evaluation methods relative to user interface complexity and focus on the user interface part being implemented or refined in the increment. The manufacturer can update the training materials during each increment. The manufacturer can develop and refine training material for formative evaluation. The formative evaluation is performed before software integration and integration testing per increment. Still, it may just as well be performed after the testing.
 
Developing a formative evaluation plan and performing the formative evaluation during each increment has the advantage that such a process can derive valuable input for the design process through the refinement of the user interface from a usability engineering perspective; an incremental development lifecycle may therefore be preferable over a waterfall development lifecycle.
 
Another advantage of incremental software development can be found when the software system design and implementation is completed, and a summative evaluation of the usability of the user interface must then be conducted. A summative evaluation must be conducted only for those parts of the user interface where the manufacturer cannot demonstrate that the software is not vulnerable to potentially harmful use errors, based on an analysis of the data found in the formative evaluations. Summative evaluation is addressed further in the section below.
 
Software validation
Validation is a regulatory requirement. See EU MDR, GSPR 17.2, and FDA guidance.⁵⁸ Validation is performed on the finished device. Software can have many forms, for example, a software-only device, as part of a larger software system, or embedded as part of a hardware device. In any case, a manufacturer must find objective evidence that the software specifications conform to user needs and intended uses. Software validation is not included in the development lifecycle of IEC 62304. Still, it is required through regulations and several standards and guidance. More importantly, ISO 13485 requires design validation to be part of the design and development process. A manufacturer must complete validation before the release and use of the product by the customer. Several documents clarify how to perform design validation, including the following:
 

• ISO 13485, clause 7.3.7 Design and development validation
• IEC 82304-1, clause 6 Health software product validation
• IEC 60601-1, clause 14.11 PEMS validation
• FDA Guidance on design control for medical device manufacturers
• FDA Guidance on Software as a Medical Device (SAMD): Clinical evaluation
• MDCG 2020-1⁵⁹ Guidance on clinical evaluation (MDR)/performance evaluation (IVDR) of medical device software
• National Medical Products Administration (NMPA) guidelines for on-site inspections on standalone software as medical devices (SaMD) (2020, No 57)⁶⁰

 
In summary, these documents recommend the following activities, among others:

• Validation plan that includes:

• A test protocol to validate that the finalized product meets the requirements for the intended use
• Methods and acceptance criteria

• Validation execution:

• Design validation must occur on the finalized product, unless substantial equivalence can be demonstrated on the aspect being validated between the version used for testing and the finalized product
• The device must be installed, connected, or interfaced according to the intended use
• The validation activities must be conducted in an environment representative of the intended use environment

• Validation users:

• The users operating the finished device during validation activities and why they are representative of the intended user population

• Validation report, including:

• Results and conclusion of validation and necessary actions to be taken
• Traceability between validation results and the use requirement that addresses the intended use
• Confirmation that the user interface requirements are met
• Confirmation that the requirements for immunity from or susceptibility to unintended influence by other software using the same hardware resources are met
• Confirmation that the requirements for basic safety and essential performance are met
• Confirmation that the requirements for labeling and accompanying documents are met
• Confirmation that the privacy and security requirements are met
• Confirmation that the professional relationship of the validation team versus the design team is documented in the validation report, to demonstrate that the examiners are representative of the intended user population and the validation team is independent of the design team.
• If, during validation, anomalies were identified in the software, these are listed in the validation report
• The validation report typically contains the following:

‒ Condition during validation or attributes of the use environment, such as location, lighting, noise, temperature
‒ Results of the validation
‒ Validation team, i.e., the assessors and the users who performed the validation
‒ Summary of the validation results

• Conclusion that the software device is validated for the intended use, based on the use requirements

 
Design validation can have many forms. Sometimes, validation comprises the confirmation that the device was tested and found to be in compliance with a state-of-the-art standard, such as IEC 60601-1.
Sometimes, validation requires testing at the user site, for example, when interoperability and compatibility testing is essential and can only be tested in an environment representative of the intended use environment. Such testing can only be performed outside of the clinical workflow unless the manufacturer obtains ethics committee approval and conducts a clinical investigation. If validation requires personal data for testing, consent is required.
 
User site testing as part of design validation is an activity requiring a validation plan and a formal test protocol that includes test requirements and test acceptance criteria. The test protocol should specify the operating conditions and simulate disturbing events that can be expected during everyday activities and use. These may include tests for a high volume of data, heavy loads or stresses, security (validation of inputs that cross a trust boundary, fault testing (avoidance, detection, tolerance, and recovery), error messages, and implementation of safety requirements. It may be necessary to include specific code into the software to facilitate such error or fault injections. The test results are recorded in the test report.
 
User interface requirements also require testing to ensure users can perform the intended functions and respond correctly, for example, to all alarms, warnings, and error messages. During risk management and formative evaluation, the manufacturer identifies possible use errors that could result or contribute to harm and creates a list of the use scenarios where these errors can occur. A summative evaluation of each hazard-related use scenario is performed on the finished or production equivalent of the software and its user interface. See IEC 62366-1 for more details. During the summative evaluation, any use errors and use difficulties are analyzed against pre-established acceptability criteria.
Typically, usability tests applied in the summative evaluation of hazard-related use scenarios covers the following, among others:
 

• Characterization of the test participants and justification of their representation of distinct user groups
• Explanation of how the test environment and conditions represent the intended use environment
• Protocols and data collection for executing the hazard-related use scenarios can include the following:

• Task compliance (confidence, time to complete)
• Descriptions of observed use errors and use difficulties
• Participants’ comments (e.g., anecdotal remarks) about their user interface interactions
• Participants’ reported root causes of their use errors and use difficulties
• Method(s) of collecting and analyzing data, targeted at the observation of use errors and use difficulties

• Objective data of observations of user interactions with the user interface and their descriptions of their experiences afterward are reported. These data can be supplemented with (simple but only descriptive) statistics, such as use error counts, task completion times, and reported use difficulty.

 
When the software is intended to be used by lay users, the safety notices are of particular interest to the usability engineering process to determine whether the user perceives and understands the information and whether the safety notices support the correct use of the product.
 
See IEC TR 62366-2⁶¹ for guidance on usability test protocols and data analysis. IEC TR 62366-2, Annex K provides valuable information for determining usability test sample size. The particular benefit of incremental development is worth mentioning here. An iterative development method that includes formative evaluations can effectively increase cumulative sample size and confidence in detecting usability defects.
 
Also, design validation may provide evidence that specific safety and performance requirements are met and that the medical device meets its intended use and clinical benefit. As such, design validation is an important component of clinical evaluation.
 
Postmarket activities
The software development lifecycle, according to IEC 62304, the usability engineering process, according to IEC 62366-1, and the security activities in the product lifecycle, according to IEC 81001-5-1, do not define processes or deliverables for postproduction or postmarket activities.
 
However, risk management continues throughout the software lifecycle, including the software maintenance process⁶² and the software problem resolution process^63,64 The software maintenance process is in place to receive, document, evaluate, resolve, and track feedback after the medical device software is released. The software problem resolution process integrates the risk management activities in the investigation of the software problem and in the evaluation of possible impacts on safety. This process is implemented, for example, as a part of the QMS per ISO 13485.⁶⁵ The QMS of the manufacturer must enable a process for reporting a problem found in the software, either inside the organization or outside. The following elements should be included in the QMS process at minimum:
 

• Investigate the problem and possible root cause
• Evaluate the problem’s relevance to safety using the software risk management process
• Document the outcome of the investigation and evaluation
• Create a change request(s) for actions needed to correct the problem, or document the
• rationale for taking no action

 
SOUP is an important aspect of postproduction information. SOUP security issues may be frequently updated. SOUP may fail, unexpected results may occur, or support may be discontinued, resulting in an unacceptable overall residual risk of the medical device software.
 
Postmarket surveillance with respect to security are defined in MDCG 2019-16. A postmarket security surveillance process should include the following aspects:
 

• Operation of the device in the intended environment
• Sharing and dissemination of security information and knowledge of security vulnerabilities and threats
• Vulnerability remediation

 
To capture postmarket information related to usability, in particular when the user interface is implemented in software, the manufacturer could add some of the following questions to the complaint/event reporting form:⁶⁶
 

• Was there harm (to patient or user)?
• Who was the user (i.e., which user profile)?
• What did the user intend to do (i.e., what tasks and what use scenario)?
• Where did the event happen? (i.e., what use environment)?
• What were the use conditions (i.e., lighting, noise vibration, temperature, humidity)?
• Were there other contributing elements (e.g., what other medical devices, if any, were being used at the time of the event)?

 
Conclusion
Defining software requirements is the single most important activity in software development. The software architecture and the risk management activities and deliverables depend directly on the software requirements.
 
Agile is commonly used in software development and is known to be flexible in handling emerging and changing software requirements and associated user needs. With medical devices, manufacturers must find a balance between user needs, intended purpose, benefit-risk, clinical claims, and clinical data. The agile development method does not provide a holistic approach to support safety and performance-related characteristics. Agile is a programmer’s method requiring user needs, software requirements, and the software architecture to be (largely) static.
 
Connected software with a user interface also must include a usability engineering process and security activities in the software development lifecycle. Formative evaluations in the usability engineering process can benefit from an agile incremental software development style.
 
Abbreviations
GSPR, general safety and performance requirements; IEC, International Electrotechnical Commission; ISO, International Organization for Standardization; IMDRF, International Medical Device Regulators Forum; NB, notified body; OTS, off-the-shelf; MDCG, Medical Device Coordination Group; MDSW, medical device software; SaMD, software as a medical device; TIR, technical information report; TR, technical report.
 
About the authors
Jaap Noordmans, BSc, can be reached at Jaap.Noordmans@Qservegroup.com
 
Citation Noordmans J. Software development. RF Quarterly. 2022;2(1):4-25. Published online 1 April 2022.
 
Acknowledgment This article is based on a chapter in Software as a Medical Device – Regulatory and Market Access Implications. The full citation for the chapter is:
Noordmans J. Software development. In: Cobbaert K, Bos G, eds. Software as a medical device: Regulatory and market access implications. 1st ed. Regulatory Affairs Professionals Society; 2021:103-122.
 
References
All references accessed and verified on 15 March 2022.
 

1. IEC 62304:2006 Medical device software: Software lifecycle processes. Edition 1. Published May 2006. International Organization for Standardization (ISO) website. https://www.iso.org/standard/38421.html.
2. IEC 62304:2006/AMD 1:2015 Medical device software: Software lifecycle processes—Amendment 1. Edition 1.1. Published May 2015. Updated November 2017. ISO website. https://www.iso.org/standard/64686.html.
3. IEC 60601-1:2005/AMD 1:2012 Medical electrical equipment Part 1: General requirements for basic safety and essential performance. Edition 3.1. IEC website. https://webstore.iec.ch/publication/2605
4. IEC 60601-1:2005/AMD 1:2012/AMD 2:2020 Medical electrical equipment Part 1: General requirements for basic safety and essential performance. Edition 3.2. IEC website. https://webstore.iec.ch/publication/67497
5. IEC 82304-1:2016 Health Software Part 1: General requirements for product safety. Edition 1.0. ISO website. https://www.iso.org/standard/59543.html.
6. ISO 13485:2016 Medical devices Quality management systems: Requirements for regulatory purposes. Third edition. ISO website. https://www.iso.org/standard/59752.html.
7. Ibid, see clause 7.3 design and development.
8. US Food and Drug Administration (FDA) Design Control Guidance for Medical Device Manufacturers. March 1997. Final. FDA website. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/design-control-guidance-medical-device-manufacturers.
9. Technical Information Report. AAMI TIR45 Guidance on the use of AGILE practices in the development of medical device software. August 2012. Association for the Advancement of Medical Instrumentation (AAMI) website. https://my.aami.org/aamiresources/previewfiles/TIR45_1208_PREVIEW.PDF.
10. Ibid, FDA recognition number: 13-36.
11. IEC 62366-1:2015/AMD 1:2020 Medical devices – Part 1: Application of usability engineering to medical devices—Amendment 1. ISO website. https://www.iso.org/standard/73007.html.
12. FDA General Principles of Software Validation. Final Guidance for Industry and FDA Staff. January 2002. Final. FDA website. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/general-principles-software-validation.
13. Op cit 3 and 4; see clause 14.11.
14. Op cit 6, clause 7.3.7.
15. Op cit 8, page 11.
16. Op cit 5, FDA recognition number: 13-97.
17. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. EUR-Lex website. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A02017R0745-20200424.
18. Medical Device Coordination Group Document MDCG 2019-16 Guidance on Cybersecurity for medical devices. December 2019. European Commission (EC) website https://ec.europa.eu/docsroom/documents/41863?locale=en
19. FDA Content of Premarket Submissions for Management of Cybersecurity in Medical Devices. Draft Guidance for Industry and Food and Drug Administration Staff. October 2018. FDA website. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/content-premarket-submissions-management-cybersecurity-medical-devices.
20. FDA Cybersecurity for Networked Medical Devices Containing Off the-Shelf (OTS) Software. Guidance for Industry. January 2005. Final. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/cybersecurity-networked-medical-devices-containing-shelf-ots-software.
21. IEC TR 60601-4-5:2021 Medical Electrical Equipment Part 4-5: Guidance and interpretation – Safety related technical security specifications. Edition 1. International Electrotechnical Commission. IEC website. https://webstore.iec.ch/publication/64703.
22. Ibid, definition 3.14: system or systems composed of communicating nodes and transmission links to provide physically linked or wireless transmission between two or more specified communication nodes.
23. Op cit 21, definition 3.6: electrical equipment having an applied part or transferring energy to or from the patient or detecting such energy transfer to or from the patient and which is: (a) Provided with not more than one connection to a particular supply main, and (b) intended by its manufacturer to be used in the diagnosis, treatment, or monitoring of a patient or for compensation or alleviation of disease, injury, or disability.
24. Op cit 1 and 2, definition 3.12: software system that has been developed for the purpose of being incorporated into the medical device being developed or that is intended for use as a medical device.
25. IEC 62443-4-2:2019 Security for industrial automation and control system Part 4-2: Technical security requirements for IACS components. Edition 1. IEC website. https://webstore.iec.ch/publication/34421.
26. IEC/TR 80001-2-2:2012 Application of risk management for IT-networks incorporating medical devices Part 2-2: Guidance for the disclosure and communication of medical device security needs, risks, and controls. Edition 1. ISO website. iso.org/standard/57939.html.
27. IEC 81001-5-1:2021 Health software and health IT systems safety, effectiveness, and security Part 5-1: Security—Activities in the product life cycle. Edition 1.0. ISO website. https://www.iso.org/standard/76097.html.
28. Ibid, standard number may be modified in the future.
29. Op cit 27. MDCG 20
30. 19-11 Guidance on Qualification and Classification of Software in Regulation (EU) 2017/745 – MDR and Regulation (EU) 2017/746 – IVDR. October 2019. MDCG website. https://ec.europa.eu/docsroom/documents/37581
31. ISO 14971:2019 Medical devices — Application of risk management to medical devices. Third edition. ISO website. https://www.iso.org/standard/72704.html.
32. Op cit 1 and 2, clause 5.1 – 5.9.
33. Op cit 31, clause 3.23 and 5.1.
34. Op cit 1 and 2, clause 5.2.
35. Op cit 1 and 2, clause 5.7.4.
36. Op cit 6, clause 7.3.2.
37. Op cit 1 and 2, clause 5.3.
38. Op cit 1 and 3, clause 8.
39. Op cit 17, Annex II, 6. Product verification and validation.
40. Op cit 6, clause 7.3.2.
41. Op cit 1 and 2, clause 5.1.1.
42. Op cit 17, definitions 57, 58, and 59.
43. Op cit 1 and 2, clause 5.2.6.
44. Op cit 1 and 2, clause 5.3.6.
45. Op cit 1 and 2, clause 5.4.4.
46. Op cit 1 and 2, clause 5.5.5.
47. Op cit 1 and 2, clause 5.6.2.
48. Op cit 1 and 2.
49. FDA Off-The-Shelf Software Use in Medical Devices. Guidance for Industry and Food and Drug Administration Staff. September 2019. Final. FDA website. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/shelf-software-use-medical-devices.
50. FDA 21 CFR 820.30. Quality System Regulation. Design Controls. FDA website. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.cfm?fr=820.30.
51. FDA 21 CFR 820.30(e) Design review. FDA website. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.cfm?fr=820.30.
52. Op cit 11, clause 5.6.
53. Op cit 11, clause 5.7.
54. Op cit 11, clause 5.8.
55. Op cit 11, clause 5.9.
56. Op cit 1 and 2, clause 5.6.
57. Op cit 1 and 2, clause 5.7.
58. FDA Software as a Medical Device (SAMD): Clinical Evaluation. Guidance for Industry and Food and Drug Administration Staff. December 2017. Final. FDA website. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/software-medical-device-samd-clinical-evaluation.
59. MDCG 2020-1 Guidance on Clinical Evaluation (MDR)/Performance Evaluation (IVDR) of Medical Device Software. March 2020. European Commission (EC) website. https://ec.europa.eu/docsroom/documents/40323.
60. National Medical Products Administration (NMPA) issues guidance for on-site inspections of independent software appended in good manufacturing practice for medical devices. June 2020. NMPA website. http://english.nmpa.gov.cn/2020-06/04/c_528674.htm.
61. IEC TR 62366-1:2016 Medical devices Part 2: Guidance on the application of usability engineering to medical devices. Edition 1.0. ISO website. https://www.iso.org/standard/69126.html.
62. Op cit 1 and 2, clause 6.
63. Op cit 1 and 2, clause 9.
64. Op cit 27, clause 9.
65. Op cit 6, clause 8.2.1, 8.2.2, and 8.2.3.
66. Op cit 61.
67. IEC 80001-1:2021 Application of risk management for IT-networks incorporating medical devices – Part 1: Safety, effectiveness and security in the implementation and use of connected medical devices or connected health software, Edition 2.0. website. https://www.iso.org/standard/72026.html.