May 14, 2021

Recently, the Human Factors and Ergonomics Society held their 10th annual Healthcare Symposium, albeit virtually. As has become a recent tradition, the US Food and Drug Administration (FDA) used the symposium as an opportunity to share their current thoughts on a number of human factors (HF) topics related to medical device development. I had the opportunity to attend the workshop presented by the FDA’s Center for Devices and Radiological Health (CDRH) human factors group, the agency team responsible for reviewing the human factors contents of medical device submissions.

(Our Human Factors Research & Design (HFR&D) team has also reported on FDA’s Center for Drug Evaluation and Research (CDER) workshop held during the symposium.)

Janine Purcell, Kimberly Konston, and Hanniebey Wiyor represented CDRH during the workshop, sharing their perspectives on HF risk management and validation testing, among many other topics. As always, there were a few golden nuggets that offer additional insights into the Agency’s latest thinking.

Key takeaways from the CDRH human factors workshop

Here I highlight some of the most interesting takeaways.

  • Use-related risk analysis continues to be deficient. The presenters again shared research that analyzed deficiencies in HF documentation that were included with submissions to the Agency. While the importance of a solid use-related risk analysis (URRA) has been a common message for some time now, nearly a third of all deficiencies were attributed to unavailable, incomplete, or unacceptable URRAs. The fact that this information is still being shared with industry is a signal of an ongoing problem. As a reminder, the FDA requires a comprehensive URRA to enable the Agency to properly judge the appropriateness of a program’s HF activities. The URRA should account for all aspects of use (e.g., intended users, use environments, all aspects of the device user interface) and should reflect results from preliminary evaluations and analyses.
  • The FDA indicated they are only concerned about the potential “clinical harm” associated with use errors. Noting that ISO 14971, the international standard for applying risk management to medical devices, defines harm not only as injury to people, but also damage to property or the environment, I asked if the Agency was not interested in harm to property and/or the environment. The presenters replied that they are concerned about “serious clinical harm,” as defined in the Code of Federal Regulations (21 CFR) Section 803.3(w):
    • “Serious injury means an injury or illness that:
  1. Is life-threatening,
  2. Results in permanent impairment of a body function or permanent damage to a body structure, or
  3. Necessitates medical or surgical intervention to preclude permanent impairment of a body function or permanent damage to a body structure”
  • “Team testing” is the way to go if it’s realistic and representative. Another attendee asked the FDA whether they approve of “team testing,” in which a single usability test session involves multiple individuals representing a team that would normally work together to use a certain medical device. Agency spokespeople said that not only do they approve of it, but they would expect it, especially if it was representative of the user interactions (with other users) and the use environment. (I was very relieved to hear this, noting that our own team has become quite skilled at conducting team tests and we do it fairly regularly.)
  • Some device warnings might not need to be included in HF validation testing. Another interesting comment related to testing warnings--usually via a knowledge task--during HF validation testing. The Agency indicated that if a warning exists across many different types of devices and can reasonably be considered “commonplace,” then that particular warning might not require validation. Of course, whether the warning should be evaluated would depend on the device, and the study documentation should include a justification for the exclusion of any warnings from validation.
  • The Agency strongly recommends in-person testing, as opposed to remote or virtual testing, when possible. Naturally, there have been lively discussions related to the benefits and drawbacks of remote testing, and what their role is in human factors--particularly when it comes to HF validation tests.  And though the Agency strongly recommends in-person testing, there is an understanding that remote testing might be the only option on occasion. When a manufacturer plans to conduct remote validation testing, the Agency strongly encourages sponsors to submit test protocols for review including justification to support remote testing. This justification should include a detailed comparison regarding how the proposed test method will differ from a traditional in-person test; how the quality of data might be impacted; and the methods to ensure accurate and comprehensive data collection.
  • Use of virtual reality technology during HF validation testing likely has drawbacks. The Agency specifically questioned whether the device being evaluated using virtual or augmented reality can be simulated with high enough fidelity. One of the presenters’ main concerns is that virtual reality technology typically lacks proper haptic feedback, feedback that might be important--if not “critical”--during actual use. The Agency also expressed a concern that use of the virtual reality equipment itself might introduce difficulties and/or other test artifacts  that could confound test results.
  • Use of HF data gathered during an Investigational Device Exemption (IDE) study likely cannot serve as HF validation data. Oftentimes, medical device manufacturers that are conducting an IDE study for a new device wonder if study data can also be used to satisfy HF validation requirements. After all, the device is being used by actual users in actual use conditions to perform actual tests. It doesn’t get any more representative than that. The Agency’s stance, though is that, typically, device user interfaces are modified based on the results of an IDE study. And, because an HF validation test must include the device’s production-equivalent user interface, an HF validation test on the updated (and, presumably, final) user interface would be required post-IDE.

Overall, the Agency did not reveal anything new or shocking. But the points they raised did reinforce their positions, and priorities, expressed during the past few years. As always, the Agency continues to encourage sponsors to pre-submit their HF validation protocols and finetune their HF validation test methods based upon the feedback. In the meantime, should you have any questions about the latest from CDRH’s human factors team or how to most effectively engage with the Agency during an HF pre-submission, please reach out to us. We’d be pleased to discuss any questions you might have.

Merrick Kossack is Research Director at Emergo by UL’s Human Factors Research & Design division.

Learn more about US FDA medical device HFE and usability requirements at Emergo by UL:

  • HFE user research support for medical devices and IVDs
  • Human factors design and prototype development support
  • Medical device and product evaluation and usability testing
  • Webinar: Human factors engineering for medical devices

Author

  • Merrick Kossack

Related