Evaluation & Usability Testing for Medical Devices

Minimize risk with expert usability testing for medical devices

Drive design decisions and manage regulatory requirements through comprehensive usability testing and device evaluations. We have experience conducting a wide variety of evaluations with a broad range of product types, including novel and high-risk medical devices, drug-device combination products and more.

Nurse writing notes on a clipboard in front of a dialysis machine

Optimize user interface design with iterative evaluations

To achieve a user interface design that provides a great user experience, it is critical to conduct effective, iterative design evaluations throughout the product development process. Product evaluations expose design strengths and opportunities for improvement that can be considered when identifying potential use-related risks and associated design mitigations.

For example, different evaluations conducted for a hypothetical home dialysis machine might reveal the following strengths and opportunities for improvement:

Strength	Opportunity
Expert review Visual alarms a displayed in a conspicuous (i.e., attention-getting) manner.	Those with dexterity or strength impairments are likely to have difficulty lifting the 40 lb. machine.
Cognitive walkthrough Users considered the set-up workflow to be intuitive.	Several users were confused about the meaning of the symbol on the backlight control.
Formative usability test Test participants with small hands and those with large hands felt that the handle provided a secure, comfortable grip.	Four out of 12 test participants were unable to install the tubing set correctly.
Pre-validation usability test 7 out of 8 users practice aseptic technique when connecting the patient line to the machine.	Two participants kinked the drain tube when placing it in its holder.

Back of a surgeon preparing surgical instruments

Tailored medical device evaluation methods for all development stages

There are many ways to evaluate a product, and the best method depends on the stage of product development and the available resources, e.g., time, budget, prototype or product status. Diverse evaluation methods, including remote testing, should be utilized for early, mid and late-stage product development.

Early-, mid- and late-stage development

Early design evaluations: Expert reviews and heuristic analyses

Conduct expert reviews and heuristic analyses to enhance user interface design during early product development stages.

Expert reviews

Frequently, we are called upon to evaluate a product’s design by conducting either an expert review or a heuristic analysis. In these cases, we apply established design principles and our judgment to identify user interface design strengths and opportunities for improvement. Such evaluations can usually be done quickly and at low cost yet produce valuable findings.

Examples:

Perform a heuristic analysis of the screens associated with a fitness app to be used on a smartphone by comparing the user interface design to recognized design standards.
Review and identify opportunities to improve a product’s packaging and labeling, seeking to maximize clarity and information presentation.

Cognitive walkthrough

A cognitive walkthrough is another technique often used to evaluate early design concepts.

This approach involves representative users reviewing product sketches, physical models and prototypes, or a series of static images or screens and thinking through and explaining how they might perform a task with the actual product.

Mid-stage design evaluations: Formative usability testing

Conduct iterative formative usability tests to provide actionable feedback for design enhancements.

Formative usability testing

We often conduct iterative, formative usability tests of products. Such tests can be quick to deliver findings for a design team that might be in the middle of a development sprint and could even follow a rapid iteration approach, in which one or two days of usability testing is followed by a quick product, labeling, and/or package redesign, followed up again by additional testing days. We can conduct formative usability tests on full products and systems, as well as smaller-scale evaluations focused on a subset of features or workflows.

Examples:

Conduct a six-participant formative usability test of two potential concepts for the homepage dashboard and sample alerts within a web-based application designed for healthcare professionals to track patients’ dialysis treatments and kidney health.
Conduct an eight-participant formative usability test of an exoskeleton used to help people suffering from stroke regain their stride.
Conduct a 15-participant pre-validation, i.e., final formative usability test of a dialysis machine, following a test protocol intended for use in the subsequent HF validation (summative) usability test.

Late-stage design evaluations: HF validation testing

Perform HF validation testing with detailed reports and root cause analysis.

HF validation (summative) testing

We have conducted HF validation tests with a wide range of participant sample sizes, ranging up to 200+ participants. Our test protocols are routinely reviewed and tacitly accepted by the FDA. We conduct tests in our own laboratories, at customer sites, in medical simulation facilities and in other research facilities worldwide. Our comprehensive test reports thoroughly describe the usability testing approach, test findings and root causes of any use error and other interaction problems. We base our testing and reporting approaches on available guidance from multiple sources, e.g., FDA, AAMI and IEC, as well as best practices and our many years of experience conducting authoritative tests as a capstone HFE activity to product development.

Examples:

Conduct a 75-participant, HF validation test of an insulin pen injector to generate data demonstrating that the product can be used safely and effectively by adolescents, adults, seniors, caregivers and healthcare providers.
Oversaw and conducted HF validation testing in China involving team use of a surgical system.
Conduct an HF validation test of a heart-lung machine used by perfusionists to sustain patients undergoing open-heart surgery.
Conduct an HF validation test of a mobile application used to track diabetes management and insulin delivery.