Substituting proxy participants for actual intended users in human factors validation testing

By OnKee Min

Including representative and intended users for a human factors (HF) validation test provides invaluable data for manufacturers because participants’ performance data can demonstrate a product’s expected safety and effectiveness once it is in the market. However, it can also be extremely difficult, or nearly impossible, to recruit exact users for a particular product due to the rarity of a condition or job title as well as the person’s health and willingness to participate. In these situations, manufacturers might consider recruiting proxy participants, or substitute participants for actual users. A proxy participant might also be referred to as a surrogate user but will be referred to as a “proxy participant” or “proxy” primarily in this article.

Although regulators generally discourage substituting proxy participants (i.e., proxies) for actual users in HF validation tests of medical devices and combination products, manufacturers might be able to justify including proxies in certain situations. This article explains when it is appropriate to recruit proxies, how to identify an appropriate proxy participant and how to justify including proxy participants in HF validation tests to regulatory bodies.

FDA perspective on proxy participants

In their final guidance, the U.S. Food and Drug Administration (FDA) states, “The most important consideration for test participants in human factors validation testing is that they represent the population of intended users” (February 2016). As such, the FDA is generally hesitant to approve manufacturers substituting or supplementing actual users with proxy participants for HF validation tests, regardless of the time, effort and resources it might take to locate, recruit and conduct sessions with actual users.

If, however, manufacturers simply cannot recruit actual users or actual users cannot participate, they can try to justify including proxies in their HF validation tests.

Justifying proxy participants

A few methods to justify a study sample with proxy participants include researching the intended user groups, exhausting recruitment efforts and providing evidence that the proxies are sufficiently representative of actual users.

Research

• Prevalence of the condition or healthcare-related occupation.
  • If the product is intended for patients with a rare disease, research the prevalence and incidence rates of individuals affected by the condition(s).
  • If the product would be used by specific types of healthcare providers (HCPs), research the number of individuals who have that job title and/or occupational background as well as the number of individuals who enter and retire from the field each year.
     
• Patients’ safety to participate. Consider any risks that an intended user, namely a patient or caregiver, might face by participating in the usability study due to their or their loved one’s condition. For example, cancer patients might have a compromised immune system due to their condition and/or treatment regimen and as such, might have limitations on participating in a usability study in a public space and/or traveling to testing facilities.
   

Recruitment

• Expand the recruit.
  • Conduct a pre-screening or recruit feasibility assessment ahead of the study to gauge how many individuals meet the recruitment criteria.
    • Pre-screening uses a short screener (e.g., up to 5 close-ended questions) to determine the number of qualified participants in a market. Pre-screeners can help manufacturers determine in which market(s) to conduct their research.
    • Recruiting firms might conduct a recruit feasibility assessment to determine the prevalence of a user group across their participant database as well as to create new connections to supplement their panel via outreach to other recruiting partners and patient support groups.  
  • Contact patient advocacy groups or foundations that specialize with the intended user group. Many of these groups have a network of patients and families (e.g., caregivers) they support and may be able to connect HF researchers with participants for usability testing.
  • Engage multiple recruiting firms and get a sense of how many firms are willing to conduct the recruit and in which markets. Some firms might agree to complete all or a part of the recruit, and some firms might decline to even attempt the recruit due to their low confidence filling the quota. Manufacturers should note the number and types of groups they contact and how many accept or decline the recruit in their justification.
  • Consider multiple markets and locations across the country to find actual users for the study.
  • Use any additional recruiting tools that might be available, including an advertising budget and referral incentives.
  • As possible, maximize the duration of the ongoing recruit and note the number of participants who were contacted and recruited during this time.  
     
• Past usability test experiences. If a manufacturer has conducted studies with the same user group in the past (e.g., for a formative study with fewer participants) but faced significant recruiting difficulties, the likelihood they will face further challenges filling the recruit for a complete HF validation test is high. In this scenario, the manufacturer can capture all details relevant to past studies and the resulting recruitment numbers for their justification.

Provide evidence that proxies are comparable

• Compare performance results of intended users and proxies. If a manufacturer has performance data with actual intended users (e.g., from a formative study), they can compare that data (e.g., number of findings per critical task) to data collected with proxy participants to demonstrate that interactions with the test product between these two groups are comparable.
   
• Explain that the proxy could become an intended user. For certain user groups, manufacturers might be able to argue that some individuals might become an intended user in the future even if they are not one currently. For example, any adolescent could be diagnosed with type 1 diabetes regardless of their background and previous medical history. As such, manufacturers who are seeking to recruit adolescents with type 1 diabetes can try to make a case that an adolescent who does not have type 1 diabetes but otherwise shares similar characteristics as adolescents with type 1 diabetes is an appropriate proxy participant for this user group.

Identifying proxy participants

Defining the intended user profiles can help manufacturers identify appropriate proxy participants. User profiles typically summarize the group’s demographic information (e.g., age, sex), as well as level of education, skills or abilities, medical history and other performance-shaping factors that might differentiate these individuals from the general population (IEC 62366-1:2015). Once the intended user profile is clearly defined, manufacturers can identify individuals who might share these characteristics and as such, might interact with their product similarly to that of an actual user.

Below are some examples of user profile information and possible proxies.

Constructing a justification for proxy participants

To summarize, manufacturers can compose a strong justification to the FDA and explain the decision to include proxy participants in their HF validation tests with the following information:

• Explain why this is a difficult user group to recruit (e.g., prevalence).
• Explain what steps were taken to recruit actual users and why 15 actual users, for example, could not be recruited despite these numerous and varied efforts.
• Explain who will be included in the HF validation test to supplement the actual user group (i.e., proxy participants) and how these individuals are representative of actual users.

Although testing with actual users is far and wide a best practice for HF validation testing, manufacturers can consider including proxy participants when actual users are unavailable for testing. With additional research and context as to why actual users could not participate, manufacturers can develop a justification to recruit proxy participants to present to the FDA.

Contact our team to learn more about how to conduct HF validation test for unique user groups. Or, sign up for a complimentary account with OPUS, our team’s software platform that provides HFE training, tools and a user profile library.

OnKee Min is a Human Factors Specialist at Emergo by UL.