DESIGNING FOR DIVERSITY: INCLUSIVE SAMPLING

research methods, sampling, inclusion, equity Experts in human factors and ergonomics (HF/E) and related fields have the ability and responsibility to broadly serve the needs and goals of diverse people, which encompasses issues of inclusion, equity, and justice. Importantly, HF/E designers, researchers, and practitioners can address these aims both as the intended outcomes of their work and how the work itself is conducted. Both pathways support progress toward more inclusive and equitable organizations and societies. This paper focuses upon one aspect of inclusive methodology—strategies for inclusive sampling. Sampling is an important focus because of its fundamental role in defining the internal and external validity of findings. Moreover, sampling is how diverse participants and perspectives are incorporated (or not), and thus represents an early way that exclusion, inequity, or inaccessibility may manifest. Three heuristic questions and six sets of strategies are briefly articulated: (1) purposive sampling, (2) oversampling, (3) community sampling, (4) removing barriers of distance, cost, communication, and awareness, (5) building trust, and (6) inclusive demographic categories. A variety of sources are cited to facilitate readers’ further consideration of these issues in their own HF/E endeavors.


Introduction
Human factors and ergonomics (HF/E) and related fields emphasize the centrality of human needs, goals, capabilities, and constraints in the design of systems that will work for people (Roscoe et al., 2019). Importantly these concerns encompass how people are included or excluded, and how they are empowered or oppressed. People need to experience meaningful autonomy and agency in their daily lives (Slemp, Kern, Patrick, & Ryan, 2018;Van den Broeck, Ferris, Chang, & Rosen, 2016). Systems of exclusion and oppression hinder people achieving their full potential and their overall well-being (Lui & Quezada, 2019;Schmitt, Bransombe, Postmes, & Garcia, 2014), such as in workplace and organizational settings (Dhanani, Beus, & Joseph, 2017;Jones, Arena, Nittrouer, Alonso, & Lindsey, 2017;Robinson, O'Reilly, & Wang, 2013). Consequently, HF/E has a crucial responsibility and opportunity to advocate for equity, inclusion, and justice (Chiou & Roscoe, 2021;Roscoe, Chiou, & Wooldridge, 2020).
Fortunately, there are multiple paths that HF/E experts can follow to support greater inclusion and equity. As depicted in Figure 1, these routes comprise both HF/E goals (i.e., what people seek to achieve) along with HF/E practices (i.e., how people do the work) (Chiou & Roscoe, 2021). First, designers, researchers, and practitioners can seek to understand critical patterns and variables related to inclusion and equity. For instance, Benda and Bisantz (2019) have shared cases in which work domain analysis (WDA) revealed social justice issues related to housing and for patient communication in emergency departments. WDA models helped investigators conceptualize complex housing and medical systems and study how those environments were driven by human goals and decision-making. In turn, the resulting models allowed designers to understand how decisions and disruptions propagated through the system. Strategic interventions (e.g., new tools to supplement professional interpreters in emergency rooms) could be proposed to encourage more equitable processes. Similarly, Rodriguez and Gaviria (2019) demonstrated how ergonomic analyses could reveal the experiences, needs, and challenges of recycler communities. Their work shed light on the interplay of individual characteristics (e.g., age and education), task demands, environments, organizations, technology, safety, and psychosocial factors that contributed to the well-being or harm for recyclers (e.g., inadequate personal protective equipment despite regulations).
The intersections of human factors, health equity, and social determinants of health are likewise increasingly important topics (Holden, Toscos, & Daley, 2019;Wesley, Boxley, Kurgatt, King, & Miller, 2019). A recent edited volume on patient ergnomics (Valdez & Holden, 2021) devoted an entire section and multiple chapters to special concerns for healthcare and design for veterans, children, older adults, and underserved populations. Such research empowered designers to recognize exclusion and marginalization, and to understand their harmful effects on human performance and well-being. Such work enables experts to develop possible solutions (e.g., technologies for older adults, Harris, Nie, & Rogers, 2019) and evaluate these tools for efficacy. In sum, HF/E experts can address equity and inclusion as the direct focus and desired outcomes of their work.
Second, inclusion and equity can also be infused into how people work-their methods and approaches. HF/E experts can strive to inclusively and equitably build teams and partnerships, collect and analyze data, and interpret findings and impact. Research has shown that authentic diversity can benefit creativity and innovation in teams (Salazar, Feitosa, & Salas, 2017;Salazar, Lant, Fiore, & Salas, 2012). When teams are demographically and intellectually diverse, they may be more prepared to generate and evaluate a broader range of ideas. Importantly, these benefits depend on teams sharing a salient team identity while also integrating the identities and contributions of individual team members.
In data analyses, intersectional approaches can explore nuanced effects of demographic variables (Bowleg, 2008;Else-Quest & Hyde, 2016a, 2016b. People define themselves in many ways and along diverse dimensions (e.g., race, ethnicity, gender, sexuality, ability, religion, and occupation). These various identities overlap and intersect, often with implications for inequity, discrimination, and oppression. Awareness of these issues allows people to develop more nuanced models, interpretations, and explanations of research findings. For example, understanding the needs and challenges of older adults (e.g., active aging, mattering, and neglect; Harris et al., 2019) can help to explain technology design and adoption beyond basic usability principles. Power dynamics and oppression also affect how and whether people participate in research, interact with researchers, and interpret and perform research tasks. For psychological scientists, Cole (2009) proposed three useful heuristic questions for thinking about social categories of participants and populations: "Who is included within this category?" "What role does inequality play?" "Where are there similarities?" These questions can be applied throughout all stages of research to offer an intersectional way of thinking about hypothesis generation, sampling, operationalization, analysis, and interpretation.
Similarly, when assessing impact, experts can think beyond scientific impact (e.g., citation counts) to appreciate societal impact on economic trends, government policy, and public attitudes (Bornmann, Haunschild, & Adams, 2019;Ozanne et al., 2017;Rogers et al., 2020;. HF/E scholars and practitioners can redefine the scope of their work in terms of solving meaningful social and societal problems and developing more equitable and inclusive systems, such as fairer elections, food and water security, sustainability, mental health care, and reducing social isolation (Rogers et al., 2020). Another important element of attaining societal impact is communicating such contributions to the broader public via education, outreach, and the news media . número 1, volume 9, jan-jun (2021) This paper focuses attention on one specific aspect of inclusive research and design. Specifically, this work addresses sampling-who we are studying and how they are recruited. Sampling is an important focus because of its fundamental role in defining the internal and external validity of findings. Moreover, sampling is how diverse participants and perspectives are incorporated (or not), and thus represents an early way that exclusion, inequity, or inaccessibility might manifest. The remainder of this paper briefly discusses the importance of representative sampling, threats to inclusive sampling, and strategies for inclusive sampling.

Sampling and Exclusion
As taught in numerous sources, sampling refers to procedures for identifying and recruiting participants as sources of data (e.g., Levy & Lemeshow, 2013;Shadish, Cook, & Campbell, 2002). Although researchers and designers often aim to draw conclusions or develop products that are meaningful across a range of people and contexts (e.g., working, learning, and playing), it is rarely possible to measure the entire population of interest. Just as doctors draw blood samples to evaluate signs of health, and ecologists draw water samples to evaluate levels of pollution in rivers, HF/E experts gather data from samples of people to make inferences about how others think, feel, and behave.
A fundamental principle of effective sampling is that the samples must be representative (Corrigan & Onwuegbuzie, 2020;Onwuegbuzie & Collins, 2007;Shadish et al., 2002). The samples must demonstrate a comparable composition and range of characteristics to the group(s) of interest. Valid inferences can only be generated for groups similar to the sample. This is true for any data gathering effort, whether for formal hypothesis testing, user testing, decision making, or design. Sampling that is too demographically skewed or exclusionary can only produce results that are skewed, incomplete, and/or biased.
In the popular book, Invisible Women: Data Bias in a World Designed for Men, Perez (2019) documents numerous examples of systems and products that were designed without sufficient attention to sex and gender variation. Specifically, she argued that men are often implicitly viewed as "the human default," with diverse and adverse effects on women's experiences with medicine (e.g., diagnosis and treatment), safety (e.g., automobile crash tests and chemical exposure), and everyday products (e.g., smart phones). Similar demographic exclusions and neglect contribute to broad inequities in technology design (Wachter-Boettcher, 2017) and algorithms that shape information-seeking, decision-making, and technology (Noble, 2018).
Most HF/E experts probably do not intend to recruit biased or exclusionary samples. However, a substantial amount of work in the social sciences relies on recruiting people who are nearby, ready, willing, and able to participate. When working in academic settings, researchers often rely on university "subject pools" or "participant pools" (e.g., students in undergraduate classes who participate in studies for course credit) (Gallander Wintre, North, & Sugar, 2001;Rocchi, Beaudry, Anderson, & Pelletier, 2016;Walker, 2020). In work with consumers, one might recruit customers shopping in nearby businesses. Similarly, in industry settings, one might test designs among coworkers. Collectively, these approaches all represent forms of convenience sampling.
A number of scholars have adopted the acronym "WEIRD" to characterize systematically skewed sampling in the social sciences (particularly in the United States) (Bergman & Jean, 2016;Clancy & Davis, 2019;Henrich, Heine, & Norenzayan, 2010;Nielsen, Haun, Kärtner, & Legare, 2017). WEIRD stands for "Western," "educated," "industrialized," "rich," and "democratic." In brief, research reviews have revealed that participants are frequently recruited from cultures, communities, and organizations defined by certain economic and societal principles (e.g., capitalism and meritocracy), higher levels of education, and higher income. These trends are partly driven by reliance on recruiting university students as participants (i.e., confounded with education and income) and participants who have the time and mobility to enroll in studies. As a result of WEIRD sampling, research and practice in the social sciences may systematically exclude número 1, volume 9, jan-jun (2021) PUC-Rio Pontifícia Universidade Católica do Rio de Janeiro Departamento de Artes & Design | PPGDesign LEUI | Laboratório de Ergodesign e Usabilidade de Interfaces marginalized and minority communities, exclude persons from different socioeconomic backgrounds, exclude persons with disabilities, exclude persons living in rural areas, and more.
In sum, based on the importance of inclusive and representative samples for valid HF/E work, it is worthwhile to consider inclusive sampling strategies. What can HF/E experts do to ensure their participants embody meaningful diversity, particularly for people often ignored, neglected, or disenfranchised? A handful of recommendations are articulated below.

Inclusive Sampling Strategies
Systematic constraints on participant availability result in systematic exclusions that restrict validity and generalizability. Numerous factors can hinder inclusive sampling (Clark et al., 2019;Ford et al., 2007;George, Duran, & Norris, 2014;Heller et al., 2014;Waheed, Hughes-Morley, Woodham, Allen, & Bower, 2015). Taking inspiration from Cole (2009), these considerations can be broadly framed as heuristic questions for HF/E experts to ask when designing experiments, focus groups, interviews, and other data collection efforts: • Who is not present to participate in the study? For instance, if recruitment takes place among students, customers, or coworkers, then one must consider who is not enrolled in those schools, not shopping at those stores, or not employed in those companies. Schools that are located in affluent neighborhoods will offer little access to students affected by poverty. Stores that are located within densely-populated urban settings will host different consumers than suburban or rural regions. Researchers must also consider how populations may change over time (e.g., throughout the work day or seasonally) and how that affects who is present and available to participate.
• What logistical barriers prevent people from participating in the study? People cannot participate when they lack the time, transportation, or technology to access the research materials or data collection sites, whether physically or virtually. Importantly, such resources are not equitably distributed throughout the broader population. Thus, logistical constraints result in systematic exclusion of individuals who are already under-resourced and under-served.
• Who chooses not to participate in the study? Some individuals or communities may possess justifiable distrust toward researchers and designers, perhaps due to concerns about privacy, fear of exploitation or data misuse, or previous negative experiences.
HF/E experts can employ a variety of approaches to overcome these challenges. Importantly, these strategies are not intended to be exhaustive, but may help people "get started" or inspire further innovation.

Strategy 1: Purposive Sampling
In contrast to convenience sampling, purposive sampling involves intentional recruitment of participants from specific populations or groups of people (Gentles, Charles, Ploeg, & McKibbon, 2015;Levy & Lemeshow, 2013;Palinkas et al., 2015;Patton, 2015). Instead of relying on chance to include diverse people, researchers can determine necessary populations in advance and strategically recruit them. Specifically, designers can review their prior work to assess who has been omitted. If participant demographics were número 1, volume 9, jan-jun (2021) skewed with respect to race, gender, ability status, language, or other factors, future recruitment can directly address these gaps. Researchers can reach out to new organizations or networks who were not previously invited. Stratified and quota sampling can increase accountability by defining who needs to be recruited, how many people, and from which underrepresented groups, for a study to be considered "complete."

Strategy 2: Oversampling
A related approach is oversampling to ensure larger or statistically adequate sample sizes of "rare" populations (Hauner, Zinbarg, & Revelle, 2014;Kalton, 2009). By definition, "minority" groups possess fewer members relative to the broader population. Thus, a sample might be proportionally representative (i.e., the demographic composition of the sample is comparable to the population) yet still unable to support meaningful comparisons or conclusions about a given demographic. For example, some estimates report that transgender individuals comprise less than 1% of adults (e.g., Flores, Brown, & Herman, 2016;Zhang et al., 2020). Thus, a proportionally representative sample of 100 people might include 1 to 2 transgender individuals; a sample of 1000 might include 10 transgender individuals, and so on. The subset of transgender participants would be substantially underpowered for many analyses. Purposive oversampling strives to recruit sample sizes that enable meaningful conclusions about people regardless of minority status.

Strategy 3: Community Sampling
A parallel approach to purposive sampling is community sampling, which involves recruiting people within broader communities defined by shared culture, identity, history, geography, or purpose (Constanza-Chock, 2020, Israel, Eng, Schulz, & Parker, 2013Jason & Glenwick, 2016;Palinkas et al., 2015;Valerio et al., 2016). All organizations are constrained-such as university entrance requirements and tuition fees, or corporate employment qualifications and hiring policies-which inherently restricts the range of people associated with those organizations. Recruiting from local communities begins to bypass those filters.
Community sampling aligns well with community-based research practices (Benda et al., 2020;Constanza-Chock, 2020). HF/E experts can intentionally strive to work with under-served communities most affected by inequity (e.g., Valerio et al., 2016;Waheed et al., 2015;Wallerstein & Duran, 2010). Community members can be instrumental in identifying and defining problems, establishing research questions, selecting methods, analyzing data, and interpreting the results. This approach can restore agency and power to marginalized groups who are normally excluded from these decisions and procedures. In such cases, the outcome is not only more inclusive sampling, but perhaps also justice for the communities (e.g., Wallerstein, Duran, Oetzel, & Minkler, 2018).

Strategy 4: Remove Barriers to Participation
Despite best intentions to recruit from diverse populations and communities, logistical and psychosocial barriers can hinder inclusion (Benda et al., 2020;George et al., 2014;Sheridan et al., 2020;Stonewall et al.., 2019). A third strategy (or set of strategies) is to enable participation-identify factors that limit participation in data collection and then remove those barriers.
• Strategy 4a: Reduce the demands of physical distance. Some participants may lack the time or transportation to visit the research site. Overcoming this obstacle may require establishing new locations closer to communities or ensuring that sites are accessible via public transportation. Another method is to employ tools for virtual or electronic data collection, such as remote meetings, número 1, volume 9, jan-jun (2021) PUC-Rio Pontifícia Universidade Católica do Rio de Janeiro Departamento de Artes & Design | PPGDesign LEUI | Laboratório de Ergodesign e Usabilidade de Interfaces chat messaging, digital diaries, or web-based surveys. These approaches require reliable Internet access but negate the need for travel.
• Strategy 4b: Reduce financial constraints. HF/E experts should be mindful that taking part in a study may require time away from work, childcare expenses, travel expenses, or other costs. People with fewer financial resources may find it impossible or unjustifiable to participate. Researchers might address this problem by scheduling data collection outside of work hours, but this may conflict with time for household chores, family, or recreation. An alternative approach is to provide meaningful financial compensation that accounts for (or exceeds) the costs of participating. Any required technologies (e.g., a mobile device) should be also made available to at no cost.
• Strategy 4c: Improve and enable communication. The preferred language of the HF/E team may not be shared by participants. Individuals who lack fluency in reading research materials (e.g., advertisements, consent forms, and instructions), writing responses, or speaking with the researchers (e.g., in focus groups) may thus be excluded. These concerns can be addressed by working with translators or interpreters who can facilitate communication across language barriers. All materials should appear in languages relevant to the target populations, and participants should have the ability to express themselves in their own language.
• Strategy 4d: Raise awareness and be inviting. Diverse individuals and communities may not have equivalent access to learning about research opportunities. Flyers placed on university bulletin boards, office break rooms, or on social media can only be seen by people who visit those spaces. HF/E experts should advertise broadly in diverse venues and learn how target communities typically communicate (e.g., signage or word-of-mouth). Advertisements must also be welcoming-not just an "announcement" of a study, but a clear invitation and encouragement to take part. Recruitment materials should clearly communicate that peoples' time and contributions will be valued.

Strategy 5: Build Participant Trust and Confidence
Some individuals and communities choose not to participate in data collection because they are justifiably wary of research, researcher motives, and ethics (Christopher, Watts, Knows His Gun-McCormick, & Young, 2008;Frerichs et al., 2018;Waheed et al., 2015;Williams & Gilbert, 2019;Wood, 2017). Differences in resources (e.g., funding), perceived prestige (e.g., corporate and university credentials), and other factors also give rise to power imbalances between researchers and communities. Participants may feel they have little control over how they participate in the study, worry about how their data will be used or interpreted, or even fear for their safety. For instance, Williams and Gilbert (2019) documented how research with neuroatypical or disabled individuals can be dominated by "proxies" (e.g., parents or other caregivers) who speak on behalf of the participants, resulting in a loss of voice, agency, and trust. As a result, findings or designs are oriented toward the needs or goals of the proxy rather than the participants, and participants feel powerless, ignored, or objectified.
To alleviate these valid concerns, HF/E experts should be transparent about project goals, methods, precautions, expectations, and outcomes. In addition, participants (and their communities), should have a substantive voice in the project-as partners, co-researchers, co-designers, and co-evaluators (Constanza-Chock, 2020;Wallerstein et al., 2018). Their needs, expertise, insights, and culture should have influence on project design and decisions (Berryman, Soohoo, & Nevin, 2013). HF/E experts should also consider how their "research goals" or "design problems" align to the target community or population. Through such authentic partnerships and collaborations, the inner workings of the research are made visible and interactive rather than mysterious or suspicious. Findings and products should also benefit people and communities in both the short-term and long-term and should never contribute to continued inequity. Trust is established and número 1, volume 9, jan-jun (2021) reinforced when the benefit of the work is meaningful and tangible, the potential for harm is reduced, and participants have the power to define what constitutes "benefit" or "harm."

Strategy 6: Inclusive Demographic Categories
A final strategy pertains to how participants are characterized during recruitment and analysis. People identify themselves using a wide variety of dimensions, categories, and labels (e.g., gender, sexuality, ethnicity, race, ability, occupation, and religion). These categories combine and intersect with significant ramifications for inequity, discrimination, and oppression (Bowleg, 2008;Cole, 2009;Collins, 2015;Else-Quest & Hyde, 2016a, 2016b. If these nuances are not respected during recruitment and analysis, then efforts to be inclusive will be undermined. Participants are essentially made "invisible" when they cannot identify themselves accurately and as they prefer. Researchers should avoid limited and restrictive category labels (e.g., only binary "Female" or "Male" options) and people should be able to describe multiple facets of their race and ethnicity without conflating the two. A common best practice is to provide multiple categories for race and instruct participants to "select all that apply." Care should also be taken to avoid labels that make people feel unwelcome or invisible (e.g., "Other" or "non-White"). When writing about different communities, researchers should respect preferences for person-first versus identity-first language (Dunn & Andrews, 2015;Gernsbacher, 2017). Person-first language strives to elevate personhood and reduce bias by not defining people by or as a disability (e.g., "person with schizophrenia" instead of "schizophrenic"). However, identify-first language highlights salient characteristics that people embrace as normal and integral to their self-such as Autistic or Deaf and Hard of Hearing-and challenges conceptualizations of what is considered a "disability." Because different communities may prefer use of person-first or identity-first language, it is essential to work with people to understand and respect their preferences. Finally, people should also have the option to not disclose their identity (e.g., "Prefer Not to Say"). Ultimately, if people cannot accurately, positively, and safely identify themselves, then it is difficult to feel authentically included.
Similarly, during analysis, it may be crucial to explore whether demographic differences influence findings, design decisions, or impact. However, these analyses are only possible when nuanced and inclusive categories can be represented in the data (Else-Quest & Hyde, 2016b). It is impossible to determine whether a sample is representative if the sample cannot be properly characterized. And, if HF/E experts fail to collect these data, then they lose the ability to detect or be responsive to the needs of diverse groups (Chiou & Roscoe, 2021).

Conclusion
Human factors and ergonomics (HF/E) and related fields have the ability and responsibility to broadly serve human needs and goals, which encompasses issues of inclusion, equity, and justice. HF/E experts can seek to reveal inequities, understand the complex causes and systems of disparities, and develop technologies or other interventions to reduce these injustices. HF/E experts can also embrace more inclusive and equitable practices for assembling research and design teams, collecting and analyzing data, and conceptualizing their findings and impact. In sum, HF/E can address inclusion and equity both through the intended outcomes of the work and how the work itself is conducted-both pathways support progress toward more inclusive and equitable organizations and societies.
This paper focused upon one aspect of inclusive methodology-strategies for sampling. Three heuristic questions and six sets of strategies were provided (summarized in Figure 2). This work highlighted sampling because of its essential and early role in establishing the validity and generalizability of our outcomes, yet other methodological concerns (e.g., study designs, elicitation and observation protocols, and analytical número 1, volume 9, jan-jun (2021)