Racially Distinctive Names Signal Both Race/Ethnicity and Social Class

,

R ESEARCHERS studying discrimination and bias frequently conduct experiments that use racially distinctive names to signal race or ethnicity. 1 This practice is perhaps most common in field experiments known as audit studies or correspondence audits, which provide the bulk of modern causal evidence on racial discrimination. Since the 1960s, researchers have conducted hundreds of audits to examine discrimination more broadly in various markets (Crabtree 2018;Daniel 1968;Fix and Struyk 1993;Gaddis 2018b;Wienk et al. 1979). This body of research has found consistent evidence of racial discrimination in the labor and housing markets over the last six decades (Bertrand and Mullainathan 2004;Ghoshal 2015, 2020;Gaddis et al. 2022b). More recently, researchers have expanded the use of audits to examine discrimination among a broader set of actors, such as bureaucrats, college admissions counselors, health care professionals, politicians, principals, professors, and members of the general public (Block et al. 2021;Butler and Broockman 2011;Einstein and Glick 2017;Gell-Redman et al. 2018;Hughes et al. 2020;Milkman, Akinola, and Chugh 2012;Pfaff et al. 2021). The popularity and usefulness of this method are reflected in numerous reviews (Baert 2018;Butler and Crabtree 2021;Gaddis and DiRago 2021), meta-analyses (Gaddis et al. 2022b;Quillian et al. 2017;Zschirnt and Ruedin 2016), and methodological examinations (Gaddis 2018a;Larsen 2020;Vuolo, Uggen, and Lageson 2018) in the last few years alone. One reason for this popularity is that audits provide strong causal evidence of discrimination. They allow researchers to (1) avoid the typical shortcomings of using secondary data to examine discrimination (Crabtree 2019;Golder, Crabtree, and Dhima 2019) and (2) bypass the possibility of social desirability bias from respondents in surveys and interviews (Butler and Crabtree 2021;. Correspondence audits, however, are not alone in their use of names for studying racial discrimination. Researchers also often use conjoint and survey experiments to uncover the natures and origins of discrimination. Like correspondence audits, these experiments have exploded in popularity in recent years Druckman and Green 2021;Hainmueller, Hangartner, and Yamamoto 2015;Hainmueller and Hopkins 2015;Jenke et al. 2019;Lu et al. 2021). Although scholars use conjoint and survey experiments for many purposes, one of their more common uses is to study racial attitudes and preferences that are likely the foundation of discriminatory behavior (Caruso, Rahnev, and Banaji 2009;Doherty, Dowling, and Miller 2019;Jankowski, Prokop, and Tepe 2020). Research findings show that respondent preferences in conjoint experiments (and discrete choice experiments specifically) generalize reasonably well to real-world behavior .
Many correspondence audits and conjoint experiments, however, do not signal race directly but rather indirectly through the use of names. The indirect signaling of race is potentially a problem for the interpretation of these experimental results because names might signal multiple characteristics in addition to race. One way of viewing this is as a possible violation of the excludability assumption (Gerber and Green 2012). In this case, the causal effect identified in a study that uses names might be a combination of the intended treatment (i.e., race) and other treatment aspects (i.e., social class) (Hansen and Tummers 2020). Another way of viewing this is as a violation of "information equivalence" (Dafoe, Zhang, and Caughey 2018) or "information leakage" (Sher and McKenzie 2006), which can occur when researchers present experimental subjects with information about one attribute in an experimental treatment (i.e., race) and this causes them to update their beliefs about others too (i.e., social class). A third way of viewing this is as a potential threat to the "construct validity" (Cronbach and Meehl 1955;Trochim 2001) of names as "race" treatments. This would mean that name treatments might not be adequate operationalizations of the underlying theoretical concept (i.e., race) because they also might measure other concepts (i.e., social class). In other words, the experimental treatments might feature a "double stimulus" (Converse and Presser 1986) or contain a "double-barreled treatment" (Kertzer and Brutger 2016).
Surprisingly, few studies empirically examine the construct validity of using names as measures to signal race or racial identity. Of those, many focus on the appropriateness of the race signal without deeply investigating whether names send signals other than race (Crabtree and Chykina 2018;Gaddis 2017a,b;Gaddis, Kreisberg, and Crabtree 2022a). This work suggests heterogeneity in how accurately and consistently individuals perceive the race that researchers intend to signal with names. Others have used demographic data on names and socioeconomic status (SES) to suggest that names likely only signal race and not SES or social class (Bertrand and Mullainathan 2004;Fryer and Levitt 2004) or that, at the very least, experimental subjects are not influenced by any signals of SES (Butler and Homola 2017;Carnes and Holbein 2019). 2 However, a potential problem with social class signaling is that people often misperceive economic reality. This misperception is particularly true in the American context, where most people have little knowledge about economic conditions (Callaghan et al. 2021), the media provides disproportionate coverage of Black poverty (Roberts et al. 2019), and rich and poor individuals hold different views about the state of economic equality across racial groups (Kraus, Rucker, and Richeson 2017).
Two recent studies examined respondents' perceptions of social class using names (Barlow and Lahey 2018;Landgrave and Weller 2022). Although these studies move the literature forward in multiple ways, they still leave many important questions unanswered. In the first study, the authors use a very small sample size of undergraduate students (Barlow and Lahey 2018), which may not capture the heterogeneity in perceptions found in larger, more diverse samples (Crabtree and Chykina 2018;Gaddis 2017a,b). In the second study, the authors use a moderate sample size of respondents through Amazon's Mechanical Turk (MTurk). Still, they only examine respondents' perceptions of 12 total names-all male, four White, four Black, and four Hispanic (Landgrave and Weller 2022). Thus, the current literature in this area has two important limitations. First, Americans' misperceptions of racial and economic realities likely limit the value of using demographic data on naming patterns to proxy for perceptions. Second, existing work on perceptions does not have adequate coverage of a wide variety of names, examine nationally representative samples of respondents, or test multiple operationalizations and components of SES and social class. For these reasons, we believe that additional research in this area will help strengthen the construct validity of future experimental work using names to signal race.
In this article, we report the results of five studies we conducted to examine respondent perceptions of individuals' educational attainment, income, and social class based on racialized names. In total, 11,530 respondents provided 151,899 perceptions on 1,000 different combinations of first and last names. Using data from four of these studies, we find that respondents perceive educational attainment, income, and social class from names in a racially tiered pattern: White and Asian people are perceived at the top of the social class hierarchy, followed by Black people and then Hispanic people. Using data from the fifth study, we provide respondents with a vignette describing the named person as having attained either a high school degree, a bachelor's degree from a public state school, or a bachelor's degree from a private Ivy League school. Even when presented with an explicit signal of educational attainment, respondents perceived individuals with names commonly used by Black people as of a lower social class than individuals with names commonly used by White people. This gap in perceptions, however, is much smaller when names are coupled with a signal of high SES, in this case being described as having a bachelor's degree from a private Ivy League school. We discuss the implications that these results have for past and future experimental work that uses names to understand racial discrimination and outline several paths scholars might take to understand more about the signaling power of names with regard to racial/ethnic discrimination and bias.

Data
We assess the construct validity of racialized names as an indirect signal of race using data from five different studies (n = 1,004; 2,002; 1,035; 5,631; 1,858) conducted at different times across four separate survey platforms (Lucid Marketplace, Lucid Theorem, MTurk, and Prolific). Our data focus on name evaluations, or perceptions of individuals based on their names. We collected survey data from U.S. residents on name evaluations across five separate data collections.
We conducted the first survey (hereafter Study 1) with 1,004 respondents via Prolific, an increasingly popular survey firm among social scientists (Palan and Schitter 2018), on February 4, 2021. We conducted the second survey (Study 2) with a national, quota-based (gender, age, region, race, ethnicity) sample of 2,002 Americans through Lucid Marketplace, an established and widely used survey provider (Coppock and McClellan 2019), from May 19 to 31, 2021. We conducted the third survey (Study 3) with a quota-based (gender, age, region) sample of 1,035 Americans who identify as first-or second-generation immigrants through Lucid Marketplace from May 18 to 27, 2021. We conducted the fourth survey (hereafter Study 4) with 5,631 respondents via MTurk from September 2014 through August 2015. We conducted the fifth survey (hereafter Study 5) with 1,858 respondents via MTurk in October 2015. 3 In Studies 1 through 3, we asked respondents to answer multiple questions about their perceptions of each name. These perceptions span four different dimensions: race, citizenship status, income, and education. 4 In Studies 4 and 5, we only asked respondents about their social class perception of each name. We provide the specific questions about name perceptions in the "Dependent Variables" section below. Additionally, we asked respondents to report on a series of demographic and background questions in Studies 2 through 5. In Studies 2 and 3, we asked respondents to report their age, gender, income, educational attainment, race/ethnicity, and whether they are an American citizen. In Studies 4 and 5, we asked respondents to report their age, gender, household income, educational attainment, race/ethnicity, sex, relationship status, employment status, zip code, and whether they have any children younger than 18.
In sum, there are at least seven important dimensions of variation across our five studies: (1) respondent recruitment platform; (2) time period; (3) sample size; (4) specific names tested, as we describe below; (5) questions about social class; (6) answer options for social class questions; and (7) whether we provided an additional signal of SES to respondents.

Names
In Studies 1 through 3, we used Tzioumis (2018) for our primary list of first names and Comenetz (2016) for our list of last names. These are perhaps the two most authoritative lists of name prevalence in the United States containing racial associations. These data sets allow us to determine the 10 most frequently used, racially distinctive (or differentially expressed) first names and 10 most frequently used, racially distinctive last names among White, Black or African American, Asian or Pacific Islander, and Hispanic people. First, we subset the first name list to those that are at least 90 percent used by Asian or Pacific Islander, Hispanic, and White people and at least 80 percent used by Black or African Americans. 5 We then subset the last name list to include those that are at least 90 percent used by Asian or Pacific Islander and White people, at least 80 percent used by Hispanic people, and at least 45 percent used by Black or African American people. 6 We then created all permutations of first and last names for each racial condition, giving us 400 distinct names (10 × 10 × 4).
In Studies 4 and 5, we selected first names used in prior experiments and then others from the Levitt and Dubner (2005) names list. Additionally, we selected last names using frequently occurring surnames from the 2000 Census, which lists the population racial composition of last names in the United States (U.S. Census Bureau 2012). In Study 4, we selected 80 White, 80 Black, and 40 Hispanic first names each paired with three randomly selected last names from the top six racially distinctive frequently occurring surnames list. Thus, we examine 600 total names: 240 White, 240 Black, and 120 Hispanic names. In Study 5, we selected a subset (from Study 4) of 80 total names: 40 White and 40 Black names.

Dependent Variables
In Studies 1 through 3, we examine respondents' perceptions of educational attainment and income based on names. One question asks respondents "What would be the highest level of education you think a person with this name has likely completed?" with the multiple-choice answer options "High school," "Bachelor's degree," "Master's degree," and "Ph.D. degree." Another asks respondents "What type of income do you think a person with this name would make?" with the multiple-choice answer options "Low income (Less than $41,000)," "Middle income ($41,000-$120,400)," and "Upper income (More than $120,400)." In Studies 4 and 5, we examine perceptions of social class. The survey question in Study 4 instructs respondents, "For each of the following names, list the social class category that you associate with that name (for example: lower class, working class, middle class, upper class, etc.). If you do not have a clear social class association with a name, you may type 'none.' " We recoded all answers into three categories: "lower/working class," "middle class," and "upper class." We removed "none" and blank responses from the analysis sample (9.7 percent). In Study 5, respondents are presented with a single vignette stating that "[Name] has a [degree]," where [Name] was one of 40 randomly assigned names (20 White and 20 Black) and [degree] was one of three randomly assigned educational attainment statuses: "high school degree," "bachelor's (college) degree from a public state school," or "bachelor's (college) degree from a private Ivy League school." The survey question in Study 5 asks respondents "What social class do you think [Name] is?" with the multiple-choice answer options "Lower or Working," "Middle," and "Upper."

Analysis and Results
We use these respondent-evaluation data to examine the construct validity of names as "race" treatments. To do this, we estimate a series of ordinary least squares models that regress respondent answers to our social class questions on a set of dummy variables that indicate the racial signal each name was intended to convey. In the figures below, we present predicted social class perceptions across racial groups. Plotted points denote predictions and bars represent 95 percent confidence intervals calculated using robust standard errors. We present the results from Studies 1 through 3 and a pooled analysis in Figure 1 with education perceptions in the top panel and income perceptions in the bottom panel. We examine heterogeneity by individual names in Figure 2. We present the results on social class perceptions from Study 4 in Figures 3 and 4. Finally, we present the results on social class perceptions with described educational attainment from Study 5 in Figure 5.   In the top panel of Figure 1, we show that, on average, respondents view individuals with Asian names as having slightly higher educational attainment than individuals with White names. Both groups are perceived as having at least a bachelor's degree. Respondents view individuals with Black names as having lower educational attainment than individuals with White and Asian names, followed by individuals with Hispanic names. These findings are robust across the three studies, with some small differences in Study 3.
In the bottom panel of Figure 1, we show that, on average, respondents view individuals with White and Asian names as being very similar in terms of incomeroughly the middle class. Similarly to the educational attainment results, respondents view individuals with Black names as having lower income than individuals with White and Asian names, followed by individuals with Hispanic names. These findings are also robust across the three studies.
In Figure 2, we show heterogeneity in education and income perceptions by each of the individual first names we used. The black bars represent 95 percent confidence intervals for each group, and the red bars represent 95 percent confidence intervals for each name. We include text for first names significantly different from their racial/ethnic group mean. This difference occurs in seven cases for education perceptions: Richard, Robert, Mary (White names); Roosevelt, Latonya, and Tanisha (Black names); and Thanh (Asian name). This difference occurs in five cases for income perceptions: Richard and Mary (White names); Roosevelt and Lakisha (Black names); and Guadalupe (Hispanic name).
In Figure 3, we show the group means of social class perceptions from Study 4. Individuals with White names are perceived as falling somewhere between middle and upper class. Individuals with Black names, on the other hand, are perceived as between middle and lower or working class, and individuals with Hispanic names slightly below that.
In Figure 4, we show heterogeneity in social class perceptions by each of the individual first names we used. The black bars represent 95 percent confidence intervals for each group, and the orange bars represent 95 percent confidence intervals for each name. We examine more names in Study 4; thus, we do not highlight specific names. However, the distribution shows that respondents perceive names across a wide range of social class categories for each racial/ethnic group. Respondents perceive a few White names as less than middle class, on average, and a few Black names as greater than middle class. However, the vast majority of the White distribution does not overlap with the vast majority of the Black distribution. Additionally, respondents' perceptions of social class for Hispanic names are clustered around the lower tail of the Black name distribution.
Finally, in Figure 5, we show the results from Study 5, which includes a direct signal of educational attainment. Even when presented with information that a named individual has a high school degree or a bachelor's degree from a public state school, respondents perceive individuals with Black names as of a lower social class than individuals with White names. This perception gap between individuals with Black and White names is reduced considerably (p < 0.05) when respondents are told that a named individual has a bachelor's degree from a private Ivy League school. However, the difference in perception between individuals with Black and White names is still statistically significant (p < 0.05). Figure 5 provides an important topical finding that merits further investigation: for respondents to perceive Black people as having (roughly) equal SES levels as White people, one has to signal that Black people have incredibly elite levels of education.

Conclusion
The results from our five studies provide strong evidence that names commonly used in experimental designs to measure discrimination convey information about race as well as education, income, and social class. In other words, signals of race covary with perceptions of social class. This simple insight raises substantial issues for experimentalists who want to use names to signal race or ethnicity. It also suggests that we should read the results of prior experimental work using names to signal race with some careful consideration. Although the results from these studies are often discussed as showing how racialized names lead to differential outcomes, our findings suggest that these documented disparities might be, to some degree, driven by perceptions of social class as well. This is particularly true in cases where researchers do not include independent cues about social class in their experimental designs, such as in most audit studies outside employment contexts.
Still, we urge some caution regarding the external validity of this research as it applies to field and conjoint experiments. The survey experiment context necessitates that we prompt respondents to think about their social class perceptions of names. Hiring managers evaluate resumes and landlords evaluate rental applications without anyone asking them to think specifically about names. Although it might make sense that participants in these fields would consider the SES of those they interact with, it is less clear that this is true in another potential study types, such as those that involve public officials or members of the public. Moreover, in some contexts, decision-makers may have access to explicit signals of social class. In Study 5, we attempted to approximate the real-world process more closely by including another signal of social class. However, future work might conduct additional survey experiments with decision-makers without directly asking them about these perceptions. Researchers could use other questions and combinations of signals to examine this issue in more detail.
Where should researchers go from here? We present two potential broader interpretations of our findings. These two interpretations depend on whether scholars see the correlation between race and social class perceptions as a mechanism of inequality or a methodological problem to solve.
First, some scholars might consider that names signaling both social class and race is a "feature, not a bug." In other words, part of the mechanism of racial/ethnic discrimination might be individuals' associations of lower SES or social class with Black and Hispanic people. In this scenario, attempting to control for or adjust the impact of social class signaling would be misguided. It could even potentially lead to something akin to posttreatment bias. Researchers could develop new name-based, factorial experimental designs that focus on identifying social class as a mechanism driving discrimination. These designs could assess the extent to which any observed racial discrimination might be the result of social class by crossing racial treatments with treatments related to social class, such as education or income. Others have implemented similar designs to examine racial discrimination alongside signals of sexual orientation and unemployment histories (Pedulla 2014(Pedulla , 2018b. This type of design would allow researchers to examine whether discrimination against members of a racial group varies as a function of social class or, considered differently, how much of any racial effects might be driven by different perceptions of social class. As we show in our experimental results, the strong link between perceptions of race and social class can be somewhat countered by providing independent signals of social class under specific conditions. Scholars could also gather more data about the possible causes of any discrimination in a study. Researchers might want to consider pairing their experimental interventions with surveys or interviews from a sample of their target population (Pedulla 2018a). In surveys, researchers could ask respondents open-ended questions about why they make certain choices. Respondent replies might help shed light on the mechanisms driving any discrimination that they exhibit and thus help researchers identify whether the discrimination they document might be driven by racial animus, social class bias, or something else entirely. In a similar way, researchers could also conduct interviews to document potential mechanisms (Hou, Liu, and Crabtree 2020). Moreover, the social class signal might explain variations in discrimination across contexts where social class is more or less important and signaled more or less explicitly (Gaddis et al. 2022b).
Second, other scholars might consider that names signaling both social class and race is a "bug, not a feature." In other words, names are poor proxies for the intended signal of race. In this scenario, researchers might see a need to find different names across racial/ethnic categories that signal similar levels of social class. Theoretically, researchers could then use these names to send unbundled treatments of race/ethnicity. These types of experiments could serve a dual purpose. They might help researchers identify heterogeneity in racial/ethnic discrimination at different levels of social class. They might also provide estimates of racial/ethnic discrimination using "crisp" signals of race/ethnicity. This "crispness," however, comes at the expense of the additional assumption that perceptions of race do not cause perceptions of class. Therefore, controlling for the latter does not eliminate the effect of the former.
To be clear, our recommendation in this article is not that researchers stop using names to signal race. After all, other alternatives for signaling race-such as language difference, photographs, or video-or audio-based manipulations-may be more bundled than name-based treatments. Rather, there are two broad lessons to draw, one theoretical and one practical. In practice, researchers using names to signal race (whether in audit, conjoint, or other experiments) should, as standard practice, pretest the names they are using to ensure that names signal what the researchers intend. Second, researchers should be clearer about what it is they are seeking to capture when they are studying racial discrimination. If they desire to estimate the joint effect of race and social class, they should use names that manipulate both of these things. If they desire to, instead, partial out the effect of social class from race, then they should choose names and make design choices that ensure that their experimental manipulations are not signaling both social class and race. Ultimately, these decisions require better theoretical clarity as to what estimands scholars desire to capture and more transparency in the choices researchers make in their experimental design and why these choices are made.
We do not believe that either interpretation is definitively correct. The best option likely depends on the research method and the context under examination. Researchers should chose name treatments that make sense in their particular application. We fully expect that much scholarly output will follow this article that branches in multiple directions. We think that the link between perceptions of race and social class means that using names necessarily will be messy. However, using names as a proxy for race likely captures or mimics real-world processes in a generally appropriate fashion. Researchers should take seriously that names are, on their own, bundled (multi-barreled) treatments (Converse and Presser 1986). Practically speaking, this means that researchers should interpret results with this in mind, explicitly acknowledging and understanding that in many cases, they cannot necessarily pull apart the "bundle of sticks" (Sen and Wasow 2016) that constitute race. Evidence of discrimination may capture actions based on racial bias, social class bias, or, as seems most likely to us, both.
Notes 1 Throughout this article, we use "race" as shorthand for both race and ethnicity. Social scientists often use them as synonyms, even if they treat the two concepts as substantively different (Bonilla-Silva 1997; Cornell and Hartmann 2006;Feagin and Feagin 1993;Omi and Winant 1994). For a critique of this approach, see Wimmer (2008). Although Wimmer (2008) would suggest using ethnicity instead of race to describe this group identity category, we use race in our data collection process and throughout this article.
4 We did not randomize the presentation order of the questions on different dimensions of perception. Respondents answered questions first on racial perceptions, followed by citizenship status, income, and finally, education perceptions. Our design may have influenced respondents' answers by asking about racial perceptions before asking about income and education perceptions. Although this non-randomized order is not ideal, we believe the results from Study 4 help adjudicate this issue somewhat. In Study 4, we only asked respondents about social class perceptions. The patterns of results from Studies 1 through 3 are similar to those from Study 4. Thus, we believe that the inclusion of other perception questions and the question order may not have strongly influenced the results from Studies 1 through 3. Scholars could test this more directly in future work by randomizing the order of perception questions.
6 Again, we needed to use a lower threshold because there are fewer names that are differentially used by Black or African American people compared with other races.