Lance Hannon, Robert Defina
Sociological Science, March 18, 2016
DOI 10.15195/v3.a10
Abstract
We replicate and reexamine Saperstein and Penner’s prominent 2010 study which asks whether incarceration changes the probability that an individual will be seen as black or white (regardless of the individual’s phenotype). Our reexamination shows that only a small part of their empirical analysis is suitable for addressing this question (the fixed-effects estimates), and that these results are extremely fragile. Using data from the National Longitudinal Survey of Youth, we find that being interviewed in jail/prison does not increase the survey respondent’s likelihood of being classified as black, and avoiding incarceration during the survey period does not increase a person’s chances of being seen as white. We conclude that the empirical component of Saperstein and Penner’s work needs to be reconsidered and new methods for testing their thesis should be investigated. The data are provided for other researchers to explore.
We replicate and reexamine Saperstein and Penner’s prominent 2010 study which asks whether incarceration changes the probability that an individual will be seen as black or white (regardless of the individual’s phenotype). Our reexamination shows that only a small part of their empirical analysis is suitable for addressing this question (the fixed-effects estimates), and that these results are extremely fragile. Using data from the National Longitudinal Survey of Youth, we find that being interviewed in jail/prison does not increase the survey respondent’s likelihood of being classified as black, and avoiding incarceration during the survey period does not increase a person’s chances of being seen as white. We conclude that the empirical component of Saperstein and Penner’s work needs to be reconsidered and new methods for testing their thesis should be investigated. The data are provided for other researchers to explore.
 | This work is licensed under a Creative Commons Attribution 4.0 International License. |
- Citation: Lance Hannon and Robert DeFina. 2015. “Can Incarceration Really Strip People of Racial Privilege?” Sociological Science 3: 190-201.
- Received: October 16, 2015.
- Accepted: November 28, 2015.
- Editors: Jesper Sørensen, Kim Weeden
- DOI: 10.15195/v3.a10
We are flattered by the intense interest that Hannon and DeFina (hereafter H&D) have expressed in our work, both in this Sociological Science piece and in a forthcoming comment, with another colleague, in the American Journal of Sociology (AJS). We are in the midst of responding to the AJS comment, and encourage those interested to read the exchange, which is scheduled to be published later this year. It will include a more complete response to H&D’s re-examinations of our research than we have time or space to cover here. As our AJS reply shows, status-related factors, including having been incarcerated, are significant predictors of racial categorization in models with respondent fixed effects across a variety of subpopulations.
We are also convinced that our NLSY results are not a “fragile” artifact of particular coding schemes because we find a reciprocal relationship between social status and racial categorization in other datasets, cohorts, and time periods. H&D fault us for not including models with respondent fixed effects in our analyses of arrest and racial classification using Add Health data, but do not mention that the results accounted for perceived skin color, a measure that is not available in NLSY79 and is frequently raised as a confounding factor (see Saperstein, Penner, and Kizer 2014). Nevertheless, models with respondent fixed effects confirm our finding that men who reported an arrest were significantly more likely to be subsequently classified as black than men who were never arrested. (Men who reported an arrest were also significantly less likely to be classified as Asian; results available upon request.) Saperstein and Gullickson (2013) also present fixed effects models demonstrating the association between occupational status and racial classification in historical linked census data. Finally, we recommend that H&D and other interested readers consult Freeman et al (2011) where we present our strongest causal evidence on the influence of status cues on racial categorization.
More generally, it is important to note that we have a very different conception of the process of racial categorization, and the relationship between race and privilege, than H&D. We would not be comfortable discussing “racially ambiguous” and “unambiguous” people without establishing the criteria for categorization that might justify these statements, and we maintain that future research would be better served by examining directly (rather than assuming) what information or which characteristics are influential when people make racial categorizations. Our research suggests that, in addition to using physical appearance or known ancestry, Americans also consider markers of social status when deciding who “fits” best in which racial category. However, we do not believe the potential for fluidity implied by using time-varying characteristics like social status to assign individuals to racial categories poses a challenge to the “durability” of racial privilege – quite the opposite. As we have argued elsewhere (e.g., Penner and Saperstein 2013), as long as racial fluidity is selective and consistent with current patterns of inequality, it will make racial privilege more durable in the aggregate, and reinforces the very idea of racial difference.
That said, we would be remiss if we did not acknowledge H&D on one point: case 1738 should have been labeled 1728. We regret the error and any confusion it may have caused.
Our replication study focused on three main points: (1) most of the analyses presented in Saperstein and Penner’s Social Problems paper cannot begin to address the question of incarceration-driven fluidity because they do not remove the race-related selection effect regarding who is most likely to be subjected to imprisonment, (2) the few analyses presented that are potentially suitable for the issue at hand produce results that do not stand up to even slight adjustments in methodology, and (3) the best approach to documenting the necessary correlation for supporting Saperstein and Penner’s causal claims involves comparing a respondent’s racial classifications when interviewed in prison/jail to classifications when that same respondent is interviewed elsewhere.
Saperstein and Penner’s reaction did not focus on these main points. Instead, they noted that in unpublished and forthcoming work significant effects emerge in respondent fixed-effects analyses for a variety of populations. However, this is not relevant for our critique regarding the published results in their Social Problems paper. Additionally, as we noted in our conclusion, even if one were to find a robust within-individual correlation with fixed-effects, doing so is just the first step for supporting their causal argument. A logical next step would involve testing whether the observed correlation varies in ways specifically predicted by the hypothesized causal mechanism. So, for example, one might expect that interviewers hearing directly about a respondent’s arrest history would be significantly more likely to classify a respondent a particular way than interviewers forced to infer an arrest history from subtle cues a respondent might give while privately entering data into a laptop (see our footnote 15).
Saperstein and Penner admit to one small error. They note that case 1738 is actually 1728. Unfortunately, case 1728 does not match up to their table either. Before noting that we could not find the classification pattern (in our footnote 5), we searched the data for any cases where 7 of 9 pre-incarceration classifications were white. None exist. This case was not simply mislabeled; the date of incarceration is also off by one period. Given the other abnormalities that we uncovered (see, for example, our footnote 11), we encourage Saperstein and Penner to publicly provide the data and code used to produce their tables.
In addition to our person-year dataset, we now provide a person-level file in our supplemental materials that summarizes the racial classification histories of all 620 ever-incarcerated respondents. In line with the results from our respondent fixed-effects analyses, these summary data suggest that within-individual variation in racial classification is not meaningfully related to within-individual variation in prison/jail interview context. For example, while 61 respondents saw their proportion of classifications as white decrease for the years they were interviewed in prison/jail (relative to before they were ever incarcerated), an equal number saw an increase in white classifications (of almost exactly the same magnitude).
Outside of the general importance of replication for sociology’s position as a social science, we believe our reanalysis is substantively important in that an exaggerated view of the permeability of racial boundaries in the United States may lead to an underestimation of the degree to which various populations face obstacles beyond their control as individuals. This is what drives our interest.