Testing Models of Cognition and Action Using Response Conﬂict and Multinomial Processing Tree Models

: Dual-process perspectives have made substantial contributions to our understanding of behavior, but fundamental questions about how and when deliberate and automatic cognition shape action continue to be debated. Among these are whether automatic or deliberate cognition is ultimately in control of behavior, how often each type of cognition controls behavior in practice, and how the answers to each of these questions depends on the individual in question. To answer these questions, sociologists need methodological tools that enable them to directly test competing claims. We argue that this aim will be advanced by (a) using a particular type of data known as response conﬂict data and (b) analyzing those data using multinomial processing tree models. We illustrate the utility of this approach by reanalyzing three samples of data from Miles et al. (2019) on behaviors related to politics, morality, and race.

T HE relative significance of conscious reflection and unconscious dispositions in explaining human behavior is a paramount, long-standing, and highly contested social scientific issue. Sociological theory has been split between "prereflective" (habitual, dispositional, skillful, embodied) and "reflective" (purposive, voluntaristic, calculated, rational) explanations of action for decades (Archer 2010;Bourdieu 1990;Camic 1986;Gross 2009;Lahire 2011;Parsons 1937). The current iteration of this debate has taken place in the context of dual-process models of cognition, with sociologists disagreeing over the relative significance of both automatic and deliberate processes for action (see Leschziner and Green 2013;Miles 2015;Vaisey 2009;Vila-Henninger 2021). Far from being theory for the sake of theory, these perspectives inform research on consequential behaviors, ranging from raceand gender-based discrimination to interactions with educational systems (Gaddis 2013; Melamed et al. 2019;Quillian 2006Quillian , 2008Ridgeway and Kricheli-Katz 2013), and are reflected in concrete practices in industry, government, and academia (Dobbin and Kalev 2018;Nelson and Zippel 2021).
Unfortunately, answers remain unclear, in part because competing models of cognitive processing are difficult to adjudicate using our existing methodological toolkits. Sociologists who have empirically engaged with theories of deliberate and automatic cognition have often used ad hoc measures with unclear relationships to the cognitive constructs of interest (Miles, Charron-Chénier, and Schleifer 2019;Vila-Henninger 2021). This means that cognitive theorizing usually far outstrips the evidence needed to support it (Hunzaker and Valentino 2019;Vandebroeck 2022). Consequently, many sociologists are theoretically invested in questions about cognition but are ill-equipped to answer them.
To make progress, sociologists need methodological tools that enable them to directly compare competing claims about how cognitive processes influence behavior. We argue that this aim will be advanced by (a) using a particular type of data known as response conflict data and (b) analyzing those data using multinomial processing tree models. In what follows, we first sketch out several major themes in sociological work on cognition and behavior, then discuss how the methods we have outlined can advance these debates. We illustrate the utility of our approach by reanalyzing three samples of data from Miles et al. (2019) on behaviors related to politics, morality, and race.

Divergent Perspectives on Automatic and Deliberate Cognition
The explanatory role of reflective and pre-reflective thought and action has been debated by sociologists from the earliest days of the discipline. Habits played an important role in classical sociological theory (see Camic 1986;Gross 2009;Lizardo 2021) but were excised from American sociology by the mid-twentieth century in favor of more "reflective" (purposive, voluntaristic, and rational) explanations of action (e.g., Parsons 1937;see Camic 1986). Over the past few decades, North American sociologists have increasingly turned toward theories of "practice" that emphasize the skillful, embodied, and pre-reflective nature of human conduct (Bourdieu 1977(Bourdieu , 1990Giddens 1984;Gross 2009;Schatzki, Cetina, and von Savigny 2001), but these in turn have been challenged as insufficiently attentive to the features of modern society that prompt regular internal deliberation (Archer 2007(Archer , 2010(Archer , 2012. This theoretical back-and-forth has not reached consensus, nor does it have clear avenues for resolution. The terms of this debate have changed with the introduction of dual-process models from the cognitive sciences into sociology. Dual-process models distinguish between two general types of cognitive processing: Type 1 or "automatic" processes execute autonomously, whereas Type 2 or "deliberate" processes require controlled attention (Stanovich and Toplak 2012). Automatic processes are often (but not always) fast, intuitive, and unconscious, whereas deliberate processes are often slower, analytical, and conscious (see Evans and Stanovich 2013). Over the past two decades, sociologists have engaged with dual-process models to explain and analyze how culture is acquired from environments, stored in memory, processed in thought, and used for action (Cerulo 2018;Cerulo, Leschziner, and Shepherd 2021;Leschziner 2019;Lizardo et al. 2016).
Currently, the foremost perspective in sociology is what we refer to as "automaticity dominance," which, in line with practice theories, assumes that automatic processes are far more influential than deliberate processes in shaping action (Cerulo et al. 2021). This has been most clearly articulated among cultural sociologists, but the basic premise can be found in other subfields as well, including race (Quillian 2008), gender (Ridgeway and Kricheli-Katz 2013), and education (Weininger and Lareau 2018). In an important early review article, DiMaggio (1997) argued that culture shapes thought through everyday schematic (automatic) processes but that people can (albeit rarely do) override schemas in order to think critically and reflexively. Years later, Vaisey (2009) argued that automatic cognition motivates action and is "usually in charge" but that people are capable of deliberation and justification when required by the demands of the social interaction (2009:1683-87). These approaches adopt what cognitive scientists call a "default-interventionist" model of cognition, which assumes that automatic cognition provides default responses (e.g., intuitions, heuristics) that generally guide behavior but that deliberate processes can-yet usually do not-intervene to endorse, correct, or override automatic responses (e.g., Evans and Stanovich 2013;Kahneman 2011;Kahneman and Frederick 2002). This view implies that (1) any given behavior is driven by either automatic or deliberate cognition and (2) that automatic cognition guides behavior most of the time (Evans 2007:332;Kahneman and Frederick 2002). Although not all sociological theorists adopt a strictly default-interventionist structure, many endorse the basic claim that automatic cognitive processes play the predominant role in producing action (e.g., Martin 2010;Miles 2015;Vaisey and Lizardo 2010).
A number of sociologists have challenged automaticity dominance. Although not denying the importance of automatic influences, these scholars argue that deliberate thinking is a routine occurrence that contributes to everyday actions and decisions (e.g., Leschziner and Green 2013;Moore 2017;Vila-Henninger 2021. Typically, their accounts emphasize the dynamism and interdependence of automatic and deliberate processes as they, for example, interact during moral decision-making (Luft 2020;Vila-Henninger 2015), communicate "iteratively" in meaning-making (Cerulo 2018), and combine during routine practices (Leschziner and Green 2013). These approaches are similar to "parallel-competitive" models from the cognitive sciences that assert that automatic and deliberate processes generally operate in parallel, rather than that deliberative processes activate only if automatic processes fail to give an adequate response as in default-interventionist accounts (e.g., Chaiken 1980;Epstein 1994Epstein , 2008Sloman 1996). In this view behavior can be controlled by one process or the other, but generally it is the product of some combination of both (Epstein 2008:25;Sloman 1996:6). Although sociologists generally do not theorize cognition in explicitly parallel-competitive terms, some share the basic assumption that deliberate processing is much more common and influential than automaticity-dominant accounts would suggest (Elder-Vass 2007Hitlin and Johnson 2015;McDonnell 2014;Mische 2014).
Of course, no sociologist believes cognition happens in a vacuum, and proponents of both automaticity-dominant and more "deliberation-friendly" alternatives acknowledge that the relative influence of each process depends on external factors. For example, theorists have proposed that automatic cognition predominates in stable or routine situations, whereas deliberate cognition and reflective decisionmaking is triggered in unstable or novel contexts (Boutyline and Soter 2021:741;Brekhus 2015:29-32;Cerulo et al. 2021:71;Lizardo and Strand 2010;Luft 2020;Shaw 2015Shaw , 2021. Relatedly, routine, habitual, skillful, or culturally scripted behaviors (often supported by stable and familiar situations) are largely grounded in automatic cognition (Boutyline and Soter 2021;Lizardo 2017;Wacquant 2004), whereas more explicit and symbolically mediated acts (often prompted by situational disruptions and uncertainty)-including developing new lines of action through explicit ideologies (Lizardo and Strand 2010;Swidler 1986), moral reasoning about unfamiliar situations (Luft 2020), deploying or constructing complex narratives and justifications (Boutyline and Soter 2021;Brett 2022;Vaisey 2009), or engaging in debates about possible futures (Mische 2014)-require more deliberate processing. However, recent work argues that context and behavior alone do not determine cognitive processing; rather, the use of automatic and deliberate cognition also depends on personal and dispositional factors, including thinking dispositions-stable propensities toward relying on either process-and the level of the "feeling of rightness" associated with intuitive responses, which can vary across individuals (Brett 2022;Brett and Miles 2021;Leschziner and Brett 2019;Tutić 2022;Tutić, Krumpal, and Haiser 2022). This suggests that the extent to which automatic and deliberate processes influence action can vary not just across situations and behaviors, but across people as well.
Sociological research thus offers multiple perspectives on how cognitive processes inform behavior, ranging from strong reliance on automatic processes to substantial interplay between processes that affords deliberate cognition a much greater role. Crosscutting these perspectives is the understanding that the operation of cognition depends on other factors, notably situational demands and the type of behavior in question, as well as individual-level differences in cognitive and meta-cognitive processing. Although some general principles have been fairly well established-for example, that deliberate processing is prompted by problematic or unusual situations-several key questions remain. We highlight four that are well-suited to the method we describe below.
First, how do deliberate and automatic cognition interact to produce behavior? Automaticity-dominant perspectives often imply that behavior is controlled by one or the other type of process (DiMaggio 1997;Vaisey 2009), whereas other approaches depict the two as operating sequentially, in an iterative back-and-forth, or combining in some other way (Cerulo 2018;Leschziner and Green 2013). Which is correct, and under what circumstances? Second, if deliberate and automatic cognition conflict, which guides action? We refer to this as control of behavior Payne 2009). DiMaggio (1997), for instance, argues that "people can override programmed modes of thought to think critically and reflexively" (P. 271), which implies that deliberate cognition is ultimately in control of behavior. In contrast, Vaisey (2009) claims that deliberate cognition is "no match for [automatic cognition] in a direct struggle" (P. 1683)-here, automatic processes are theorized to be in control. Third, how often is each type of cognition employed in practice? To return to the previous example, DiMaggio argues that deliberate cognition can control behavior, but notes that "[s]uch overrides are necessarily rare" (P. 271). Consequently, in most instances action is guided by automatic processes, so the total amount of influence attributable to deliberative processes is small. Other scholars disagree, arguing that deliberate influences on action are much more common, which suggests that their total, over-time influence is greater (Leschziner and Green 2013;Vila-Henninger 2015). Finally, how much variation is there from person to person? Given that people differ in how strongly they rely on automatic and deliberate cognition, it seems possible that answers to questions about how these types of cognition interact, which is in control, and how much total influence each exerts could vary from person to person as well.

Adjudicating Competing Perspectives
Sociologists have (at least) two choices about how to advance debates surrounding cognitive processes and action. These choices are not mutually exclusive, but their merits will be clearest if we treat them separately. The first is to continue to read deeply in the cognitive science literature and use that knowledge to adjudicate between different claims. Although we believe that extensive familiarity with the cognitive science literature is beneficial-and in some cases might be sufficient to resolve particular points of disagreement-we are skeptical that this approach alone will be adequate. The primary reason for our skepticism is that questions about cognition and action continue to be debated by cognitive scientists, which means there is rarely (if ever) "settled science" for sociologists to draw on. Cognitive scientists have proposed numerous dual-process models that differ in many ways including their intended explanatory scope (e.g., perception, attitudes, morality) and their structure (e.g., default-interventionist vs. parallel-competitive; Evans and Stanovich 2013;Gawronski, Sherman, and Trope 2014;Greene 2017;cf. Vila-Henninger 2021). Cognitive scientists also continue to debate concepts that are fundamental to dual-process research, such as how to define Type 1 (automatic) and Type 2 (deliberate) cognition (Stanovich and Toplak 2012) or whether it is even sensible to talk about there being two types of cognition at all (Amodio 2014;Melnikoff and Bargh 2018). Other scholars argue for models that emphasize different numbers of processes, such as the single-process unimodel or models that include additional forms of processing (Dijksterhuis et al. 2014;Kruglanski et al. 2014). Given this variety, it is no wonder that sociologists have proposed and defended so many perspectives on cognition and behavior-our disagreement reflects the lack of consensus that exists in the cognitive sciences.
The second approach sociologists can take is to empirically test questions about cognition and action themselves. This strikes us as advantageous for three reasons. First, sociologists (sometimes) ask different questions and study different outcomes than other social scientists, which means we need to be able to test whether our cognitive models apply in the cases we care about, and we need the capacity to modify those models when necessary. Neither of these tasks can be done well if we leave the empirical work to cognitive scientists. Second, developing the capacity to rigorously test cognitive theories will help sociologists contribute to interdisciplinary conversations, something that arguably we have not always done well but that has great potential to advance research on culture, cognition, and action (Lizardo 2014;Vaisey 2021;Vaisey and Valentino 2018; but see Lamont et al. 2017). Finally, it is worth reminding ourselves that although sociological work on cognition and action currently draws heavily on work from the cognitive sciences, questions about automatic and deliberate cognition have been a part of sociological thought from the foundations of our discipline (Dewey 1922;Weber 1920), and our commitment to answering them is reaffirmed every time we invoke concepts like habit, agency, schemas, habitus, embodiment, reflection, imagination, reflexivity, or implicit bias (Archer 2010;Bourdieu 1990;Giddens 1984;Gross 2009;Hitlin and Johnson 2015;Hunzaker and Valentino 2019;Lizardo and Strand 2010;Quillian 2008). In short, these are our questions too, and we should be willing to take ownership for answering them.
Reliance on cognitive science and testing theories "in house" are of course not mutually exclusive options, and some of the best work in cognitive sociology takes both seriously. For example, Vila-Henninger (2021) developed a dual-process model of moral outcomes that is deeply rooted in the cognitive science of memory and tested it using a technique called Rapid Serial Visual Presentation that allowed him to estimate the effects of both deliberate and automatic cognition. He found, unexpectedly, that moral decision-making relied on deliberate but not automatic cognition. Other notable examples include Srivastava and Banaji's (2011) use of implicit and explicit measures to investigate collaboration in organizations and Miles' (2015) work on values that employed cognitive loading to isolate automatic processing. More often, however, sociological work on cognition and action is strong on theory but uses methods whose relationships to deliberate and automatic cognition are unclear, such as observational, interview, or forced-choice survey methods Mohr et al. 2020:27;Vila-Henninger 2021). This ambiguity makes these methods generally ill-suited to answering questions about how cognitive processes produce judgments and behavior. Sociologists have responded by promoting a number of alternatives, such as implicit measures of attitudes, identities, habits, and related constructs, as well as various measures for capturing cognitive schemas (Boutyline 2017;Miles et al. 2019; Shepherd 2019; Srivastava and Banaji 2011; see Miles 2019 for a review). We believe these methods hold great promise for advancing disciplinary work on cognition and will also make it easier to contribute to wider interdisciplinary conversations.
Of course, no method is the right tool for every job. Most methods advocated by sociologists are measures of constructs, like attitudes or schemas. These are useful for answering questions about those particular constructs (e.g., can automatically activated racial associations lead to discriminatory behavior?) and perhaps can be used to compare the effects of automatic and deliberate cognition as they pertain to those constructs (e.g., do implicit racial associations or consciously held racial attitudes matter more for discriminatory behavior?). However, measures of individual constructs are not well-suited for evaluating many of the general claims and models about cognition that are circulating in the sociological literature (e.g., Vaisey 2009 or Vila-Henninger 2021). These general claims often focus on types of processing-for example, automatic or deliberate-rather than the particular constructs that are processed-for example, attitudes or schemas. They specify when particular types of processing are likely to execute and how they interact (or fail to interact) to produce behavior. Testing these models therefore requires methods that capture the total effects of automatic and deliberate cognition, rather than the effects of specific constructs that are automatically or deliberately processed. Furthermore, testing general cognitive models requires methods that can evaluate the dependencies among processes-for example, what happens when automatic and deliberate cognition are in conflict, if (and how) cognitive processes interact to jointly produce behavior, and so forth. See Appendix A in the online supplement for a further discussion of the limitations of standard sociological methods.

Response Conflict and Multinomial Processing Tree Models
A powerful approach to testing claims about cognitive processes is to analyze response conflict data using multinomial processing tree (MPT) models. The fundamental assumption underlying response conflict data is that any given behavior can be produced by multiple processes, such as automatic or deliberate cognitive processes. Response conflict data record individuals' behavior under conditions in which the processes are expected to influence that behavior in either similar or dissimilar ways. Presumably, respondents are more likely to perform a behavior when all processes favor doing so and will be less likely to perform the same behavior when processes give conflicting messages. This creates variation in how often respondents perform the behavior that can be used to understand the underlying processes. Response conflict data can occur naturally but are often produced using tasks specifically designed to manipulate the processes under study. These response conflict tasks (RCTs) can be designed to produce data at either the aggregate or individual levels, which means RCT data can be used to empirically test whether cognitive processing is universal or if it varies across individuals or groups. Examples of RCTs include the hiring and voting tasks in Miles et al. (2019) and well-established tasks used in psychology and neuroscience including the Stroop task, the process dissociation procedure, and the Eriksen flanker task (see Kleinsorge 2021; Littman, Keha, and Kalanthroff 2019; Payne and Bishara 2009 for reviews).
MPT models extract process information from response conflict data (Calanchini et al. 2018;Erdfelder et al. 2009). MPT models are formal mathematical models that require researchers to specify how processes interact with each other to produce an outcome. Processes are arranged in a tree-like structure, and a given behavior might be produced by traveling down the one or more branches of the tree. Processes lower in the tree only control behavior if processes higher in the tree fail to produce a response. Once the model equations are specified, they can be solved to produce estimates of the probability of relying on each process. MPT models can be fit to both the aggregated and individual data from RCTs, making it possible to examine variation in cognitive processing at both the individual and group levels.
The logic of MPT models and their utility for understanding cognitive processes is best illustrated with an example. Consider the weapon identification task introduced by Payne (2001), which is an RCT. In this task, respondents were shown pictures of weapons and tools and asked to classify them quickly but correctly. Prior to each picture, an image of a Black or White face was briefly shown. Payne assumed that the race of the faces shown would automatically prime associations of safety for Whites and danger for Blacks. Thus, seeing a Black face would make it easier to subsequently identify a weapon but harder to identify a tool. Following a White face, the opposite would be true: tools should be easier to identify but weapons harder. Across multiple trials, respondents were presented with four combinations of faces and objects: Black-weapon, Black-tool, White-weapon, and White-tool. These combinations varied in whether the prime facilitated or impeded correct identification of the object in question. Bishara and Payne (2009) tested several MPT models to determine which best explained responses during the weapon identification task. We consider two here with strong theoretical affinities to the deliberation-friendly and automaticity-dominant perspectives we discussed earlier. The first model is what Bishara and Payne call the process dissociation model. This model is shown in Figure 1. 1 The model assumes that responses are determined first by a deliberate process-that is, respondents recognize weapons and tools and can carry through on their intention to identify them correctly. The probability that they are successful in doing this is given by the parameter C (C for "controlled processing"). If that fails (with probability 1 − C), then with probability A their response will be determined by the prime, which is assumed to run through automatic cognitive processes. Because deliberate processes have initial control over response behavior, this model is a deliberation-dominant model (cf. Payne and Bishara 2009).
Two points should be emphasized about the structure of the process dissociation model shown in Figure 1. First, during each trial the response is controlled by either deliberate or automatic cognition, but not both-that is, there is no path in which A follows C rather than 1 − C. This is not a limitation of MPT models generally but rather a feature of this specific model. Second, the structure of the model refers to control of behavior and not necessarily to the temporal order in which processes occur. In fact, scholars generally agree that automatic processing is faster than deliberate processing and hence activates first. However, people might deliberately ignore, selectively attend to, or override these processes before acting. In such cases, deliberate processing is in control of behavior even though it occurs later than automatic processing. The process dissociation model makes predictions about how respondents are expected to answer, given any combination of prime and object. These are shown in columns along the right side of Figure 1. A plus sign indicates that the model predicts a correct response, and a minus sign that the model predicts an incorrect response. For example, the first column in Figure 1 indicates that if a White face prime is followed by a tool, respondents could provide a correct response if (1) deliberate control succeeds and they can intentionally identify the tool as a tool or (2) deliberation fails but they are successfully influenced by the prime, which, being a White face, should lead them to expect a tool. MPT models assume that these processes are independent, so the probability of each possibility can be calculated by multiplying the probabilities along the pathway leading to it, and the total probability can be calculated by summing the probabilities for each possibility. Thus, the probability of giving a correct response in the White-tool condition is P(correct | White, tool) = C + (1 − C) × A. Similar equations can be constructed for each of the other three conditions and are shown along the bottom of each column. Collectively, these equations constitute the process dissociation MPT model.
An alternative to the process dissociation model is what Bishara and Payne referred to as the Stroop model. As evident from Figure 2, it includes the same parameters as the process dissociation model (C and A) but reverses the order of priority for deliberate and automatic processes. That is, the Stroop model assumes that automatic cognition has control of behavior, and deliberate cognition only guides behavior if automatic processes fail to produce a response. Thus, it is an automaticity-dominant model.
The process dissociation and Stroop models have different structures and therefore imply different sets of equations. These equations make different predictions about how often respondents give correct responses in each condition. The fit of each model can be assessed by seeing how closely these predictions correspond to the actual patterns in the data. Consequently, comparing the fit of the process dissociation and Stroop models can yield insight into whether automatic or deliberate cognitive processes are in control of behavior. Bishara and Payne (2009) fit both models to data taken from four different studies. They found that the process dissociation model generally provided a better fit to the data than the Stroop model, both when data were aggregated across the entire sample and when models were fit separately to data from each individual. Thus, their analyses suggested that deliberate processes were in control of behavior, although they emphasize that their work only applies to the weapon identification task and that other types of behaviors could return different results.
Additional information can be obtained by examining the parameter estimates from MPT models. To illustrate, consider Figure 3, which shows average estimates of C and A from models fit to aggregate data that are presented in an appendix in Bishara and Payne (2009). 2 The probability of deliberate cognition guiding responses during the weapon identification task is 0.61. If that fails (with probability of 0.39), then the probability of relying on the automatic influence of the prime is  0.52. Hence, the overall probability of relying on prime-related automatic influence is 0.39 × 0.52 = 0.20. This suggests that respondents were much more likely to rely on deliberate cognition than automatic cognition when responding during the weapon identification task, at least on average. 3 No estimates from models fit to individual-level data were reported, making it impossible to determine if this conclusion holds universally or only for certain respondents.
It is worth highlighting a few points about these analyses. First, the deliberate and automatic influence captured by C and A are not construct-specific. A, for instance, represents the influence of all automatically processed constructs that are activated by the primes-these could be race-related attitudes, identities, stereotypes, and so on. The parameters from these MPT models therefore provide the level of generality needed to test the general claims about cognitive processes that are often made by sociologists.
Second, these MPT models provide information relevant to answering three of the four questions we highlighted above. MPT models operationalize claims about control of behavior through the placement of processes in their tree-like structure, with processes lower in the tree being dependent on processes higher in the tree. Models that arrange the same processes in different ways thus make competing claims about control of behavior, and those claims can be adjudicated by determining which model structure provides a better fit to the data. The estimated parameters from MPT models capture how often behavior is produced by following each path in the tree. Parameter estimates thus provide insight into the total influence of each process across repeated opportunities to perform the behavior. Bishara and Payne (2009) also fit MPT models to data from each person. For clarity in presenting the method we did not emphasize these results, but in theory these estimates could be used to examine individual variation in control and/or total influence. Bishara and Payne's analyses did not address whether deliberate and automatic cognition interact to jointly produce behavior. Doing so is possible with MPT models but would require specifying a model in which both types of cognition lie along a single path-for example, deliberate cognition activating automatic processes that then guide response behavior.
Third, there is nothing magical about MPT models that allow them to generalize beyond the data they are fit to. That means that Bishara and Payne's (2009) results only apply to individuals like those in their samples and, perhaps more importantly, only to behaviors similar to the weapon identification task. Other behaviors-say, habitual acts-might be better explained by an automaticity-dominant rather than a deliberation-dominant model and could rely much more heavily in practice on automatic cognition. This suggests the important point that validating general models of cognition requires testing them using many types of people and many types of behaviors.
In the remainder of this article, we illustrate the utility of using response conflict data and MPT models for adjudicating between different cognitive models. To do this, we analyze response conflict data from three samples related to politics, morality, and race that were originally presented by Miles et al. (2019). These data are not perfect-a point we elaborate on below-but they do allow us to demonstrate how MPT models can be used to address important questions about cognition and action. As a secondary benefit, our analyses will contribute several data points toward current theoretical debates around models of cognition and action. We recognize that studying only three behaviors and other data limitations restricts our ability to offer strong support to any of these theoretical perspectives or to generalize our findings. Nevertheless, we hope that our work will be the first step in a larger accumulation of evidence from which generalizations can eventually be made.

Data and Methods
Data come from the three samples used in Miles et al. (2019) and are available at https://osf.io/jexs9/. We describe only the relevant features of the data for our analyses below and refer readers to the original article for additional information.
All three samples used RCTs with the same basic format. In each sample, the RCT had multiple trials and required respondents to make decisions as quickly as they could. On each trial, a prime was shown for 100 ms, followed by a blank screen (for 100 ms), then a target about which study participants had to make a decision (for 100 ms), and finally an image of static that remained on the screen until a response was entered.
Because the format of the RCT is the same across samples, we fit the same MPT models in each. We adapted these from the models used by Bishara and Payne (2009). 4 The two models are shown in Figure 4. Panel A shows a deliberationdominant model derived from the process dissociation model, and panel B shows an automaticity-dominant model derived from the Stroop model. The primes and targets used in each sample are shown in rows along the top.
The C parameter is meant to capture deliberate processing related to the task (i.e., how a person intended to behave), and the A parameter is meant to capture taskrelevant automatic cognition, which we assume is generated by the primes. The E parameter is a catch-all parameter that captures other influences and is included to make our assumptions about what C and A represent more plausible. 5 We discuss these assumptions in greater depth momentarily, along with their implications for how estimates are interpreted.
It is worth emphasizing that these MPT models are not perfect representations of any of the dual-process models advanced by sociologists (e.g., Vaisey 2009; Vila-Henninger 2021). In part this is practical: the data created by Miles and colleagues (2019) are not rich enough to support much complexity. However, it also reflects our desire to speak to the larger questions we presented above, which crosscut models. The models shown in Figure 4 allow us to test questions about both cognitive control of behavior and the total influence different processes exert and to illustrate how scholars can test for variation across individuals. We address the question of fitting more complex MPT models that more faithfully capture the contours of particulars theories-including models that allow deliberate and automatic cognition to jointly produce behavior-in the discussion.
Sample 1 focused on politics. Respondents were shown pictures of potential Democratic and Republican candidates for the 2016 presidential election and asked to vote for or against each one. Primes were words related to liberal or conservative political ideology. This design created four unique prime-target combinations: liberal-Democrat, liberal-Republican, conservative-Democrat, and conservative-Republican. Following Miles et al. (2019), we assume that self-identified conservatives intended to vote for Republicans and self-identified liberals intended to vote for Democrats, and that all respondents would follow through on their inten- tions if deliberate processing prevailed. We also assume that a conservative prime word would encourage voting for a candidate if a respondent was conservative but discourage voting if a respondent was liberal. Similarly, liberal prime words would encourage voting among liberal respondents but have the opposite effect among conservatives. Because liberals and conservatives are assumed to respond differently to the same primes, we followed Miles et al. (2019) and modeled them separately. The models shown in Figure 4 are for liberals and show when controlled and automatic processes are expected to produce a "vote" response (indicated with a plus sign). The models for conservatives are shown in Appendix B of the online supplement.
Sample 2 examined morality. Rather than voting, respondents were asked to "hire" fictional applicants based on whether they possessed an appropriate degree. Respondents learned which degrees were "hirable" prior to the task. Primes consisted of images of caring or harmful behaviors. This design creates four unique prime-target combinations: caring-hirable, caring-not hirable, harmful-hirable, and harmful-not hirable. We assume that respondents intended to hire only those applicants with the approved degrees and that they would follow through on that intention if fully in control of their actions. We also assume that caring prime images would generate positive affect and predispose respondents to hire whichever applicants followed, whereas prime images of harm would have the opposite effect. The pathways through which deliberate and automatic processes are expected to produce a "hire" response are shown as plus signs in Figure 4. Sample 3 focused on race. The RCT was identical to the RCT used in sample 2, except that the caring-harmful prime images were replaced with pictures of Black or White faces. Consistent with past work on race and hiring (Bertrand and Mullainathan 2004;Quillian et al. 2017), we expected White faces to predispose respondents to hire whichever applicant followed, whereas Black faces would do the opposite. The plus signs in Figure 4 indicate when deliberate and automatic processes are expected to produce a "hire" response.

Analyses
Our analyses proceed in three phases, which illustrate how MPT models can be used to answer the types of questions about cognition and behavior we outlined above. In phase 1, we test for the overall control of cognitive processes on behavior by comparing the fit of the deliberation-dominant and automaticity-dominant models to the data in each sample. The data for MPT models consist of counts of the decisions respondents made in each condition, aggregated across respondents. For example, in the morality sample there are four combinations of primes and targets (caring-hirable, caring-not hirable, harmful-hirable, harmful-not hirable), so the data offer four data points for analysis.
The level of correspondence between a model and the data can be captured with the G 2 statistic, which compares how closely the counts predicted by the model match the actual counts found in the data. Smaller values indicate better fit. G 2 statistics can be compared to a χ 2 distribution, with a non-significant result indicating adequate fit to the data. 6 The degrees of freedom for this test are equal to the total number of unique prime-target conditions minus the number of parameters in the model, and at least one degree of freedom (d.f.) is needed to perform the test (Riefer and Batchelder 1988). In our case, each sample contains four conditions, and our models include three parameters (C, A, and E), so d.f. = 1.
Phase 2 examines how often each cognitive processes guides behavior across the multiple trials in the study, which we refer to below simply as total influence for brevity. We denote total probabilities of relying on deliberate cognition as C total and on prime-related automatic cognition as A total . The total probabilities of deciding based on other factors are given by E total and (1 − E) total , which represent, respectively, the total probability of voting/hiring for reasons other than those captured by C and A and the total probability of not voting/hiring for reasons other than those captured by C and A.
Phase 3 demonstrates how MPT models can address issues of individual difference by fitting separate MPT models to the RCT data for each respondent. This produces individual parameter estimates for each respondent that can be used to empirically examine how control and the total influence attributable to deliberate and automatic processes vary across individuals.

A Note on Interpreting Model Parameters
A key consideration for any statistical model is understanding what the estimated parameters represent. We assume that C captures deliberate cognition and that the A parameter measures automatic cognition induced by the primes. These expectations mirror assumptions made in related work that uses MPTs to study cognition (e.g., Bishara and Payne 2009) and have reasonable face validity-after all, researchers routinely assume that people intend to follow task instructions and carry out those intentions deliberately and that primes influence automatic cognition (Bargh 2006;Wegner 1994). However, a little thought shows that other processes might also be at play. For instance, respondents might intentionally choose to enter random responses or might misremember the correct answers and intentionally make incorrect selections. Responses might also reflect automatic processes unrelated to the prime, such as spillover effects from a pre-existing mood state, associations evoked by the target rather than the prime, or involuntary reactions (e.g., twitches) produced by the rapid nature of the task. These extraneous processes, if present, might bleed over into model estimates, much like omitted variables can influence coefficients in a regression model. Thus, it is problematic to assume that C and A parameters reflect only accuracy-related intentions and the influence of the primes without further validation.
The typical way to validate MPT model parameters is to administer an RCT under conditions that should selectively affect parameters and see if those parameters change as expected. For example, do estimates change if respondents complete the RCT while under cognitive load? If the C parameter really captures deliberate cognition, it should diminish when cognitive resources are depleted, but the A parameters-which rely on resource-light automatic processing-should remain unchanged (e.g., Cameron et al. 2017). Unfortunately, Miles et al. (2019) did not include a cognitive load condition in their studies, so we cannot test our interpreta-tions of C and A using this approach. 7 A second approach-and the one we use-is to add parameters to the model to represent the additional processes that we think might be operating. With only four degrees of freedom for analysis we cannot add many processes, so we opted to add a single "catch-all" parameter E, which represents the probability of voting for a candidate or hiring an applicant for any reason other than those captured by C and A. Although adding E does not provide a direct test of what C and A represent, it does make it more likely that C and A represent what we assume them to.
Practically, this means that although our interpretations of C and A are plausible, we cannot be certain that they are correct. For expository purposes we will continue to assume that C captures intention-related deliberate processes and that the A parameter captures automatic processes generated by the primes, even if we suspect that they do not do so as cleanly as we would prefer. This will make it possible to pursue our primary aim of illustrating the utility of response conflict data and MPT models as clearly as possible. We return to the issue of parameter validation in the discussion, with a particular eye toward evaluating how much our analyses contribute to the empirical record.

Phase 1: Assessing Control
Our first question is whether deliberate or automatic processes are ultimately in control of behavior, which we can assess by determining whether our deliberationdominant or automaticity-dominant model better fits the data. Table 1 shows fit information for these two models in each sample. The fit of both the deliberationdominant and automaticity-dominant models is adequate in all samples, as indicated by the non-significant G 2 statistics. Thus, we have little reason to prefer either model, and consequently we cannot resolve the issue of control for the behaviors under consideration.
A closer look at Table 1 reveals that in every sample the fit of the deliberationdominant and automaticity-dominant models is not just adequate, it is identical. Describing the reasons for the identical fit here would take us too far afield, so we instead discuss them in Appendix C of the online supplement. The upshot of that discussion is that the identical fit is likely due to features of the data combined with the simplicity of our models. Fortunately, this means that identical fit is not a problem with MPT models generally (see, e.g., Payne and Bishara 2009;Sherman et al. 2008) but rather a feature of our samples and analysis. Regardless of the reason, the practical implication for the present analyses is easy to grasp: our data and models do not allow us to determine whether deliberate or automatic processes have control of behavior in cases where the two types of processes conflict. Table 2 presents parameter estimates from the deliberation and automaticitydominant models fit to data in each sample. Both types of models display a similar pattern of results. Across models, C estimates are invariably larger than estimates of A and roughly equivalent in size-or slightly smaller-than estimates of E. Estimates for A are largest in the two politics samples, smaller in the morality sample, and 0 in the race sample. Although these estimates hint that individuals rely more on deliberate than automatic cognition, we must remember that individual parameter estimates do not account for the tree-like structure of the model. We address the question of total influence in phase 2 of our analyses.

Phase 2: Assessing Total Influence
The parameter estimates in Table 2 do not account for the nested nature of the processes in each model's processing tree. To gain a fuller picture of the total influence of automatic and deliberate cognition, we must examine the total probabilities of relying on each process, given the structure of the processing tree. Because the deliberation-dominant and automaticity-dominant models have equivalent fit to the data, they also return identical total influence estimates. These estimates are shown in Table 3.
What is immediately evident is that, regardless of the task, individuals relied on deliberate processes more often than they did automatic processes induced by the primes. In the two politics samples, prime-related automatic processes guided behavior 12 to 13 percent of the time, whereas in the morality and race samples primes had essentially no influence on respondent decisions (A total = 0.03 and 0.00, respectively). Deliberate processes, in contrast, guided behavior anywhere from 29 to 50 percent of the time, depending on the sample. Table 3 also displays how frequently respondent behavior was guided by processes other than those captured by C and A. E total captures the frequency with which these unspecified processes led respondents to vote for or hire a candidate, whereas (1 − E) total reflects the frequency with which they led respondents not to vote for or hire a candidate. Thus, the total probability of relying on these unspecified processes to make either type of decision is given by E total + (1 − E) total . Values of E total + (1 − E) total are shown along the bottom of Table 3. In the politics samples, the frequency with which respondents relied on unspecified processes is anywhere from 66 percent larger to twice as large as the frequency with which they relied on their deliberate intentions, making these processes the most common determinant of behavior. In the morality and race samples, individuals relied on deliberate cognition and unspecified processes about equally often-50 percent versus 47 percent of the time in the morality sample, and 47 percent versus 53 percent of the time in the race sample. To our mind, these results suggest two key points. First, deliberate cognition was much more influential than prime-related automatic cognition, although the use of both types of cognition varied depending on the behavior in question, as previous research suggests. Second, there was much more than just intentions and prime-related automatic cognition shaping decisions during the voting and hiring tasks, as evidenced by the large total probabilities associated with E total + (1 − E) total . We consider the implications of these other, unspecified processes further in the discussion.

Phase 3: Assessing Cognitive Models at the Individual Level
The analyses to this point have demonstrated the utility of response conflict data and MPT models for understanding the cognitive processes underlying three situated behaviors. However, we are also interested in knowing whether the patterns we have uncovered are universal or whether individuals vary in the extent to which they rely on deliberate and automatic processing.
To address this question, we can fit MPT models to the data for every individual separately. In this way, we obtain a unique model for each individual along with an individual set of parameter estimates. However, obtaining reliable and efficient estimates requires having enough data points. Data for an MPT model consists of the number of times a person performs an act under each unique set of conditions. The data from the politics, morality, and race samples had between 48 to 64 data points per person, divided over four unique prime-target conditions, for a total maximum count of 12 to 16 per condition. In some cases, individuals had one or more 0 counts-that is, no data for a given prime-target combination. Consequently, individual MPT models fit to these data sometimes did not converge or had extreme estimates.
Our view is that individual-level estimates from the politics, morality, and race samples are unreliable and should be viewed with skepticism. However, because our primary goal is to demonstrate how MPT models can be used to study cognition, we proceed with an analysis of individual-level effects to show the possibilities that this sort of approach offers. In each sample, we present results only from the individual-level models that converged.
As in the aggregated data, the deliberation-dominant and automaticity-dominant models always had equivalent fit to individual-level data. 8 Thus, we cannot determine if individuals differ in whether deliberate or automatic processes take precedence in guiding decisions when the two conflict. We note again, however, that equivalent fit is not a feature of MPT models generally, so studies of other behaviors using richer data might well find variation in model fit across individuals.
What about total influence? Table 4 shows the proportion of respondents for whom C, A, or E + (1 − E) had the largest total influence on their behavior. 9 The crucial point is that variation exists across all samples. Among liberals in the politics sample, for instance, the unspecified processes captured by E + (1 − E) guided behavior more often than either deliberate or automatic cognition for 81 percent of respondents, whereas deliberate intentions and prime-related automatic Notes: Estimates of total influence are calculated from MPT models fit to data for each individual. Estimates from individual models that did not converge were excluded. Remaining sample sizes are N liberal = 115, N conservative = 50, N morality = 190, and N race = 186. A is assumed to capture prime-related automatic cognition, and C to capture task-relevant deliberate cognition. Rows may not add to 1 due to rounding or if some estimates were of equal size. Notes: Estimates of total influence are calculated from MPT models fit to data for each individual. Estimates from individual models that did not converge were excluded. Remaining sample sizes are N liberal = 115, N conservative = 50, N morality = 190, and N race = 186. A is assumed to capture prime-related automatic cognition, and C to capture task-relevant deliberate cognition. Rows may not add to 1 due to rounding.
cognition was most influential for only 10 and 8 percent of respondents, respectively. This distribution shifts across samples, although in all cases respondent behavior is guided most often by either deliberate or other, unspecified processes. Table 5 restricts focus to the comparison between deliberate and prime-related automatic cognition, which are of more direct interest than the processes captured by E total + (1 − E total ). In all samples, most respondents relied on deliberate cognition more often than automatic cognition. However, the relative influence of deliberate and automatic cognition shifts across samples. Deliberate processes were predominant for roughly 50 percent of respondents in the liberal and conservative samples, whereas prime-related automatic processes were more influential than deliberate processes for about one-third of respondents. In the morality and race samples, the influence of deliberate cognition was much higher compared with automatic cognition. Deliberate processes exceeded the influence of prime-related automatic cognition for 91 percent of respondents, and automatic processes guided behavior most often for only four to five percent of respondents. Again, given our concerns about the sparseness of the individual-level data we do not place great emphasis on the exact estimates shown in Tables 4 and 5 but instead highlight what to us is the key point: that individuals varied in how often they relied on deliberate and automatic processes.

Discussion
Sociological dual-process models tend to fall into one of two camps. There are automaticity-dominant models that emphasize the power of automatic processes in influencing behavior, and there are deliberation-friendly models that place greater emphasis on the role of conscious, deliberate thought in guiding action. Although these perspectives diverge on the relative weight attributed to automatic and deliberate processes in explaining how people think and act, there is broad recognition these processes vary based on other factors, such as the features of the context, behavior, and the individual.
We proposed that sociologists use response conflict data and MPT models to disentangle cognitive processes in a way that addresses several key questions that are unresolved in the literature. These include (1) whether (and how) deliberate and automatic processes interact to jointly produce behavior, (2) which type of cognition is in control, (3) which most often shapes behavior across repeated opportunities to perform an act (total influence), and (4) whether (and how much) the answers to these questions vary across individuals. We illustrated this approach with an extended example from the psychology literature and through a reanalysis of the data used in Miles et al. (2019).
Data from Miles et al. (2019) involved simulating voting and hiring behavior under rapid-response conditions. Primes intended to evoke automatic cognition included political, moral, and race-related content, and we analyzed these data at both the aggregate and individual levels. Turning first to models fit to the aggregate data, we found that deliberation-dominant models and automaticity-dominant models fit the data equally well, suggesting that our analyses cannot determine which type of process controls behavior in cases where the two conflict. Control is not equivalent to total influence, however. We subsequently found that both deliberate and prime-related automatic cognition could guide responses in three out of four samples, the exception being decisions in the race sample that were never guided by automatic processes. However, in all samples behavior was most often guided by either deliberate cognition or else the unspecified processes captured by the E and 1 − E paths in our model (see Figure 4). Individual-level analyses were also uninformative about questions of control but yielded a range of values for total influence across individuals in every sample, suggesting that individuals differ in how often they rely on deliberate and automatic processes.
As noted above, our primary aim in presenting these analyses was to introduce sociologists interested in cognition to the benefits of response conflict data and MPT models by showing how these methods can address key questions in the literature. Our analyses demonstrated how researchers can assess control, total influence over time, and variation across individuals, but given data limitations we were unable to present models that show how deliberate and automatic cognition might interact to jointly produce behavior. We therefore briefly describe how MPT models could test interactive hypotheses now. In an MPT model, joint influence on a behavior means that both types of processes must be active along the same path in the tree-like structure of the model. For example, a model might specify that deliberate processes (C) call up task-relevant information that in turn prompts either a positive (A P ) or negative (A N ) automatic evaluation of that information and produces a behavioral response. In either case, both types of cognition are implicated in producing the outcome, which would manifest in the model equations as a multiplicative effect: C × A P or C × A N . Constructing MPT models with joint effects will be particularly important moving forward for testing the interactive/iterative processes that have been featured in "deliberation-friendly" accounts (Cerulo 2018;Leschziner and Green 2013;Vila-Henninger 2021). 10 A secondary aim of this article was to provide evidence that could speak to the various questions circulating in the literature about cognition and action-that is, not just to show how to answer these questions, but to actually start answering them. This was complicated by the sparseness of the data. MPT data require observing many behavioral decisions across a range of conditions that variously facilitate or impede the processes under study. The data from Miles et al. (2019) contain just four unique conditions and hence four degrees of freedom for analyses. Practically, this limited us to models containing three or fewer parameters to ensure that we could estimate stable models with accompanying fit statistics. 11 Simple models risk omitting relevant processes, and as with any statistical model omitted variables can bias estimates. Fortunately, we were interested in just two types of processes-deliberate intentions and prime-related automatic cognition-so our models were able to accommodate a "catch-all" E parameter that allowed the influence of unspecified processes to be included. Although we cannot be certain that the E parameter fully captured all extraneous processes, it does make it more likely that our estimates are close to their true values. 12 Another challenge is that we were unable to directly verify that our C and A parameters capture deliberate intentions and prime-related automatic cognition, although as we noted previously our study design and the use of an E parameter make these assumptions plausible. This plausibility is enhanced when we consider that tasks and stimuli were new to respondents, which suggests that respondents could not rely on learned habits but instead had to exert deliberate control to recall and apply decision rules-this makes it more likely that C captures deliberate intentions. Furthermore, subjects were instructed to respond rapidly, which should have placed additional demands on deliberate cognition and accordingly left fewer cognitive resources to execute other processes. Because resource-light operation is typically associated with automatic processing, this implies that the A parameter captures automatic cognition. Of course, plausibility is not the same as direct evidence, and the evidentiary value of our findings will remain largely hypothetical until we can compare them with the results of future studies with more informative data and direct tests of parameter interpretations.
Individual-level analyses faced the additional challenge of having few data points per condition. This led to issues with model non-convergence and possibly unreliable and extreme estimates. Although we excluded models that did not converge from our analyses, we still have no way to tell whether the remaining range of estimates represents true diversity in individual-level cognition or if it is an artifact of estimation issues. However, the pattern of results we observed is consistent with prior work that suggests that individuals vary in how they make use of their cognitive capacities (e.g., Brett 2022; Leschziner and Brett 2019;Tutić et al. 2022). Given this, we suspect that the main empirical pattern of variation is accurate, even if the specifics about how much variation exists and where that variation occurs is not. Obtaining more accurate estimates will require additional research using richer individual-level data.
Another important task for future research will be determining which processes are captured by the E and 1 − E paths in the models. As noted previously, the novelty and rapid-response format of the decision tasks makes it more likely that all processes other than respondents' deliberate intentions to answer accurately relied on automatic cognition. This suggests that whatever E and 1 − E represent, they are likely to be automatically processed. If so, the total contributions of automatic cognition to behavior might be much higher that our analyses suggest. The crucial question is whether these additional processes are meaningfully related to behavior-such as thoughts or emotions generated by the target-or not-such as involuntary reactions. That is, are these unspecified process part of the signal or are they just noise? Again, the key to untangling E and 1 − E is collecting better data-in particular, data with enough unique conditions so that analysts can explicitly add and test additional processes in their models.
Let us assume for a moment that future research substantiates our results. What are the implications? First, our results agree with prior work that argues that deliberate cognition is more influential in novel situations (Lizardo and Strand 2010;Luft 2020;Shaw 2015Shaw , 2021. As noted above, both the voting task and hiring task were new to respondents, which meant they were unlikely to have task-specific automatic processes that they could rely on. Second, the differences in the level of automatic influence between the politics sample and the morality and race samples could indicate that automatic influence is greater when the behavior in question relies more on personal preference rather than objectively verifiable decision criteria. In contrast to the hiring task, where correct answers were unambiguous, the voting task asked respondents to vote for the candidates they preferred. We assumed that they would want to vote for candidates from the political parties that matched their political orientations. However, in the absence of objective decision-making criteria, respondents might have been more susceptible to using other information in making their choices, such as feelings or other associations generated by the primes, or information derived from the pictures of the candidates, such as attractiveness. Although the voting and hiring tasks were artificial, our results agree with work in the empirical literature on hiring that demonstrates that systematizing the criteria used to evaluate candidates reduces bias by limiting the role of gut feelings and stereotypes in evaluations (see Stephens, Rivera, and Townsend 2020). We note, however, that even during the voting task respondents relied on prime-related automatic processes relatively infrequently, which could suggest that automatic processes (e.g., implicit biases) are less influential than has sometimes been asserted (cf. Forscher et al. 2019).
Perhaps the most important implication of our analyses is that researchers should assume less and measure more. Often, sociologists assume automaticity dominance and use survey, experimental, and network data to measure the effects of automatic processing on behavior (Leschziner 2019). Given both the ongoing debates in the literature and our finding that deliberate cognition guides behavior more often than automatic cognition, we suggest that taking automaticity dominance as a premise is unjustifiable. By the same token, the limitations in our data, ambiguity around what other processes were left unspecified, and the fact that we only studied three behaviors performed under artificial conditions also caution against taking deliberation-friendly models as the default. Furthermore, our results indicate that cognitive processing varies considerably across people, even when those people are performing the same behaviors. We therefore suggest that taking any one cognitive model as a theoretical given is a mistake and urge scholars to directly study rather than assume the cognitive processes involved in the behaviors they are interested in. This will allow scholars to speak with confidence about the processes at play in their particular cases and will contribute to a body of evidence that crosscuts different types of behaviors, contexts, and individuals. These data will be invaluable in our efforts to adjudicate perspectives and answer unresolved questions about cognition and behavior that continue to be debated.
Determining how cognition shapes behavior is an ongoing project, made even more complex by the potential for variation across contexts, individuals, and behaviors. Currently, sociologists are far better at grappling with this complexity theoretically than empirically, which means sociological contributions to our understanding of the interplay between the cognitive and the social remain a mere shadow of what they could be. Response conflict data and MPT models are not the only way to study cognitive processes, but they are a powerful addition to our toolkit that can provide traction on fundamental questions. 13 Notes 1 This model is equivalent to the process dissociation procedure used by Miles and colleagues (2019) and prevalent in psychological social psychology ).
2 Parameter estimates are for the Lambert et al. (2005) data. The estimates shown in Figure 3 are a simplification. The MPT models fit by Bishara and Payne (2009) included two C parameters and eight A parameters to reflect the unique features of the study design in Lambert et al. (2005). The two C estimates were similar (0.58, 0.64), as were the A estimates, although to a lesser degree (0.47, 0.47, 0.60, 0.56, 0.44, 0.43, 0.63, 0.59), so to keep the exposition simple we used average values of C and A. These values are found in Appendix C of Bishara and Payne (2009).
3 These interpretations are only correct if the assumptions underlying the model are accurate-notably, that C captures the deliberate implementation of respondent intentions and A captures automatic processes generated by the primes. Consistent with our discussion later in this article, we suspect that A captures much more than prime-related automatic cognition. Bishara and Payne also fit models with a catch-all "guessing" parameter (equivalent to the E parameter we use in this article), which allows us to test this claim to some degree. Bishara and Payne find that the process dissociation model without a guessing parameter best fits the data. However, the estimated parameters presented in their Appendix C show that once a guessing parameter is added to their models, estimates of A become quite small. This indicates that the estimates of A in the process dissociation model without the guessing parameter are likely a combination of cognition generated by the primes as well as other, unspecified processes. Regardless, the relative size of the C parameter relative to the A parameter means that the main finding remains the same: behavior during the weapon identification task relied most often on deliberate cognition.
4 Specifically, we use the same parameters (C and A) but differ in how we code our behaviors. Our models predict behavioral choices, such as voting for a candidate or hiring an applicant. Bishara and Payne (2009) predicted whether respondents made "correct" decisions, or in other words behaved in a way that reflected their intentions. We could have coded our data in that way (e.g., coding whether respondents hired only those applicants with the right qualifications), but that approach struck us as being less intuitive. Practically, it does not matter-both approaches return identical results.
5 The deliberation-dominant model shown in Figure 4 is similar to the process dissociation procedure used by Miles et al. (2019). However, Miles and colleagues estimate two A parameters, whereas the current model estimates only one. Furthermore, the two approaches differ in how they attempt to isolate automatic cognition produced by the primes. Miles and colleagues take the difference between the two A parameters, whereas the model in Figure 4 includes an E parameter to represent extraneous processes.
6 G 2 statistics can also be used to compare nested models using a likelihood ratio type of test. When models are not nested, model comparisons can be made using the Akaike information criterion (AIC), Bayesian information criterion (BIC), and the Fisher information approximation (FIA). AIC, BIC, and FIA all impose fit penalties to the G 2 statistic based on the number of parameters in the model and (in the case of FIA) other features of model complexity. Practically, this means that when two models are equally complex-for example, by having the same number of parameters as in our case-these additional fit statistics provide no information above and beyond the G 2 statistic.
7 Other tests might be needed to test other model assumptions. For instance, the assumption that respondents intended to answer accurately could be assessed using a follow-up question that asks about their intentions, and the assumption that respondents were able to answer accurately could be evaluated by directly testing their knowledge of the answers in a task given before or after the RCT (see Miles et al. [2019:317-18] for an example). In cases where deliberate but inaccurate answers are likely to be common, the possibility of deliberate control producing inaccurate answers could be built into the MPT model. Additionally, "automatic cognition" and related terms are often used as catch-all terms to refer to many types of processing features that do not necessarily co-occur; for example, processes might be efficient, unconscious, unintentional, uncontrollable, and so forth (Melnikoff and Bargh 2018). Scholars should therefore be careful to design validation tests that accord with the features of the processes that they theorize are operating in their model. For instance, a process that is unaffected by cognitive load is likely efficient because cognitive loads reduce certain types of working memory. However, this says nothing about whether the process is unconscious.
12 Another possible source of bias is a misspecification of the processes that are already included in the model. For example, analysts often allow a single type of process to vary across conditions if they suspect that those conditions modify how (or how much) it operates Singmann and Kellen 2021). Bishara and Payne (2009), for instance, estimated separate C parameters for data that included different response deadlines, because deliberate cognition is likely to be affected by the amount of time available for thought (see Appendix C in their article for further details). In our analyses, the most likely source of variation is in the effect of the primes. See Appendix D of the online supplement for a discussion of this issue.
13 For researchers interested in applying these methods, we suggest beginning with a few key readings. Riefer and Batchelder (1988) and Batchelder and Riefer (1999) are two foundational pieces that explain MPT model estimation, inference, and related issues. A high-level overview of MPT models in cognitive and social psychology is given by Calanchini and colleagues (2018), whereas Singmann and Kellen (2021) give a brief overview of MPT models, model inference, and model selection as well as details on how to estimate MPT models using the statistical software package R (another useful tutorial is at http://doi.org/10.31234/osf.io/gh8md). This literature should provide a solid foundation in the logic of RCTs and MPT models. In our experience, the major challenge comes next, when researchers must translate their theories into testable models and design studies that allow those models to be tested. We include a guide to this process in Appendix E of the online supplement, which includes a discussion of the data requirements for estimating more complex models.