Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Political Analysis. First view

Recipient of the 2022 Clifford Clogg Award for Best Graduate Student Paper from the American Sociological Association Section on Methodology

Forced-choice conjoint experiments have become a standard element of the experimental toolbox in political science and sociology. Yet the literature has largely overlooked the fact that conjoint experiments are used for two distinct purposes: to uncover respondents' multidimensional preferences, and to estimate the causal effects of some attributes on a profile's selection probability in a multidimensional choice setting. This paper makes the argument that this distinction is both analytically and practically relevant, because the quantity of interest is contingent on the purpose of the study. The vast majority of social scientists relying on conjoint analyses, including most scholars interested in studying preferences, have adopted the average marginal component effect (AMCE) as their main quantity of interest. The paper shows that the AMCE is neither conceptually nor practically suited to explore respondents' preferences. Not only is it essentially a causal quantity conceptually at odds with the goal of describing patterns of preferences, but it also does generally not identify preferences, mixing them with compositional effects unrelated to preferences. This paper proposes a novel estimand—the average component preference (ACP)—designed to explore patterns of preferences, and it presents a method for estimating it.

Pre-Print | Appendix | Replication materials

Know it when you see it? The qualities of the communities people describe as “diverse” (or not)

with Maria Abascal

City & Community. Forthcoming

We explore what people mean by “diversity” when they use the term to describe real communities. “Diversity” can refer to multiple differences—ethnoracial, economic, etc. It may also refer to multiple dimensions of the same difference, i.e., heterogeneity or group representation. Analyzing a survey of Chicago area residents, we ask: (1) When people describe a community as diverse, on which kinds of differences are they drawing? (2) Within each relevant difference, are evaluations of diversity predicted by heterogeneity, the share of specific groups, or both? Findings suggest respondents associate diversity primarily with a community's ethnoracial attributes and secondarily with its economic attributes. Within ethnoracial attributes, both heterogeneity and the share of disadvantaged ethnoracial groups, especially Blacks, predict assessed diversity. Within economic attributes, income inequality predicts assessed diversity, albeit negatively; the representation of poor people does not. Qualitative responses reveal varied understandings of diversity while confirming the dominance of ethnoracial attributes.

Pre-Print | Appendix

The cumulative risk of jail incarceration

with Bruce Western, Jacklyn Davis, and Natalie Smith

PNAS: Proceedings of the National Academy of Sciences of the United States. Vol. 118 No. 16 (2021)

Research on incarceration has focused on prisons, but jail detention is far more common than imprisonment. Jails are local institutions that detain people before trial or incarcerate them for short sentences for low-level offenses. Research from the 1970s and 1980s viewed jails as “managing the rabble,” a small and deeply disadvantaged segment of urban populations that struggled with problems of addiction, mental illness, and homelessness. The 1990s and 2000s marked a period of mass criminalization in which new styles of policing and court processing produced large numbers of criminal cases for minor crimes, concentrated in low-income communities of color. In a period of widespread criminal justice contact for minor offenses, how common is jail incarceration for minority men, particularly in poor neighborhoods? We estimate cumulative risks of jail incarceration with an administrative data file that records all jail admissions and discharges in New York City from 2008 to 2017. Although New York has a low jail incarceration rate, we find that 26.8% of Black men and 16.2% of Latino men, in contrast to only 3% of White men, in New York have been jailed by age 38 y. We also find evidence of high rates of repeated incarceration among Black men and high incarceration risks in high-poverty neighborhoods. Despite the jail’s great reach in New York, we also find that the incarcerated population declined in the study period, producing a large reduction in the prevalence of jail incarceration for Black and Latino men.

Working papers

Greater Diversity or Fewer Whites? Disentangling Heterogeneity and Non-White Share at Macro and Micro Levels

with Maria Abascal and Delia Baldassari. Appendix by Daniel Lacker

Scholarship on the consequences of racial/ethnic diversity often claims that diversity undermines trust, participation and cooperation. This work has been criticized for its inability to discern the causal effects of diversity. We draw attention to a more elementary issue: most studies are unable to interpret associations between their outcomes of interest and diversity, as these may be due to associations with non-White or immigrant shares. We make the practical and theoretical case for preserving the distinction between diversity—i.e., mixture—and non-White or immigrant shares—e.g., percentage Black or percentage foreign-born—, and we warn scholars about the dangers of using language and measures associated with diversity, especially in contexts, like North America and Europe, where diversity strongly overlaps with non-White or immigrant shares. On a practical front, the policy recommendations that follow from the claim “greater diversity is associated with less prosociality” are different from those that follow from the claim “greater non-White share is associated with less prosociality among Whites.” On a theoretical front, most studies of diversity rely on theories that are predicated on ingroup/outgroup shares—most commonly intergroup conflict/threat theory—rather than on diversity; the predictions implied by popular theories, however, contradict those implied by claims about diversity. Importantly, two empirical obstacles undermine our capacity to disentangle associations with diversity from associations with non-White or immigrant shares. The first stems from the underrepresentation of predominately non-White (or predominately immigrant) communities in the real world. The second concerns our ability to infer that individual attitudes and behavior are correlated with diversity from correlations between macro-level outcomes and diversity. Finally, we spell out the kinds of data required to draw empirically sound conclusions about associations between diversity and social outcomes.

columbia twitter github