Philosophy of Social Science Roundtable

Predictive Invariance is routinely proposed as a condition that a fair algorithm must satisfy. Dietrich et al. (2016) defend the COMPAS algorithm against critics (Angwin et al., 2016) by appealing to predictive invariance. Kleinberg et al. (2016) also take predictive invariance as a sine qua non and argue that it is incompatible with the fairness criteria endorsed by Angwin et al. Hedden (2021) argues that predictive invariance is the only candidate for a necessary condition of fairness. We depart from this consensus and defend the view that the algorithmic fairness discourse typically requires violations of predictive invariance. We propose that measurement invariance, a concept common to psychometricians but overlooked in the literature on algorithmic fairness, is a better fairness condition.
In the relevant applications, an algorithmically generated score S is used to make a consequential decision. Typically, there is a single threshold t such that everyone scoring above t gets the desirable outcome, e.g. admission to a university or parole. Additionally, there is a sensitive variable R which stands for a demographic category relevant to fairness, e.g. race, or gender. Finally, there is some contextually relevant outcome O that will be observed in the future. For example, O might be GPA after two years at university, or it might be a binary variable indicating whether the prisoner is rearrested after release.
Predictive invariance requires that individuals receiving the same score S have the same expected outcome O, regardless of the value of R. In other words: students receiving the same score on an assessment exam have the same expected GPA, regardless of sex; or candidates for parole receiving the same risk assessment score have the same probability of being re-arrested, regardless of race. We claim that, unless social outcomes are already fair, predictive invariance requires punishing people for outcomes that reflect social disadvantage, and not individual merit. Perversely, it claims to do so out of considerations of fairness. For example, we would expect that if there were any racist patterns in policing, a black candidate for parole would have a higher chance of being- rearrested than an equally law abiding white candidate, even though they pose no more danger to their community. To satisfy predictive invariance, black candidates must be penalized for racist patterns of policing.
The problem is that O is not an error-free measurement of an underlying unobserved feature L that tracks the contextually relevant notion of merit. O reflects L but also other, undeserved, social advantages or disadvantages. Measurement invariance requires that individuals with the same underlying value of L receive the same score S, regardless of the value of R. If S satisfies measurement invariance, decisions based on S would reflect individual merits and not an undeserved social disadvantage. Measurement invariance is satisfiable, but typically incompatible with predictive invariance: the two conditions can only be jointly satisfied in extremely unusual circumstances (Millsap, 2007, Borsboom et al. 2008). When the two conditions compete, any fair algorithm should be designed to satisfy measurement, and not predictive, invariance.

A great deal of recent work in philosophy of science and social ontology on the metaphysics of racial kinds understands racial kinds as objective in the sense that one's racial membership obtains independently of what one thinks. For example, metaphysical accounts might hold that race is constituted by practices of conferring racial membership (Ásta 2018), or by conceptualized social roles (Mallon 2016), or by heirarchical social relations (Haslanger 2000), or by thin, unexplanatory properties (Glasgow and Woodward 2016), or by membership in a biological population of a certain sort (Spencer 2014), or membership in a racial culture (Jeffers 2019), or by structures (Mallon 2018, in press). In membership in a racial culture (Jeffers 2019), or by structures (Mallon 2018, in press). In each case, one's membership in a race floats free of one's identification of oneself as a member of a race.
In contrast to such objective kinds, a great deal of humanistic work focuses upon social identities (e.g. Alcoff 2006; Appiah 1996; Hacking 2007; Mallon 2017) where this involves, at least, the use of a shared understanding of a kind of person to shape one's projects. Richard Miller has argued that self-identification should play a role in our employment of social scientific kinds (Miller 2000). This paper considers the existence of racial kinds in light of recent work arguing for the objectivity of such kinds as well as Miller's argument, and argues for a role for racial identification in constituting a sort of racial kind. I argue that our explanatory purposes can dovetail with normative concerns regarding identificatory kinds, and push us towards pluralism about racial kinds.

Numerous social-scientific studies suggest that race is a cause of various inequalities. These studies all rely operational definitions of racial concepts that are then deployed to find evidence for generalizations about the inequalities of interest. Philosophers have argued that these studies' success allows us to infer that race is real. However, philosophers drawing on these studies have not investigated how race is measured and how these measures license commitment to race's reality. We remedy this by closely examining the different social-scientific measures of race and the burgeoning literature on the philosophy of measurement. Based on this, we defend a position we call placeholder realism about race. On our view, races are real and play a role in social-scientific investigation, but function as a placeholder for a combination of: (a) an ontologically thin notion of race, and (b) a motley of other social and psychological processes that frequently do the explanatory heavy-lifting in social-scientific research. Consequently, race need not have a substantive biological or social ontology nor need race be a kind unto itself.
We motivate placeholder realism by critiquing "measurement arguments." These arguments purport to show how the widespread use of different operational definitions of race in a variety of studies justify commitment to the existence of race. These arguments hinge on independent lines of evidence converging on a single concept of race. However, we present evidence from the social sciences showing how different operational definitions of race yield divergent empirical results or else are not independent of each other. This would seem to undermine measurement arguments.
We suggest that placeholder realism is a way of preserving realism about race in the wake of measurement arguments' failures. On this view, there is meager convergence, which only licenses a thin notion of race, i.e. (a) above. However, the significant divergence of these measures speaks to (b), i.e. the different social and psychological processes that do the explanatory heavy-lifting in different studies. We contrast our view with the basic realism defended by Joshua Glasgow and Jonathan Woodward. Like placeholder realism, basic realism is a thin view of race. However, basic realism holds that racial categories are real but scientifically irrelevant (or arbitrary). By contrast, our view entails that race is a thin concept but still scientifically relevant.