Philosophy of Science Meets AI Ethics

Criminal justice systems around the world, especially in the United States, are increasingly using evidence-based predictive algorithms to influence decisions about individuals who are accused or convicted of crimes-including decisions about bail, sentencing, and probation. The use of algorithms for such tasks is sometimes justified by their supposed neutrality. On this view, algorithms are "stabilizers of trust, practical and symbolic assurances that their evaluations are fair and accurate, and free from subjectivity, error, or subjective influence" (Gillespie 2014, 179). This view is highly implausible, as I show here; but it is perhaps less implausible to think that algorithms, while not completely neutral, could be less biased than the human beings who would otherwise perform these tasks. On this account, for example, algorithms that assess risk of recidivism might not be value neutral, but they could be less influenced by racial and ethnic biases than human judges.
Recent scholarship in philosophy of science and technology has shown that scientific and technological decision making is laden with values, including values of a social, political, and/or ethical character (e.g., Biddle and Kukla 2017; Brown forthcoming; Douglas 2009). This paper draws upon this work, as well as interdisciplinary research on machine learning (ML), to argue that ML systems are value laden in ways similar to human decision making, because the design of ML systems requires human decisions that involve tradeoffs that reflect values. Given this, the important questions surrounding the development and use of ML systems are not whether they are value laden (they are) but how they are value laden, whose interests they serve, whose values they reflect, and how they might by designed and implemented to effect positive change.
In order to explore these questions more deeply, I examine recent work on the development and implementation of recidivism-prediction algorithms in penal systems. I examine the following steps in the construction of recidivism-risk-assessment tools: (1) collection of base-line population data; (2) operationalization of the concept of recidivism; (3) model construction; (4) output selection; (5) choice of a fairness criterion, and (6) decisions about transparency and opacity. At each of these steps, designers must make decisions that involve tradeoffs that reflect values-decisions that can impact significantly the lives of human beings. I conclude by raising some practical suggestions for designers of these tools, if they are to be used in ways that reduce bias and discrimination.




References:
Biddle, J. B. & R. Kukla. (2017). The geography of epistemic risk. In K. Elliott and T. Richards (Eds.). Exploring inductive risk (pp. 215-237). Oxford: Oxford University Press.
Brown, Matthew (forthcoming). Science and Moral Imagination: A New Ideal for Values in Science. Pittsburgh: University of Pittsburgh Press. 
Douglas, H. (2009). Science, policy, and the value-free ideal. Pittsburgh: University of Pittsburgh Press.
Gillespie, Tarleton (2014). "The Relevance of Algorithms." In T. Gillespie, P. Boczkowski, and K. Foot (Eds.). Media Technologies (pp. 167-194). Cambridge, MA: MIT Press.

This paper uses ideas from the values in science literature to interpret recent debates about transparency in machine learning (ML), and explores to what extent value transparency provides a feasible approach to managing value-ladenness in ML systems.
Several philosophers have highlighted value transparency as a plausible approach to managing value-ladenness in science. When scientific research involves methodological decisions, the value judgments guiding these decisions should be made explicit and transparent to policymakers who rely on the conclusions of this research. This is supposed to compensate for a lack of epistemic transparency, i.e. that policymakers often cannot fully assess the complex chains of justification and uncertainty underlying scientific conclusions. Value transparency is supposed to enable policymakers to nonetheless determine whether accepting a given scientific conclusion is conducive to their preferred values.
Advanced ML systems base decisions on complex statistical correlations derived from some data set. Similar to scientific research, this complexity makes it difficult to assess the justification and uncertainties of the systems' recommendations. Much current technical research focuses on making ML systems more 'interpretable' or 'explainable', i.e. increasing epistemic transparency. Law and policy scholars have criticised this approach, arguing that governance frameworks should instead require developers to make transparent the aims ML systems are designed to achieve – i.e. ensuring value transparency.
I distinguish three types of value-ladenness in ML systems.
(1) Explicit Goals. ML systems are designed to achieve some specific goal, operationalised as a mathematical objective function that the training process seeks to optimise. These are clearly value-laden choices. Making these transparent is (relatively) straightforward.
(2) Inductive Risk. Deciding trade-offs between different error-types is inextricably value-laden. In ML, how to weight false positives and false negatives can be explicitly encoded into the objective function. Moreover, there are well-established methods (e.g. ROC curves) for illustrating an algorithm's false positive and false negative rates on the training data. However, predicting how this performance extrapolates to out-of-sample contexts is less straightforward.
(3) Unintended Optimisations. ML systems only optimise for goals that are explicitly encoded into their objective functions. This can lead to unintended solutions. For instance, a dating app was found to systematically match users with people of their own ethnicity, even if they ticked "no preference" for ethnicity. According to the developer, this was unintentional: the algorithm was merely designed to suggest the matches that were likely to produce a successful date. Apparently, the algorithm had found a correlation that efficiently optimised for this goal, but did so in a way that compromised another important value. Thus, the problem concerns neither the algorithm's explicit goal nor its overall inductive risk profile. 
Applying value transparency to (3) is challenging, as it is difficult to predict which correlations a powerful ML system may discover. While it is possible to test post-hoc for any given correlation, this requires developers to know which correlations might compromise any values that matter to potential users. Thus, this type of value transparency requires higher levels of stakeholder engagement than (1) and (2).

Humphreys (2004, 2009) argued that most, if not all, algorithmic models suffer from epistemic opacity: there are epistemically salient processes of the model that are inaccessible. The case of machine learning models (MLM), makes this problem worse. Often MLMs are so complex and opaque that even the modelers do not fully understand their inner workings. The presence of such epistemic opacity undermines the amount of understanding that is possible from MLMs. For a model to enable understanding, transparency (Dellsén forthcoming; Strevens 2013), simplicity (Bokulich 2008; Kuorikoski and Ylikoski 2015; Strevens 2008), and the ability to manipulate the model (Kelp 2015; De Regt 2017; Wilkenfeld 2013) all seem necessary. 


However, recently Sullivan (2019) has argued against this line of thought: model opacity qua opacity need not limit understanding. Even if this is the case, might there be other ways MLM opacity limits understanding? In this paper, I argue that the presence of inductive risk requires greater model transparency in order to satisfy the epistemic demands of explanation and understanding.


Particular MLMs use cases-medical diagnosis, tracking criminal activity, recommending which news to read-vary greatly in personal and social significance. Philosophers of science generally agree that the stakes and other social influences can impact scientific practice (Anderson 1995, Potochnik 2015). This has led theorists to argue there is inductive risk in science, especially in high stakes cases like chemical pollutants or climate change (Douglas 2000; Elliott 2013; Parker and Winsberg 2018). The higher the risk, the higher the demand for evidence. Relatedly, in epistemology, pragmatic encroachment theories of knowledge suggest that the stakes of a situation influence our attributions of knowledge or when someone knows something (Hannon, 2017; Fantl and McGrath 2009). In this paper, I explore how lessons from both these literatures helps us to understand how the level of inductive risk or height of the stakes impacts understanding, in the machine learning context (and in general). Holding the level of MLM opacity constant, I will assess whether the stakes of the phenomena changes the amount of understanding possible. I will look at three cases: medical diagnosis, recidivism risk, and advertisement recommendations. I will explore differences between individual and societal risk and how that might translate into greater local model transparency (i.e. how a specific decision was made) with less global model transparency (i.e. how the model in general works).

This paper argues against one popular design solution to the problem of inexplicable algorithms: model-based counterfactual explanations. I argue that this solution does get something right, namely, that inexplicable algorithms are compatible with important political values, under certain institutional conditions. However, it focuses narrowly on the actual bases of the algorithmic decision, instead of broader questions of political morality, in this case, how institutions should be designed to achieve important political values in non-ideal scenarios.
Recent research in computer science has aimed to build techniques to recover model-based counterfactual explanations from even very complex models. Model-based counterfactual explanations isolate a subset of features such that, if the values of those features were minimally changed, the output of the model would be a desired value (such as a different classification, or a certain score).
Such explanations are taken to achieve many important political values enabled by explanations. By providing data subjects with a single or small set of counterfactuals that link specific inputs to a desired output value, model-based counterfactual explanations provide individuals with the ability to make a guided audit of the correctness of the input data, and challenge decisions on that basis, or to change individuals' future behavior in order to achieve a better result. Indeed, since the counterfactuals are "personalized" to the particular inputs about the individual, model-based counterfactual explanations seem to serve this function especially well.
Model-based counterfactual explanations, however, are not the best means to enable political values. And that is because these values are not best enabled merely by individuals being scored in some way according to the model, or knowing how a model's output depends on a few relevant features in their particular case. In the case of political values that require individuals to change their behavior, what those values require, and what individuals should want, is to actually have the relevant properties. Individuals want to be creditworthy, and to receive a loan on that basis, not to be scored well according to a model. Furthermore, many of the relevant political values are not enabled by correcting mistakes or changing one's behavior, but are instead enabled by understanding abstract descriptions of the rules by which one's political community functions.
However, what the model-based counterfactual view gets right is that inexplicable models are sometimes compatible with explanation-enabled or constituting political values. They are so compatible under the condition that the algorithmic system have a disjunctive set of properties that reliably correlate with the "right reasons" on which the outcome should be – and, hopefully, is – based. In that case, the decision-maker can provide the individual with the explanation based on the right reasons, be it a relevant rule, or population-level causal facts about what properties people have that tend to cause the desired outcome.