 Hi everybody. Welcome to the first data AI and society seminar talk of 2021. I'll start by honoring the first Australians who are the traditional custodians of the land on which we're meeting. For those of us calling in from an you that's the none of all and memory peoples, and we should pay our respects to their elders past present and emerging and acknowledge that sovereignty was never seated. I'm pleased to introduce Brian head and who has recently moved an you as associate professor of philosophy to give our first talk. Take it away Brian. Okay, thanks Serena. Let me just share that. Is that all working properly. You can see my screen. Okay, great. So I'm going to be talking about statistical criteria of algorithmic fairness, and the plan for today. Let's see. There we go. The plan for today is I'm going to start off by looking at the most famous case study of algorithmic fairness which probably many of you have seen multiple times namely the case of the compass algorithm. I'm going to survey some proposed statistical criteria of algorithmic fairness. Then I'm going to state some impossibility theorems, not proven by me proven by others. But these impossibility theorems show that these criteria these proposed criteria of fairness are not jointly satisfiable, except in marginal cases where we can predict things perfectly, or where base rates are equal across the relevant groups. So that's a very informative question. Do we take these impossibility theorems as showing that fairness dilemmas are inevitable. We basically can't help but being unfair in some respect or another. Or do they invite us to look again at the criteria and decide that some of them are not in fact necessary for fairness, after all, and we have to separate the wheat from the chaff. The only one is that only one of the criteria that I'm going to consider is plausibly necessary for fairness. And I'll do this by using a simple case in which something that I suggest is a perfectly 100% fair algorithm can violate all of these criteria, except, except for the one. And moreover, it can violate these criteria simultaneously, and it can do so even when base rates are equal across the relevant groups. And that's important because unequal base rates is often taken to be the root of all our problems in this domain, but I think it's actually a bit less important than it's often taken to be. Okay, and then I'm going to close with some general reflections on algorithmic fairness. Compass is an algorithm used to predict recidivism recidivism is committing a crime, and it's used in jurisdictions around the US to make decisions about bail parole alternative treatments and the like so it's used in various elements of their criminal justice system. In a major report for ProPublica. These authors and when at all argued that compasses quote biased against blacks to quote their headline. And what they argued was that the reason why compass was biased against blacks is that compass yielded a higher false positive rate for blacks and for whites. That's to say, of those who weren't going to commit a crime, a higher percentage of them were falsely predicted that they were going to commit a crime for blacks than for whites, and it yielded a higher false negative rate for whites, then for blacks that's to say, of those people who were going to commit a crime who did commit a crime, a higher percentage of them were falsely predicted not to recidivate for whites, then for blacks and that seems like it's a sort of bias or unfairness going on. Okay, but then that's not the end of the story and there were some responses so the north point the company behind compass argued that there was in fact no bias in their algorithm because compass was equally accurate for blacks and for whites in this sort of technical statistical sense that I won't go into. Another group of authors florist at all argued that there was no bias because compass was equally calibrated for each group. That's to say that for each risk score. This is the percentage of individuals assigned that risk score who recidivated was roughly the same for blacks and for whites. So what it looks like it's happening is ProPublica tied fairness to one set of criteria, equal false positive rates and equal false negative rates, while north point and florist at all tied it to a different set of criteria so right as I said fairness to equal false positive rates and equal false negative rates across groups north point said fairness requires just equal accuracy in this technical sense florist and co authors argued that fairness requires just calibration within groups. So which criteria are necessary for an algorithm to be fair or unbiased. That's the question for today. So I'm going to be focusing on algorithms like compass that make predictions first and foremost, and not decisions. And that that I think is important that I'm focusing on predictive algorithms. In the first instance, and these algorithms take as input a set of known features which is often called a feature vector, and the output either a risk score, or a binary prediction, yes no positive negative, or both. And for simplicity, let's allow the risk score to fall in the interval 01 so that we can think of it as something like a probability that the individual caught falls into what's called the the positive class where positive might be recidivating, or it might be defaulting on a loan, or, you know, not finishing university or whatever it depends what we're trying to predict. Okay, and I'm going to focus on algorithms that output both a risk score in the 01 interval, and also a binary, yes, no positive negative prediction. Okay, so some criteria fairness might concern the inner workings of the algorithm so for example, you might have a criterion which says that the algorithm has to be blinded to race, or more generally to protected class status right. That would concern the concern the inner workings of the algorithm. But other criteria just require that certain relations between prediction and actuality be the same across the relevant groups, and that's what I'm going to focus on here. I'll call these statistical criteria of algorithmic fairness, and they're attractive in part because they don't require us to look at the inner workings of the algorithm like the date on which it's trained or how it operates. And that's, that's important because the inner workings of the algorithm might be proprietary, or potentially just opaque, right, we might not be able to just stare at a huge amount of code and figure out from that whether it's fair or unfair. But with statistical criteria, we just look at what was predicted, and what happened, and how that all shook out. Okay, so here are a bunch of statistical criteria of fairness that have been considered in the literature. Do not worry about remembering all the details I'm going to survey 11 of them. I'd recommend if your eyes start to glaze over, just try to focus on a few two or three and remember those and then I think the general lesson will be pretty clear even if you don't remember all the details. So calibration within groups. This criterion says that for each possible risk score, the percentage of individuals assigned that risk score who are actually positive must be the same for each group, and also equal to that risk score. So the idea here is that fairness requires that a given risk score, quote unquote, mean the same thing or have the same evidentiary value, regardless of the grouping question. So this criterion governs risk scores but it has, it can be generalized to the case of binary predictions, and these are called criteria of equal positive predictive value and equal negative predictive value. So equal positive predictive value says that the percentage of individuals who are predicted to be positive, who are then actually positive must be the same for each group. And equal negative predictive value says the percentage of individuals individuals predicted to be negative, who are actually negative should be the same for each group. So this requires that a prediction of positive mean the same thing or carry the same evidentiary value for each group in question and similarly for a prediction of negative. Okay, next we've got equal false positive rates and equal false negative rates. So equal false positive rates says that the percentage of actually negative individuals who are falsely predicted to be positive must be the same for each group. And similarly, the percentage of actually positive individuals who are falsely predicted to be negative must be the same for each group. And these criteria can be motivated by the thought that individuals who behave in the same way they either recidivate or don't write, they should on average on balance be treated the same in terms of how they're predicted, regardless of the group to which they belong. These criteria govern binary risk scores but they also have natural analogs for risk scores. And these criteria that are known as balance for the positive class and balance for the negative class. So balance for the positive class says that the average risk score assigned to those individuals who are actually positive should be the same for each group. So the negative class says that the average risk score assigned to those individuals who are actually negative should be the same for each group. And so again, you know, take the take the individuals who are actually not going to recidivate the ideas on average. You know the blacks who don't recidivate should on average be assigned the same risk score as the whites who don't recidivate, and so on. Okay, then we've got a few just a few final criteria some people will defend the claim that fairness requires an equal ratios of the false positive rate to the false negative rate. So ratio of false positive rate to the right rate to the false negative rate should be the same for each group. And the idea here is that fairness maybe requires assigning the same relative weights to false positives and false negatives regardless of the group in question. Equal overall error rates. So this would say that just in this set in the following sense the algorithm should just be overall equally accurate for each group. So the number of false positives and false negatives divided by the total number of individuals should be the same for each relevant group. Statistical parity then says the percentage of individuals predicted to be positive should be the same for each relevant group. So this is a really controversial condition and in fact, most people in the literature reject it because if base rates are unequal across the groups that is to say, if the groups differ with respect to the percentage of members of that group who are positive, who say go on to commit a crime, then statistical parity would be violated even by an omniscient predictor who could perfectly predict everyone's behavior. But we can do better by moving to this final criterion, which says that the number of individuals predicted to be positive, divided by the number of individuals who are actually positive, must be the same for each relevant group. So this final criterion allows that you can have a different percentage of each group predicted to be positive, but only provided that those differences in the percent who are predicted to be positive are driven by underlying differences in the base rates. So you can predict whites to recidivate at twice the rate that you predict blacks to recidivate provided that whites have twice the base rate of committing crime as blacks do, for example. Okay, so that's a mouthful. It's a ton of statistical criteria. I'll just put them up here again. Like I said before, don't worry about trying to remember all of them or get your head around each and every one. What I want to do is just give you the sense that there's a lot of different criteria out there. They all have some sort of prima facie motivation, but we're going to face a problem in a second where we learn that they can't all be satisfied together, right? So we've got these three statistical criteria for risk scores, calibration within groups, balance for the positive class, balance for the negative class. And then we've got eight that govern binary predictions. So equal false positive rates, equal false negative rates, equal positive predictive value, equal negative predictive value, equal ratios of the false positive rate to the false negative rate, equal overall error rates, statistical parity, and equal ratios of predicted positives to actual positives. Impossibility results. So there are several theorems in this literature that have a sort of very pessimistic feel, right? So they say that we can't have it all. We can't satisfy all of these criteria fairness together. So the first theorem says that the criteria one, two, and three, that's to say calibration within groups, balance for the positive class and balance for the negative class are not jointly satisfiable unless either base rates are equal across the groups. So again, unless the same percentage of each group is in fact positive, or if prediction is perfect. But those are very marginal unlikely circumstances to get into. John Kleinberg and some co-authors, and in some ways this is like the seminal paper in this literature. A related theorem due to cold to cova says that criteria for five and six are not jointly satisfiable again unless base rates are equal, or prediction is perfect. So this is equal false positive rates, equal false negative rates, or sorry. Yeah, equal false positive rates equal false negative rates and equal positive predictive value. And then there's another theorem saying that no algorithm can satisfy more than one of the conjunction of four and five that's equal positive predictive value equal fall negative predictive value, six and seven that's equal false positive rates and equal false negative and 11 which is equal ratios of predicted positives to actual positives, again unless base rates are equal or prediction is perfect. Okay, now I'm not going to go through the proofs of these or really say a whole lot more about them. The main takeaway is just, these theorems say that we can't have it all, we can't satisfy all of these intuitively attractive criteria fairness, except in these very marginal scenarios where base rates are equal, or we can just predict things perfectly. Right. So we can't have it all. So what do we say about that how do we respond to impossibility. I think the prevalent response in the literature is a pessimistic one. It's to say that fairness is impossible, except in marginal cases fairness dilemmas are all but inevitable. We cannot help but be unfair in some respect or another. So, Kleinberg and co authors say, any assignment of risk to doors, can it principle be subject to natural criticisms on the grounds of bias. That's the conclusion of their impossibility result. Gabriel Johnson says that these results show that there is no such thing as an unbiased program. Sandra Mason says, they show that race neutrality is not attainable. And Burke and co authors in an important survey article, say the implications of the impossibility results are huge. The goal of complete or race or gender neutrality is unachievable. There's also a more optimistic response, you could look at the impossibility results and take them to show that just not all of our criteria one through 11 are in fact genuine necessary conditions on fairness. Some are just specious. Some look like they're important for fairness, but in closer inspection, they're not. We could if we were sort of tempted by this optimistic response we could go criterion by criterion and subject each one to scrutiny to try to separate the wheat from the chaff. Or what we could do is find a perfectly 100% fair algorithm and just see which criteria it can violate. And if some criterion can be violated by a perfectly 100% fair algorithm that proves that that criterion isn't necessary for fairness. Now this methodology this ladder methodology might seem impractical since you know you might think it's always going to be controversial whether a given algorithm is fair or not. So how could you even implement this, maybe the best we can do is go criterion by criterion and stare really hard at each one and try to figure it out. But I think we can do better. I think we, we can in fact employ this second sweeping methodology. So here's my argument. I'm going to argue that none of the statistical criteria, except perhaps the first one calibration within groups is necessary for algorithmic fairness. And this is because all of the other criteria can be violated by a predictive algorithm that is perfectly 100% fair. And moreover, all these other criteria can be violated simultaneously by a perfectly 100% fair algorithm, and moreover they can be violated simultaneously by this perfectly fair algorithm, even when base rates are equal across the relevant groups. Okay, so here's the case people coins and rooms. There are a bunch of coins of varying biases towards heads. The individual in our population is randomly assigned one of these coins. And then each individual is randomly assigned to one of two rooms and be intuitively think of them as our demographic groups. And our aim is to predict for each individual, whether their coin lands heads or tails. And this is to say whether their behavior is that of a heads person or a tails person. Right, so we're predicting the behavior we're predicting is heads or tails. Right, we've got these demographic groups room a and room B. Okay, so that's that's our task. Now luckily for us, each coin comes labeled with its bias towards heads so it just says how biased towards heads, it is. It does that with a with a number between zero and one representing the objective chance that it lands heads. Okay, here I submit is a perfectly fair and moreover uniquely optimal predictive algorithm for each person take their coin and read its label. If the label says X assigned that risk person a risk score of X. If X is greater than point five, make the binary prediction that the individual is a heads person. And if X is less than point five make the binary prediction that the individual is a tails person. What about when X equals point five I mean then we face an issue of do we just randomize our prediction, or do we arbitrarily assign them all a prediction of heads or what. And then a sidestep that issue by just assuming that none of the coins is labeled point five none are perfectly fair coins. So that just skirts that issue. Okay. Now, this algorithm, I submit is perfectly 100% fair, its risk scores are not sensitive to the room from which a person comes. And on which its risk scores and predictions are based, namely the labeled bias of the coin is clearly the relevant one to focus on, and is not a proxy for room membership. And not only is the algorithm not unfair to any individuals in virtue of their room membership, but there's seemingly no unfairness anywhere at all in the situation. Of course there's plenty of randomness, but there's no unfairness, I take it. So, and moreover, the algorithm is uniquely optimal so no alternative predictive algorithm could be expected to do to do as well or better at predicting heads people versus tails people. Now, here's the lesson. Our fair algorithm cannot violate criterion one. It cannot violate calibration within groups, at least assuming that relative frequencies match coin biases so that, you know, about 90% of the coins labeled point nine. In fact, Landheads, about 30% of the coins labeled point three Landheads and so on. So assuming relative frequencies match coin biases, our algorithm can't violate calibration within groups, but it can still violate all of the other criteria two through 11. It can violate them simultaneously, and it can violate them simultaneously, even when base rates are equal across the two rooms. That's to say, the same percentage of people in each room are heads people versus tails people. So here's a case right here's here's just filling in the numbers. So let room a happened to have contained 12 people with coins labeled point seven five. So these people are all assigned risk score point seven five and predicted to be heads people. And because we're assuming that relative frequencies are going to match coin biases, 75% of them, that is to say nine of them are heads people. And it also contains eight people with coins labeled one eighth or point one to five, and they're all assigned that risk score point one to five and predicted to be tails people are negative. And in fact, one of them is a heads person right one out of the eighth, one out of the eight people with coins labeled one eighth, in fact as a heads person. Let room be contained 10 people with coins labeled point six, they're all signed risk score point six and predicted to be heads people. And in fact, six of them are heads people. And it also contains 10 people with coins labeled point four, they're all assigned risk score point four and predicted to be tails people or negative. And in fact, four of them are heads people. Right. So that's just the setup. And note that in the base rates are equal across the two rooms in each room, 10 people are heads people out of a total of 20 people. So each room has a base rate of point five. So we've got equal base rates here. Okay. Now, just crunching the numbers. We're going to see that this given this population as input are perfectly 100% fair algorithm is going to violate all of the other criteria of fairness except for calibration within groups. I just have a table here. I won't go through each and every one of them. But, you know, what we look at is, you know, what's the average risk score that was assigned to people who are actually positive. Well, for room a it's point six eight seven five for room B it's point five two. Similarly, you get a difference for average risks who are assigned to people who are actually negative. The false positive rate of 3 tenths for room a for 10 for room be a false negative rate of 1 10 for roommate for 10th for room B different positive predictive values different negative predictive values. You know different ratios of the false positive rate to the false negative rate different overall error rates different ratios are predicted positives actual positives and different percentage of people who are actually predicted to be positive. which is perfectly fair. If it works on this population, it's gonna violate all of our criteria except for calibration within groups. So if you agree with me that the algorithm is perfectly fair, you have to reject all of these other criteria as necessary conditions on fairness. They're specious. Okay. That I just said. Now note that I don't claim that calibration within groups is sufficient for fairness. I think that's an open question. It's just that it's the only one left standing in the face of my perfectly fair algorithm. And I also, so one lesson here is that when the other criteria, other than calibration or violated, this could be due to unfairness in the algorithm or it could be due to the distribution of evidence and risk in the population and not to unfairness in the algorithm. It's also possible that violations of these other criteria provide evidence for unfairness without actually constituting or entailing unfairness. I think that's very plausible that a violation of one of these other criteria still could provide some prima facie evidence that there's unfairness, but that's not to say that the criterion is a necessary condition on fairness. Okay, so I'm gonna close briefly with just considering two upshots of this argument or two lessons. The first has to do with inframarginality. So most of the criteria violated by my fair algorithm fail for reasons related to the so-called problem of inframarginality. The idea here is that at least an important part of fairness is treating marginal cases the same for different groups. So for example, the least suspicious African-American person who would be predicted to be carrying contraband should be exactly as suspicious as the least suspicious white person who would be predicted to be carrying contraband. The least promising woman who would be predicted to succeed in the job should be exactly as promising as the least promising men who would be predicted to succeed and so on. But many of the criteria that we've considered and now rejected concern in part how things turn out with non-marginal or inframarginal cases. So for example, if you want balance for the positive class, you want people who are actually positive to be assigned the same average risk score, well that concerns everyone who's actually positive, not just the ones who are close to the margin where you would tip the balance between predicting one way versus predicting the other way. It also concerns how many clear cases there are. Now in real life, it's always gonna be controversial whether a given case is or ought to be considered a marginal case. But with coin flips, things are clearer. So basically what's happened is a lot of our criteria were violated simply because room A contained relatively few marginal or unclear cases, everyone had a coin with either a heavy bias towards heads or heavy bias towards tails, whereas room B contained predominantly marginal or unclear cases, everyone had a coin that was roughly pretty close to 0.5. So they were just harder to predict. And then a little bit of asymmetry just gets us to violate the other criteria. The second thing I wanna talk about and this is the sort of closing point is the importance of the fact there are multiple possible intervention points in any real life system like this. So when a predictive algorithm is used to make decisions with distributional consequences that we deem to be unfair or unjust or just otherwise disvaluable, this doesn't necessarily mean that the predictive algorithm itself was unfair or biased. The unfairness or disvalue or injustice could instead lie with the background conditions of society or with the way that decisions are made on the basis of the predictions out quoted by the algorithm. And so as a result, the best response also might sometimes be not to modify the predictive algorithm itself, but rather to intervene elsewhere. And here's an analogy that I think is illuminating. Suppose we face two problems in our city, traffic and inequality. And we're deciding whether to adjust, adopt a congestion pricing scheme where we aim to reduce traffic through higher tolls or fees for driving during rush hour. Okay, I think congestion pricing schemes are really attractive, but they're also really controversial in part because one might worry that congestion pricing schemes would be unfair to low income people, right? After all, low income people would have less flexible work hours, have to drive for their work and therefore they'd wind up paying a higher percentage of their income in these congestion fees. So we might in response be tempted to either just abandon congestion pricing altogether or adopt some more complex scheme where low income drivers are exempted from the fees or something along those lines. But there's a better response available. We should adopt the original congestion pricing scheme along with something like an across-the-board tax cut for low income people or insert your favorite anti-poverty or wealth redistribution scheme here, right? We can do multiple things at once. We can walk into gum. We have multiple goals, reducing traffic and reducing inequality, but we've also got multiple points where we can intervene and we shouldn't ask the congestion pricing scheme itself to do all the work addressing traffic and inequality at the same time. Now, of course, maybe it's gonna be politically infeasible to adopt this more optimal divide and conquer strategy. So it might be second best to adopt the more complex version of congestion pricing that exempts low income drivers, but we shouldn't be confused into thinking that in fact, fairness itself requires this second best scheme. I think the fairest and best response is just to do the congestion pricing with an across-the-board tax cut for low income people, right? Divide and conquer, intervene at multiple points. Okay, now similarly with predictive algorithms, right? We've got multiple goals. We want fair and accurate predictions. We want fair decisions. We want adjust and flourishing overall society, but we've also got multiple points where we can intervene. We can modify the predictive algorithm. We can modify the way decisions are made on the basis of predictions or we can intervene elsewhere in society. We shouldn't necessarily ask the predictive algorithm itself to achieve all of these goals on its own by itself. Now, of course, again, if it's politically infeasible to adopt some fair predictive algorithm where while making suitable big picture, heavy-duty interventions elsewhere, it might be second best to fiddle with or modify the predictive algorithm to get the distributional results that we want. But again, we shouldn't be misled into thinking that fairness itself requires the predictive algorithm to be tailored to get these results on its own. Okay, now in actual fact, I think there may be good practical reasons to try to achieve distributional outcomes in part through our predictive algorithms. Algorithmic designers don't have the authority typically to intervene elsewhere. So if you're just designing a compass algorithm, you don't have the authority to pass a sweeping nationwide criminal justice reform act or pass reparations or do massive changes to the tax code, right? You're just able to intervene in a small one area. And you might think it's also more just unlikely that these big sweeping changes will be made. But I think it's a hard empirical matter how predictive algorithms should be designed in light of anticipated failure to take more drastic steps to address these inequalities. And it's not gonna be something we can capture with a simple statistical measure like saying that false positive rates should be equal across the two groups. One reason, and this is the final point for this that we can't capture this complex sort of goal we might have with a simple statistical measure is that whether the violation of some statistical criterion is bad for some group overall or not, that's gonna depend on the payoff structure associated with the use of that algorithm. It's gonna matter what's the cost of a false positive for each group or a false negative or a given risk score. What are the payoffs? What are the effects not only for people whose behavior is being predicted by the algorithm but also for third parties, right? And all this stuff is gonna vary case by case. So it's not gonna be something that we can capture with a simple statistical measure. Okay, so in conclusion, I've surveyed a bunch of statistical criteria of fairness that have been discussed in the literature. We've seen some impossibility results saying they can't all be satisfied together. I've given a case which I think shows that at most the first criterion calibration within groups is necessary for fairness, none of the others are necessary for fairness. So that's the optimistic response to the impossibility results. And I've closed by talking about the problem of inframarginality and the importance of the fact that we have multiple points where we can intervene in a system. Thank you. And here are some references. And I'll stop sharing my screen. Thank you so much for that lovely talk. I now now open the floor to questions. So those of you in the audience, please use the Q&A function. And panelists, you can use the raise hand feature. And I'll start with Atuza. Thanks very much, Brian. So I was wondering if you can say a little bit more about the philosophical or conceptual basis of this 100% fair algorithm. Because I fear that we can have some worries like similar or parallel to worries about ideal versus non-ideal theories of fairness or justice. And so I was just wondering, it seemed to me that you are adopting maybe implicitly or maybe you know, what is this ideal conception of fair algorithm? And so if that is the case, if you are kind of like using this ideal theory and then you're saying that for that ideal theory or that ideal conception, this statistical criteria or many of them are not relevant, then someone who is more the defender of a non-ideal conception of algorithmic fairness and wants to develop some accounts of fairness that take care of the context relevant features and sophisticated social democratic values would say that, okay, maybe this is great for this super ideal conception of fairness and algorithmic fairness, but the ramification of it to the real concerns of practitioners who care about algorithmic fairness are not super evident. Okay, yeah. So one thing is certainly that my case involves a pretty cleaned up idealized model, right? So I don't claim that real life cases like the case of the compass algorithm look a lot like my case of people coins and rooms for all kinds of reasons, right? But, you know, human behavior is not like random in the way that coin flips are. We don't have the same kind of evidence about individuals as we have in my case about the coin flips. And also in real life cases, you know, well, my demographic groups, the rooms are sort of, not the basis of historical oppression. They don't play any sort of causal role in determining the feature vectors that we have available to base our predictions on or, and they also don't have any causal role to play in, you know, determining people's behavior. Whereas all that stuff could be true in the case of other, you know, black and white and we're talking about crime, right? There's gonna be, there are demographic groups or the basis of historical oppression and they play a causal role in determining who's got what features and also how people behave in all kinds of ways. So what I would say in response is that what I'm doing with this model, this idealized model is trying to abstract away from some of these issues to try to focus on the issue of is false positive, equal false positive rates, for example, a necessary condition on fairness, right? And I think it's useful then to abstract away from these facts that, you know, in real life cases, the demographic groups are the basis of historical oppression, for example, right? It's also the case that to argue that something is not a necessary condition on fairness, all you need is one case, right? You just need one case where the criterion is violated, but there's no unfairness and that's what I've aimed to provide. So I take that I've just proven that these are not necessary conditions on fairness. Now I think someone, you know, so someone might listen to my talk and they're, you know, they're trying to design their own predictive algorithm and try to ensure that it's fair. Now it doesn't give a recipe for how they should do that, right? It doesn't, because, you know, their case doesn't involve people wearing, you know, their objective risk label on their forehead or anything like that. So what do they do? Well, I take it the lesson isn't that I haven't aimed to give them a recipe for designing their own algorithm. What I have said though is, or what lesson they can take when they're doing this kind of stuff in a non-ideal scenario is say, well, okay, we probably really should worry about making sure that it satisfies calibration within groups. That's at least plausibly necessary for fairness. But if it violates equal false positive rates or equal negative predictive value or equal ratio of false positive rate to false negative rate, that doesn't mean that our algorithm is unfair. It could still be fair. And we, you know, we shouldn't, you know, try to ensure that somehow or other we're gonna get equal false positive rates or somehow or other we're gonna get equal ratio of the false positive rate to the false negative rate. Cause those are just not necessary for fairness. Now, what I really think is also I'm fairly consequentialist about these things. So I think I wanna see a lot more sort of empirical social science kind of work to kind of try to figure out what do we want to, you know, suppose we are concerned with wanting our predictive algorithm to, you know, benefit historically disadvantaged groups or harm them as little as possible, then I think it's just a hard empirical question how to do so, right? But I don't think it's gonna boil down to any trying to ensure the satisfaction of some simple statistical criterion. Right, next question, Seth. Okay, thanks Brian for your talk. That was fantastic. So I kind of wanna pick up on those last couple of points and just sort of press you on whether it's actually the case that someone should take those two lessons from the paper. So I thought that, you know, like you were careful to say that calibration within groups is perhaps necessary for fairness, but it's perfectly possible when imagines that someone could come up with another case that was, you know, given the creativity and imagination involved in coming up with this case, something else similarly creative and imaginative, which leads to a violation of calibration within groups, at least for all that you said in the paper, right? So that seems something that's kind of possible. In fact, you could have a real life case, for example, that didn't exhibit calibration within groups, but which we thought was perfectly fair, conceivably. And then on this point about, you know, if we show that these other conditions aren't necessary conditions on fairness, then we shouldn't worry so much if they're not satisfied. I'm not sure that that follows either because it could be the case that some, you know, satisfying some statistical property is importantly productive of the values that we want to achieve when we're aiming at fairness. It might not be a necessary condition on fairness in as much as we can imagine a perfectly fair scenario with certain features that doesn't exhibit that statistical property, but it could also still be, you know, like a good-making feature with respect to fairness in many other cases and also departures from it could be a bad-making feature with respect to fairness. So then I wonder whether, you know, one should take comfort if one's algorithm doesn't satisfy one of these criteria from the notion that it isn't a necessary condition on fairness. Yeah, good, excellent question. So I think on the first case, yeah, so it certainly, I haven't foreclosed the possibility that someone could find another case where there's a perfectly fair algorithm that violates calibration within groups. I mean, I do think it's useful to think about this model where people are actually like, you know, labeled with their objective risk, so to speak. And I think, you know, you can't predict in accordance with, you know, known sort of objective risk and violate calibration within groups. So if you think that that's, you know, I would sort of want to resist. So any, if you give me an algorithm and it violates that calibration within groups, I would resist the claim that it's perfectly fair, right? Cause it can't be predicting in accordance with sort of known objective risk. But yeah, having said that, I mean, I haven't definitively foreclosed that possibility. I should also add one reason you actually might want to reject calibration within groups as necessary is actually that it might conflict with blinding this condition that says that the algorithm should be blinded to group membership, right? So there would be some cases where the only way to satisfy calibration is to base predictions partly on group membership, right? So that would be the case, for example, if you modify my case so that make it that, you know, people don't come labeled, coins don't come labeled with their bias, but you're just told that, you know, everyone from roommate has a coin labeled 0.75, everyone in room B has a coin labeled 0.4, right? The only way to achieve calibration then is to base predictions partly on room membership. And then, so then we have a conflict between blindness and calibration, and you get aside which one goes, and that's it. So one other feature that might, you know, one other problem that might be raised for calibration within groups as well then is if the sort of the boundaries of the groups are determinate and there are problems with, are indeterminate and there are problems with the actual assignment of groups. Yeah. That's also a situation where, you know, it might carry problems with it. Yeah, yeah. Now your second point was maybe I was too quick in fully dismissing some of these other criteria as saying that algorithm designers just don't need to really worry about it. And perhaps I did overstate that a little bit. So I think your point is well taken that as I would put it, as a contingent matter, and depending on the context, it might be that satisfying one of these statistical criteria is important for fairness. But the gloss on that, so I acknowledge that that's a possibility. What I would wanna say about that is that it's still not that say equal false positive rates is sort of constitutive of fairness. What I would wanna say is, you know, maybe we've got some independent gloss on fairness, you know, here's a crude version, right? Fairness requires, you know, increasing the aggregate wellbeing of the historically disadvantaged group. Now that's obviously much too crude to be a full analysis, but just suppose that we're working with that. But then it's like, it'll be the sort of thing where like, well, depending on the case, it could be that equalizing false positive rates redowns to the aggregate benefit of the historically disadvantaged group. But that's gonna crucially depend on this bridge principle linking satisfaction of the statistical criterion with effects on sort of aggregate wellbeing of the disadvantaged group or something like that. So I would, maybe what I should have said is just, if you're an algorithm designer and you sort of worry, hey, my algorithm violates one of these other statistical criteria, I think it's maybe more, maybe I shouldn't have said, oh, just don't worry about it. Maybe I should have said, look again and try to think, is there some contingent relation between satisfying or violating this criterion and something else that we independently care about? That's fairness. So I mean, in a way one could say then that the, maybe there aren't any real standards of statistical fairness as such. Maybe fairness is a broader value that any given kind of algorithmic intervention can only kind of touch on kind of tangentially. And maybe there are no, I mean, it's possible too that like, so like Atuza was saying, perhaps there's no real sense in which something can be statistically fair, intrinsically. It's much more a question of fairness as a property of societies rather than of statistical methods. Yeah, and it's worth also just bringing in, so I've talked about fairness, but it's certainly related to a notion of discrimination. And there is a literature actually in legal scholarship distinguishing between discrimination as disparate treatment and discrimination as disparate impact. And I think if you tie, say discrimination or unfairness to disparate treatment, then at least my algorithm counts as perfectly fair in that sense, right? Cause it really isn't treating people differently depending on their group membership. But disparate impact is just a matter of, does it kind of unbalance, negatively affect some group in a somewhat unjustified way, right? And that's gonna be just super contingent and it's not gonna be, yeah. So if you buy into disparate, I mean, there's a debate in the law about whether disparate impact is discrimination, right? But if you say it is discrimination or it's unfair, then yeah, it's just gonna be very contingent which things are unfair because it's gonna be, it's gonna depend on whether some fact about your algorithm actually yields a disparate impact or not. And that's gonna be case by case. So we have a question from Kathy Reed which asks, are there lessons or contributions here to emerging best practice for machine learning practitioners? Is your argument by extension, reducing the responsibility or accountability that machine learning practitioners have for ensuring fairness in models? That's a good question. I wouldn't wanna quite say that I wanna reduce their responsibility. I think I wanna shift their focus a little bit, right? So again, I mean, this is kind of piggybacking on what I was saying to Seth. I think it's plausible to think that they should really try to make sure that they satisfy calibration. Although I haven't fully argued for that climate. Again, that was just the last man standing in the face of my argument. But as for the other criteria, I think what they need to do is not just focus on simple statistical criteria but focus on the sort of contingent effects of the use of this algorithm on different populations. And that's gonna, basically it's gonna require a methodology that's less kind of a priori statistics, right? And more empirical social science, right? So, you know, we need to know, for example, for compass, right? I don't wanna foreclose the possibility that compass is unfair, but I feel like if it's calibrated then what we need to see to know whether it has any other objectionable features isn't just whether it violates, whether the false positive rates are unequal across the groups. I wanna know sort of what on balance are the algorithms effects on the African-American population and the white population, right? And that depends again, not just on how it behaves with respect to people who actually get fed into the algorithm but also effects on third parties, right? How does it affect crime in the broader community? How does it affect people's behavior and incentives? So I think it's not, I don't wanna lessen responsibility. I wanna just shift the focus of it. Like basically these impossibility results are less important than I think they're often taken to be. Okay, next question from Jenny. Yeah, hey Brian, great job. That was a really interesting talk. So I guess one thing I wanna sort of push your question is from kind of the very foundations of what you were saying and sort of this juxtaposition of pessimism and optimism with regard to data bias. And so sort of the way that you set it up is if we can't be statistically fair, this is sort of, if fairness is impossible and bias is inevitable, then this is sort of a pessimistic view of fairness to machine learning. I wonder about potentially repositioning that pessimism into an optimism about alternative pathways forward. And I think this builds on a bit of what others say a bit of what others have been saying. And in particular, if we were to sort of do away with criteria, if we were to sort of do away with the value or the prerogative of fairness and accept an inevitability of bias, then could that be an access point to do something intentional with that bias? Right, and to say, if we are inevitably biased, how do we channel that bias in a way that surges resources to those who've been historically disadvantaged? Just creating or constructing a just society, if not a fair one? Yeah, I think I'm quite, right. So one point is just that I framed this response that a lot of people have as pessimistic. And I think that is actually the flavor that you get in there, papers, but yeah, you could agree with the point that fairness dilemmas are inevitable and say, hey, that's not so bad because we just wanna make sure we're unfair in the right ways, right? And we wanna make sure we're biased, let's say, but biased in favor of historically disadvantaged groups, right? It's biased, but it's biased in the right direction. Yeah, that's certainly one less pessimistic reading you could have on the alternative interpretation of the impossibility results that I suggested. But I just wanna say, I don't, I'm not convinced that fairness dilemmas are inevitable. Like I resist that conclusion because I don't think it's been shown. I think what's been shown is that you can't satisfy all of these criteria together, but I just don't think they're all necessary for fairness. So for me, it remains to be seen whether fairness dilemmas are inevitable. But I guess too, and I think there's other hands, but I guess, I mean, I guess too to sort of push on that a little bit more, whether or not like it feels like that sort of hanging on by fingernails to fairness, right? So if we say I don't accept that fairness is impossible, right? Why are we hanging on to fairness as the value rather than say just redistribution of resources, right? So like, if fairness is a real struggle to hang onto or we're sort of fighting to keep it, why? Why fight to keep it? Okay, yeah, yeah. So, say, maybe I'll make two points. One is actually, I'm very sympathetic to like not talking about fairness as much and instead talking about just redistribution of resources and so forth. And in fact, I would go further, like I'm a consequentialist, so I'm happy to talk about just optimal promotion of value. But I also don't think that, if you do wanna talk about fairness, I don't think it is the sort of thing where we're hanging on to fairness by a thread, right? And that it's really hard to satisfy. I actually think if you buy my argument, like for everything I've said, it might be pretty easy to satisfy. Like maybe all you gotta do is satisfy calibration. And certainly you don't have to satisfy all these other things. So I think if I were a sort of a certain kind of non-consequentialist who really cares about fairness, I would take away from my talk that, hey, this is a lot more, I think we really can be fair, right? On the other hand, if you're a consequentialist, like I really am, then it's like, well, okay, yeah. Fairness, I think then the lesson is, if fairness isn't as hard to achieve as you might have thought, but also we should independently think that fairness isn't the super important notion in the vicinity. Yeah, great. Yep, that's where I was going. Thank you. Okay, we have another question from Seth. Sure, yes, and anyone else who wants to go should, but so I just was thinking about the sort of walking into chewing gum point that you're making, Brian. And this connects up a bit with Cathy's point as well, but I guess the notion that we should try to kind of remedy the shortcomings of algorithms or shortcomings of any particular intervention by thinking about social policy more generally, obviously I completely agree with that. And I completely, I guess I would add as well that you don't necessarily want a particular group of kind of technological elites making social policy choices in the way in which they design these algorithms. There's a question of legitimacy there as well. At the same time, the sort of the phenomenon that emerged out of the Compass algorithm is kind of, it occurs in many, many different places where the similar types of algorithms being used and they're being used in an increasing number of different places. Similar underlying phenomena lead to all sorts of things from the ways in which silly apps that take your face and turn it into an oil painting work through to sort of things with much higher stakes, through to things like where, what type of ads you receive from Facebook or whatever. So I do wonder whether you might wanna be, whether you would be happy kind of sticking with the central point, that as far as the actual algorithm design goes, it's a matter of calibration within groups and you're good, or whether you would sort of want to take, whether you think there's anything general that one can say, because the problem seems to be pretty general and it's not only the problem of the background injustice that exists, there's also something that just, you take these similar methods, you apply them in lots of places, you get lots of similar problems, we should be able to aim at something to do a bit better in that respect, but I'm just sort of skeptical about whether calibration within groups would be sort of sufficient in terms of resolving that sort of range of problems. Right, right. And one thing, and this is another way in which I think my argument is more limited than maybe I sold it as is, I'm really just focusing on predictive algorithms. And I think, so one of the things another intervention point is, we could change the way we make decisions on the basis of the predictions. And I think with these other algorithms, where it's deciding what ads to show you or what translation of a bit of another language to show you in Google translate or how it, what oil painting sort of thing it gives you in response to your photo or I think there was something with a, some app had a hot filter and it tended to just lighten people's skin, right? Those are all, I mean, there's sort of, maybe there's a predictive element in there, but it's a lot of just making decisions. And I do think that, if we're talking about decision making, then I think that raises a bunch of other issues of fairness that really aren't addressed by my talk. And it's not gonna be as anything like, I mean, all the criteria that I considered and rejected are just criteria governing predictions. So once you bring in decisions, there's a bunch of other kind of possible criteria that you might appeal to. And I think we're gonna have to decide, what is fair action? Again, I'm kind of tempted towards something about, something where it's really emphasizing, sort of aggregate wellbeing of historically disadvantaged groups, but I fully admit that that's, a lot of people will think that's much too crude and maybe I would, I probably couldn't see that before long. But yeah, I think once you bring in decisions, you gotta, my argument really is just talking about the predictive side of things.