 Awesome. Hello, everyone. I'm Bhakti, and I work for Google Research. And I come from the responsible AI team. I hope all of you are enjoying your conference. And I'm here to talk about flagging and fixing bias in machine learning. So today, we're going to be talking about bias in ML. And then we're going to talk about some open source offerings from Google that you can use in your workflows to make your workflows fairer. And then we'll end with a conclusion. So let's begin with the riddle. Here's the riddle. A father and a son, they are in a horrible car crash that kills the dad. The son is rushed to the hospital. And as he's about to operate, the surgeon says, I can't operate. That boy is my son. What do you think is happening here? And how would you go about solving this riddle? Any answers? Go ahead. The surgeon is his mother, OK? But that's one way to look at it. And any other answers? That's correct, by the way. Grandpa, OK. But it's a son, right? So now the grandson. But yeah, close. We're getting there. But you are right. It could be that the doctor or the surgeon is the mother and the doctor is a she. And this was exactly what the study that was conducted in Boston University showed, that when you think of a surgeon, a lot of people who identify themselves as feminists also forget or overlooked at the fact that the surgeon could be a she or the boy's gay second father, and so on. So that's bias. And I'm going to play a very quick video for all of you to think about machine learning and bias. Let's play a game. Close your eyes and picture a shoe. OK, did anyone picture this? This? How about this? We may not even know why, but each of us is biased toward one shoe over the others. Now imagine that you're trying to teach a computer to recognize a shoe. You may end up exposing it to your own bias. That's how bias happens in machine learning. But first, what is machine learning? Well, it's used in a lot of technology we use today. Machine learning helps us get from place to place, gives us suggestions, translates stuff, even understands what you say to it. How does it work? With traditional programming, people hand code the solution to a problem step by step. With machine learning, computers learn the solution by finding patterns and data. So it's easy to think there's no human bias in that. But just because something is based on data doesn't automatically make it neutral. Even with good intentions, it's impossible to separate ourselves from our own human biases. So our human biases become part of the technology we create in many different ways. There's interaction bias. Like this recent game, where people were asked to draw shoes for the computer. Most people drew ones like this. So as more people interacted with the game, the computer didn't even recognize these. Latent bias. For example, if you were training a computer on what a physicist looks like, and you're using pictures of past physicists, your algorithm will end up with a latent bias skewing towards men. And selection bias. Say you're training a model to recognize faces. Whether you grab images from the internet or your own photo library, are you making sure to select photos that represent everyone? Since some of our most advanced products use machine learning, we've been working to prevent that technology from perpetuating negative human bias. From tackling offensive or clearly misleading information from appearing at the top of your search results page, to adding a feedback tool on the search bar so people can flag hateful or inappropriate autocomplete suggestions. It's a complex issue, and there's no magic bullet. But it starts with all of us being aware of it so we can all be part of the conversation. Because technology should work for everyone. So I hope that gave you an intro of what machine learning and how human bias can creep in into our ML systems as we deal with data and algorithms that are not hand coded. They run completely on the data we feed in. So this is your typical ML workflow. Most of you are aware about all these steps. It begins by defining a problem. You collect and prepare your data. You train a model, evaluate, and integrate and monitor. And obviously, I trade through the process to ensure that you are not drifting away from concepts and your models are not becoming stale and you're getting the right data at each time. But essentially, bias can creep in at each of these steps in your typical ML workflow. For example, when you're defining a problem, you can have automation bias or historical bias or interpretive bias. Historical bias is something where the world itself is biased and you're defining the problem in a wrong way. Or automation bias. You're overly relying on a machine to make judgment calls. For example, here, this was an Eastern Airlines flight 401. And the pilots entirely relied on this one light, gear indicator light. And they thought that the machine would indicate whether they were at the right altitude. And they did not know that they were disconnected throughout and you had 101 depths because of that. So when you're defining a problem, you really need to think, is it making sense to rely completely on computers or should computers be augmenting the human decision instead of making it all together? Likewise, when you're collecting and prepared data, you can have sampling bias where you're just sampling from a particular population. Representation bias where you're not looking at a diverse set of users. Or measurement bias where you're only measuring things in a way that complies to your own inherent biases. So yeah, if most people, like this meme here, most people are taking surveys, then you know that people like taking surveys. So that's a bias that can creep in during your prepared data phase. And when you're training the model, a lot of times you can have algorithmic bias or experimenters bias. And what this means is you either overfit or underfit your model to suit your own judgment. And you often try to correlate things. And this can inherently show up as spurious correlations in your ML workflows. And evaluate or implicit bias. Imagine you're trying to design a gesture recognition system and this is me doing okay gesture, which is prominent in North America. But if I go and do the same thing in Brazil, they will boo me out. Why? Because it's a very offensive gesture right there. So you need to understand how you're evaluating it and how you're evaluating over different groups as well. And finally, deployment bias. Are you even deploying this model for the right audience? And that's where deployment bias comes in, like we saw for the gesture recognition model. So in your typical ML workflow, to make things fairer, you would need to add special considerations at each and every step to ensure that you have a responsible and fairer workflow throughout. So when you're defining the problem, think if automation should exist. Should you have a AI model that predicts criminality of people? Probably not. That's a bad idea to have automation for. Who's the intended user of the system? Are you designing it for the right user? Or is it just a subset? And what are the risks associated if the model gets it wrong? What's gonna happen? What's the fallback? And what would success look like? We think about business metrics like accuracy, latency, but have you thought about fairness as a metric when it comes to ML models? So that's on defining a problem. And when you're collecting and preparing the data, think about where the data is coming from, how it's sampled, how diverse it is, and are there correlations in the data that give you indications on what the biases are? And of course, the data, is it privacy-protecting? And finally, when you're training the model, think about who trained the model and how, because the experimenter himself or herself might be introducing bias into the system, and ask if you're making special fairness considerations while training your models. And finally, while evaluating, ensure your evaluation is done on a wide variety of users and you're using exhaustive data sets and if fairness goals are being met when you're evaluating the model. And finally, when you integrate and monitor, ensure that your model is not stale, you're still catering to the user group you are designed to serve. And then you also need to communicate the pitfalls of your model, so transparency becomes very important when you are finally launching. And to see all of this in action, we have a lot of offerings that can help you solve these problems and some of them are open source so you can use them right away into your ML workflows. And today we'll be covering some of them. So let's start with defining a problem. So when you're defining a problem, when you're defining a problem, we have the people plus AI guidebook and Google's AI principles. So think about using AI for socially beneficial causes, ensure that it's not reinforcing bias and it's tested for safety. So these AI principles and a lot of work essentially when you're designing fairness goes into just doing this step right and hence this is a very socio-technical concept. It's not about tech users diving into the core and doing it all by themselves. It involves putting a lot of stakeholders in the same room and getting that problem definition right. And this can be a good way to start. And this also includes things you shouldn't pursue. So if your AI application is likely to cause harms, drones and military and that getting wrong can have harmful consequences, creating it for weapons, surveillance, violating internationally accepted or legal norms, you should not do that. That's our principle and this is one way to get started with defining your problem. There's also the people plus AI guidebook that you can use and like I said, defining success is a major part of getting fairness right and people plus AI guidebook can help you get there. So it can tell you how do you define what success looks like for the user? How should you put feedback and control and how should you fail gracefully? Because a lot of times these models will go wrong. There's no two ways about it. What happens then? How do you incorporate a feedback loop in the process and make them better? Because it's an iterated process, remember? So people plus AI guidebook can help you with that. And again, the most important part is understanding what fairness looks like in academic research, there are 21 definitions of fairness. What works for Google Maps may not work for YouTube and it may not work for the model you are trying to design. So you need to think about what works well, what fairness definition works well for your problem use case and think of fairness as a metric. So I'm gonna take you through some examples here. So this is one definition of fairness where you could essentially be group unaware. Imagine you are trying to design this ML model that decides whether you should give a loan to a population, blue and orange are the populations. And let's just say 55 is the threshold and 55 is the credit score of the population. And you can see that group unaware makes the bank this much profit. And let's look at another definition of fairness which is demographic parity. You can say that we are not biased because this time we are not unaware about the identity of the group, but this time we are gonna ensure that the positive rates of people getting the loan is comparable in both the cases. So in this case, 37% and as you can see that the loan thresholds are different for both these groups. And what that means is the profit is also going to be different. So what I'm trying to get here is the moment you change your fairness definition, it takes a different turn when it comes to business metrics and you need to maintain that trade-off. And you can explore these playables on our website as well to understand how this impacts the models under the hood. There's another one called equality of opportunity. And again, this one is about getting the true positive rate, right? So people who pay back are essentially same for both these groups. And again, this has a different impact on the business metric, which is total profit for the bank as well. So as you can see, getting that user metric right is the fairness metric right is critical and doing a right trade-off with your business metrics is crucial to getting this part right. And finally, think about who the users are. When you're defining a problem, like for example, when you're designing this feature for Pixel that essentially does the portrait mode on Pixel phones, you need to think about who are the users, how are they segregated, what's their age group, gender, race, and you should be thorough in defining this so that eventually you test it exhaustively on these user groups as well. So now that you have defined the problem, this is the piece that's hard and takes most of our time at Google, the next piece is collecting and preparing data. So you can use this tool called Know Your Data. So this is a tool, again, it's available to use and you can play around with it. So what this does is it basically gives you a bird's eye view of what your data looks like and it also has some inbuilt signals that can tell you if there are human faces in your data or if your picture has been blurry or if it has been taken in the night or in the day and so on. And the most important piece here is it tells you about correlations. So now, people who are smiling are oftentimes, are they oftentimes the people who are wearing a hat? And this can tell you that and quantify it as a metric and it can tell you this. This is particularly important because you may think, oh, my model is not even having gender as an input, but oftentimes if your model has some other input like has makeup, you can easily tell what the gender is because correlations exist. So with KYD, you can look for skewed samples. So do you have more examples of women than men in your data? And that's a skewed representation, you can find that out. For example, here you can see that the word, it's a text-based data set and you can see the word attractive. Oftentimes correlates very much with females and a beautiful, again, correlates very much with female and so on. And you can see these correlations exist and you might think your model is group unaware, but it might be picking up on these proxies and causing bias downstream. Then you can also look for other signals that exist with KYD. So you can essentially look for features that are not a part of your data set, but you can build upon those. Like I said, if this is a self-driving data set, are there people in this data set and you can use KYD to figure that out? And you can also look at tainted samples and proxies. So like I said, even if a sensitive attribute is not used for training an ML system, other features can often turn up as proxies and you can essentially map that out using KYD. For example, this is another text data set and you can see that when you correlate words like, active words like riding, playing, jumping, you can see that it's skewed towards the younger population and not so much towards the elderly population. That doesn't mean that the elderly population can't do any of these, but you can clearly see this skew in the data. So this is something that you can use KYD for. And finally, you can also see sample size disparity with KYD. If you have a majority group and a minority group, how many examples are coming from each group? This is something you can also look with KYD. So for this example, there are fewer women who are in the data set than men and so on and you can easily probe that using KYD. And yes, getting a good, fair representation in your data is very important. I mean, it's the hard part, but you can get a lot of bang for the buck if you get this process right. Now imagine that you have done your best to collect and prepare data, but there's only so much you can do and now whatever considerations you have to make are at the training step. There's not much you can do. Then there are some fairness considerations you can do at the training step to make your model fairer. So essentially, you can remediate your model if you have identified bias. So essentially, if you have a data set and it's skewed, you can collect more data, but if you don't have the budget, you can generate synthetic data or you can sample the data in certain ways so that the distribution is same. So these are data pre-processing techniques. You could also do training time modeling techniques to remediate bias. So essentially, you can change the model's objectives in itself. You can add constraints to the model while training it or you could do some post-training methods to remediate for bias. And this is again at the model level. So you tune the model's outputs in a certain way so that the interpretation of the outputs can improve the performance across the fairness metrics that you have set. So this is one other technique and this is also open source. This is from our model remediation toolkit. This is Mendeff. Imagine that you are trying to train a model and you want the performance of these two distributions to be the same. You don't have the data resources, but you are trying to do something at the model level. So you can use this Mendeff law so that you can add to your model and then you can see that the majority and the minority group have equalized distributions in this case. This is particularly helpful if you are say training a toxicity classifier and it's trying to call a certain group more toxic than the other. So you can use this Mendeff laws in your training and you can observe that it improves the performance there. There's also the TensorFlow constraint optimization. So now a lot of times these fairness goals that we talked about, the 21 fairness definitions, they can be written as constraints. So you can say that the true positive rate of majority needs to be equal to the true positive rate of the minority and that's demographic parity essentially and likewise for equality of opportunity. Now you can take all these constraints using the TensorFlow constraint optimization toolkit and add them to your training objective and this is again open source and you can see that this helps improve the model's performance over the various groups. On your right you can see that you're trying to train a classifier which essentially predicts whether a person is smiling or not and without constraints you can see that it oftentimes thinks that all people are not smiling or their false positive rates are higher and then with the constraints added you can see that the gap reduces and the overall false positive rate for both the groups also reduces. So you can use this toolkit or you could use counterfactual logic pairing. So again, you're training this toxicity classifier and you put these sentences in a toxicity classifier and you can see that all these sentences, none of them appear to be toxic but the model is learning these correlations, right? So it's seen as toxic and the toxicity scores are very different for all of these groups. What do you do then? So then essentially you can use this counterfactual logic pairing loss which is again open source. You can use this and you penalize the model for predicting any of these sentences differently and when you train it over and over you will see that essentially it not only brings the toxicity score of the model down for all these terms but also closes the gap between the various groups. So you've trained your model, you have tried your best to prepare the data in a diverse way. Now it's time to evaluate the model and again, this should be in an iterative loop, like I said. There are several tools to iterate or flag bias in ML models. So one is fairness indicators. Again, this is open source. So essentially you can look at sliced metrics across the various groups and you can see how you're performing over the female group or the male group or the LGBTQ group and so on and you can slice it across the user groups, whatever user groups you have that you thought about in defining the problem stage and see how you're performing. And oftentimes when you deal with fairness your data is going to be very sparse. You're not gonna have enough information about each of these groups. So then fairness indicators toolkits comes with confidence intervals. So it tells you how confident it is about each of these metrics. So you can even work with sparse data when it comes to these evaluations. You can also use the what if tool. Again, this is open source. So essentially you have a model, you have its predictions. What if I just change the age group and is that causing my model to flip its decision? You can find out all these what if scenarios using the what if tool and understand if there's bias, if it's over indexing on age to give a particular response and if it's over indexing on some other sensitive attribute to predicted decision. And there's also the language interpretability tool. So if you're working with languages and then here you're looking at a sentiment analysis model, just changing the word waiter to waitress is causing the model to behave differently. Sentiment is the same. But it's just that you're changing the gender so you can see how much a term is causing the model to flip its decision or to wait in one way or the other. You can use the language interpretability tool which is also open source. And once you're done evaluation you can essentially use model cards to basically communicate what your model is doing to the broader audience. Not just to model developers, of course, as coders we know what the model does and we can read into code but for the lawmakers, for the model users it might be hard and transparency and communicating what the model is good at and what the model is bad at is just as important as building the model itself. So we have the model card toolkit. So what this does is it basically tells you what is the model good at? What are the intended use cases? For example, phase detection. What should you be using this for? What is it good at? What data sets has it been trained on? And what are its limitations and ethics and considerations? And you can do this for your model as well. And this again is an open source tool that you can use. So with that, there are several offerings that we have to make your workflows fairer. And a lot of these offerings are open source. So be sure to use them. And with that, thanks a lot. And yeah. Oh, you saw it too. Oh, gosh. Yeah, I think I just made it in time. But any questions? Yeah, go ahead. I'm curious how many of you have considered fairness in your AI or ML workflows? Maybe a show of hands. Oh, wow. That's great. Where any of these findings or toolkits new to you? If yes, then show of hands, perhaps. Okay, great. So this talk was helpful. Are you saying, do we have tools that can automatically annotate? What kind of pictures are? Oh, right. Yeah, I think the question is around, do we have tools on identifying where data sets or how well represented a data set is in terms of geographical locations? We don't have any annotators that can do this out of the box. But if you have those location annotations as to where the image was taken, then you can obviously explore them using no data and see how well represented that is. We also had this competition called open, not a competition, but a crowdsource challenge called the open images challenge. And we saw that a lot of times when you source data sets, oftentimes it's from North America or it's not as diverse as it should be. So we did this challenge and we got people to crowdsource images, including a lot of Googlers and then we created this data set. So you had diverse representations of what a wedding meant. It could mean a Christian wedding or a Hindu wedding and so on. So it was a more deliberate process. But yes, we did go through some data auditing to come to that point and realize that there was a skewer, there was misrepresentation. Any other questions? Well, thank you very much for your time and I hope you use some of these in your workflows. Thanks.