 Hello, everyone. Welcome to this talk on responsible AI in banking. Artificial intelligence or AI has remarkably improved our ability to predict outcomes much more reliably than ever before. The progress and availability of these tools in a simple and scalable way have meant that businesses can have far deeper insights and look forward capabilities than, say, 10 to 15 years back. Used in the right ways, these capabilities can enable businesses to provide far more relevant offers and better experience to their customers. This has enabled businesses to improve their revenues, improve NPS scores of their products and services, and generally establish a more trusted relationship with their customers. However, as with all great powers comes greater responsibilities, we need to be mindful of the dangers and the pitfalls which we may come by if we are not careful. Many advanced machine learning models are not as transparent and easy to understand. They can encapsulate a lot of complex logic. And between the crevices of these complexities can lie dangerous biases and bugs. They can hide seemingly innocuous but potentially illegal and unethical reasoning. In order to be responsible users of this powerful paradigm called AI, we need to be on the top of and in control of these risks. The good news is that in recent years, there has been a number of different tools and techniques which have come by. And these can enable us to control and mitigate these risks. All the while, we can focus on improving revenue and continue to get our customers' love. So in today's talk, we will be exploring some of these tools and where and how we can use them to stay safe. So with that, let's introduce ourselves. I'm Ronadip Chatterjee. I'm a data analyst specialist working in Google Cloud. I'm based in London and I mainly work with financial services customers. And then Christos Anichtos, a machine learning specialist working for Google Cloud, also based in London. Let's look at a story to guide us through this talk. This is a work of pure fiction, but any resemblance to the real life is intentional and not merely coincidental. So our two friends, Jess and Nick. No, they're not a couple. This is not a romantic story. Let's see what happens next. But let's consider our two friends, Jess and Nick, two young professionals. They don't know each other and they are both dreaming about buying their first homes and getting on the property ladder. So they don't know too much about mortgages and credits. And though they have had a few years of pretty steady relationship with their banks, mainly through current accounts and some deposits. So here on the top on this slide is Jess. No, she's not always upset. It's just me being lazy and not finding a smiling picture of hers. So she is fine. So Jess has been working for the last three years and has had a pretty stable job. She banks with Conce bank and she has had a steady relationship with the bank for all of this time, mainly through her current account and some savings. And lately she has seen this new apartment block come up close by and she is she fancies a two-better there and she's looking to get onto the property ladder being a first time buyer. And not knowing much about mortgage, she just approaches her normal, everyday bank, Conce bank for the mortgage. And down here is Nick. So here on the other hand is a little bit more adventurous. So he likes the idea of being independent and having his own business. So he works contracts for the money and is saving and also slowly building up his business, mainly working in between contracts and additional money that he's getting. So he has also been working for the last four, five years and has been lucky to get regular work. And he banks with Tellubank. And again, all this while the last four, five years he has had a steady relationship with Tellubank. So coincidentally, Nick also decides to get onto the property ladder and he's also attracted by this nice apartment block that's coming up. And he also plans to buy a flat there. So he's slightly more savvy than Jess about finances being in business and all that, but he's very busy and he doesn't have time. So he just goes to his standard bank Tellubank and requests for mortgage. Now both Can't Say and Tellubank use instant mortgage approval processes using AI engines to encapsulate their mortgage approval process. So as in every nice story, here comes the twist. Jess's mortgage application gets rejected by Can't Say Bank. And true to their name, they haven't provided a reason. They said they can't say the reason for the rejection. It's against their company policies. So Jess is left wondering why she doesn't know what she can do to get that mortgage and she's really disappointed that she cannot get onto the property ladder and she cannot buy her dream home. Neither does she like her bank anymore because of the unhelpful attitude that they have shown. Now looks can be deceptive. Even though we see here Nick is smiling, but his mortgage has got rejected too. But he has some reasons for being a little bit happier than Jess because his bank has told him the key factors that affected the decision. Like his credit score is too low. His monthly outgoings are too high for his income and maybe the period left in his current contract is too short. And what's more Tellubank has told Nick how he can improve his chances of qualifying for that elusive mortgage. So now he can plan his next steps. He can prioritize what he wants to do and he knows exactly what sort of steps he might take in order to go on to the property ladder and own his dream home. Now let's imagine Can't Say Bank picks on board Jess's feedback and realizes that they are losing their customers trust and customers are not happy and going to their competitor Tellubank. So they quickly crank up some transparency features in haste without due consideration of the risks. So what can go wrong? Well, problems or weaknesses in their models can manifest in many damaging ways. Maybe they had some issues with the input data that was used to train the model and now the model is biased against women, for example. Now was it intentional of the bank to do this? Is it even ethical? There can be other unintended behavior as well if we are not careful. For example, the model might say that Jess's income is too low. Now that might be the result of skewness in data. Maybe the data that was used to train this model had a whole lot of data coming from New York because there might be more customers based in New York for this particular bank. Now what the model has learned here is that the average salary drawn by New Yorker is the standard salary for even smaller towns. And so when Jess applies for the mortgage, even though her salary might be quite fine for this apartment that she is looking for, the model has wrongly rejected her application. And there could be other things as well. Like it might be discriminating based on physical disabilities, which may be completely unintended and might even be illegal in some jurisdictions. It might be considering completely absurd factors like whether somebody is a coffee drinker, say for example. And this might be the reason because absurd data has not been removed from the trading data and due to some coincidental patterns in this data, the machine has actually picked up that as one of the significant factors. So in order to ensure that such unintended behavior is not passed on to the AI models, a robust machine learning model lifecycle needs to be established with the right software and organizational checks in place to ensure consistent and intended results. Internal processes in AI, such as AI project governance, for example, can establish an ethics approval process where when there is a new idea for using AI. And then data scientists, analysts and managers, they need to be educated on the concepts of ethical AI, risks of biases and bugs, and potential negative outcomes that can come out of not being careful. Additionally, we need to put in a framework in place that ensures that all the right controls and checks are in place when a model is built. For this framework to work well in a scalable way, we need to leverage tooling and techniques. Hence we need to invest in tooling. At Google, we innovate and create tools that can help understand and validate data in ML models. We open source these tools as we want to enable the AI community to build products that are ethical and responsible. We open source tools for data exploration, model analysis, explainability and reporting. In the rest of the talk, we will take a look through some of these tools and techniques that can be used to adhere to responsible AI principles. As an example of principles that can guide the use of AI in a responsible manner, here we show Google's AI principles. This can be accessed online by going to ai.google forward slash principles. There are a set of objectives against which we assess our AI applications. For example, AI applications should be socially beneficial and avoid creating unfair bias. They should be built and tested for safety and accountability should lie with humans. On the other hand, they should not be used in applications that are likely to cause overall harm or that violates internationally accepted norms and human rights. Every organization that uses AI should create their own set of principles based on their ethics and beliefs which they should honor. With that, I hand over to my co-speaker, Christos, to take us to the rest of the presentation. Thank you, Rana Deep. Let's see how we can leverage responsible AI in a machine learning lifecycle. A typical machine learning lifecycle consists of several steps that take care of data ingestion, data analysis, training of a machine learning model, deploying the machine learning model, and then assess the model. Given the previous example of a loan application, then we can first of all gather previous loan applications' data and the outcomes. So if the loan was approved or not, we can do data analysis and feature engineering. Then we train the loan approval model and we evaluate that the model is accurate enough before we put it into production. Then we deploy a machine learning model into production and then after that, when there is a new loan application, we can send the application details to an API, for example, and get back a prediction that says if an individual should be approved or declined alone. With that in mind, let's revisit the scenario of bias. How could we ensure that we do not introduce bias in our data? Responsible AI expands beyond explanations or suggestions. In this scenario, we can see that the model is biasing based on gender. Why is gender so important for a loan approval? One should wonder why gender is even part of the equation. Do we discriminate? You need to ensure that AI models perform in an ethical and responsible manner. Of course, you do not need to do that simply because the end user can see the explanations provided by the model. You need to create a fair environment for your customers regardless if they know how the decisions are made or not. Biases can exist in data. For example, selection bias is when a selection of data points happen without proper randomization. Let's now go back to our loan application use case. Imagine that we're training our machine learning model using loan applications from a pool of New York applicants only. This might result in misrepresentation of applicants in other cities as our AI model will not know much about those people. Another example is when historic data include marginalized groups that were treated unfairly in the past and had their loans declined. By the use of such data, the AI model will naively learn to discriminate those groups. So to sum up, biased data introduced to an AI algorithm will produce a biased machine learning model. Of course, another issue that can be introduced is the one of unintended treatment. When we are analyzing our data and creating features, we can use tools that can help us better understand our data. For example, facets. Facets is a tool that we use in order to slice and dice our data. Notice that this can be used before we build our machine learning models. You can also use it when you want to analyze your data without building machine learning. Simply load your data and explore. What you can see in this animation is how easy it is using facets to explore different age groups from your data. You can visualize ages in different baguettes, color code based on relationship status and split based on martial status. With this approach, you can easily tell if there is a group missing. What if you have no data for people between ages 6 and 70 years old, an age that a lot of people retire? What would your model assume for this age group? Would it be accurate enough? Lastly, facets is a tool which you can use for both structure and unstructured data. So you can visually explore images and text. This might discriminate indirectly, like the fact that our customer receives a disability allowance. The reasons may vary. Whether there is bias in our dataset or an underrepresented demographic group, the problem is still there. While we train our model, we need to ensure that it is performing well overall, but also for different slices of the data. We need to be sure that it's treating all groups fairly, and for that we need fairness indicators. Fairness indicators allow you to explore how your model performs with different groups of data. In these examples, we see the false negative rate for three different ethnicities, A, B and C. Assuming that these data are for our loan application use case. What this chart tells us is that our AI falsely declines loan applications of ethnicity C way more often than other ethnicities. The same we can check for age groups, genders, disabilities and so on. Not always the results would be unreasonable. Some attributes might be relevant. It is reasonable to reject very big loans of individuals with very low income, because those might be of higher risk. But how about ethnicity? We need to do more investigation in order to understand the reasons and if unfair treatment is introduced. Another concept we can use to better understand our models is accessibility. Explanability is a process of understanding how and why a machine learning model is making predictions. And it happens during modeling and prediction phase of the machine learning life cycle. Explanability is a concept that you can apply with different modeling techniques. And it is also applicable when you're using state of the art deep neural networks. As mentioned, explainability is a tool which you can use during model training as well as during prediction time. That is when the model is deployed in production. So back to the example of our coven drinker. Yes, maybe this attribute should be part of the model. But for the purpose of this presentation, assume that this was not filter out and then made it into the model training step. Using explainable AI tools, you can quantify how much of an impact each feature has for a particular prediction. When looking for each predictions individually, we're referring to local explainability. If we then aggregate all the predictions together and average the impact of the features, we get what we call global explainability. In the first example here, you can see that the loan was declined mainly because individual applied drinks coffee. Capturing that early, we can understand why and fix such defects. Additionally, by serving explanations to the customer, you give them the right to dispute any decision and escalate to humans in order for their application to be revised. In the second example, we can see that another loan was rejected this time because of gender. The reasons may vary, as I said earlier, not enough data or maybe biased in the data. Maybe the aggregation was problematic. Either way, capturing these allow us to investigate further. Another important asset in your toolkit is the what if tool. This is another open source tool that, with the help of explanations, allows you to explore data and their predictions. Compare with previous model versions and finally test different what if scenarios. Looking at an individual whose loan application was declined, you can check different scenarios. For example, what if their age was 50 instead of 38? How does this change the prediction? Would they now be approved for the loan? The what if tool helps you answer all those questions. And finally, the last piece of the puzzle is to be able to retrospectively evaluate your model performance after it is deployed to production in order to ensure it is behaving. Additionally, you need to look predictions explanations in order to be able to debug and report. And for this, you can use continuous evaluation. Continuous evaluation is more of an architecture decision than an open source tool. Having said that, CoolCloud has a ready made solution that you can use when you deploy your models. Continuous evaluation automatically logs all predicted attributes for a machine learning model and allows you to complement data with the truth at a later state. Either by using human labelers or by having feedback loop from a given platform. For example, if you're automating loan application processing, you should look 100% of the predictions along with their explanations. You can then loop back the customers that disputed the AIS decision and want the dispute. So we now know how responsible AI fits the machine learning life cycle. During this life cycle, there are multiple steps. We explore data, we analyze data, we train models, we push models into production. We have tools that we can leverage during the whole process. Those tools can help us understand our data, our model, and how the models perform, how accurate they are, and if they discriminate between different slides of the data. And with that in mind, let's get back to run it. Hello again. So let's wrap up the story. So coming back to the story, let's see what happened to Jess and Nick. So it so happened that Jess and Nick, they bumped into each other on one of their visit to the estate where they were looking to buy their apartments. And Nick, smart guy, he stroke up a conversation with Jess, and then they both started chatting and went for a coffee. And during their coffee, Nick told Jess about his experience with Telubank. And that gave Jess a direction. Now Jess knew that there was somebody out there who can help her. So Jess now went to Telubank and applied for a mortgage and got some reasons of how she can actually make things work better. And eventually they worked hard towards getting the mortgage and eventually got onto the property ladder. And that made her smile. But again, being lazy, I couldn't find a smiling picture of her. So that's why she still looks glum here. And Telubank was actually able to expand their business by getting a customer from Consay Bank, Jess in this case, and also by increasing trust on their own brand by their own customer, Nick in this case. As for what happened next, we'll leave it up to the audience to think about how you want to end the story. Thank you very much for being with us and look forward to seeing you somewhere sometimes again. Thank you.