 Hi, everyone. Again, so today, I will be talking about responsible AI and explainable AI. It's more like a one-on-one in terms of responsible AI and explainable AI and how they are both connected. So I will start talking about responsible AI, of course, what are the principles, what are the main developer phases, how it connects with explainable AI. And at the end, I will showcase a paper that it's making a new approach in terms of explainable AI. It's more like related with human explainable AI. So that will be mainly the talk. So first, I know it's 11 AM, so we should be OK to see a pizza now. We are not so hungry. So the idea is to use the analogy of making a pizza to explain responsible AI. So we have principles. When we would like to use responsible AI or when we need to meet something using responsible AI, we have principles. Of course, in the papers and all the documentation that you can find, there are multiple principles. I prefer to highlight the main four principles that you can see in most places because the documentation is pretty new. And the concepts, you can find multiple concepts and multiple different concepts everywhere. So the first one is fairness. The pizza, if we're talking about the pizza, it talks about having the toppings equally distributed in each slice. In this case, I think it's arugula and cheese, yes. Hopefully, it's not pineapple. I don't know if you are big fans of pineapple. I'm not a big fan. But in that case, it will be equally distributed. So the same amount of toppings to each particular person, each particular slice. In AI, it is if you would like to do... This is mainly related with bias. So you can have a data set that is biased and you would like to avoid it. You should be avoiding training your model with a biased data set, of course. You have tools that we will see later. Second one is transparency, which is in the pizza analogy, is if you would like to go to a place, you should know how they cook the pizza, how they prepare the pizza, what is the recipe. If it's a Neapolitan pizza, or if it's an American pizza, or if it's... I don't know. They have different kinds of pizzas. So you should know or they should publish you or they should show you how they are preparing this pizza. So you will trust in this pizzeria because they are sharing the recipe. In terms of AI, it's sharing the models that they are using or how they are working with the data or how they are dealing with the data that it could be from the preprocessing, from the modeling, from the training, the modeling, and of course, how to use the data in the model. The third one is accountability. In the pizza, if you receive a pizza that it's overcooked, you need to know that someone is in charge of this pizza. They would like to replace you. The pizza they would like to give you a new pizza. Or probably if you don't like the pizza, you can ask for a new one. So someone has to be responsible for making this pizza and to give you a different pizza with different toppings. You'll probably ask for pepperonis, and if you get pineapple, you won't like the pineapple, so you would like to ask the pepperonis. And in AI, of course, someone has to be responsible of this application of this model. Now we are all seeing the LLMs hype, and the question here is who is responsible for this model. It's the person that is training the model, it's the person that is implementing the model, the person that is doing the fine-tuning or the company that is doing the fine-tuning. So someone should be the owner of this application. The last one is privacy and data protection. In the PICS analogy, it is to protect the data of the employees, so you don't know the names of the people that are working there because it's protected, or you probably don't know the exactly cheese provider or the exactly person that is providing the cheese. So this is internally protected. In the AI, it's more about protecting how you work with the data. For instance, if you are training a model to detect, to make a fraud detection system, you don't want to share the data of your customers, so I like to keep the data hidden and to protect the data. This could be related to something with GDPR, but this is more in that line. So what you will get at the end is if you go to a PICS idea and the PICS idea is following these four principles, you would choose this principle. You will go to buy a PICS in that place. The same can happen with AI. If you will trust to a model or to an application, you will give your data to this application if they are following the four main principles, four main principles or more principles. But now, that's OK with the PICS, and so we are done with the PICS, and we are mainly developers. So if you would like to build a model, it's OK with the principles, but you need tools to build this model to meet each particular four principles. So without going into details, the first one is you need to check if your dataset is biased. The way that you check it is if you know the data, you can see or you can do some query and you can see if it's biased by gender, age or whatever. And you have some tools, what is the IBM i360 or the Google, what if they are open source tools that you can go there and you will give, and these tools will give you if the dataset is biased or non-biased. And they can also de-biased the dataset if it's oriented to one particular gender or whatever. The second one is you have to detect or you should detect proxy variables, which is sometimes it's not implicit, the bias for instance. It's easy if you like to detect gender or age or country or whatever, but sometimes you can get this information through other information or through different information. That can be for instance, if you see the monitor status, you can get the sexual, someone can get the sexual orientation of this person. Or if you have a geographic patterns, you can see the ethnicity of this person. So you would like to avoid that to make your dataset less biased. The last three or the main ones is you should have some, you should have the dataset documented, how you treat the data, where the data come from in LLMs for instance, or it could be LLMs or computer vision or any use case is, which was your dataset to train the model, like in LLMs, you train the dataset for a particular use case. So you train for finance. So you get the data from particular banks and you train your model or healthcare or whatever, but where the data came from, you should have it documented because I didn't talk yet, but probably you can use it to make your model better, to work better, or it could be some regulations that you need to meet, that you will need to explain when you're making a decision or the model is taking a decision, you should explain why the model did that. The last two is explainable AI that will go deep later. That is, you can have the principles and you can have everything, but you need tools or you need something to help you to explain why the model is taking that decision. There are family, it's very advanced and there are multiple algorithms and multiple ways to do it, but explainable AI is basically that what you need. You need something to give you insights about the decisions. And the last one, which is pretty important, that is, while you are implementing your model and it can help on the data protection part and in the privacy part, let's suppose that your application is centralized or working in multiple devices, you need to protect the data or you need to protect the model when the model is working. So in that case, you can use tech techniques like PPML, which is privacy protection machine learning. So it's encrypting the machine learning process. Of course, I will not go in detail by just to let you know that we have that. And OpenFL, which is a federated learning framework. So when you're doing, when you're training a model in a federated way, you should protect the data of each particular node. So there's another technique that could be useful, but you also should be aware of, okay, it's not just following the principles, I need to protect the data when I'm working with the data or when you are training this model. So just to show an example of the first one to detect bias, this is just an example of something that I wanted to highlight. This is the case of the IBM. So you have a data set. In this case, it's the German credit scoring. And you will like to see if it's bias or if it's not bias. So what happened here, here it's saying that you have multiple features that equal opportunity index and so on. And you can see, you can visually see if the data set that you are working, it's biased or not. This is just a visualization tool, but you can go and step ahead and you can say, okay, could you please de-biased this data set for me? And this tool has the chance to do it. But this is a step before training the model. It's something that you do before that also. So it's not just one tool, it's not just one thing that you say, okay, I have one tool, I put everything here and I'm protected about responsible AI, explain it by AI, and I can get everything in just one place. It's not the case, you have to use multiple tools and depending on your use case and your need, you will probably use different toolkits. So once you have a model, I suppose that you have a model trained to detect if an email hits a fraud or it's not a fraud. And you can have some questions. Let's suppose that you have a model and it's saying that yes, this email, it's fraud. But you can make some questions. I mean, it's okay, you probably need to know, you probably need to understand why did the model took that decision? Based on what? Based on what? Features, variables, why not something else? Because you don't know. And the second one is how do I correct an error that probably if you're not an AI user or if you're not developing the model, you would like to say, okay, you said that it's spam, but it's not fraud. So how can I correct that? Or how can I modify that to make it more trustable? And the last one is should I trust it? Which is more like a human perception because at the end of the day, we are humans using these models. So we need to create models that are trustworthy for people, basically. So why do we need, explainable AI to try to answer those questions? This is a quote that I think that's pretty useful. If you go to the papers and so on, the terms could be used interchangeably. You can say explainability, you can see interpretability, you can see multiple things. But basically what is explainability, it was providing size to a targeted audience to fulfill their need. So it's a targeted audience. You are doing something to a particular audience, which means is that if you have a solution that works for healthcare, the explanations that it will give to some particular verticals will be different. I mean, it could be different or it could not be different, but for sure it should be different because the audience is different, the use case is different, the explanation that they need that they are different and probably for some people is different, probably some people need some kind of explanations and some other people needs other kinds of explanations. So as I said, it's more related with perspectives. It's to who we are explaining that. So we have five angles that could be more, but here I wanted to highlight five angles. The first one is the regulatory perspective. So you need to write the explanation, like GDPR, you are probably, you should be forced to explain why the model did something. If you are using Uber or if you are using a fraud detection system in Europe, that GDPR is used, you need to do it. So it's not that you are forced to have an explanation on that topic. So you need to find something that can give this kind of explanations to regulations, which is could be understandable, could not be understandable, but it's what they need. And the other explanations could be related with scientific or for the person that is developing the model. If you are working with LLMS, you probably will need to understand why the attention layer or why the detail of the model did something. And this insight will help you to improve your model, to make it better and to improve your model. So if I go to these explanations to an end user, probably the end user will not understand what is an attention layer, of course. But for an AI developer of this scientific perspective, it's pretty important. And last one, there are the end users. So as I said, the people that is consuming your application, people that has no idea about AI that they need to understand or visually understand, okay, why did you do that? So as you can see, there are multiple ways to explain a model and you need to find the right one for each particular use case. This has a lot of details, of course I will not go into details, but the main idea to explain here is that you have an explanation, you would like an explanation. Okay, it's not working. You like an explanation, right? And this explanation, it's mainly divided in three parts. The first is data explainability. The thing goes, explain the data, as we said with the bias and so on, which is previously of your, it's before you are training your model, so you need explanations about your data, related with your data. And you have a family of things that the main idea is to remove the bias, remove the overfeeding, or try to avoid these kind of problems that you can have on your training model. Second one is the model explainability. Everything, as I said, related with the model, going in details of the model, trying to understand how the model works and how to make the model to perform better. And the last one, that is the one that I will go in details later, is the post hoc explainability. Once you have the model trained, once you have a model with node bias, once you have a model perfect, someone will need to know, okay, why did you do that? Or why did the model did that? So this is the post hoc explanations. So it seems confusing at the beginning, but it was pretty easy to read. But now, you probably know that the models that are out there, decision trees, regressions, transformers, GPT, neural networks. And the challenge that we have is if you like to get better or what is going on in the last years, is that we are improving in accuracy. We are doing models that are better, that perform better, like GPT, I mean, LLMs are with computer vision. Compared with five years ago, it was pretty different. So now they are performing much better. But the thing is that the interpretability or the explanation that you can get from those models, those models, it's almost impossible to understand what's going on inside. And it behaves like a black boss and it's for scientists and it's something that is really hard. We can understand how they work internally, but it's pretty hard to really go into details. So the transformers or even the neural networks, you cannot see what's going on inside. And the easiest part is the decision trees. So decision trees is something that you see. So you know which part of the model, which leaf of the model, the algorithm is following to give you an answer. So it's pretty visual. So we have a challenge. We need to explain these models are the most complicated models that most people is starting to use, like deeper learning. If we start talking about the 200, the 2000s, most people started to use neural networks. We started to use more neural networks compared to the decision trees because we like to solve a problem. So we are, most of the time, we try to find algorithms that solve our problems. If they are complicated, we'll see later, but they solve our problems. So the challenge that we have is what we can do with all these models that we have or that we are using. And this is another complicated slide. Seems to be complicated, but it's not complicated. So as I said, the Potshawk, which is you have the model and you have multiple approaches to try to find an explanation. You have mainly, we can separate in two families. One family is I have the model and for me it's like a black box and I would like to understand not how the model works, but how the model took that decision. So I will not go into details of the model. I will not touch the model. I will try to find a way to find an explanation using the inputs and the outputs. Just looking at that thing. This is what shop is. I will explain later, but this is this part of, but this is not working. It's this one, right? Shop. And the other common or most common use is lime. Lime, it's a bit different. Instead of having, as I showed before, these complicated models, let's suppose that you have a transformer or you have a neural network, the attribution models or lime or this family that is up there, they try to create a parallel model that is simple, that is understandable, that could be a decision tree, to try to explain what the main model is doing. So instead of using BERT or a transformer or a neural network, you can probably get the same results using a very explainable model. So and since the model is explainable, you can use this model to do the predictions instead of using the neural network. It could be more challenging because sometimes you cannot get the same results, but it works well, let's see. But these are two approaches. And within these two approaches, you have tons of multiple variations that some of them they use parallel models with shop and parallel models with an alternative models and sample of models. But this is our main concept or I go in detail of the model or I just see input and output. And this is what shop is. You treat the model as a black box, so you just see the input and the output. You have features, of course, in this case, HX, BP, but I don't know what it is, and BMI. And what you will get is the weight of each feature for one particular prediction, you can say. So in this case, I wanted to know something and it says that H was the most important feature and sex agenda was the less important feature in that case. But this is just technical, right? What you can do after, once you have this information, this is going to the beginning that is it important for the end user? Probably not, probably it could be useful, it could not be useful, but this is a kind of explanation that you can use. If you like to go deep in this shop prediction that is not using the model, we have a toolkit that is open source with its Intel explainable AI tools, which is, we try it, because one of the challenge that you have with this is that if you like to use Lime, you need to go to the Lime site or download the Lime GitHub. If you like to use Shop, you have to go to Shop. If you like to use any others, you have to go to each particular GitHub. What we are trying to do here is we are trying to centralize everything so you can use just in one place, you can use Shop, you can use Lime, and you can use multiple use cases. I wanted to do a demo, but I didn't have the time to do it because it could take time, but this is an example that is, let's suppose that we would like to know what the heart disease system, and we would like to know which features it was important to make that classification. So these are the inputs, and the label is at the end, which is if the heart disease is fixed, normal, reversible, which I don't know what it is, but it's the kind of labeling, right? So you train a model, as you normally do, you use TensorFlow, you compile a model, you fit your model, this is as usual, could be TensorFlow, could be PyTorch, could be PsychoTrain or whatever. This is the model that you get, of course, this is rated with TensorFlow, so it's as usual. And what you do after that is once you have a model trained, you ask for the explainer. So you use the API, and you send the model, and you send a particular part, or you send them the line of the code that you would like to get explanation. And what you get is the answer. As I showed before, we can see what was more important and what has less importance. So three table spoons we have. Know what it is, what's the most important part, or the most influenced feature in that particular decision, and old peak was less important part, what was the less important feature. And same as before, so it's just information that you can use or you cannot use, but it could be important to, even for you if you are training a model, you can say that these features are important, or some other features are not important. But it also works with BERT, or with GPT or these other models, as it's a black box, and you are not going in this black box, and you can use whatever model. You can use the latest model, whatever. In that case, I would do the same with BERT, so you train your BERT model, you download your BERT trained model, you invoke for the explainer, same thing. You send the data set and you say everything. This case is a text classifier, so you say it's like a sentiment analysis if it's positive or negative. So what you will get is, you get which parts of the paragraph or the text the model took as positive and the model took as negative. In this case, the red ones are the negatives, you see not bad, not good and so on, it's negative, and the positives are the blue. So this part of the, and the model usually waits this to see, okay, this phrase of this part is a positive or is a negative part, but this is also important to know if the model is performing well. In text, if you are doing that things, it's not able to detect ironies and so on, so you can see here if it's working well, the model with the ironies or it's not working so well. But this is another way to get explanations. But as I said at the beginning, we have everything that we developed is by ourselves, by ourselves. It's we need to create an explanation for us and we create those explanations based on what we think that could be useful for the audience. It could be finance, healthcare or whatever. But this is a human interaction, so you need to put the human interaction or the human-computer interaction in the middle of the wave. So you can have the best part of the best model to explain something, but if it's not useful for the end users, or they will not trust your model or they will not use your model for whatever you are trying to sell. So you need to add the human part here. And how we can do that is by as simple as just interviewing your audience, as simple as that, trying to see what they like, what they would like to see if it's useful or not. And this is what this paper did, which is a paper that this is from Intel Labs and it's an interview that they did and they picked this Merlin application. It's pretty useful. I mean, I tried to use it here and it's pretty useful. It's an application to detect birds. So in that case, I have a goose, a Canadian goose, I guess I'm from Canada. And it detects the goose, right? And the idea is to talk to the end users of this application and give them explanations and try to see why they think about those explanations. So the questions are if it's important, how they would like to use the explanations and if it's useful for them. They show these four explanations. But for me, for instance, the last one is the better, but because I'm more close to AI, but they propose heat map, example basis or concept basis or prototype basis. Heat map is like in computer vision, you can have the same explanations. So you will get that heat map off with the most important parts that the model see to take that decision or to classify or to identify the bird in this case. So they give the heat map examples. They said, okay, I took that decision because this bird is similar to these other birds. Concept base is similar to a shop. You get values, you get numbers by each feature. And the last one is, this is why it's what I like the most because it's highlighting which zones of the bird were important to classify the bird. Here we can see, okay, sometimes probably it's not important for the user, but this is what we can see. And the perception where they divided into people, into kind of knowledge people between high knowledge of AI and low knowledge of AI. And you see that the results were different. So for high, if you have a high AI, you think that the heat map is intuitive because it is for us, it is. But for a person that has no idea about the AI, it's related to weather for them because the blue, the red, so. And the other one is same with the concept bases. Concept base, this is great for us, for me because you have data, you have the features, you have the value and you have everything. But it's not so useful for people that has no idea about it. So stuff like this will go right over my head and make no sense because it's numbers and so on. So what we get at the end is we get a gap. We get a huge gap in the middle. And we have one solution, but we have multiple end users or multiple consumers of the solution. And we need to find a way to fill this gap. For instance, the creators and the end users, if they both know about AI, of course they are satisfied with explanations. But the users with low AI, they don't care about explanations or they would like to see practical examples and because they would like to collaborate with the system, okay, I see that the model is detecting that it is a particular kind of bird. Okay, I would like to help the model to improve or to make it better. And they are, they don't like it, they didn't like it to be honest. So we can say two things from this slide. The first one is it's perfect for people that have an AI knowledge. So we are doing something or we are usually building software or models that are good for us, not for the end users or for the people that has no idea about how to use AI. And mainly we need to fill this gap between the AI knowledge and the low AI knowledge. But we need to see it and the only way to see it is we need to talk to people, we need to understand what they need and something that for us is really easy to understand or really useful for some people it's not and they will be the consumers of our application. So we need to be aware of that. And okay, recapping, how does it fit everything? So you have the principles that I talked at the beginning with the pineapple pigs and so on. We have fairness, privacy. Once we have these principles, but explainable AI is something that it's like feeding those principles. And once you have everything in the middle, once we are working with everything almost, you will get the responsible AI or we will meet the responsible AI goal. As I said, it depends. Sometimes most, some of them are important. Some of others are not relevant for the use case that you are working. But you need to know that at least there are some steps that you should follow. We didn't talk about ethics. Ethics is another huge topic, but we didn't know any detail, but it also goes within this explanation. Conclusion, so we have it's known fits all solution. As I said, you need to find the right solution for you, for your use case, for your audience. The real use cases that have happened in this use case, in this example, can expose pitfalls in explainable AI existing methods. So probably we don't see it, but we need to do that. One of the most important is explainable AI has to answer the why, not the what. So we need to know why the model is taking that decision, why the model did that, not what or not how, because the users probably don't care about that. And the explanations, which is the last one, is very important, are part of your value proposition. Even if we need to meet with some requirements of GDPR, which is good, because we need to meet it, but if you're all building a solution, your value proposition is, okay, my model, I am totally transparent with you, the data set that I'm using, how I trained, it's not biased, and it's part of the value proposition when we are explaining that. Call to action, of course, we encourage the community to try to reduce, to help us to reduce the gap between the last gap, between the high AI knowledge audience and the low AI knowledge audience. As I said, there are multiple models, some of them works for healthcare, some of them works for finance, some of them works for multiple verticals, but what I used to see is they are not too much collaboration between most of them, so they are working, you have the finance or you have the healthcare community, but probably some healthcare problems were solved using finance models, so they can work for both of them, and there is no one centralized place. As I said, that could be a good project to try to centralize, but really to centralize everything in one toolkit, toolkit or place or project or something like that. And the collaboration here is also, I mean, of course, is key. And that's it, thank you so much. Any questions? Thank you for your time. What I was asking myself is for large language models, like JetGBT, which have a large variety of use cases, how can this topic be applied? I think it's a bit harder, and also expectations might be very different from different users. Yeah, usually with large language, good question, thank you. And I think for large language models, we have, when we try to use large language models, we download something that's generic, and we don't know what it was trained. Even if it says the model card, if you download for high defense, for instance, they say that, okay, it was trained on one particular data set and so on, but these models, they could be racist or they could, because they were trained in the entire data set of the world, for instance. And probably the ethics in that part is important. You need to be careful about, you need to add an extra layer, because even if the model would like to comment something that could be racist, you need to put this layer to try to avoid that. And it could be racist or it could be bias or whatever, because the models they are prepared for, they answer. And I see that this part is very important to use it. There are some toolkits that you can use for that to try to avoid that. And after that, if you like to use it in your environment, finance, healthcare, you probably will need to fine-tune that. And this part of the remove the bias and remove everything is also important to have your model just for your data, for your environment. But I believe that the most important part is to set the guard rules of those models, because you will check with all the models if you try it with all of them. They can all be biased, and they are biased. And they have a disclaimer that they say, okay, this model can be racist, so it's up to you if you like to use it or how you use it. So even if the model is great, you need to build this extra layer of security. Did I answer your question? Yes. All right, thank you. All right, all right. Thank you so much for your time and enjoy the rest of your day. Thank you.