 Hello and welcome everyone to this Kubernetes AI day, which is running as part of the KubeCon virtual event this year. And hopefully, you know, we can all meet in person next year. It's always exciting to be part of KubeCon and giving, you know, talks, but also, you know, listening to a lot of exciting talks which are happening. So what are we here for? So myself, I'm Animesh Singh. I'm the chief architect for data and AI open source platform. And with me, I have Andrew. Andrew, do you want to introduce yourself? Yeah, sure. I work mainly on Kubeflow, integrating some electricity projects into Kubeflow and then some other like Kubeflow, KFETecton stuff as well. Great. So we are part of this team, right? And today here we are to talk about, you know, how to stand up for ethical AI. And what we mean is essentially, you know, with the proliferation of AI systems and more and more models being deployed in production, right? We want to make sure as we are building AI, right? We are bringing and building it in a trusted manner. And what it means is that, you know, we have techniques for bias detection and mitigation or being able to explain your model predictions or, you know, being able to detect adversarial attacks being generated on your models, right? So that's the focus of this talk. And then, you know, we are going to talk in the context of Kubeflow, right? And we are aware it's a very popular open source project. So both Andrew and I work for a group in IBM called CODE, Center for Open Source Data and AI Technologies. What you see is a very nice picture of our San Jose Silicon Valley lab. We have a cricket pitch as well. If you are into that, right, we can have tournaments. But it's an amazing place, right? It's surrounded by green hills and large part of our team is centered there, but definitely, you know, the team is very globally distributed across the U.S. as well as other parts of the globe. And we've contributed to open source projects, you know, across the end-to-end AI life cycle, right? So for example, you know, we are the second largest contributor to Kubeflow. We contribute heavily to TensorFlow and PyTorch, Spark. We have our own open source projects like Illyder, Data Asset Exchange, Model Asset Exchange. So essentially work on strategy open source projects which are, you know, being consumed by our products as well as take, you know, the innovations we are doing inside and move it in open source like some of the trusted AI projects which we are going to talk about today. Okay. And you might be wondering, you know, why this topic and why not, right? So I think we are all living in this, you know, unprecedented time in history, right? We are all in the midst of this global health crisis. And hopefully, you know, we are now all seen, light at the end of the channel, but still quite a long way to go, right? And as part of this within IBM, right? We have had our own brainstorming and discussions around, you know, how we can help, you know, make the situation a bit better, right? We have been also focusing heavily in terms of bringing trust and transparency into AI, right? But does it really matter? I think it matters a lot more in the times we are in, right? Even, you know, a simple Google search can show you that, you know, even at the time of this pandemic and crisis, what we have seen is, you know, the trusted AI techniques are all the more relevant, right? Because when you are actually, for example, at this point, right, we are doing vaccine distributions. How do you make sure there is no bias in that distribution system, right? Things like, you know, being able to get tested, right? How do you make sure that, you know, the people who are being approved and being approached, they are not being done and approved in a manner where we are discriminating based on sex or race or, you know, based on the nationality of someone. So how do we handle it, right? So within IBM, right, we have our own vision for trusted AI, but, you know, we are also making sure that this is something which we are aligning with in the industry. We are taking feedback, always expanding it. But when we began and at this point, right, our focus has been around four pillars of trusted AI, right? What we call robustness, fairness, explainability, and lineage, right? So what do we mean by these different things? Robustness means that can anybody tamper with your AI systems or AI models? Fairness means that is it fair? Is it not biased towards a particular race or gender or nationality? Can you explain your model predictions, right? What it is doing? If it is making these decisions which are impacting my life, can it explain its decision? Last but not the least, right? Lineage, can I trace back and make sure that, you know, if a model is giving life-changing decisions for the stakeholders, we can trace back all the way. On what data set was it trained on? What were the hyperparameters used? And, you know, what version of a particular framework was used so that, you know, you can create accountability where it is needed, right? So having, right, lineage is very, very important. Just to give you an example, right? Since 2008, right, nearly every arrestee in a Broad County in Florida, right, was being assigned a criminal sentence which was based on, you know, if they are likely to reoffend, it was based on not points compass algorithm, right? And based on research and you can trace the link here, right? Which was found that data set was found to be heavily biased towards non-caucasians, right? The other thing is explainability, right? Now, explainability is not only important for someone who is consuming the models. Obviously it's the most important for that particular person, right? If I'm being denied admission to a university or if I am being given a loan or, you know, I'm being accepted into a company or denied into acceptance into a company, the model should be able to explain to me, right? Very, very critical because these are life-changing decisions for me, right? I cannot buy a house, I cannot get into a company, I cannot get into a university. Thou shall explain why you did that. But more importantly, I'm not the only stakeholder, right? There are people in the middle, right? So there are government regulators, people who run compliance and safety. They need to be convinced that over a period of time what the model is doing, it's able to explain itself why it is doing and why it is leaning towards those certain predictions, right? And last but not the least, right? When we talk about adversarial AI, what do we mean? A very simple example could be like you are depositing a check of $151, let's say, right? But a hacker could adversarily modify your image and instead of, you know, $151, like to the machine, it appears as $757 because it's very easy to fool a machine learning system, right? And adversarily generate an image which can make a one look like a seven for a machine to understand. Now, that's undetectable to the human eye. So the person, you know, instead of who gave you the check, instead of having $151 debited from his account, he has now, you know, $757 gone, right? That's, you know, more in the financial industry, but if you look at, like, you know, the impact of this can be life-changing and more scary in examples like this. For example, you know, a lot of the self-driving vehicles are being trained over stop signs images, right? Now, either if these stop sign images are adversarily modified more explicitly or implicitly, you know, over a period of time because of the wear and tear, right? These images are not great. Or, you know, they have lost their actual accuracy. Then, you know, the way the AI model will learn to detect them will be less accurate, right? So that might mean that your vehicle, the self-driving vehicle, actually running over a stop sign, right? And you can definitely imagine the consequences of a behavior like this. So we talked about the four pillars of AI and, you know, gave you some quick examples why we think, you know, they're really, really critical in real world and to handle that, right? IBM actually created four projects and then moved it in open source, right? So projects around adversarial robustness for both, you know, being able to detect if the models are robust and being able to mitigate them against adversarial attacks, that project is called adversarial robustness 360. For fairness, we have launched a project called AI fairness 360. Again, you know, this is around bias detection and not only detection, but, you know, being able to mitigate bias in your AI models as well. AI explainability 360 is essentially, you know, being able to explain AI predictions right across for different stakeholders, right? And last but not the least, right? Project which we have not yet moved the code, right? But a lot of the research papers in the material that already being made public is around AI fact sheets, which is around, you know, creating this lineage and governance model so that it can trace back the origin of a model right from the dataset on which it started. So adversarial robustness 360, as I mentioned, right? It's a project that actually allows you to rapidly craft and launch attacks on your models. If they are found, you know, vulnerable, you can have defense mechanisms and defense algorithms which you can implement. There are quite a bit of different kinds of, you know, evasion attacks and defense and detection methods. You can assign metrics, core robustness metrics to different kind of models. So definitely try it out. It's one of the most popular projects which we have out there, right? We are also working jointly with DARPA in terms of advancing the tools and the techniques and the algorithms which are available here. The IFN is 360. A library to actually detect bias across different metrics. So at this point, we support around 70 plus fairness metrics across AI lifecycle, right? So what we call pre-processing, in-processing and post-processing phase. So that means, you know, even before you start creating your model, you should be able to coin this tool at your dataset and be able to detect if there is bias in your dataset. Once you have, you know, trade your model, you should be able to then validate that model, right? Which is in-processing phase. And last but not the least, right? Once you have deployed your model, right? You should be able to then, you know, monitor the model predictions and say, and figure out right if they are biased or not, right? So you can work across this lifecycle, right? And not only this, right? So based on these 70 plus fairness metrics, if you are able to detect bias, there are 10 plus bias mitigation algorithms which are provided as part of this, which essentially you can take and then, you know, mitigate bias across the AI lifecycle. And last but not the least, right? AI explainability 360, as I mentioned, right? There is no one explanation which fits everyone. There is the end consumer who is directly impacted by the model predictions. But then there are, you know, regulators and people who make sure, you know, things are compliant who are in the middle. There are next generation of developers who are retraining these black box models and positioning it towards a customized use case or towards a customized industry. Plus you want explanations at different levels, right? So you want to be able to also explain datasets and the features of the datasets, what do they mean? And then when you are looking at, you know, some of them are very simple models, some of them are black box models. So in case of black box models, you might want to have a surrogate model, right? Which is learning based on the explanations and creating a parallel model, right? Which over a period of time, as it is seen the basic explanations, right? It's creating its own robust explanation model. Now, apart from moving these projects and open source, right? So as a mission of trust area, right? As I mentioned, like we created this project and we realized, right? These techniques being propriety doesn't make sense. So they need to be in open source, but not only in open source, the whole IP of these projects need to be in a neutral place under a neutral governance model, right? So move these projects to Linux Foundation AI and they are now part of the Linux Foundation AI hosted in a neutral place. So hopefully that most of you have either used or using Qflow or trying Qflow or probably have heard about Qflow, right? So it's an end to end machine learning platform, right? Which goes across the whole AI life cycle, right? So you can essentially build a model using Qflow notebooks. You can then, you know, run distributed training on those models using different libraries like TensorFlow, PyTorch, XeBoost. We have MPI based algorithms like Horoword, et cetera, inbuilt as well. You can use hyperparameter optimization engine like Catev, which also gives you capabilities around neural architectures. So if you need more auto AI capabilities, and then, you know, you can deploy models in production, right? So using Qflow serving. So end to end spectrum and tying all these things together is pipelines or machine learning pipelines, which allows you to automate this whole end to end life cycle and not only allows you to automate, but also allows you to trace the lineage, right? So I'll briefly go into them. So the two projects, right, where we focused from the perspective of trusted AI are essentially, you know, pipelines and serving, right? And the reason, you know, for focusing on these projects was twofold. One, like when we talk about pipelines, what we want to do is, you know, make sure that as we are running through this AI life cycle, there are inbuilt capabilities into these pipelines, which can then help detect things like bias or, you know, the models which are being trained are they vulnerable to adversarial attacks, et cetera. So they can help you manage the whole life cycle and ensure that, you know, they're able to detect if the models are being built in an ethical and trusted manner. And serving, which is care serving, right, can then once your models are finally rolled out and deployed in production can then help, you know, to see if your models in real world are actually adhering to these trusted AI principles and you can monitor and create metrics, right? So it has the importance of these two projects. So very quickly on pipelines, right, it's the most popular project in the pipeline, Qflow umbrella, targeted towards data scientists. So there is a Python DSL also allows you to craft pipelines in a programmatic way. Each of these steps in the pipeline is called a run, which essentially is backed by a container, which essentially has the code, right, which is doing that specific part, right? And it's very, very popular in the Qflow community, right? Because for data scientists, they can now use a Python way of running machine learning pipelines on Kubernetes and they can also leverage underlying Kubernetes constructs like volume secrets and other things. But also, you know, for operational folks, right, they can actually go, there is a very rich dashboard which shows you the streaming logs in real time, et cetera. And you can, you know, look at the real time logs and debug. And not only this, right, there is a strong lineage in governance, right? So you can track back, like, once your models are produced on what data set it was trained on, what were the hyperparameters used, what was the version of TensorFlow. So very, very robust set of capabilities in this, right? And what we have done is, right, essentially created some pipeline components, as I was mentioning, right, pipeline components are the building blocks, which essentially allows you to run. And you can take these components around adversarial detection or, you know, bias detection, plug it into your own workloads. And then, you know, identify as part of that, whether your workloads are, you know, adhering to the Trusted AI principles and then you can detect if there are bias, et cetera being found. So with that, I will pass on to Andrew to talk a bit about Kubeflow Pipelines and Trusted AI and show you, you know, how we can leverage this in practice. So, Andrew, over to you. So basically, we're going to take a look at for first Kubeflow Pipelines and see some of the components in action in that. So we have two set up a model fairness check, which works with AIF 360 and then an adversarial robustness evaluation with the ART tool that we have. And the first training step we're going to do is on a gender classification. So it takes images and tries to classify a gender based off of that. So the way you would run this through is click creating run and then specifying some parameters that you need for this. You might specify the attack epsilon for the fast gradient. Sign method parameter are favorable label, which AIF 360 needs to know what is a favorable label, what they would like to see, and then you can also specify which groups you should check the bias for. So in this case, we've chosen a race with the label zero and the unprivileged group is the race of label four. So then we just can run this through. This experiment takes a while. So we won't watch it go through the whole way, but it'll just start up like this. And then some nodes will start to show up and run through, but we already have a pretty set up one. That is executed already. So we can kind of look at what has happened. So first, let's look at the training step. See what training logs we can get. We can see all this is going on. So we can see for the failure condition is evaluated default. So it hasn't failed yet. And then the success condition is also false. So it hasn't succeeded yet either. And it will continue to check that until one shows up true. And in our case, it succeeded. So we've succeeded in training it and then it'll. Store all the training parameters so that we can. Go back into the model fairness check and the average or robustness evaluation and kind of take a look at what we've got there. So here we see some of the metrics. And there's a bunch of metrics calculated. We're just going to look at one for time constraints. So for the disparate impact. So the way that the disparate impact works is that it takes the probability of a favorable outcome for the. Unprivileged group and divides it by the probability of a favorable outcome for the privileged group. So values of less than 1 generally indicate bias towards the privilege group. And values close to 1, 10 to show that there's no bias. So generally we go by the forefist rule, which means anything less than 80% shows some sort of bias. And so we have a 77% here, which would indicate that likely there is some bias here and we would need to go and mitigate it in some way. And then let's take a look at the robustness evaluation. See how that went. So the model accuracy on test data in general was 85%. And then on our adversarial samples, it comes out to only 15%. So this is not very good. It indicates that we would need to do more robust training to kind of deal with adversarial. And to go a little bit deeper into that, you can also get the average perturbation on misclassified samples. So small values show that your model did not perform well at all. And in fact, very little perturbation will cause a misclassification, which is not good. And if this label is very high, it shows that it took a lot to mess with your model and get it to mispredict. So you would prefer higher numbers. So yeah, in this case, you would find that likely you need to both check on the bias and to check on robustness in your model training step. And so you could go and rerun this training step, perhaps with new data that you've created based off of the model fairness and adversarial robustness data that you created in both of these steps. So jumping back in. Thanks, Andrew. That was great, right? So I think in the interest of the time, we will probably, you know, speed up some of the rest of the slides, right? So I think I gave a quick background around K of serving, right? So it's the solution in our QFLU community for production model serving, right? So it gives you a serverless machine learning inference capability built on top of K native, right? And then, you know, there are capabilities around model explanations under which we have built our rest of the trusted AI umbrella, right? So as part of that, right? So Andrew has been working heavily in terms of integrating some of these projects into K of serving, including, you know, explainability 360 or any FNS 360. And, you know, we had the first version or done and delivered as part of the K of serving these 0.5. So if you're interested, you can definitely try it out. A lot of times it's not possible to do the analysis on a single transaction, right? So you cannot just based on a single transaction say that the model is biased or not, right? So in those kind of scenarios, right? You need robust payload logging capabilities. What that means is that, you know, you're able to capture the requests which are coming to models, the responses which are going back from the models, collect, curate, and then, you know, over a period of time, once you have that data, then you can run more advanced analysis with it. Over the course of last hundred predictions, the model is biased, right? Or, you know, 80% of these inputs which are coming there, you know, adversely modified and model is getting full, right? So to do more advanced analytics like this, you need payload logging, which we have enabled using, you know, the cloud events standardized protocol, which is a standard which is being developed by the C and CF community. Okay, so I'm going to talk a bit about these capabilities as part of the KF survey. Yeah. So what we've implemented in KF serving so far for the AX is using a method called locally interval model explainability in Lyme. And the idea is you take an image on the left like an MNIST image and the explainer will highlight pixels that it finds are highly indicative of a certain classification. So you can change certain parameters inside of Lyme to get different highlighted pixels if you want something that is like the pixels that are most indicative of a classification, then you would raise a parameter called minimum weight. And then the next project, every server bus install box is using the square attack method. So you take an image, you add some sort of perturbation or noise. And you'll be given with another image, which is very similar to the first, but makes your model misclassify or at least is the goal of the art tool. And in most cases, that'll just be small perturbations that look identical to the human eye. For AF 360, the idea is to collect a bunch of payloads, as we mentioned, and then through the payload logs, we can create bias metrics based off of a large collection of instances and outputs. And so there's a number of specifications that you need to have here. Like we saw before in the KFP example, there's the favorable label, unfavorable label, privileged groups and unfavorable groups, and this just lets you know what is a favorable outcome and what are the bias groups that we may need to look out for. And so the same deal with the spec here. The left is the same as you would deploy a normal predictor. And then the right is all the extra commands and information that you may need in the explainer to give you something valuable. And so the flow of the AF 360 is you send your prediction request to a model, then the model sends their logs to some sort of persistent storage. In our case, when we show the demo, it will be a message dumper. And then when the bias detection request comes through all of the logs and the persistent storage will be sent to the AF explainer, which will then be able to calculate metrics based off of those logs. So we'll show that here. So this is housed in the KF serving repo. You can just go to the samples and look for it. But the basic idea is to deploy the model, the bias the predictor and the explainer here and the message jumper. We've already had that set up so that it doesn't take a lot of time to spin them up because they are fairly large images, at least for someone. Yeah, so there they are. And then the idea is to connect to it and send simulation prediction requests. So I have that here. So we'll just simulate sending multiple requests just to pull some logs into the message dumpers. You can see we have about six requests and this is the responses we expect to see a few predictions and both the requests and their response will get logged. And then the idea here is that we need to create adjacent from the logs. So let's just take a look at what that looks like. It's created this data.json, which is just adjacent instances, which are all the logs that we sent to it. There's quite a few because we added some information before just for added information without taking up too much time. And then the outputs here are all the prediction responses that we've got. And so now that we have those, we can go back and we can actually query our explainer to check what bias do we see. So there we go. And that's pretty quick. So now that we have that, let's look at the disparate impact. So we can have a kind of comparison to the KFP. So again, disparate impact is measuring favorable outcomes for privileged groups and non-privileged groups the ratio. So this is much lower than the fourth fist rule. So we have a very low value of 0.5. So we would say that the model that we have deployed in camp serving is very biased. And we, what we can do since we have the payload logs for this is we can go back and we can see what the biased examples were. And then we can check if our model behaves differently when we change those examples to make them the privileged class. And if they do, then we can save new data for when we would like to retrain and keep the same examples before but say that those examples should be marked with a favorable label rather than the unfavorable label that they're giving. And this will expand our data set and make it a little bit better because we know that if it is given to the privileged class they are getting acceptable results. So now we'll hop back in. Over that, right, we are also doing a lot of great work and advancing a lot of the principles of trusted AI as part of the Linux Foundation AI trusted AI committee. We meet, you know, monthly where, you know, a lot of the partners like, you know, the folks from the companies which are listed there or, you know, folks from Microsoft and others, right, we come together and talk both about the principles. What should define the principles of trusted AI but also, you know, technical solutions like we discussed today. Right, so we are interested in this. Definitely, you know, join this. It's an exciting place where a lot of us, you know, people who are actually interested in advancing this field are getting together and bringing technologies and tools and use cases together as part of this Linux Foundation AI trusted AI. Here are, you know, some of the links in terms of, you know, some of the GitHub repos or the design document, right? This is the KF serving payload logging. Plus, you know, if you do need to reach out, like, you know, or Twitter or LinkedIn, URLs are listed here. So definitely, you know, try and reach back to us if you have any questions, comments, feedback. We would love to get your feedback and make sure that, you know, you're using these capabilities, advancing these capabilities and the ratios which you'll find or your suggestions, we would like to incorporate and advance these projects together, right? So in there, you know, I would like to round up by saying we definitely want to stand up for ethical AI. We are doing quite a bit of efforts that are both as well as through IBM, but also, you know, Linux Foundation AI has been a huge partner, the two-flow community has been a huge partner and we would like to invite all of you right into this mission in terms of bringing ethical and trusted AI to the masses and building, you know, platforms which are essentially, you know, making this core part of their DNA. Thank you. And thanks, Andrew.