 Okay, in terms of AI adoption and why we are talking about bias, you know, now one of the reason is, you know, in the early stages of adoption, right, you know, in terms of machine learning and AI, bias was not even thought about because everybody was just in a hurry to get the result, right? The outcome was more important than, you know, the quality of the outcome itself, right? But today, if you look at, you know, you can create a model maybe in 10 minutes, right? You may take a whole year to do the same stuff, right? Because what matters is the quality of the output you are getting. And one of the quality parameters is how good is your algorithms in terms of bias detection, right? In terms of making sure that, you know, if you give the, a new set of data, how it responds to the new set of data. And that's why we are here. And if you look at, you know, what we are trying to say is, when the AI adoption is increasing, one of the biggest issue in terms of the success of the AI itself is, because people don't trust the outcome, okay? There was a pretty good joke about IBM, you know, a couple of years back, IBM created a very simple application when you go to the airport. It tells you, you know, remember they do the screening, right? And at the moment, you know, basically the security guy will just stop you and say that, okay, I want to screen. And people thought, oh, this was bias because, you know, he looked at the person and based on that, you know, he will randomly pick. It was supposed to be random, but a lot of time, everybody, you know, who went through that screening process thought he was bias, right? So what they did is they asked IBM what they can do and we created a small app, right? It's like an iPad app and, you know, you just go through, tap in, you go straight or you go through the screening process. Now that app was like, you know, a few million dollars, okay? And someone put on the internet that, you know, it was written by maybe some interns or something, or maybe an hour, okay? So a million dollars for just, you know, a few lines of code, but that was not true, right? You know, inside, you know, we did a lot of stuff to make sure that we mitigate those biases and the very reason the application was created, okay? So what I wanted to say to you is, when you're embarking on the journey of, you know, creating a model, trust is very important, right? The correctness of the model is very important. It's not just about the model itself, okay? And that's what I'm going to focus today. Now for some of you who are new to machine learning, I heard a lot of you, you know, some of you said you are here for the first time, you are hearing something, okay? We don't have, like, you know, I'll say time to go through the whole machine learning exercise, but I just put up, like, what I call a cheat seat for machine learning, and I'll take one minute to walk you through, right? Basically, related to, like, for example, I want to predict about someone going to a college degree after 10 plus two, right? After your A-level, whatever you say. Now for that purpose, you know, because your data is already kind of categorical, right, you'll be using logistic regression instead of using basic regression, right? The same way, you know, if I want to do any kind of prediction, right? And basically the recommendation engine you see, where someone is trying to figure out whether you'll get a loan or not. For that purpose, we are using stuff like, you know, decision tree or the random forest kind of algorithms, okay? The idea behind no matter what you are using, the step what you go through is the same, okay? These algorithms are much more, I will say, trusted. The reason is when I'm doing a decision tree or a random forest, it gives me a pretty good, like, a diagram, like, on what basis the decision came, okay? So that's why if you look at, you know, the success of these algorithms are much higher as compared to deep learning algorithms. Because deep learning algorithms, you know, I'll talk about that, where they use the concept of neural networks and other things to, you know, come to a conclusion. And because a lot of things are black box over there, right? The trust is not that high. So, you know, if someone is starting, you know, it's good to learn, deep learning and all, but if you really want to be successful and if you really want to develop something, right, which you can put in production, which people can trust, I will say start from here, all right? So, yeah, so what I was saying is, you know, you got much better off trust and other things with this algorithm, because this algorithm itself to give you on what basis you came to a conclusion. But specifically the deep learning algorithm doesn't tell you, right? So here, we are going to talk about how do we, when we build a model, how do we develop trust, okay? Now, the beauty.ai application, which I showed you, you know, you can say, okay, even if it goes wrong, it was made for fun purpose, right? So, you know, doesn't, we'll have like a life-changing impact. But trust me, the algorithm which you build can have a life-changing impact, okay, just to give you an idea, okay? So this was one of the, you know, use keys regarding, you know, machine bias, okay? And this was really researched a lot, and what they figured out is just to, I'm giving you a summary of the result. So what happened is there was this algorithm put into like the, you know, criminal system, where they basically help you to identify, like how many years of imprisonment you have to go through, okay? So in this case, what happened is they were too tapped, okay? And this algorithm, which was being used by the police department, you know, looked at the picture and looked at the profile of the person as well. And based on that, it basically says that this guy is a low risk. And the other person, you know, Brisha Borden, high risk. If you look at the risk category, it's pretty high. So this is like on a score of 1 to 10, the risk basically tells you the algorithm was about what is a good chance that this person is going to do another crime, okay? So they waited, you know, high, okay? But what really happened, right, after, like, you know, you can't look at in the future, but you know, after some time you can look in the past and see what happens. So this is really what happened, okay? This guy, whom they risked, like just a lower risk, already has some past experience, okay, to armed robbery. And he did some subsequent offense as well, okay? In this case, there was no offense, right, either before or either after. So what it shows is your algorithm was totally wrong. And it happened because the bias, right? So what happened is the algorithm was trained based on historical data set. And one of the things which happens is, you know, even the beauty.ai, what happened is they trained the model based on a certain set of data. And this data was mostly from, let's say, you know, European countries or America, right? And that's why this bias happens, okay? So what I'm trying to say is it's not just fun, right? The trust and the bias, it's a very serious, it's a life changing impact on someone. So when you're developing your code, you need to be a responsible person. And make sure that you abide this on day one. Now, here I got a life cycle of how you make sure that there's no bias in your code, okay? Pretty complex stuff, but I'm trying to simplify. Not oversimplify, but trying to simplify so that we can cover in this session. So we talked about building a model. Let's say we are building this model for the same stuff, right? Trying to find out whether this person will do a crime again or not. The first thing is, of course, you have to do some, this is a life cycle of your machine learning model development as well. I've just added the bias steps as well over here. So first you do the preprocessing. What I mean by preprocessing is, you may get data, right? For example, if I'm looking at some web references, I may be getting the data and some HTML format, some JSON format, something else, right? What you need to do is you need to make sure that the data can be utilized for your algorithm. That basically means either you're doing the categorizations, right? Sometimes you're doing a standardization, and sometimes just grouping itself. You're converting your categorical into a numerical value, right? Just to give an example, for example, if I have age of every individual, if you give machine learning an age itself, it doesn't mean anything. So what you have to do is you have to categorize this age into age group, right? And that's part of the preprocessing, right? So you have to go through the preprocessing, and then what you do is you have some training data. So you get a samples of data, and based on this sample, right? You split the data into what we call as a testing data, test data, right? And doing the training part, you train your model using those algorithms, okay? And after that, you are testing it, okay? And then usually you go ahead and deploy it. Now, there are a couple of ways where you can do the mitigation. One is, of course, when you deploy, you learn from the experience. So in this case, it's very straightforward. I didn't do anything in the past. I deployed the model, but I have a mechanism by which I can monitor the model, okay? Figure out if the model is biased. If it is biased, I take that feedback, do some cleanup, right? And we deploy the model. So if you're going with this approach, that basically means you're monitoring the outcome, each and every outcome, okay? Over a period of time. But this is more like I will say reactive approach, right? Because what is happening is already someone's life has been impacted, and now you're trying to mitigate it. The proactive approach will be before even if you go to that step, right? You have the training data, okay? Do what I call as preprocessing on that training data and try to go through data, what I call is going through algorithm that mitigate biases from the data itself, okay? What it means is, let's see if I give you a data, right? And I'm just trying to simplify in the sense that, let's say I give you a data. And we are trying to say that, okay, the earlier example I took, how many people will go from 10 plus two to college, right? And if I'm picking a particular city, okay, let's say Singapore. And I'm going to use this model somewhere else, some other part of the world. Already I'm biased because over here, right, the living style, the economic is very different than the rest of the country. So when you're mitigating this kind of bias, what do you need to do? You need to generalize your data set, okay? You need to make sure that your data set is general enough to represent a larger population, okay? So that's what you do into this. If it is very simple, right, you can do that, but certain point of time you'll find that it's very complicated, okay? I will start from here, right? The proactive effort and then we'll go to the reactive effort. So for the proactive effort, we put something called AI Fairness 360. And this is a open source toolkit, okay? So all of you have access to this, okay? What the AI Fairness 360 Open Source Toolkit does, basically it gives you more than 70 different algorithm which can help you find risk in your data set itself, okay? Also, it provides more than 20 different matrices, which will basically tell you, right, whether if you develop an algorithm based on this data set will have a problem or not, okay? Also, you quickly, you know, how you can use this. So let me go to this. So this is the URL called AI Fairness 360. I'll put it on my PowerPoint if it is not visible. It's called aif360.mybulomics.net, okay? And this assets was developed by IBM Research, okay? And it has been there for last, like, we published it. We open source it six months back. We're getting pretty good, you know, interaction and what I would suggest if you are into open source, please contribute back. Use this and contribute back to this as well, okay? Now let's look at, you know, how this helps you to mitigate a risk, okay? So what we provide you, the API, we provide you the algorithm. But for this sake, you know, just to make it easier for you to understand, we also created a small demo. I'll just walk you through, this demo is available, okay? I'm going to use this, you know, this data called German credit scoring. And this is the same data sets we are going to use in our lab, okay? So I'm just trying to make sure that you get to understand this data set better. In this case, I have, like, you know, basically we are predicting individual credit risk, very common thing, right? We all go through that. The banks are doing on us. So we are going to do a credit risk prediction. Now, in this case, the first step is checking the bias, whether the data is good or the data is having some bias, okay? Now, how to find out bias, right? And this is not like rocket science, but again, it's a lot of mathematical, you know, equations as well. I will take the simplest one, okay? I'm not going to explain the complex one, the simplest one. So let's say, for example, there's something called a statistical pavity difference, okay? This basically tells, like, okay, there's this concept of privilege group and unprivileged group, right? So what you're saying is, let's say, you know, in our beauty.ai, we said, you know, the guys with the whitest skin is a privilege group, other is unprivileged group, right? Or we can say age, right? This algorithm, the statistical pavity difference, basically tells you, right? In terms of the outcome, where you are biased, okay? So basically, if you're getting a value of zero, that means you're neutral. If you're getting anywhere, then basically it means bias. So what we did is, we created more than, you know, 70 different, you know, such matrices, right? And over here, like, on this data sets, when I apply this algorithm, what we do is, we apply not just one algorithm, but multiple set of algorithm behind. And this is giving me a few matrices, right? Which will be, whether this data set is a problematic data set, or it is good to use, okay? So in this case, it says that, okay, statistical pavity difference, you know, the difference is 0.01. Means almost it is fair, right? But in certain cases, you know, you'll find that, you know, dis pavity impact, or in case of, you know, thrill index, right? I see, you know, a much higher or something. So if you find, in this case, you know, the data set is like, if you look at, you know, for, we are looking for two different entity, okay? One is the sex, right? Whether it's a male or female. So if you look at over here, it's basically that if I'm looking at protected attribute sex, right? You know, everything is pretty okay. The data set is fair, right? But if I'm looking at age, right? In this case, what you find is the data set is highly biased, okay? And this is not a rocket science, you know, means, you know, people figured out very easily why. Can anyone tell me why? Why the data, so this was actually based on real data set, right? The credit score of lot of people in Germany over a period of time. And they found that the data set was biased against age. Why? Exactly, right? Because lot of, you know, young people were applying for the credit card. And this is the data, you know, they had, sorry, other way, not lot of young people were applying, right? Because most of the time, by the time you make money, you know, 10 years back, you're already into a certain age group. So this is why it shows bias against, you know, younger population, okay? So now we know that, okay, there's some bias over here. How do we mitigate the bias? So this is where, you know, you go to this next process, okay, you can check some of those, okay? Now I have a different bias mitigation algorithms over here. I can look at, you know, which bias mitigation algorithm I'll be using it, right? One of the simple thing is re-weighting, right? That basically means, okay, earlier, if your age was, you know, 30 to 35, I was giving you a risk of, let's say, five. Now I'm making it maybe a bit lower, right? A bit higher, depends on the use case, right? So that basically means I have those 10 attributes which identify a result and I'm re-weighting them, okay? So that my data becomes much generalized and it supports every age group, okay? So that's what, you know, you can do as well. Now if you do manually, you have to prepare the data sets. In this case, what happens is I take the data sets, I pick up the algorithm and the AIF 360 helps you to generate the new data sets for you, okay? It will take some time, you know, to do that. And this basically saying that, you know, you have a certain set of data, you check if there's a bias on certain protected group and if you find the bias, you run through the next set of algorithms which mitigates and give you a new set of data and this new set of data is good for building your algorithm, okay? So very, you know, strong and very powerful, you know, set of algorithms over here, it makes your job much easier instead of you trying to, you know, change manually the data. And a lot of time it happens, right? If you look at, you know, even when people, machine learning was not popular, people say statistical analysis, there the sampling was very important, okay? And they were trying to take the right set of data for the sampling, right? Here, you're doing the same, but what you're doing is, you know, if you need to make some modification in the data sets for making sure that your algorithm is not biased, we are doing that as well, okay? So now once I have a new set of data, you can see that, you know, you can compare, right? I'm not going to the, this part, I look at the age part because this was where the bias was. And, you know, if you look at, even after certain changes, I'm still seeing some biases, okay? So what I can do is I can go back, had a different algorithm, so re-weighting is not helping me, okay? So I'll go back and pick the other algorithm and go through that. When I see that, this bias is removed or very close because it's not possible to have a perfect world, okay? You're never going to have an algorithm will be just perfect, right? You'll always have some issues, but we are trying to make sure you close, come as close as possible, okay? I'll pause over here for a moment. Any question on this? Yeah. So we're getting a new copy of the data set, right? And we are really changing the weight, or, yeah, okay? So different algorithm does this. In case of re-weighting, we are changing the weight. So that basically also means, let's say, you know, if I had like five records, right? I'll introduce, you know, 10 dummy records, right? So that will automatically, you know, when you're doing the analysis, right? Because you have more record of such type, your weight will change. That's what it means. You can say that, you know, some means, it's a similar approach, right? But what you're doing is we are looking more from a data bias perspective. Like there the, you know, objective is different, right? Here we are trying to look more from the bias perspective. Yeah, right, from the existing record set. Yeah, you can look at, you know, so that's why, you know, I said this is open source, the data set is open source. The algorithms are all open source. You can try yourself, right? And just bookmark this, aif360.mybloomix.net, okay? Let's go back to the other part of the agenda. So this was like proactively trying to clean up your data. Now let's say, you know, I built the algorithms. Now I put up in a production. What else I can do? Okay, so I went through this. Now for the second part, right? Where we talked about, you have built a model, right? You put it in a production. You deployed the model. You want to check for the biases. For that purpose, I like to introduce, you know, our technology, what we call as open scale AI, okay? And this is a technology which we, you know, is a product actually, which is available on the cloud and on-prem as well. Okay, that basically means you can do with your secure set of data as well. And the concept over here is you build your model, you deploy your model, and after deploying the model, you decide what are the protected attributes, what are the unprotected attributes, and based on the result, based on the outcome, it will continuously monitor your model and tell you about any kind of biases. Now it doesn't just, you know, do the monitoring, but also it gives you a kind of, I will say, insight, which is helpful in a production environment. So you can set a threshold, right? So for example, I can say that, okay, if the bias on this, let's say age group becomes, you know, goes beyond 80%, right? Then send me an alert or stop this model for running further, right? That is very important. So you can put those threshold over here as well, okay? So apart from the model, you know, the bias part, it also is giving you another important of trust, which is a lineage, okay? So what we mean by lineages, like when you do a prediction, there may be, you know, 20 different data elements which you pass to do the prediction, but what was those two data element which was most important for the algorithm to make decision, it will show you that as well, okay? So normally if you're doing, you know, algorithms like decision tree and other, it automatically shows you, right? But those are meant, you know, for the guy who is building the model itself. This is something we are targeting also for the guys who are consuming it, okay? So what I mean is you see a credit risk and you want to know how this credit risk was applied, right? And this is where, you know, today it's not happening, but you know, going forward, we see this future where you can very much ask, right? The bank, okay? You came up like you, I requested for a loan of this amount. You gave me only this, why? Okay, because the decision will be made by machine, they need to justify through the machine, right? And this is where the open scale will actually give you a pretty good idea, like you had these three things, maybe your last loan was rejected and that's why your credit score goes down and this is why this is your score, okay? So that's what, you know, it does. So two things, one is the lineage, one is giving you the, you know, the bias reports and mitigating the bias report. Now, how do you do that? Like, let's say you want to build such a thing yourself, how do we do that? So there are two parts, three parts to this, one is developing it. So this is where we are going to, we have a tool called Watson Studio, okay? And specifically, if you look at doing the DevOps process for a data science and AI, it's all about, you know, open source, okay? So here we build a platform using open source, right? And this is like, you know, whole, like if you're looking for Python, you know, you're looking at Anaconda packages, whether you're looking at TensorFlow, you know, more than 75 different, you know, AI framework, the machine learning framework we embedded into this and we build a development process where you can actually build the code using your favorite tool. So whether you're using R, whether you're using a Jupyter notebook, or whether you're using R Studio, right? Japlin, okay, you know, you can use this to build the code. So once you build the code, the next part is how do I do the deployment into the production? And this is where we have something called Watson Machine Learning. This is a platform for you to deploy your model, okay? Building the model, you can build through the Watson Studio or even some other tools, like for example, if you're using SAS or if you're using, you know, maybe, you know, just your notebook on your laptop, right? To build the model, that's perfectly fine. When you have to deploy the model, you want to deploy in a way which is skillable. It is easy to consume by the application. So when you deploy the model in Watson Machine Learning, two things is happening. One, a very secure way of assessing your model. So what we do is we provide you the rest API for anything which is deployed over here. That means as soon as you deploy the model over here, it's ready for consumption, okay? So anybody, you know, from your mobile apps, from your IoT applications, right? You can consume those model in a very secure way. So we have this authentication mechanism, another thing, all built in. What it means, this is very important. What it means is from your built to deploy is just a matter of few hours, okay? Until now, the built to deploy was a matter of at least couple of weeks. Why? Anyone over here? Why the built to deploy is a big challenge? You think something? Okay, so you got a spot on, right? So basically, if you look at, you know, when I'm building this model, right? You know, I may be using different kind of library. When I'm building on my laptop, right? I have like, you know, Python 2.7 is still on my Mac, and I build using that, right? When I'm doing it into the production, what happens is that machine may not have the same sort of library, right? Then what do you do? You try to figure out the library, right? And a lot of times, it happens that the code which you wrote for one package is not available into other package. And in open source, it's happening every day, right? You know, all my code in Python 2.7 doesn't work in Python 3.7, right? So a big change. So what we do is, we use this concept of, you know, we had this K native session, right? And most of you were here. You understand the concept of Docker, right? So we are using the same concept over here. So when you're building this library over here, instead of, you know, giving you the code on your host machine itself, we provide that as a containerized image. That means, you know, it gives you flexibility to build a model using different version, right? And in one tool, you have different environments. So I can have, you know, Spark, you know, to Spark 1.6 if you had some old codes as well, right? All in a single environment. And when you're building, going to the deployment part of it, I can use the same container to deploy it over here, okay? And you don't have to do anything manually. Automatically, as soon as you do the deployment, the containers are pushed to the production machine and you're all set to go. And that's what I'm saying, you know, from few weeks, three to four weeks depends on the maturity. Some people may take three months as well, right? Here we are doing the development to production in just a matter of few hours. Yeah, yes, it does allow you because, you know, even though we provide you like pre-built containers, five, six containers, but that's not good enough, right? You know, someone may come up with some new set of algorithms, right? So then only I saw one of our research team built an algorithm which basically fast track, like the performance of your normal logistic regression increases by 45 times. And that's their proprietary algorithm, but they want to develop on this platform. So they can containerize it. So what we do is we provide our toolkit, okay? So you can download the existing environment we have, right? And you can modify it, push it back over here. Okay, so you can very much bring your own environment, okay? So that helps you to do the production part. Once the production is done, right? You know, the next part is the monitoring thing. And this is where the open skills comes into the picture, right? And this, as I mentioned earlier, we are talking about bias, right? Which is the fairness part of it. Then we are talking about lineage, which is the explainability part of it. Apart from that, it also look at the model health, the model accuracy, right? And also the performance as well. Let's say for example, or 10,000 people suddenly start hitting your API, right? Of course, the response time will get slow. It will tell you whether the response time is slowing or increasing. The best thing is based on the feedback. So today you have 10,000, you know, concurrent user, suddenly it become a million concurrent user. Because we're using the container technology to deploy over here, automatically it will scale more containers and it will take care of your workload as well, okay? So that's why the build, deploy and monitor, right? So we are providing this with a set of tools to you and these are the tools we'll be using in the lab as well, okay? A bit more around this open scale and then I'll move to the lab part of it. So as I mentioned over here, it's doing your model explainability, right? One important thing is to use the open scale, it doesn't mean you have to deploy it on IBM Watson machine learning itself. You may have deployed on, like let's say, what about the existing deployment? So if you have some existing deployment, open scale can connect even to that, okay? And we do two way, either you give us the locks or either you give us the URL, okay? So if you have deployed it, there will be an endpoint URL. If you give us the endpoint URL, we can connect and start monitoring those algorithm. Of course, there are certain limitations, right? Because normally, you know, those kinds of bias and other things work pretty much well with logistic equations kind of algorithms and some of the things I talked about like recently. We are building up to match this, means basically generalize it to more common algorithms as well. But right now, you know, when you're using it, it supports the 70% of the algorithms which I talked about, yeah? Yes, so Kidman, you know, just put his laptop aside, I'll let him answer it, okay? So we have this, you know, whatever you have over here, we have a solution which we call as IBM Cloud Private for Data, okay? Which is like our container technology along with the applications, the data and AI applications all bundled together. And that you can deploy anywhere. You can deploy on your, you know, data center. You can deploy on Google Cloud, Microsoft, you know, anywhere, right? So the answer is yes, okay? So apart from, you know, the open skill, the other technology, right? And this is where I talked about, you know, deep learning in the beginning. Just wanted to close onto that. So deep learning, if you look at the most common use case for that is around two things. One is around basically picture classifications and the other is around natural language, right? Understanding. So TensorFlow, you know, how many of you heard about TensorFlow? Most of you heard, right? So it has been a common way and if you are doing the stuff right, you know, it takes a bit of effort to do some coding over here. There, right? And then you can build the model, the first thing very easily, but if you have to really get to a level where the algorithm is very accurate, it takes a lot more effort. So to simplify that, we came with something we call AI for AI, okay? And Kirtman is giving a talk later today at two o'clock, I think, which is about will AI take over developers, right? This is like, you know, just a preliminary stuff around that. You know, what here we are talking about is here we have a tool where you give an image, right? And it automatically built the whole neural network for you. Okay, of course, you know, when machine is doing and also this is just, we just got it started, right? So don't expect magic, right? But you know, this is doing pretty good job. Like if you give, I tried with some natural language, a classifier, and if you give like a set of text and say, you know, classify as a positive, negative, or classify based on the sentiment, right? You know, you can get the outcome in, you know, like maybe an hour or something, you'll be all set. Whereas you'll have to build the code, you know, it may take some more time. So that's a new net. And in the lab, you're going to talk about that as well. So that was my chart. Let's go to the hands-on workshop, what we're going to do on the hands-on workshop. So this is our scenario, right? So let's assume that we are a credit card. We are going to do the same thing, like a credit risk analysis. For that, I'm using the same data set which I showed you earlier. So we have certain population data from the past where it automatically classified, whether this guys are, what is the credit risk for them? Okay, so we are going to use that data. And based on that, we are going to create a risk model, okay? And on this risk model, we will deploy our open scale to figure out, right, if there is any kind of bias, okay? We are not going through the AIF 360 where we were mitigating the bias, okay? Here we'll build it, just deploy it, and then look for if there's any bias, okay? So we'll be doing mostly on to the monitoring part of the bias. So how does this work? So we are going to use, you know, okay, let me go back over here first. So we are going to use these four tools. One is, you know, Watson Machine Learning, which will be basically used for the deployment of the model, open scale for the monitoring part, and in the backend, you know, our data set is in a DB2 warehouse, and for processing purpose, we are going to use Apache Spark, okay? If you, you know, most of the codes are, like, already there, so if you're new, just, you know, try to spend some time going through the stuff, and you'll get it, how we are doing it. We are using the, this is openly available data sets, so why I'm bringing this, you know, just to give you an idea, like we are not using anybody personal data, this is open data sets what we have, and we are using this open data sets for the algorithms, okay, and these are certain, like, data elements which are available over here, which we'll be using to predict, and in this case, our protected group is going to be personal status and age, on which we are going to do the prediction for, which are going to monitor for the biases, okay? How the things will work in production, once the model is deployed, there'll be request, right? And this request goes to the Watson Machine Learning, where it is deployed, and the AI open scale is going to look for, you know, the training, look for the biases, and as I mentioned, if it finds there's a bias or something, which is happening over a period of time, we can look at a new training data sets and try to re-evaluate our model so that the problem can be fixed, okay? With that, I think we are all good to get started, okay? So I need a couple of things from you, everybody connect to the internet using this Wi-Fi, if you haven't done yet. So this, you want to join, and do you want to walk them through the content? Okay, where do I have? So let them note this first, and then I'll show you, just note this guys, because I'm going to show you another URL where you need to download some content. Let me know once you are done, okay? You guys are smart, so 30 seconds should be good enough to note this down. Let me go to the other URL. This was the one, right? You gave me that shortcut, okay? Where did you give me? On WhatsApp? Or in the PowerPoint? Did you copy it or no? I don't want to open WhatsApp here. Okay, I'm just noting it down over here, so you can note it down yours. Okay, don't note it down, let me just make sure that it works first. It was sttps-bit.ly2-d-r-i-l-capital. All right, this is correct, all right? So sorry, I have to ask you to do a bit of work, so I want you to put this on your browser, okay? And this will take you to a folder where you can download the content, okay? So once you get to this URL on the top, you'll find that you got certain data sets over here. Then we have two labs, okay? We will request you to start from the lab one, okay? The lab one basically, you know, goes through the building the model, deploying to open a scale, monitoring the model. Once you are done, then you can go to the lab two, which is about building a deep machine learning using the new nets, okay? The account information all there in the, okay? You want to go? Okay, I'll ask, you know, while you do this, right? I'll ask so there's just to walk you through the steps, how to get started on to this. Actually, so once you open the lab one, right? So you'll see there is a scenario that is given. But in terms of practically doing it, from the previous session, you already have a registered IBM cloud account, right? So you go into that account, right? And you'll see there is a catalog. Are you logged into cloud? Are you logged into the cloud? So in the previous lab, you didn't actually really create any services, right? So whatever was given, you just continue with the exercise because the clusters were already given. In this lab, you'll be creating these services. So, you know, once you go into the main catalog here, you'll see you have this whole IBM catalog, right? So what we are doing is for this exercise, we are using some of the APIs, right? So if you click on the AI tab, right? So you'll see there's all these different APIs. So for example, one of the services that we are using in this lab is let's say the machine learning. So you click on the machine learning, and then it'll come up with some name and some details about it. So what you do is you choose the light version, which is a free version for your account, and then you just click a create button, right? So that'll actually instantiate a machine learning service for you, which is needed for the lab. So, okay, so one more thing. So in your last exercise, some of you would have changed this account to IBM, right? You change it back to your name. Suppose you have logged in as ABC at yahoo.com, you change that back to your account, right? Because you may not have permissions in the previous account to create services, right? So you change it back to your account, okay? First thing, everybody is able to download the PDF. Everybody is able to download the PDF, right? So once you have downloaded the PDF, right? So you make sure the first step is you're able to log in to the IBM cloud, right? So that's the next step. So once you are there, you'll see you need to create some of the services, right? So make sure you are in the right organization. In your case, it'll just display your name here, right? Or the email address. Once you are there, you click on, let's say the first thing is instantiate Watson Studio Service. So you see here, there's a Watson Studio Service. You click here, right? It'll come up with some name here. And then make sure you're always in, for this exercise, make sure you're in Dallas. And then once you're done, you just say create. The same thing you do, you create multiple instant, you create multiple services, Watson Studio, Watson Machine Learning, Watson OpenScale. Just follow through the lab. You should be able to get it through. Because there's no code that is needed to be written here. You just follow the scenario and then just progress. Okay, a couple of common things what I'm saying most of you are going through. So I'll just explain why it is happening. Some of the things you have to take care of while doing the lab. The number one is, I can't get the projector back, but I'll talk through. Make sure that all your services are in the same zone. Okay, so a lot of, a couple of guys I have seen when you're creating the services, the three services, make sure that they are in the same zone. Yeah, so that's one of the things. So over here, like if you look at, I have the Watson Studio. The other one which you need is the OpenScale and the machine learning. All the three services create in the same place. Okay, right now, we have this limitations where if you are running this OpenScale in a different zone, we have some network issues and other things. So you'll get some authorization or authentication kind of ever. Okay, so to avoid that, make sure that you get it in the same zone. So if you have done this by mistake, like if you have gone to some other zone, delete the services because you cannot create more than one services on the free account. So it will not allow you to create multiple OpenScale services. So delete the old services, create a new services and then you should be fine. Once you have done that, the next step is, the best way to move quickly is go to your Watson Studio, okay? And in Watson Studio, when you're creating the project, first you need to make sure that you have linked Watson Studio to the Watson Machine Learning, right? And that you do by adding it as a services. So what you need to make sure that you should be able to connect to your object to storage and the Watson Machine Learning services. There, the steps, what we have given to you is like going through the different experience, but typically what you're doing is either you can build your own model or you can go over here and say new Watson Machine Learning model. And when you do that, you have the option to select from the sample. So either you have your own model or you do it from the sample. In this case, the credit risk model is already there. So all you have to do is pick credit risk as the model to be deployed, okay? Give a name and that's all. Once the model is deployed, you do the testing, okay? You got a REST API, the JSON interface to test it, right? Typically that is given just for the testing purpose. In real case, you'll be using that API and include in your front-end application, okay? Now, once you do that, when you're going back to your open-scale configuration window, there'll be bit of latency. And this is very practical, right? Because what is happening is, these are two different services and you're trying to connect, okay? And the idea about monitoring is not like instantaneous, right? We are looking at the history of bias based on the history of prediction. So it doesn't need to be real-time. That's the reason there's a latency over there. The typical latency can be like an hour. So if you go to the open-scale, it says that whatever prediction is available for only last one hour and the latency may be like five minutes, okay? So if you have done a single prediction by the time it is available in open-scale to monitor, maybe around five minutes, okay? So there's no harm predicting it a couple of times, okay? So you can predict a couple of times so that it has got more transaction in the open-scale. And once it is done, then when you will come to the open-scale, right? I'm going to walk you through this interface because some of you were not able to go to this step. And also considering that, you know, we are serving lunch at 1.20, okay? So I want you to quickly finish, don't miss your lunch, and then be ready for the next session. This lapse you can also do later as well. So let's go through this interface, what this interface does. So in this case, the first step is, you know, if you look at, you know, here they are a couple of a step. One is like configuring the model itself, okay? In this case, you know, when you do the edit, you'll pick up your existing Watson machine learning model, whatever you have deployed, and then, you know, you'll pick up, right, what is those bias columns you want to detect the bias, okay? Once you have done this configuration, one of the things you can also do over here, not the chat, you click on that model once it is deployed. And one of the things I typically do is because I want to see some more transactions, right? You have the option of, you know, add feedback data, okay? So you can give some feedback data over here. So what it will do is, it will do actually the testing on each one of them. So it's like a batch scoring, okay? And that gives you a lot more transactions so that, you know, when you're looking at the output in OpenSQL, it makes more sense. At least when you're doing the testing part of it. So normally I'll put some feedback data over here. Let's go to the, you know, the interface itself. So in this case, my model is a bit different than what you're trying to do. Your model was around risk, right? Credit risk. Here, I've got a model about drug test. So basically, in the scenario over here is, I've got a set of drugs called ABC, right? And a set of like disease. And we are trying to figure out if my model is biased against a certain population, right? Based on, you know, age or based on actually over here, based on blood pressure, okay? So in this case, you know, if you look at, I don't have any new set of data because I haven't done any training. So you look at the timeline. The way to look at over here is, this is showing historical, right? So over here, it shows the accuracy, fairness, right? And on over here, it shows over a period of time. So if you have transaction, lot of transaction, you know, when I will move around, I'll get a lot of transactions over here. But I haven't done anything over here. So I'm saying for some last transaction, which was done on March 8th, okay? Now here, basically it shows me that, okay? Let's look at this. I'll go to the details, okay? Now in this case, if you look at, I have the, you know, some summary over here. It says 1% of the group is normal, right? And there is certain bias as well. And if you look, you know, look for the data element, it will tell you, you know, what happens over here. So in this case, what it is saying is, if the blood pressure was low, right? You know, 26% per time, a particular drug was given. And what we're trying to say is, the problem is the same, right? You know, the symptoms and everything is the same. Why we are giving different medicine, right? So in your case, you know, we were looking at age and sex, right? In this case, we are looking at blood pressure as one of the element. And whether we are seeing that, blood pressure influenced the decision, okay? You'll also be able to see the transactions, where it will do the transaction explainability as well. In my case, you know, if you look at, you know, I have some, the model, and it shows me, like, you know, the performance of that, okay? If I had more transactions, you know, I can actually look at the transactions over here. So I don't have the transactions history over here, because typically, one or something, okay? If you, that's by default, I can change the parameter to keep it for a month or whatever I want. And when you click on a particular transaction, over here is the scoring, okay? It will tell you how the prediction happened, okay? So that's something you'll be able to look at in your stuff. Okay? Also, I told you about, you know, the model, this is your production model. But if it finds the, there's a bias, this is the good part. You know, it also give you a de-biased model, okay? So basically, it is, you know, modifying some of your, I will say, optimization parameter, right? And based on that, you know, it is creating, suggesting a de-biased model as well. If you want to take this model to production, you can take that as well, okay? So this was about your open-scale, and I apologize. I think some of you are having some issue with the services, okay? I think the latency from the machine learning to here shouldn't be more than five minutes when you're doing the configuration. But right now, make sure that you are putting up in the same zone. If you're putting up in different data center, we know that's a non-issue, all right? Also, one other thing I wanted to show you, like in this case, the model is deployed on our machine learning, you know, in our environment itself. But this is important, right? You can do it for, your model can be anywhere, okay? Your model can be deployed on Amazon, okay? Your model can be on Azure ML Cloud, right? The reason we are trying to do is because we understand that you're not going to bring, you know, whole thing just for the sake of using the open-scale, right? So if you are already using AWS or Microsoft, you know, services, you can still use this environment. The custom environment is nice. In this case, you know, even if it is somewhere else, right, it's, what you can do is you can give the individual a scoring endpoint and this way it will be able to connect to the custom services, okay? So that basically means, you know, I don't have to deploy everything in Watson machine learning. You know, you can deploy it anywhere and you can bring this stuff over here. And the new net thing, right? Because none of you did it, right? You know, what you'll do is there's a service in open-scale at the last. It will say new nets. And when you go to this, you can, you know, if you have provisioned the new net, it will say synthesize a model, right? And over here, it's basically two steps, right? One is you upload your data. So you give some certain data over here and then it will do the rest of the things for you, okay? So yeah, I'll really like, you know, thanks for your time. I hope, you know, it helps you to learn something new, right? And, you know, basically you can play with it. Even this account will be there for a month, this services. If you get wrong anywhere, just delete those services, create a new services, okay? And, you know, that's all. So we have another 15 minutes, right? At 1.15, we'll break, right? They are serving lunch down at 1.30, so we'll break around 1.15, okay? Thank you. It'll be all good, man. Thank you.