 Live from Miami, Florida, it's theCUBE. Covering IBM's data in AI forums. Brought to you by IBM. We're back in Miami, everybody. You're watching theCUBE, the leader in live tech coverage. We go out to the events and extract the signal from the noise we're here. Covering the IBM data in AI forum. John Thomas is here, a many time CUBE guest. He's not only a distinguished engineer, but he's also the chief data scientist for IBM data and AI. John, great to see you again. Great to see you again, Dave. I'm always excited to talk to you because you're hardcore data science. You're working with the customers and you're kind of where the action is. The watchword today is end-to-end data science life cycle. What's behind that? I mean, it's been a lot of experimentation, a lot of tactical things going on. You're talking about end-to-end life cycle, explain. So, Dave, what we are seeing in our client engagements is actually working with the data, building the models. That part is relatively easy. The tougher part is to make the business understand what is the true value of this, so that it's not a science project, right? It is not an academic exercise, right? So how do you do that? In order for that to happen, these models need to go into production. Well, okay, well, how do you do that? There is a business of, I've got something in my development environment that needs to move up through QA and staging and then to production. Well, a lot of different things need to happen as you go through that process. How do you do this, right? See, this is not a new paradigm. It is a paradigm that exists in the world of application development, right? You got to go through a DevOps life cycle. You got to go through continuous integration and continuous delivery mindset. You got to have the same rigor in data science. And then at the front end of this is, you know, what business problem are you actually solving? Do you have business KPIs for that? And when the model is actually in production, can you track, can you monitor the performance of the model against the business KPIs that the business cares about, right? And how do you do this in an end-to-end fashion? And then in there is retraining the model when performance degrades, et cetera, et cetera. But this notion of following DevOps mindset in the world of data science is absolutely essential. So when you think about DevOps, you think of Agile, but when you, so help me square the circle. When you think end-to-end data lifestyle, you think chewy, big waterfall. Yeah, yeah, yeah. So, but I'm inferring you're not prescribing a waterfall. No, no, no, it is. Obviously, so how are organizations dealing with that holistic end-to-end view but still doing it in an Agile manner? Yeah, exactly. So I always say do not boil the ocean, especially if you're approaching AI use cases. Start with something that is contained, that you can define and break it into sprints. So take an Agile approach to this. Two, three sprints. If you're not seeing value in those two, three sprints, go back to the drawing board and see what is it that you're doing wrong, right? So for each of these sprints, what is the specific success criteria that you care about and the business cares about? Now, as you go through this process, you need a mechanism to look at, okay, well, I've got something in development. How do I move the assets? Not just a model, but what is the set of features that you're working with? What is a data prep pipeline? What are the scripts being used to evaluate the model? All of these things are logical assets surrounding the model. How do you move them from development to staging? How do you do QA against these set of assets? Then how do you do third-party approval, oversight? How do you make, how do you do code review? How do you make sure that when you move these assets, all of the surrounding mechanisms are being adhered to, compliance requirements, regulatory requirements, and then finally get them to production, right? So there's a technology aspect of it, obviously, there's a lot of discussion around cube flow and ML flow, et cetera, as technology options, but there is also a mindset that needs to be followed here, right? So once you find a winner, business people want to scale, because they can make more money, more times they can replicate that value. And I want to understand this trust and transparency, because when you scale, if you're scaling things that aren't compliant, you're in trouble. But before we get there, I wonder if we could take an example of, you know, pick an industry or some kind of use case where you've seen this end-to-end lifecycle be successful. Yeah, so I mean, it's across industries. I mean, it's not just a specific industry related, but I'll give you an example. This morning, Wonderman Thompson was talking about how they are applying machine learning to a very difficult problem, which is how to improve how they create a first-time buyer list for their clients, right? So, but think of the problem here. It's not just about a one-time building of a model, right? The model needs, okay, you got data, understand what data sets you're working with, what are the data, what is the lineage of that data? Once I have the understanding of the data, then I get into feature selection, feature engineering, all the steps that are needed in your machine learning cycle. Once I am done with selecting my features, doing my feature engineering, I go into model building. Now, it's a pipeline that is being built. It is not a one-time activity. Once that model, the pipeline, has been vetted, you got to move it from development to your QA environment, from there to your production environment, and so on, right? And here comes, and this is where it links to the trust and transparency discussion. Well, the model is in production. How do I make sure the model is being fair? How do I make sure that I can explain what is going on? How do I make sure that the model is not unfairly biased? Right, so all of these are important discussions in the trust and transparency, because people are going to question the outcome of the model. Why did he make a decision? If a campaign was run for an end individual, why did you choose him and not somebody else? If it's a credit card, fraud detection scenario, why was somebody tagged as fraudulent and not the other person? If a loan application was rejected, why was he rejected and not someone else? You got to explain this, right? So it's not an explainability that term has a lot of, it's overloaded at times, but the idea here is you should be able to retrace your steps back through an individual scoring activity and explain an individual transaction. You should be able to play back an individual transaction and say, version 15 of my model used these features, these 100 features for its scoring. This was the incoming payload, this was the outcome, and if I had changed five of my incoming payload variables out of the 500 I used or 100 I used, the outcome would have been different. Now you can say, you know what? Ethnicity, age, education, gender, these parameters did play a role in their decision, but they were in within a fairness bracket. And the fairness bracket is something that you have to define. So if I can play that back, so first take fraud detection. So you might have the machine telling you with 90% confidence or greater that this is fraud, but it throws back a false positive. When you dig in, you might see, well there's some bias included in there. And so then what, you would kind of refactor the model? So a couple of different things, right? Sometimes a bias is in the data itself, and it may be valid bias, and you may not want to change that. It's an, and that's what the system allows you to do. It tells you, this is the kind of bias that exists in the data already, right? And you can make a business decision as to whether it is good to retain that bias or to correct it in the data itself. Now if the bias is in how the algorithm is processing their data, again it's a business decision. Should I correct it or not? Sometimes bias is not a bad thing, right? But you know, it's not a bad thing. So no, because you are actually looking at what signal exists in the data. But what you want to make sure is that it's fair. Now what is fair, right? That is up to the regulatory body or your business to define. Okay, you know what? Age range between 26 and 45. I want to treat them a certain way. And that's if it's a conscious decision that you as a business or your industry is making, that's fair game. But if it is, you know, this is what I want the model to do for that age range, but the model is behaving a different way, I want to catch that. And I want to either fix the bias in the data or in how the algorithm is behaving with the model. So you can inject the edicts of the company into the model, but then as long as, okay, and then appropriately and fairly apply that, as long as it doesn't break the law. Exactly, exactly. Which is another part of the compliance, right? Yeah. So because you know, this is not just about compliance. Compliance is a big, big part here, but this is also just answering what your end customer is going to ask. You know, I put in an application for a loan and I was rejected. And I want an explanation as to why it was rejected, right? So you got to be transparent. I got to be transparent. Exactly, exactly. And if the business can say, you know what? This is the criteria we used. You fell in this range, and this in our mind is a fair range, that is okay. That may, it may not be okay for the end customer, but at least you have a valid explanation for why the decision was made by the model. So it is not some black box making some. So the bank might say, okay, well the decision was made because we don't like the location of the property. We think they're overvalued. It had nothing to do with your credit. Exactly. We just don't want to invest in this. Exactly. And by the way, maybe we advise you don't invest in that. Yeah, right, right, right. The feedback loop is there. But you know, this is being able to find out for each individual transaction, each individual model scoring, what weighed in into the decision that was made by the model, right? This is important. So you got to have atomic access to that data. At the transactional level. At the transactional level. And then make it transparent. Or organizations take banks. Are they actually making it transparent to their consumers? Because everyone is. I know in situations that I'm involved in, it's either okay, go, or you know, but we're not going to tell you why. Everyone is beginning to look into this space. Healthcare is another one, right? Where you guy would love more transparency in healthcare. So this is happening, right? This is happening where people are looking at, oh, we can't just do black box decision-making. We have to get serious about this. And I wonder, John, if a lot of that black box decision-making is easy to not share information, healthcare, you're worried about HIPAA, financial services is so highly regulated. So people are afraid to actually be transparent. Machine intelligence potentially solves that problem. So internally, at least internal to the company when the decision is made, you need to have a good idea why the decision wasn't made. As to what you use to explain to the end client or to regulatory body, it's up to you. At least internally you need to have clarity on what decision, how the decision was arrived at. When you were talking about feature selection and feature engineering and model building, how much of that is being done by AI or things like auto AI versus humans? So it depends, right? So if it's a relatively straightforward use case, you're dealing with 50, maybe 100 features, not a big deal, right? I mean, a good data scientist can sit down and do that. But again, going back to the Wonderman Thompson example from this morning's keynote, they're dealing with 20,000 features. You just can't do this economically at scale with a bunch of data scientists, even if they are super data scientists doing this in a programmatic way. So this is where something like auto AI comes into play which says, you know what, out of this 20,000 plus feature set, I can select, no, this percentage, maybe a thousand or 2,000 features that are actually relevant. Two, now here comes interesting things, not just that it has selected 2,000 features out of 20,000, but it says, if I were to take three of these features and two of these features and combine them, combine them, maybe do a transpose, maybe do an inverse of one and multiply it with something else, or whatever, right? Do a logarithmic approach to one and then combine it with something else. XOR, whatever, right? Some combination of operations on these features generates a new feature which boosts the signal in your data. Here is the magic, right? So suddenly you've gone from this huge array of features into a small subset and in there, you are saying, okay, if I were to combine these features, I can now get much better prediction power for my model. And that is a very, and auto AI is very heavily used in the Wonder Woman example, right? In scenarios like that where you have very large scale feature selection, feature engineering. You guys use this concept of the data ladder, collect, organize, analyze, and infuse. Correct me if I'm wrong, but a lot of data scientists' time is spent collecting, organizing, they want to do more analysis, and so ultimately they can infuse. Talk about that analyzed portion and how to get there, and what kind of progress the industry generally and IBM is making to help data scientists. Yeah, so analyzers, you know, you don't jump into building machine learning models, right? The first part is to just do exploratory analysis, you know, age-old exploration of your data to understand what is there, right? I mean, people jump into the sexy bit first, and it's normal, but if you don't understand what your data is telling you, it is foolish to expect magic to happen from your data, right? So exploratory analysis, your traditional approaches, you start there. Then you say, okay, in that context, I think I can do model building to solve a particular business problem, and then comes a discussion of, okay, am I using neural nets, or am I using classical mechanisms, am I doing this framework, XGBoos, or TensorFlow? All of that is secondary. Once you get to exploratory analysis, looking at framing the business problem as a set of models that can be built, then say, what technique do I use now? And AutoAI, for example, will help you select the algorithms once you have framed the problem. It says, okay, should I use light GBM, or should I use, you know, should I use something else, you know, should I use logistic regression, whatever, right? So it is something that the algorithm, the algorithm selection can be helped by AutoAI. So, John, we're up against the clock. Great to have you. Wonderful discussion. Thanks so much. Really appreciate it. All right, good to see you again. Yeah, same here. All right, thanks for watching everybody. We'll be right back right after this short break. You're watching theCUBE from the IBM Data and AI Forum in Miami. Right back.