 Welcome back to SuperCloud Six, I'm Paul Gillan, and when you're talking about AI, it goes hand in hand with data science. Data science is the foundation of AI. And our next guest is a data scientist, a real live data scientist. Shell Carlson is head of Data Science Strategy and Evangelism at Domino Data Lab and is intrinsically involved in AI projects at a variety of large enterprises. Shell, welcome. Thank you very much for having me. Tell us a little bit about Domino Data Lab. Certainly, it is an enterprise AI platform that provides all of the tools, infrastructure necessary in order to be able to complete the end-to-end AI lifecycle. So being able to access the data that you want, being able to go in and share version code, be able to access the IDE that you want to use, the infrastructure, the distributed compute framework, enables you to go all the way from that data to deploying a model and then being able to monitor that model. But doing this across all of the major tools, all the major clouds, on-prem, you name it. So you've been working with AI for years, right? Do machine learning, deep learning, all the manifestations. But year and a half ago, everything changed with ChatGPT. How did the ChatGPT change the world for AI? I mean, so as you mentioned, I mean, data scientists have been really, really excited about these quote, unquote, transformer models. We now call them large language models since they were introduced in 2017. I wrote a report, I think it was in 2020, 2021, which was talking about AI 2.0 and how these things were the best things since sliced bread and how enterprises should be using them. What ChatGPT did was that it created an experience around this. And with all of that funding that they got, they could make it free and available to everybody. So now everybody realized, hey, this is real. This is usable. I can start actually implementing this in my organization, at least so we thought. And it was sort of like the spark that was heard around the world. It was the call to action. So now it's actually amazing time to be a data science leader and AI leader because everybody's excited about this. You're getting the investment, the willingness to change and do all of these things. But it's also the scariest time to be one of those leaders because there's so hyped expectations around this that you're running headlong into that trough of disillusionment. And six months to a year's time, folks are gonna be really asking, okay, well, so what have we been doing with AI? What have we achieved with all of this? And it's not really fair on the data science leaders and the AI leaders. So we've seen over the last year just a frenzy of investment in AI, every major corporation is doing some kind of AI project right now. What are the big impediments they're running into? Well, they're a little bit like the impediments that we had with traditional machine learning and quote unquote predictive AI since time immemorial just on steroids. The same challenges that we had around, well, we're not really certain on the business use case when it comes to generative AI, even larger. The challenges that we faced when it comes to managing data and with data privacy and security with being able to train models, with being able to ensure governance across that life cycle, being able to operationalize a complex pipeline, being able to leverage the latest and greatest technologies from different sources. That has always been a problem when it comes to data science and machine learning AI historically. When it comes to generative AI, it's just so much harder. The models are so much larger. The data challenges are so much larger. The innovation that's happening is so much faster. So for organizations to really get ahead of this, the ones that I see who are being most successful with this are actually organizations who have already very well-established data science teams and AI products and are able to go in and leverage that know-how for now this new category of gen AI applications. So those successful organizations already have the structures in place. What about culturally? What are these organizations? How do these organizations differ culturally? The ones that are getting AI versus the ones that are still in the starting gates? I would say they think about data and AI as products. So that these are things that are not just the insights, the one-off, it's not just the incredible individual who is going and pulling out some incredible information or some insight off of the data that they're analyzing. But instead, this is crafting what is an AI or data-driven application, something that is going to live and continue to exist, something which is going in and making decisions or automating actions on an ongoing regular basis. And this requires sort of a knowledge of, again, the data engineering, the how to operationalize AI machine learning models, how to monitor and test them, how to govern all of that process. And so that's, I think, where they're coming from. They have a maturity around creating these GenAI products and experiences that a lot of other organizations are just approaching this de novo, and they're hoping, okay, well, this is great. I don't need to worry about the model anymore. That's going to be handled by one of the cloud vendors or OpenAI. So I don't need to go in and train and develop it. I don't need to go and host it. So therefore, I can just give this to business users and developer teams, and they're going to be able to figure out how to create something magical out of this. And if only it were that simple, if only it worked that way. So is the fast follower syndrome appropriate here? I mean, should companies wait for the automation to emerge, for the vendors to take care of some of the startup problems that they're having? Is there any disadvantage now to being a fast follower? I wish I could say that that was a viable strategy, but it hasn't really been a viable strategy when it comes to predictive AI and traditional machine learning to begin with. There's just too much variation across every company's data, the systems that they're working with, the kinds of patterns they're looking to detect for there to be enough transference from one organization to another organization to another organization. This is the reason why we remember Salesforce Einstein and we remember those, we call them AI, but advanced analytics and machine learning case capabilities that were being embedded into CRM. It's the reason why they never really delivered. It was because even there where you've got control over that data and less variability of your use case, even there there was so much variation that there was a limit to how much value you were getting out of these things. When it comes to gen AI, there's certainly a hope and expectation that these models are more generalizable, but when you actually go in and operationalize them, the gen AI pipeline that you have to build when it comes to bringing in your data, putting that through embedding models, storing that in vector databases, passing that to other generative AI models, potentially getting consensus amongst generative AI models, potentially then summarizing that consensus with yet another one. That is vastly more complicated and unique to your organization. So on the one hand, there probably will be consulting firms and others that will come in and be able to go in and help bridge a bit of that gap of customization. But right now, all of the success stories that I'm seeing are organizations that have, yes, used external components, but have built very substantial internal capabilities to make all of this work. Sadly, I wish I could tell you it was much easier. Well, all the talk has been about LLMs for the last year, but AI is, of course, much more than just generative AI. What, outside of generative AI, what are some of the applications you've seen that impress you the most? Well, I guess I would like to back up and say it's a bit of a myth that there is such a thing as a generative AI application or that there should be a generative AI application. It is an application in itself. It's an application in itself and it's usually actually a combination of predictive machine learning models working with generative AI models, at least if you're doing it well. So to give you an example of this, if you were doing customer service automation, yes, you could go and ask the LLM to go in and classify what was coming in. Is this a complaint? Is this somebody wanting a refund? Is this somebody noting that there's something dangerous about the product that they receive or those things like? You could get the LLM to do that, but you'd actually get a lot more accuracy and to be able to do it cheaper and faster if you actually pull that out and had a traditional predictive AI model, a machine learning model to go and do that. When it comes to then saying, you could ask the LLM to go and say, well, what should the resolution be? Should I give them a discount? Should I go in and offer them a refund? But actually you'll get much better accuracy and again be able to do it cheaper and you'll be able to use much more customer data and other things like that. If you pull that out of the LLM and again have a traditional machine learning model do that and just use the LLM for what it's really good at, which is creating the verbiage around this, creating that conversation that's going on there but leveraging all of the other models for going in and making the decision around it. So to a certain extent, one of the really exciting things here is that combination of predictive AI models and generative AI models functioning to create these powerful GenAI applications. So I would highlight that, that first of all the idea of a GenAI application on its own is a bit of a myth. The other thing I would point out to as this, certainly the coolest thing that I've seen is what's happening with, they're not technically large language models, but they're the same architecture, they're the same transformer models and the bio-pharma space, folks are leveraging these in order to go in and accelerate the development of synthetic proteins for things like heart disease and diabetes. And I've spoken to five different bio-pharma companies and really they've all been doing this and they've all been doing this for in the one to two year timeframe, at least the ones who are advanced in this. And so they're taking something which looks like a really primitive LLM. It's a think of a BERT style model from 2018, 2019, which is trained on language data, but they're then fine-tuning it with a massive boatload of molecular data on top of this. So by the end of this, this looks nothing like what a traditional LLM is. But by going and doing that, you go in and not just have a model which can give you a prediction about a particular protein. You know, is this protein gonna cause an averse reaction or gonna be effective against the target? But if you're using this cleverly, it also gives you information about, well, what other tests should I be doing? Where's this model not confident? Where can I go in and boost the knowledge of this model? And so it ends up directing where you should do your drug discovery, what real-world tests you should be doing, which you then take that data and you feed it back into it, into the model, and then it goes and suggests more and more of these. So there's this really cool way that you're leveraging these models to go in and massively accelerate drug discovery, new therapies, but this approach also works for other types of unstructured data. So certainly the hope is that this is going to apply in advanced materials and elsewhere. But to me, this is the coolest thing ever because you're going in and really bridging across very, very different types of data and different types of problems. You're sort of implying that we may expect too much of LLMs. I mean, is there the risk that we, because they are so human-like, we think they can do more than they can really do? Oh, absolutely. It's very, very easy to go in and sort of think, okay, well, ah, so these things behave human-like, so I can go in and put them and leverage them everywhere where I'd like to use a human being. And almost invariably, the success stories, certainly the ones which are low-hanging fruit ones are ones where actually you're augmenting the intelligence of usually a very sophisticated, or very an expert human being. So thinking about use cases in the legal profession where you're augmenting a lawyer by enabling them to go in and draft contracts or go in and create legal briefs faster and be able to do the research for this faster than ever before. Or within the healthcare space going in and augmenting a life sciences researcher by enabling them to go in and understand the literature or be able to go in and identify more appropriate compounds faster than they were able to do before or in engineering, being able to enable that engineer to be able to find, access, summarize all of the data that they need in order to be able to say, for example, if you're a geological engineer, go in and identify effective candidate areas for oil exploration. So right now, very much think about these, I like to think about them as Ironman's suit. So I think of the Jarvis AI and Ironman suit, which is going in and identifying things that the person wouldn't have been able to go in and detect anomalies that they haven't seen or go in and pull information that the person cannot feasibly go in and synthesize. But also there's the suit component of this, which is being able to turn this into action in a way that, and at scale in a way that us as human beings, repetitive activities we're just not very good at. For organizations that are just getting started and they wanna do something safe, they wanna do something that they have good structured data for, what kinds of applications are natural starting points for them? I mean, I like to think of Clay Christianson's jumps to be done and so like, what would you hire an LLM to go and do for you? And when I've looked at it, there are a couple of different jobs. The top three I would say is a first draft generator. So whether that be code, whether that be draft interview questions, whether that be draft essays, images that you want to use for marketing content. All of those kind of things, these models are very good at creating that first, maybe the second, third or fourth draft of. And a starting point for humans. Exactly, a starting point for humans and with human intervention, even potentially all the way to the ending point of this. I would say information retriever summarizer. So again, think of you the ideal research librarian or research assistant. They do that extraordinarily well. And I haven't seen this used exploited anywhere near enough but a customer listener being able to look at omni-channel customer data. So all of the reviews that you have out there across all of those different websites, being able to analyze what folks are saying on social media, being able to analyze all of the emails that are coming in, being able to analyze those transcripts that are coming into a call center. These technologies are extraordinarily powerful for getting that view of the customer, understanding the voice of the customer in a way that it just has never been feasible previously. What are some common misconceptions you're hearing, particularly over the last year about AI? Oh, ah, they're the ones that are fairly harmless, like generative AI will replace all of AI. It's like, mm, yeah, you go try doing that time series data and all this with that forecasting with it and come back to me and see whether you still think that's the case. So there are ones that are fairly innocuous like that. There are ones which are all of the doomsday predictions of it's like, oh, this could lead us towards the extinction of humanity. And you're like, how, like, what's the pathway in which you get from these to that? Well, what do you make of that? I mean, we had a Google researcher come out last year and say that he believes AI has become sentient in some projects at Google. We have these 200 data scientists publishing a manifesto about AI as dangerous as nuclear weapons. I mean, what do you make of all that? Is it just hysteria? Mostly, I think it is hysteria. I mean, certainly the models are incredible and they can absolutely convince you that there's a human behind this or that there is more in it than we think it is. But to a certain extent that's telling us a little bit more about us as human beings, those are human beings. Actually, one, we're probably not as intelligent as we think and we're certainly not using that intelligence as much as we think and we're not really as complicated and sophisticated in what we do and we're also really, really easy to trick. So I would say there are all sorts of real concerns around this technology. Don't get me wrong. I mean, for fraud, this is the most powerful tool ever and the cat's out of the bag. There are open source models out there that scammers are using and you can expect very, very convincing scams and frauds and also just misinformation. I mean, this is absolutely going to be used to go in and manipulate elections and disseminate misinformation as it already is being used. And those kind of challenges we absolutely should be tackling and we should be making them illegal and we should be going and enabling regulators and law enforcement to be going after those, I would say. But worrying about the extinction of humanity that feels a little bit like, was it Andrew Young who was talking about worrying about pollution and overpopulation on Mars? It's way, way too early to be worried about that and a distraction from worrying about just the real world challenges that we are already facing with these. Which kind of makes the question of government regulation. Do you think government regulation is appropriate in some capacity? Absolutely. Again, there's all of these different ways in which you can cause harm with these models. So for example, when we're talking about election interference or fraud, absolutely for cyber crime, absolutely. These models do enable that. But unfortunately, so much of the regulation has been kind of knee-jerk and not really seem to, or has been very disconnected from the technology itself. From an understanding of the technology. Well, yeah, exactly, are we shocked with this? And so when I've been really pleased with the regulation and the EU has a lot to be commended for going, not so much for the fines, that's ridiculous. But when it comes to going in and looking at the use cases and trying to regulate the use cases, I think that totally makes sense. Where we completely fall off the deep end is when we start going in and trying to regulate the underlying technology and go in and start proposing things like watermarking should be a requirement or go in and say that, well, we have to have red teaming going on. And there's very little evidence that this is actually gonna make us any safer while potentially having a lot of burden and overhead around this. Introduce complexity without solving the problem. It's not necessary to say that maybe those will become part of the solution, but at the end of the day, you should be going in and regulating these use cases irrespective of what technology is being used. And when you try and get into it and start talking about specific technology that you are or are not allowed to use or techniques that you do and do have to use as part of this, again, I'm not sure we're making ourselves any safer while at the same time going in and potentially, you know, throwing the baby out with the bathwater and killing a lot of innovation. So you're a data scientist. Data science is cool now. There's been a shortage of data scientists for years and it's becoming a bigger and bigger problem. Do you see any end to that? I mean, that really threatens to stifle innovation in AI. Do you see any end to that skill shortage? Yeah, and we were talking previously about one of the myths is being that, well, I don't need data scientists anymore. Now I can just go in and- You need more data scientists, don't you? Yes, sadly, I think there's no way around it. It's probably a different data, well, it's for certain a different data scientist to the one that we currently have today. Does it make sense to go in and train somebody to be an extremely advanced applied statistician and then go in and train them data mining and then go and have them start working with these tools? It doesn't. There are ways in which we can shortcut this and we can focus on the education about how you leverage these technology components, how you work with generative AI models, how you get LLMs to behave in a way that this is what the data scientist does and today the data scientist is the person who's best set up to do these. But we can make that education far, far faster and more effectively. And it's really cool when you then start looking at what people are doing with LLMs on the education side, where you're using those to go in and help people learn more effectively. So one of the things that I'm most excited about is where that's going to, where that's going to take us. Not sure that it'll make its way into the public schools anytime soon, but certainly for improving our own intelligence, I think LLMs are probably going to be the, again, the best thing since light bread. Optimistic point of view, quick plug for your podcast. Please, yes. I'm the host of the Data Science Leaders podcast. It is for going and sharing those best practices about those poor individuals, those AI and data science leaders who have the misfortune and good fortune of being able to work with these on a daily basis. So please come take a listen and let me know what you think and what you want to hear more about. Watched a couple of episodes. It's very in depth. It's hour long episodes. You really get into the weeds and with a great, great lineup of guests. So, Joel Carlson from Domino Data Lab. Thank you so much for being with us at SuperCloud Six. Thank you so much for having me. We'll be back soon.