 This is George Gilbert from Wikibon. We're back on the ground with Aman Neymat at Demand Base Hey, and we're having a really interesting conversation about building next-gen enterprise applications. It's getting really deep So so Let's look ahead a little bit sure we've talked we've talked in some detail about the foundation technologies right and you've told me before that we have So much technology, you know still to work with. Yeah, that is unexploited that we don't need, you know a whole lot of breakthroughs But we should focus on customer needs that are unmet. Yeah, let's talk about some problems yet to be solved but that are Customer-facing with as you have told me existing technology, right can solve. Yes Absolutely, I mean, you know, there's a lot of focus in selling this like in Lally about like scaling machine learning and investing in you know GPUs and what have you but I think there's enough technology there. So where is the gap? The really gap is in in understanding how to build AI applications and how to monetize it because it is quite different than building traditional applications and Has different characteristics and I can you know, so it's much more experimental in nature Although, you know with lean engineering. We've moved about towards iterative software development for example Like for example 90% of the time I you know after 20 years of building software. I'm quite confident I can build software it turns out in the world of data science and AI driven or AI applications You can't have that much confidence. It's it's a lot more like in discovering molecules in pharma So you have to experiment more often the methods have to be discovered there's more discovery and less engineering in the early stages is the discovery Centered on do you have the right data? Yeah, or are you measuring the right thing? Right is you thought you were going to maximize my work the model to maximize revenue, but really maybe the The end function should be increasing engagement with the customer. So often we don't know the end Objective function or incorrectly guess the right or wrong objective function and the only way to do that is to be able to build an end-to-end system in days and Then iterate through the different models in hours and days as quickly as possible with the end Gold and customer. This is really fascinating because we were some of the research. We're doing is on the Really primitive capabilities of the sort of analytic data pipeline. Yes, you know All the work that has to do with coming up with the features. Yeah, and then plugging that into a Model and then managing the model's lifecycle, right that those that whole process is so fragmented Yeah, and it's you know chewing gum and bailing wire sure and I imagine that that slows dramatically that Experimentation process. I mean it slows it down, but it's also a mindset, right? So Now that we have built You know, we probably have a hundred machine learning models now demand-based that I've contributed or build with our data scientists and in the end We found out that you can actually do something in a day or two With extremely small amount of data Over using Python and a skill earn today very quickly That will give you and then you know build some simple UI that can a human can evaluate and give feedback or whatever action you're trying to to get to and Get to that as quickly as possible rather than but worrying about the pipelines rather than worry about everything else because in 80% Of the cases it will fail anyways or you will realize that either you don't have the right data or Nobody wants it or it can never be done or you need to approach a completely different From a completely different objective function. Let me parse what you use what you've said and in a different way And so I see if I understand it Traditional model building is based not on sampling but on The full data, that's right and what you're saying in terms of experimentation Stop doing that. Yes is to go back to samples. That's right. Go back to there's a misunderstanding that we need You know while demand-based processes close to a trillion rows of data today. We have found that almost all big data AI solutions can be initially proven with very small amounts of data and Small amount of features and if they don't work that if you cannot have 100 rows of data and have a human look at some rows and make a judgment Then it's not possible most likely with 1 billion and with 10 billion So if you cannot work it now, there are exceptions to this But in 90% of the cases if the solution is not at you know Few thousand or million rows of data. Now the problem is that all the the easy You know libraries and open source stuff that's out there It's all designed to be a workable and small amounts of data So what we don't want to do is build this whole massive infrastructure Which is getting easier and worry about data pipelines and you know putting it all together only to realize That this is not going to work or more often. They doesn't solve any problem So if I were to sort of boil that down into terms of in product terms, yeah, the notion that you could have Something like spark Running on your laptop Yeah, and scaling out to a big cluster. Yeah It's running a laptop that yeah, you're right. You don't even need spark I was gonna say not even spark just use Python despite its scale learn is much better It's almost like it's this is so so it's back to a visual basic, you know You're not gonna build a production app. I wouldn't go back far Well, is it better time and for the the prototype gooey app? Yeah, you do in visual basic sure and then you know When you're gonna build a production one you use Microsoft most often. Yeah, right more often You don't have the right data. You have the wrong objective function or your customer is not happy with the results or wants to modify it So and that's true for conventional business applications the old school Whatever internet applications, but it is more true for here because it's much the data is much more noisy the date the Problems are much more complex and ultimately you need to be able to take real-world action and And so build something that could take the real-world action be it for a very narrow problem or use case and get to it even without any model and the first model that I recommend or I do or my data scientists do it's like just do it yourself by hand just label the data and say as if like let's pretend that This was the right answer and we can take this action and the workflow works like this Do you something good happens? You know will it be something that would satisfy some problem and if that's not true Then why build it and you can do that manually, right? So it's I guess it's no different than any other entrepreneurial endeavor, but it's more true in data science projects firstly because they're more likely to be Wrong then I think we have learned now how to build good software much the imperative software and And data science is called data science for a reason. It's much more experimental right like in science You don't know a negative experiment is a fine experiment This is actually of all of all that we've been talking about it might sound the most abstract But it's also the most profound because what you're saying is This elaborate process. Yes, and the technology to support it, you know this whole pipeline That it's like you only do that once you've proven the prototype That's right and that and get the prototype in a day. You don't you don't want that elaborate structure and process When you're testing something out. No, yeah, exactly And and you know like when we build our own machine learning models obviously coming out of academia You know we were there was a class project But it took us a year or six months to really design the best models and test it and prove it out intrinsic Intrinsic testing and we knew it was working But what we should really have done which we do now is we build models we do experiments daily and Get to in essence the patient with our molecule every day So, you know, we have the advantage given that we entails a marketing that we can get to test our molecules or drugs on a daily basis and we have enough data data to test it and We have enough customers thankfully to test it and some of them are collaborating with us So we get to end solution on a daily basis So now I understand why you said we don't need these radical algorithmic breakthroughs or you know new Super turbo charged turbo charged processors So so with this approach of really fast prototyping, right? What are some of the unmet needs in that's you know, that it's just a matter of Cycling through these experiments. Yeah, so I think one of the biggest unmet need today able to understand language we're able to predict Who should you do business with and what should you talk about? but I think natural language generation or creating a personalized email Really personalized and really beautifully written is still something that we haven't quite You know how to have a full grasp on and and to be able to communicate at human level Personalization to be able to talk, you know, we can generate ads today, but that's not really, you know language Right, it is language, but not as sophisticated as as what we're talking here Or to be able to generate text or have a bot speak to you, right? We can have a bottom we can now understand and respond in text, but really speak to it to you fluently with context about you It's definitely an area where we're heavily investing in or are looking to invest in in the near future And and with existing technology with existing technology, huh? I think we think if you can narrow it down we can generate Emails that are much better than what a salesperson would write in fact We are we already have a product that can personalize a website Automatically using AI reinforcement learning and all the data we have and it can rewrite a website To be customized for each visitor personalized to each visitor give us an example of so you know for example if you if you go to Seamans or SAP and you come from a pharma it will take you and surface different content about pharmaceutical and You know in essence at some point that you can generate a whole page That's personalized to if somebody from comes to pharma from a CFO versus an IT person It will change the entire page content, right to that in essence the entire buyer journey could be personalized because You know today buying from B2B. It's quite jarring. It's filled with spam. It's you know, it's not a pleasant experience It's not concierge level experience and really in an ideal world you want B2B or marketing to be personalized You want it to be like you're being you know guided through if you need something you can ask a question And you have a personalized assistant talking to you about it So that there's the the journey is not coded in there's the journey or the conversation Conversation response reacts to the to the customer to the customer right and B2B buyers one You know they want something like that. They don't have time to waste towards who wants to be lost on a website, right? You know you should go to any fortune 500 companies website, and you it's just a mess Okay, so let's let's back up to the demand base in it in the Bay Area software ecosystem sure so Salesforce is a big company. Yes marketing is One of their pillars. Yes Tell us what is it about this next-gen technology? That is so we touched on this before but So anathema to the way traditional software companies build Yeah, I mean Salesforce is a very close partner. They're a customer We we work with them very closely I think they're also an investor a small investor for demand base. We have a deep relationship with them And I myself come from the traditional software background, you know, I've been building serum So I'll talk about myself because I've seen how different and mice, you know I have to sort of transition in a very early stage from a human-centric CRM to a data-driven CRM or human driven versus data driven and It's you have to think about things differently. So one Difference is that when when you look at data in in a human driven CRM You can you trust it implicitly because somebody in your org put it in you may challenge it It's old it's still but there's no there's no fear that it's a machine recommending you and driving you and It requires the interfaces to be much different you have to think about how do you build trust between the person, you know who's being driven in a Tesla also a similar problem and You know, how do you give them the controls so they can turn off the autopilot, right? And how do you you know take feedback from humans to improve the models? So it's a it's a different way the human Interface even becomes more different and simpler the other interesting thing is is that if you look at traditional Applications, they're quite complicated. They have all these fields because you know Just enter all this data and and then you type it in but the way you interact with our applications is we already know everything We're a lot. So why bother asking you? We already know where you are way who you are what you should do So we are in essence guiding you more of a again using the Tesla autopilot example It already knows where you are if knows you're sitting in the car And it knows that you need to break because you know, you're gonna crash So it'll just break by itself. So, you know, the interfaces really an interesting analogy Tesla is a data-driven piece of software. It is. Whereas, you know, my old W or whatever is a Human-driven piece of software and there's some things in the middle. So and you can you know I recently have been looking at cars. I just had a baby and Volvo is something in the middle Where if you're gonna have an accident or somebody comes closed it blinks. So it's like advanced analytics, right? Yeah, an analogous to that Tesla just stops if you're gonna have an accident and that's the right idea Because even when I have an accident, you don't want to rely on me to look at someone light What if I'm talking on the phone or looking at my kid, you know, some blinking light over there Which is why advanced analytics isn't hasn't been as successful as it should be because that the handoff between The data-driven and the human driven is a very difficult very difficult handoff And whenever possible the right answer for us today is if you know everything and you can take the action Like if you're gonna have an accident just stop or if you need to go go Right. So if you come out in the morning, you know, and you go to work in 9 a.m. Just it should just pull out put itself out like, you know, why wait for human to You know get rid of all the monotonous problems that we ourselves have, right? That's a great example on that note let's break and This is George Gilbert. I'm with and having a great conversation with Amanda Mott senior VP and CTO of demand base and We will be back shortly with a member of the data science team. Thank you George You