 Welcome back everyone to theCUBE's live coverage here at Google Next in San Francisco. I'm John Furrier, Dustin Kirkland, I'm a CUBE analyst, CUBE contributor. We also got Lisa Martin here and Rob Stretchy who heads up our CUBE collective research team. We've got a great guest here talking about search, value of data, Elastic is here. We've got Kathleen Walker, senior director of product marketing on the search side for Elastic. It's you, it's Sudeep Joseph, principal engineer at Cisco Systems. Welcome to theCUBE, thanks for coming on. Welcome. Thank you, thanks for having us. You're now CUBE alumni's, where'd it go? First time in the CUBE. Look, this is, we're in the middle of all the action here. The ecosystem's booming. We're seeing a lot of traction around Google's messaging around, hey, get cloud scale. The AI wave is here, and the demos are impressive. I mean, you're seeing like how AI is crossing first party applications in the demo, providing action, and so having that data and surfacing those insights and unlocking that value has been the promise of the big data for a decade and a half. We're finally here. Well, it's interesting, because I would say from the Elastic's perspective, we've been democratizing search for over a decade and search the amount of data that you have to process in order to do that continues to grow. We've been in AI for a long time, but there's really interesting things happening in the market right now, like it's a really exciting time to be in search if you think about the explosion around generative AI and the kind of improved customer experiences that you can build off of that. There's just the possibilities are endless. You guys are very well known, Elastic's great company. We're a small customer, but it's been great benefit to us in our CUBE data and our transcripts and just the agility and the flexibility of getting value. Now that AI's come here, you're seeing large language models coming, you hear vector databases, you hear embeddings, you got connectors, you got other mechanisms to create new ways to stand up more functionality with AI. How are you guys looking at this? Because you're in the position. Again, you guys have been doing this for a long time. Unstructured data, structured data is good too, but the unstructured data is even better. How do you guys see this AI wave? How well positioned are you for this? Well the good news is that we've made some great product decisions over the last several years and we're in a position because we've built our platform to be so flexible that we can really meet teams where they are in terms of their development resources, their data sciences backup, all of that. So if you think about a customer who's really still in the early phases of adopting something like Symantec Search, we just released our own machine learning model which is actually pre-trained so now that takes that burden completely off of you and you can start enjoying the benefits of that without having to do the training yourself but then also get the benefits of the increased relevance that it can provide and then you consider all the way on the other end of the spectrum, you might have somebody who's been very sophisticated and wants to build their own model as part of the differentiation for their experience. We're flexible enough that we can use that model in our search with our platform as well. Awesome. Yeah, so tying that to Cisco, can you talk a little bit about the architecture, how you've integrated Elastic and these models into your solution? Absolutely, so three years back during the COVID time, we started a journey of migrating our Enterprise Search platform to Elastic. So Elastic being the foundation search engine, we build an ecosystem of components what it is needed to build a platform on top of GCP. For example, all of the connectors are based as SparkJob, whether it's a crawler, REST API, they're all based as SparkJobs, they're run as DataprocJobs in Google Cloud. Now, we have content management systems pushing in data in real time to Google Cloud storage and sending the links to PubSupp that goes in an event-driven architecture based on GCP and components like GKE, data flow for text extraction and all the business logic transformations. And ultimately, they get indexed in different Kubernetes clusters in different regions in Asia and US so that we have global coverage and the latency is low in APGC regions. So for us, it is the humongous transformation where we moved away from an on-premises code data center into a cloud native platform based out of Elastic as well as AI and based on the foundation of Google Cloud. And when you guys did that architecture, what was the main thing that you were trying to solve? Was it you wanted flexibility with the data? Was it you were knitting a solution together? What was the driver? So I worked on Enterprise Search for 19 years. So I've seen variety of vendor products over these years and we knew what it takes to build something in what product to use like Elastic. Elastic is the most stable search engine I've seen across these years. So building upon that stability and the functionality that Elastic offers, we were able to design an architecture which helps us in such a way that I could sleep well at night. I don't have to worry about busy duty calls. So trust me, it's not easy to get things right. It's not easy to get the speed right. It's not easy to get the accuracy right. With the combination of AI and Elastic and GCP, you know, we did it right. And some of the applications we have seen, humongous, tremendous feedback, positive feedback on what we have rolled out. I think Cisco is a really great example of a company that was able to take an emerging technology and execute on it really quickly to create a great experience. And I think I'm hopeful that the work that Elastic has done to build our product and then the partnership with Google because we're obviously both very interested in providing these great experiences for our customers to build off of. You know, allowed that company, you know, Cisco allowed you to kind of build and execute more quickly. The big hype right now, obviously it's AI, and I hate to use the word shiny toy, but it's bigger than a shiny toy. It's like the most intoxicating revolution for the young people coming in. So as you have more experience or stable solutions like Elastic, people are worried about making the wrong bet right now. And it can be fun to jump on the new wave and go the hot, the cool kids. And there might be some great starts that emerge out of this, but from the boardroom, they're like, okay, we're architect, is it going to be reliable to your point about pager duty calls? So there's an airman experimentation with the new stuff. And as Google brings more stuff out, how is that working with you guys and the partnership because Google's trying to create trust. You guys have a trust relationship with your customer and vice versa. Trust and innovation have to go along, but now you've got compliance and legal challenges. So, you know, the embeddings on a vector database, does that cross-pollinate into another cloud? You know, these are the kind of questions people are worried about. So how do you guys navigate that together as a customer? And what advice would you have for practitioners out there that are looking at this who want to jump in the deep end of the pool, so to speak, and get in and start building with confidence? Well, I think it's interesting because I think most people were introduced to things like chat GPT and generative AI through like a more consumer experience. And so it's, you know, really exciting, really wet the appetite of the possibility, art of the possible. But then when you think about it as a business, all of a sudden you have to consider the fact that I'm only going to be able to provide the best answers, the most relevant answers, if I point this towards my private data. And that kind of changes the game because to your point, all of a sudden you have to think about privacy and security and is somebody going to see information that they don't have the right to see and how do I protect my customer's private data as well? And so that's where a company, a product, a platform like Elastic comes into play where we can serve as an intermediary. If your customer, if your employee has a question, query the information that you already have in Elastic and then we'll provide just the information that's appropriate to that user, serve it up in a context window to a large language model and then they'll build that conversational experience off of just the relevant data that that person has the right to access. So as you can imagine, that takes a lot of the risk out of pointing your private data to something like a large language model. And that's your recommendation for customers. Yeah, absolutely. Yeah, so connecting that to the Cisco solution, Kathleen's talking about those consumers who did, what is the consumer experience, your customers, the Cisco customers, how are they interacting with this generative AI back end? So what is interesting is we've been rolling out generative solutions right when we launch this new platform. So we call this reimagined search. So one of the first things that we did is we were used to chunk these documents in smaller paragraph. I have two sets of indices, one for chunks and one for documents. And what we had is an AI model, back in the day it was UniLM model from Microsoft that we used to generate questions out of it. And these questions becomes auto suggestions to the UI. And then we store these questions back into the chunk. Now we know where, which chunk cost, created this question. So when the user clicks on that auto type ahead, you're directly directed to the right chunk. Or if there is a similar question, based on elastics of similarity and semantics of similarity, we are able to bring in the right answer to the question. So here's the thing, bringing the right answer to what the user is asking for and leading them to a paragraph within the document. And that's the key experience that we are able to bring in using AI. On to the support side of things, like for example, we powered the support search at Cisco, all of the historical 10 years worth of support tickets, historical bugs in the last 10 years, we search it using elastic. And it's almost like 50 million large documents, one of the documents you're talking about, and it's not easy to find the needle in the hair stack. Getting relevancy right with 50 million documents is extremely difficult. So what we've done is we have rolled out things like birth attention weights for keyword based ranking. We've used embeddings based key phrase extraction to augment the original query and then send this query to Elastic to get the right set of results. And Elastic's able to help with the citations and attributions and ensure that trust is there, that there's an answer coming, but is it the right answer? Is it a believable answer? Absolutely, I mean, if you think about other use cases like legal situations as well, like you have a large corpus of data that you're trying to search for the appropriate information, and it's critical that you have all that citation information returned along with the information. So yeah, absolutely. I mean, I love that some of the language on the signage in and around the place. Unlock value, I love that. Always unlocking value is really what we're seeing. You guys are enabling. The question I have for you guys is how hard was it to go into production? Because, I mean, again, we're familiar with Elastic, we're a customer of a small level, but I can imagine how great it could be for you. You want to be, you want the headroom, you want the future value of the data. You don't want to be foreclosed, right? So that's the promise. But then getting stuff into production with some of these AI tools out there could be challenging. How does it work for you guys? How did it work with Elastic? Take us through that little experience of ideation, grinding it out and putting it in production. You will be surprised to hear that we built this platform in six months. It just took two months of the POC, and then six months to take it from there to production, the entire platform they're talking about. Now, most of the time, ideation needs proof of concept to be developed, and you've got to do a little bit of risk-taking to see that there is a value to the consumers for the use case. Now, as I mentioned earlier, if you don't mind me asking, what made it so easy? Two things. One, the extreme support that we had from Google Cloud, our cloud team, and as well as from Elastic. So, before we started this, none of us had any knowledge in the cloud. We were all in on-frame shop, and we had experience in Docker because we used to use Elastic services service using Elastic Cloud Enterprise. So that, I had Docker experience, but the team was new to Kubernetes. The team was new to cloud. What we did is we mandated everyone who would join this project to pass the CNCF certification for CKA, that is Kubernetes Administration, or the CKAD's application development. So, within two months, pretty much, we had 12 to 15 people in the team. Everyone passed it so that we avoided learning on the job. We spoke the same language, and the cloud, the Google Cloud Doc conditions, amazing enough to be learned. We did the right way. And one of the most important things to get right is security. We focused on that from get go, and we had amazing support from the Google account manager. And talking about the AI things, like, for example, taking an AI model and doing a piece is easy, but taking the production, you need to optimize the model. We did things like TensorRT back in the 2020, Triton inference server from NVIDIA were the ones that is available to serve it out in a very low latency scenario. Like, so it seems to be fast, and if you have to have a good user experience, the AI alerts in front of it needs to be super fast as well. So, we always focused on latency. We wouldn't take anything to production if it doesn't meet our cutoff. Yep, yep, in terms of latency, yep. Awesome. As you can imagine, we have a lot of people within Elastic who are really interested in search experiences and are the possible there as well. And so, really quickly out of the gate, we had some very creative engineers who realized the possibilities of pairing with our security solutions. I'm getting an alert that this is happening. We also have a lot of data as you can imagine on what should you do if that's happening and instantly matching the two, I'm getting this alert. Here is a set of information about what you might want to do next and the steps you might want to take right away. So, that was our first, the use case that we started exploring there. We actually took that and built our AI assistant off of it, which we just recently launched. And it's gotten a lot of really great feedback from customers. Kathleen, if you don't mind me asking, I'd love to get your perspective on, and again, following the success of Elastic from the beginning to where you are now and quite a cloud journey. What was it like inside the company with the engineers and the staff when the chat GPT wave wakes up the average person in the middle of a main street tech world? Oh my God, magic. Look at AI and so like, and then also the market started moving much faster. Will you guys change me moment? Oh my God, this is our time. What was it? We got to build more stuff. What was some of the conversations inside Elastic? Because I can almost imagine the whiteboard frenzy going on and the excitement inside the company. Really in all parts of the organization because you can imagine that from the top down, they recognize this is a great opportunity because there's so many eyes on search right now because this is a market moment. We really want to capitalize on that. But even outside of those conversations, we had an engineering organization that was like, this stuff is cool. And so the awesome thing about that is that we really empower the engineers to experiment, have space time to really see what can you do with this technology. And we started getting some amazing blog content and articles coming out of the team that we've just been continuously dripping out to the market. It's just a really exciting time and I think all parts of the organization from engineering to marketing to all areas are like, I see a potential for us to use this in my part of the org to make my life easier. So G, talk about Cisco from your perspective, to getting everyone on the same page with Kubernetes. That's a hard task in and of itself. So one of the things that's interesting is even before TATGBT was rolling out, right? We visited General DA, for example, the question generation, the summarization. We had something or the SEO query generation from support tickets using fine-tuned. So I have fine-tuned T5 models which were taking a support ticket, generate a bunch of search queries. Then we used that to hit elastic and then generate recommendations and cashed in, file store, et cetera. So when TATGBT came out, what we realized is how fast the LLM world has moved from the early days of birth into a very powerful ecosystem which can turn to an AI agent. Now imagine the possibilities of running an AI agent to do things that you had to manually do earlier, improving the employee productivity a lot. So that's the power that TATGBT and Vertex AI is going to bring into the enterprise. Yeah, so have you, I assume you've looked at and played with both. You've clearly decided, at least for this product, to go with Vertex AI and the Google APIs. But compared to open AI, how did you and your team evaluate those two? So we do evaluation across all the major APIs, whether it's Azure Open AI or Google, Vertex AI, Bison models, or even Lama 2 models, right, fine-tuned Lama 2 models. So the thing is, depending on the use case, you've got to decide when it makes sense to use a fine-tuned Lama 2 model on your own hosted inference rather than paying by token. You know, ultimately, and you know, you cannot compare GPD 4 with Bison or you cannot compare Bison to Unicorn. So it all depends on the use case and what are the only main differences that you have to decide to choose the right product. And for search in particular, you found a good match with Elastic and the Google infrastructure. Yeah, so. That makes sense. A lot of use cases that require generate AI using the RAG approach or retrieve logman generate AI. We have done full-fledged constables soon go live in two months to Cisco.com where anyone in Cisco would be able to ask, anyone in internet would be able to ask questions on Cisco.com content. What you also see is internal content, you know, for our sellers, for example, you know, they have an ability to generate PowerPoint presentations on the fly by just putting in a prompt, leveraging the data from a variety of places and generating, and you know, it amazingly improves their productivity and that's an exciting time to be in. Yeah. Thanks so much for coming on theCUBE. Really appreciate you guys sharing the story and the insights on around data. Kathleen, give us the last word, take a quick 30 seconds we have left, put a plug in for less. What are you guys are doing? What's the pitch? What's going on? What's new? Yeah, I think, as you can imagine, lots of talk about AI at the show. So we're really, you know, making sure that people understand how we play in this space as well. So we feel really strongly that again, this is a great opportunity to both increase the relevance of search, which you know, is a journey and that will probably never end, but this is a moment where you can really leapfrog and create better experiences by leveraging this technology. So we're here to talk about how we're doing that with our, in our own house and then also with great customers like Cisco to kind of take this, take this new experience, point it towards your private data in a safe way and then also in a cost efficient way as well. You know, you got to get the concepts, you got to get the data, it's got to be smart, it's got to be reasonable, but you have the reason and be intelligent and get the right results. Secret of search, right? Yes. Good to see you, thanks for coming on. Okay, I'm John Roy Dustin Kirkland here in theCUBE, winding down day one of our two and a half days of coverage. We'll be right back after this short break.