 every company every business out there is eventually going to become a real-time business but you know they are slowly realizing that I mean if you fundamentally look at how data is created data is never created in batches so and the reason why we were actually processing data in batches was because the technology was not available and right now it's it's a good time because the technology is there to actually process real-time data. Hi this is your host Sopnil Bhartiya and welcome Muthia for a let's talk and today we have with us Manish Devgan Chief Product Officer at Hazelcast Manish it's great to have you on the show. Thank you Sopnil good to be here. Yeah it's my pleasure to host you and today we are going to talk about the maturity model of esteem processing before we deep dive into this topic we folks of course we have covered you folks earlier but it's very good to just remind our viewers what is Hazelcast all about so talk about the company. Yeah thanks Sopnil so Hazelcast is a platform company we have a unified data platform which allows you to leverage real-time data from both storage and compute and streaming analytics perspective we basically are the platform based on which real-time applications are built and some of these applications involve you know we actually operate in various particles including financial services, retail, banking and a lot of the real-time applications are centered around opportunities and threats you know which businesses can work on. When we say esteem processing what does it exactly mean? Sopnil I always think about it from a business perspective so I'll give you a little idea on so let's talk about the customers and what it means to their business so when I look at let's look at our customer base so we have you know from retail like dominoes and target we have big financial services like Deutsche Bank, BNP Paribas, you know other brands like Volvo and Nissan so these these customers are essentially trying to leverage data right and derive the most value from data and what we have realized with the customers is that the value of these insights actually perish over time so you know the longer you wait the less valuable they get so from a business perspective you know these companies are trying to you know improve customer experience and how do you use how do you improve customer experience you know a lot of the retailers for example look at customer experience from how can I provide a hyper personalized kind of experience to the retailer so let's say you are you're transacting at target and you know is there a real-time offer I could present you because you were looking at sweaters for example so so basically they're trying to figure out how do you generate more opportunities for the business how do you you know there's a lot of things about card abandonment even when you're actually buying online a lot of times you you forget to complete your order because you got you got distracted but you know this is the age of distraction right with all the mobile devices so how these businesses are trying to figure out how do I leverage the most value of data and guess what the most value of data is actually generated in real time so being able to leverage data both in motion and data at rest so this might be historical information and generate the most value and the most value comes from from data which is in motion so that's where streaming comes into play and as as we all know you know data is not created in batches it's actually streaming in so that's that's kind of the the value behind stream processing and how can I process real-time streams and provide that hyper personalized experience or how can I improve a product service like in banking or how do I count of threats like fraud detection for example it's all about processing streams of data and then we look at real-time data uh is real-time data going to be the kind of future of successor of data or there are certain industries because not every industry every use case is generating real as you gave some examples it could be this you know your your uber it could be your food delivery it could be there are some industries that do rely on real-time but what I understand is that are there specific industries use cases where real-time data is going to be relevant or it's going to be the future also so does the question make sense yeah yeah thanks thanks for that I mean real-time we are living in a real-time economy I mean look at the customer expectations right when you order your uber you want it to be there in 10 minutes not not 10 hours right so the customer expectation has fundamentally changed and a lot of businesses have realized that and historically you know the the fan companies are the companies who are actually leveraging real-time data right and but now because of you know the onset of the value the opportunities and the threats the other rest of the industry is kind of catching up and you know we fundamentally believe that every every company every business out there is eventually going to become a real-time business but you know they are slowly realizing that I mean if you fundamentally look at how data is created data is never created in batches so and the reason why we were actually processing data and batches was because the technology was not available and right now it's it's a good time because the technology is there to actually process real-time data I mean just imagine when when you go about your day you go grocery shopping then you go fill gas in your car and then you do another transaction you know that credit card company does not look at all those transactions at the end of the day right there's value very little value left so when you are at the grocery store a store shopping what is they were able to actually generate a real-time offer and give you today so it's basically seizing the moment and generating more opportunities for you and on the threat side it is fraud right there's a rampant fraud in financial services and they you know that needs to be countered and that can only be countered when you're looking at data as it is created I also want to talk about the security here because data is the real asset applications can come and go when it comes to real-time data or streaming data what does security mean for it yeah so security is very very important I mean when we work with our customers and most of the application where we get deployed in are what we call tier one applications so these are applications which are mission critical these applications might be things like fraud detection countering fraud or it could be generating the real-time offer so these are generating revenue and also preventing from threats right so because we are mostly handling customer data transactional data securities of paramount importance there's a platform inherently supports everything from you know transport level security to authentication to authorization and all that but what we are also beginning to see is that you know that is not enough because with the advent of AI and off-the-shelf tools where you can actually you know mimic somebody if there is a voice level authentication you can actually mimic somebody right so you basically we are basically in a catch-up game right so how can you actually use AI and ML models to actually predict the outcome so a lot of time security especially in fraud detection use cases security the fraud threats are countered not just because this transaction is more than 50 dollars but it's because of the real-time features which the predictive models are able to seek and it has to happen in that moment so basically you are able to detect fraud before the person leaves best buy with the TV right so that it's that in the moment and I think security is very very important because because of you know because of the the damage it can have to the business to the three person the great history and all that but more importantly now there are tools available to be able to act in that moment since you mentioned AI I mean this is the hottest topic of technology is Genetic AI you know chair GPT I mean you folks have been leveraging AI for a long time and it kind of became boring but Genetic AI kind of brought the interest back in AI talk a bit about what kind of scope do you see of Genetic AI in streaming real-time data or for his cost you know in our customer base you know especially in the fraud detection space in the financial services they are they are very hyper focused on the tools which are now available off the shelf tool as I was especially the gen gen AI tool where you can actually create or generate data right now the way what we have seen how generated AI has actually come into play is that we actually our customers can actually use gen AI to create what is called synthetic data and we are actually working with a partner of ours where you can actually create synthetic data to kind of mimic so let's say a predictive model can only be so accurate based on you know what data it's been trained on right so using Genetic AI you can actually have synthetic data which can train the model for better accuracy because after all you're you're countering you know a set of tools which are created by Genetic AI and I think you know the chat GPT and the tools like chat GPT have really shown the power of of gen AI and actually brought focus into core AI what I call predictive AI and predictive AI has been in use for many many years it just takes it to another level where you can actually have you know synthetic synthetic data be created and and make your predictive AI model so much more accurate how much adoption you are seeing of streaming real-time data in the market where you are like happy that you know what the industry is that as you said you know that kind of seems to be the future but you're like happy or you feel like a lot of education awareness drives have to be done to actually tell companies that no you should move migrate to real-time streaming data I think great question for actually we we discussed this quite often with our customers and internally as well there is definitely a shift but we also realize that not everybody is that you know far in the journey of becoming a real-time business and as customers realize that there is so much opportunity to be had or so much of threat to be countered using real-time data platform like Hazelcast you know they they get on that journey quickly but then there are customers who are struggling with you know I have a system already how do I bring in without ripping and replacing bring in this new technology to give me this more you know a better position in the in the market or be able to counter my competitors so we are very conscious of that in fact one of our large customers had a system in place already so they were actually already taking data and pushing that into a data lake through you know through system built on IVM and Q and and Kafka and we are able to actually help these customers kind of almost like plug in these new capabilities so that they can start to see the power of the real-time real-time data so it's not like a rip and replace but it also depends on where the customers are starting their journey from typically what we see is the customers are first they want to collect all the data and move data from whatever event streams they are getting transactions and and customer information and whatnot and they push that into a data lake for future analytics that's that's the one step of the journey and then they realize yeah we I need to analyze the data but that is all delayed action you would analyze the data and then make an action here I want to do a price increase or I want to do XYZ right but as they move from being able to take real-time action then they really start to look at platforms like he will cast or other technologies where they can actually process incoming streams so it's it's really a journey and I think it's a lot from vendors like us you know and people like you to kind of educate the market on what's possible what's the art of the possible how does a company know that they are ideal candidate for real-time data stream data or the real-time stream data is ideal for them and once they do recognize that what is your advice how they should approach it it does dealing with real-time streaming processing different from static or whatever the term we use for data talk about that aspect there are two questions bundled together we talk about this world of batch and the patch and the world of streaming right and if you're already doing batch you you you probably are already collecting a lot of this real-time data but you're putting that into let's say a data lake or a data warehouse now the actions you take are delayed right you look at reports you look at models you're trained and it probably will be few hours to a few days now if you kind of open up your mind and think think what is possible right the the the possibility of actually taking immediate action on something which happened in your system you know that's when people you know eyes glow up because you know what if I have a customer who is actually transacting at an ATM ATM machine and as they are transacting I'm actually able to create a real-time loan offer based on their their their credit history the the the transaction they are making and we have seen like one of the customers we have been people about they are actually able to successfully increase their loan origination by 400% because they are actually catching the customer right in the moment it's about in the moment how can I better that experience right because that that's a big thing and it's it's always seizing the moment can you seize the moment because as you know you know our attention spans span these days is ever decreasing in shrinking so uh so there's there's a lot of opportunity there so you know I really advise the the customer CIOs and CTOs I meet that you know the art of the possible for moving into real-time businesses it's real right and as you know customers kind of embark on their real-time streaming data journey can you also advise what kind of tech stack they should start building how they should prepare themselves from technology point of view to start leveraging streaming real-time data the real-time applications which get built on our platform and the and the customers we have we have figured out that there are some key ingredients to the tech stack which are important while building a real-time application you know one of the big things is real-time messaging so first of all you need to figure out the the really plumbing on messaging right on on how to capture and ingest data and then the other big ingredient is can I actually have a real-time data store you know data stores which are not bottlenecked by disk-based databases so these are you know data stores which are very fast so can I get a query back in let's say two milliseconds or 10 milliseconds you know that level of real-time data management and then stream processing is a key thing can I process streams as they're coming in not only after they are stored in a data store but as they are coming in right and then the other big ingredient is real-time machine learning this is fundamentally changes the value of of use cases we have seen because can I actually predict in real-time whether this transaction is fraudulent or not or can I actually effectively create a real-time offer and that means having access to real-time feature stores real-time features based on which the machine learning models operate so these are the few ingredients which are very important to have in the tech stack and as we talk about text you know whole stack I also want to talk about people and cultural aspect do companies also need to have cultural or mindset changes as well so that their team are once again looking at streaming data from different perspective versus you're like no this is not an area where we need DevOps like moment yeah so that's that's very key in fact mindset is something which is so important while while changing your business and that's key to success right I mean there are technologies and tools which are plenty in the market that you can you can adopt you can stitch them together or you know go for a consolidated platform like handle cast but I think it's a it's a it's mindset shift and the shift is between batch and real-time so just imagine the data is created in real-time so why are you processing the data in batches or micro batches right that's a fundamental shift we always wanted to collect data and store that in a data warehouse so that we could run a report or train a model in the future you know those use cases are there but the world is moving to an automated world right where I want to take an action as soon as something happens right so as soon as my car actually is beginning to skid the automatic the the ABS system kicks in and your anti-lock brakes kick in right that's real-time right so there are there are things in our world which are real-time so why shouldn't businesses be real-time right I think that's a fundamental mind shift that you know I don't want to collect the data so that I can process that tomorrow and run an analytics report and then change my business I can actually I take an action right now so that's a mindset shift and the shift is is actually very very important because now there are tools in the market which can help you get on to becoming a real-time building Manish thank you so much for taking time out today and of course not like talk about Hazel Cosworth also real-time string data as well and I would love to chat with you again because as we see there is a lot of need for education in the market and you know that's where we can you know really collaborate and help you folks as well and help the whole ecosystem but I really really really in you know appreciate your insights today thank you yeah thank you