 Hi, this is Hoseb Nibhartiya and welcome to brand new episode of T3M, our topic of this minute, topic of this month is observability. And today we have with us Asaf Iqal, co-founder and studio logs IO. Asaf is great to have you on the show. Great to be here. Thanks for having me. I would love to discuss with you, how have you seen before, how have you seen the evolution of observability? Let's try to hear from you. How would you define observability in today's, we have been talking about observability for so many years, but it's kind of evolving. How would you define observability? What is it? Yeah, it's a good question. I think a lot of people define observability and a lot of them, a lot of customers that we see that say they do observability by what they basically do is monitoring. I think the key thing that differentiated two of them is whether you set a goal, whether you set the KPI, whether you set an objective to do it. And not just like, hey, I'm gonna put all these monitors on my CPU and my memory and all my servers. And when things go bad, I'm gonna wake up. The way we define observability is log.io is I wanna be able to measure how my technology is impacting the business. I wanna measure my service level. I wanna be able to be alerted when my service level is not within the objectives that I set. And then I wanna troubleshoot it. So that's kind of like how we define observability. If you just look at how you define it, it's very closely tied to the success of business as well. How different is observability from, let's say, monitoring, logs, tracing? And you also talked about service level. We also talked about SLOs, service level objective, not just SLAs. So let's look at these different things and let's once again, that the whole idea is to business continuity is important, services are running properly is important and that ties to the success of business itself. I think the way I look at it is monitoring is, I have objects and then I need to monitor them. I have servers, I have my cloud environment, I have my Kubernetes cluster. I have my code that is running. These are things that I need to monitor. I need to make sure they're alive. I need to make sure that they're not reaching the limits. And that's kind of like how we define kind of like monitoring the difference of it and going into observability is tying all the pieces together because eventually when I have a service, when I as a company run a service, let's say I am in e-commerce website and I have a service about purchasing where a customer adds stuff to its cart and then he goes to a purchasing and payment and it includes tens of different microservices of authorization of checking and price and conversion and taxes and checking credit card and fraud detection and stuff like that. The overall process, if you have an objective and saying, hey, I want my purchasing process to be less than 200 millisecond, less than a second, I want my purchasing process to be have an error rate of less than 0.01%. Now you set up observability, you set up goals, you set up objectives, and then you can basically create observability and the underlying components would be logs, would be metrics, would be trace information, but tying all of it together and saying, hey, these are logs that are coming from my payment service, they relate to my overall purchasing, these are logs coming from our authentication, they also relate to it. Tying everything together is what brings, kind of like from monitoring to observability. Now let's talk about the evolution of observability as you have seen over the years. Yeah, so I think over the years, most companies started with logs, metrics and traces and some companies started with logs, some companies started with metrics and they kind of like built a way to getting logs, metrics and traces. I would say that probably most companies today have logs and metrics, a large percentage of them also have traces, like distributed traces, but now comes the observability. How do I create a layer on top of the data, the data components that gives me what I need in order to carry observability? And that's, it's not only technology, it's also processes, it's about the people, it's about training, it's about how do I create alerts in the system? How do I report up to my managers? How do I report to the business? Do I have automation in place? Do I have automation for code? All of it is part of observability and the evolution of it. So we used to have separate kind of like pillars, if you will, they all brought together and now the observability is the layer on top of all these pillars. How mature, like when we look at Kubernetes-like technologies, you know, of course it's in production now, those early days are gone. When we look at the observability landscape, how mature is the adoption of observability where you feel that, hey, customers, they know about it, all they are doing at the next level of it, or you feel that, hey, we still have to go and educate, hey, they should have the right observability practices in place. From what we see, there is a relatively low to medium adoption of observability in general. I think obviously you mentioned Kubernetes and it's widely adopted. I mean, I think we rarely see an organization or company that doesn't want it to production. Kubernetes creates a lot of challenges just because of its flexibility. And it's an amazing service and it's amazing open source service that everybody's using, but it does create challenges and when you want to create observability. So this is, I think, where we come into play on addressing the challenge. Obviously the more flexibility you have in your environment, the more layer of obstructions, the more you need observability. If you have one server running one process, then you just log into the service and see what's going on. Since you talked about those challenges that Kubernetes creates, I also want to talk about, you know, if you look at some of the technological advancements that are happening and what impact they are having on observability, we can talk, of course, observability is a big player when it comes to security in for a second, you know, of course, you know, and then we talk about Kubernetes and then we are also talking with Genity AI. So if you look at some of these recent technological developments and you look at, hey, these are, we're at observability is going to play a very important role, but they're also kind of creating some challenges for the observability players. So I think obviously InfoSync and security is playing a big role in observability. We just announced a couple of months ago the integration of a threat detection within our Kubernetes cluster. It is challenging for organizations today to actually know what's running inside the cluster just because developers can download images from the internet and they use a lot of pre-made services. So I think this is coming into it. I think generative AI is an opportunity rather than a threat in this space. But the other, I wouldn't say threat, but kind of something that needs to pay attention to its cost management has become a major thing in the past probably six to 12 months. So it's both cloud cost management, but it's also observability cost management because all these layers of obstruction, they spew out so much data that it just costs a lot of money and organizations are now more concerned than before about the cost of the cloud as well as the cost of observability. What are some of the big challenges that the observability community is facing? It could be about first of all, the adoption of Kubernetes like things are growing, which means a lot of new use cases are coming in. I think one major challenge with the observability that there's our tackling is the simplicity of data collection. And you see it today again with the wide presence of open telemetry. It's one of the biggest projects under CNCF. And this is definitely a major challenge for all the observability vendors. How quickly and how simply can I collect information? And again, coming back to Kubernetes, there are so many layers of obstructions. If I want to collect the logs, I need the logs of the cluster. I need the logs of every pod. I need the logs from the actual nodes that are running and so forth from metric and tracing information kind of like cutting across all the obstruction layers of this component. So data collection is a problem. The entire industry joined forces under CNCF to solve that problem with open telemetry. It's definitely making strides in the right direction and hopefully it's been a mature over the next two to three years. Can you also talk about why companies should care about observability? They should care about the observability because there is no other way. So companies, what they should care about today is business agility. Today, if you're an online business and most companies today are online businesses and that means that you need to have business agility. It means that you have to release 10, 20, 30, 50 times a day. In order to achieve that, that means that you have to run on the cloud and you have to run on Kubernetes. You have to run Docker as you have to run Kubernetes. Running on Kubernetes, if you don't have observability, you're basically flying blind and that restricts your ability to achieve the business agility that you want. So you want to be an agile business, you have to have observability in order to support that agility. So it's not a question of like, should I miss is definitely a mandatory solution that every company should have. Of course, we were talking about what kind of challenges these new technology like Genetic AI are posing for observable space. But they can also kind of not, they can almost everybody is trying to use these technologies. So talk about the impact of Genetic AI on observability. If you look at observability, what you in essence have is a lot of data that you need to make sense out of very, very quickly. And today most of the observability, especially around the open source, the ability to query the data is limiting and requiring you to know specific language like PromQL, like you've seen and access to the data is basically limited by your ability to know what to ask. I think this is a place where we see a future in generative AI and say, hey, I want to ask my data, tell me where the problem is. And I want generative AI to be able to construct the right query and give me all the problems in my environment or give me highlights of the problem. I want to ask, hey, give me all the servers that crashed last night. I don't need to know how to construct that query. I can just interact with it as basically if you will chat with my data that is constantly streaming. So I think there's a lot of application to generative AI in the observability space. And I'm excited to see what us and other vendors are going to bring into it in the coming year or two. What role is Logz.io playing in this space? Yeah, so Logz.io is definitely, we're an open source observability. We're contributing to all the open source components. We're strong believers in open source, especially when it comes to observability. We're working hard to make the life simpler for engineers, devops and executives and companies and the ability to get from monitoring to observability, the ability to define service level objective, the ability to view the process and to end with all its underlying microservices. This is kind of like where we come into play. What advice do you have for companies in respect of where they are in their observability journey very early on, they have not been considered or they are like advanced, that they should have these are the right practices when it comes to observability. Yeah, it's a good question. So first of all, my recommendation is to set on objectives. So when you come to a problem, you wanna make sure that you know what you're gonna solve. So whether you're type of a company, a gaming company, an e-commerce company, an IT, whatever, just set the objectives of what do you want the observability to report to you. And so at the high level and understand the business impact because without the business it doesn't make sense. The second thing is invest a lot in data collection and data hygiene. We see, we come across a lot of organization and they just collect everything that they can just like piles and piles of data. It costs a lot of money. It creates a lot of mess in the environment. It clutters the ability to search and gain insights. So make sure that you're very, very detailed about the data that you collect, where you're gonna collect it, how is it gonna tie to the rest of the data? Make sure that it's standard and synchronized. And if you have the objective, if you have this, you're probably like 80% there. Asap, thank you so much for taking time out today and talk about observability. And as usual, I would love to sit down and chat with you again. Thank you. Thank you very much. Thanks for your time.