 Welcome to the CUBE's coverage of KubeCon EU 2024, live from Paris, France. Join hosts Savannah Peterson, Dustin Kirkland, and Rob Strecce, as they interview some of the brightest minds in cloud-native computing. Coverage of KubeCon cloud-native con is brought to you by Red Hat, CNCF, and its ecosystem partners. The CUBE's coverage of KubeCon EU 2024 begins right now. Hello and welcome back to KubeCon cloud-native con EU Paris. Glad to have you on board here, and we got some exciting stuff. We're continuing down the observability path, and really unpacking this, and trying to get a better understanding because I think there's still a lot of confusion about how you get started, how you get into things. There's a lot of projects out there, and today we're joined by Alois Rapebauer. Yes, hello. Chief technology strategist from Dynatrace. Hello, thanks for having me. Thank you, and again, joined by Dustin Kirkland, who's going to help me unpack this. I think this is, I mean, one of my passions is observability, and I think that when I come here and understand all of the stuff that's going on, and really is being unpacked, it's incredible, the amount of work that's going in on the observability side, and I think there's still a lot to be done, but I think there's progress being made. Now, Dynatrace, when I think Dynatrace, I don't immediately think open source. Help the people sitting at home who are like, why is Dynatrace on, and what do they have to do with cloud-native, open source? What is really Dynatrace contributing back in here? Yeah, it's actually good that you brought this up, because a lot of people don't associate us as a commercial software vendor with open source, while we are like very active in the open source community, it has been for a very long time. Open source, I wanted, but also contributing to standards for even a longer time, and we contributed to several W3C standards along the way, and within the CNCF, we were part of OpenTelemetry from day one. We even were part of the group that was working on the observability topics before, like the project merged and OpenTelemetry eventually merged. We have people in the governance board, we have people in the technical committee. We this week actually launched our own distribution for the OpenTelemetry Collector, where we bundle everything up, make it secure, make it enterprise-ready, take care of end-to-end testing, and even providing it open source back to everybody who's out there who wants to use it. We also funded two of our own open source projects within the CNCF, Captain, which has been around for quite some time. It's observability for GitOps, basically, when you roll out applications, getting tracing, getting your metrics and Dora metrics when you deploy something. So either it works for GitOps scenarios or Cube CTL scenarios, understand really why your deployment is not working or going wrong or what's going on in there, and also being able to hook into these processes. Obviously, it works out of the box with Argo and other GitOps tools. And the OpenFeature project, which has been very successful. So OpenFeature, the goal is to have a unified SDK for feature flagging that you put into your application because up to now it was either proprietary solutions that companies built or it was a vendor-specific solution that was out there. We kind of figured out all of these SDKs are doing the same things. All of them are actually open source. So why don't create the standard around it? We've got a lot of people in the industry joining in working on this project and also have a feature management solution. And this week announced the availability of our web SDKs. Yeah, the feature flag feature is quite interesting. That's not an easy thing for many projects to simply add. Can you tell us just a little bit, unpack one more layer of how you're able to do that in a general case and then provide that observability from, I'm guessing what you're able to do is see what's happening going down one path or the other path, right? So what do you think about deployment and release? These are two processes we kind of separated. Deployment is where I think of a blue-green deployment. So you're checking for a new release whether the code actually works. But you do not necessarily turn on the feature and then you start to gradually, progressively turn on the feature. It might just, I'm turning it out for you but not for you. So it sounds very simple when people think about feature flag, they think about an on and off switch. But it's actually more, you can pick and choose which functionality you're seeing. And in the top of the flag, you have what is called a targeting rule. The targeting rule basically specifies who's seeing the feature. People say from Paris who's seeing it but not from other countries. And usually applications do not have just one feature but they have multiple, even thousands. I think at Dynatrace we roughly run about 2000 features in parallel. So what does this mean for observability and for application execution? In the past you thought, okay, if you understand how like the transaction is working, like buying a product or doing something online you kind of understood how it's working was pretty much the same for all of us. But most likely in an advanced environment each one of us has different features turned on and off or different features with running different what you call variants, like behaviors. So suddenly every transaction starts to be different. And just from seeing which version of software is processing your transaction or you really request does no longer help us. So you get more or less this massive vector of, okay, for this transaction this was on, this was off, this was set to this value, this was set to that other value. So the analytics challenge and the observability becomes key. That's also why observability and open telemetry is natively built into open feature. So you get all that observability data and then you can figure out, okay, why is this feature working for you but not for you? Maybe because you have another feature enabled. So the analytics complexity is actually massive that you have to deal with. So that's really interesting. So it's not only that you're helping with the feature flagging actual project, it's how do you look at that from an observability standpoint so that you can actually understand why different things are happening underneath the hood. Exactly, because there's just so many possibilities, like how features can be turned on and off. It's very hard to analyze it from a, especially for rest of these, it might just even feel like random some of all the applications behaving. So that's why the analytics component is key. And that's also where open feature really helps because once you have a standard SDK that you're using, you can switch out the backend provider you could use open open source project, the commercial provider, but all the observability is really there. And it's also, you're also getting the same data across your environments because like for testing you might choose another feature flagging provider than you're running in production. Yeah, let's bring this to AI, okay, right? What sort of, can you tell us just a little bit about your capabilities and how your roadmap forward, how you're thinking about observability around AI integrations for code? I think for observability in AI, that's actually two directions that you can think of this. Like one is how can observability, or how can AI help us to get better observability? We at Dynatwit have been using AI, not Generative AI, but AI in general for like roughly 10 years. So this is more on the backend, like these are prediction models, these are fault tree analysis models, these are like traditional machine learning models and kind of like combining them together and now adding Gen AI as a layer on top. I think Gen AI can help in a couple of areas. One is user input. People ask more and more questions, they want to explore the data more and more, which means that the queries that you have to write are getting more and more complex and powerful, and you want to make this easier and make it easier for people to access it. That's one area. Another area is especially around remediation. When you have like root cause analysis, you know how things need to change or how you need to modify. For example, in Kubernetes manifest, have Gen AI simply rewired it. And also configuration overall is a key area. I think you want to build a dashboard. Building a dashboard is a lot of work. So now by combining different sorts of AI, you can ask AI, just build me a dashboard for all my conversion critical services and AI can then go in there. So this is not like that one JetGPT type of query. These are really AI programs or AI agents that check back what does your environment look like, what is critical in there, what needs to be modified, and then these AI programs are really built at runtime. So that's more how AI can help observability, but observability can also help AI. If you think about all these new solutions in the market, this is another layer you add to your replication where you also need observability on a number of fronts. And can you tie that back to the feature flag work? Is that feature flag capability, is that also tied back to some of the AI insights that you're able to derive? I think especially on the analytics side, you want to understand what features to roll out, even like creating optimal configurations for you by learning on how it actually works. Which configurations work? Or even handing over to an AI agent, like now thinking a bit further ahead, thinking also about generative AI, more about its planning capabilities versus the pure generation capabilities, asking an AI agent to safely roll out a new application feature by always like asking other observability data, then deciding on the strategy, what to modify, and getting analytics feet back in, and automating more and more of those processes. Because I think that's what we can really achieve with AI. There is a certain amount of daffops and there's a rework that is not just simple automation, but there is this cognitive step in between where you have to do some analysis to decide in which direction to go and what to do exactly. And there the combination of AI can actually help to make this simple and also start automating these tasks. Yeah, I think again to your point and exactly the questions that Dustin is asking is you've had Davis AI for quite some time and now exposed it and you're doing AI for using AI for tracking of LLMs and helping with that observability. You're also, you made an acquisition of RuneCast recently and you're bringing in security into that. Talk about how the feature flag and security kind of play together because that to me would make a lot of sense where the feature flag might also be, hey, we need to make sure that this is never turned on in these situations and things like that and where observability could kind of tie that all together. Yeah, I mean obviously you can look into whether like a feature flag is like enabling a security fault or not, but also think about feature flags not just about turning on our features but also providing more or less an API how to reconfigure your replication at runtime. These are the so called operational flags like how you at runtime want to change the behavior of your application. So it's giving application developers also to entirely new API how to change behavior. Like I could say as a remediation action like for like this IP address which is the targeting role enable this feature which rerout traffic for example not to the production database but to even like a honeypot type of database where they can work on. So even this type of application configurations you can build in and by those feature flags you're really extending the capabilities in your application to do this and creating like why are those feature flags as additional API. The very nice thing about it is as well it doesn't require any complex deployment that would take time. Feature flags can be switched immediately so you can have much faster response times in the case of security incidents. If you give me a list of IPs I could turn off the entire application functionality immediately at the application layer or even if you already have an intruder in the system you can block out all traffic and all API calls coming from that service host whatever in real time. Yeah I think that's one of the things that hasn't gotten a lot of airtime this week so far is security hopefully we have open SSF tomorrow and some others but let me kind of roll back to open telemetry as well because I'm interested in you guys obviously there must be an uptick in open telemetry use that you went and built that package and secured it and again a test station at a station of that source so that people can go and use it. How do you see the uptick of open telemetry? We see more people use it especially for custom instrumentation that they put into their code. More cloud native environments that just expose more data in various formats and open telemetry is definitely one of them. I think in the past it was pretty much a boutique thing for people to do you built telemetry in there. Now people are way more open to add this telemetry code to their application. Obviously it's done by open telemetry these days. We also see enterprise starting to adopt it but obviously we get more enterprise requirements which was driving a lot of our work for the Donutways version of the hotel collector. You want it enterprise ready, you want it fully secure and we want to support people in both directions so if you want to use our own agent technology you're fine. If you want to go all in on open telemetry you're good to go either. So that's why we will see more and more support from the observability providers and also creating their own distribution because we heard this from our customers and there's actually a large number of customers who are using the hotel collector but they wanted us to take care of it just to give you an idea and the popularity of this. In the first couple of days since the release we had over 3,000 downloads already of our open telemetry collector distribution. Well, and I think the fact that it's open, I mean again, that others can use it, I think that's amazing then and it's great contributions back into the community that you're making. It seems like Donutways is making a lot of contributions back for not getting a lot of airtime on it but you are today. We are working more behind the scenes on it but I think we contribute to open source where it makes sense for us. We are not the traditional, we take an open source project and commercialize it. But we think okay, this is where the industry as a whole needs to evolve, where we need to have a standard, where we need to collaborate. That that's usually where we also put our open source efforts, where we think it makes more sense for a larger group of people to collaborate on something versus having a pure open core business model around it. And that the contributions on open telemetry have always been there and for us, it was actually natural to make it available openly. Many only difference really is if you are a Donutways customer, you can file a support case and get like the support contract response times that we agreed on. But beyond that, it's exactly the same code so everybody can now run our version of the collector that is secured, it has all of these components in there. And we also take the complexity out of the upstream development for some of our customers because sometimes they want to see components on the open telemetry collector, like receivers that are not yet GA. So we then also take on the further development of those components to reach GA status and maintain them. Yeah, I think that's fantastic. And I think again, it's always eye-opening to me to see people who are contributing back in, like you said, where it makes sense, where you have actual capabilities to contribute something back into the community where it totally helps everybody, including your customers. I think that's fantastic. And I want to thank you for coming on board today. And thanks, and hopefully you had a good time here. Definitely, thank you. Yeah, and thank you, Dustin. Thanks for joining us again for this one. And this is going to be a great day. We got a little bit more coming up. So stay tuned here at KubeCon CloudNativeCon EU Paris. Thanks and stay tuned.