 Hello, KubeCon. My name is Eduardo Silva. And as part of this KubeCon Europe Fluent Bed Lightning talk, we're going to introduce a little bit about the Fluent Bed project, which is part of the Fluent Ecosystem, but also talk a little bit about the roadmap and exciting news that we have for this year. I'm going to share the screen right now, so you can see the presentation. Well, as I said, Fluent Bed is one of the projects of the CNCF, right? And actually the goal of this Lightning talk, as I said, is to share some news, but also give you a little bit of background of what kind of problems the project solves. So my name is Eduardo Silva. I'm the founder of the company called Caliptia, where we offer enterprise offerings on top of the Fluent E and Fluent Bed Ecosystem, and I'm also the creator of this project called Fluent Bed. Fluent D and Fluent Bed both are part of the CNCF graduated project. Actually, in technical terms, Fluent Bed is a graduated sub-project under the umbrella of Fluent D, both under the batch license, and both actually are part of the same huge ecosystem that you're planning to use or you're using now. Let's talk about logging pretty briefly about how this operates, how this works, so you can get a clear vision about what is expectations from a logging tool. Actually, logging tried to solve the problem on how do we collect messages from application services, even hardware, and try to process this information and centralize it in a place where we can deal with it or perform the analysis. Most of the log information comes as a text format, but nowadays the biggest problem is that with microservices and with the new cloud infrastructure and the new ways to deploy applications, we have more data that we used to have before. And you have more data, you have more challenges. How do I analyze this information but to analyze it and to centralize it, but also collect it? When application generates a message, this message also can come in different formats and usually there's no unified format for logging messages, right? There's sometimes the market to try to unify, but this has been there for more than 30 years. Actually, but what actually is really important for the scope of this presentation and for the project itself is that in an environment, it could be a cluster environment with Kubernetes distributed or just normal environment, right? What is important is that we have to understand that our data, most of the time will come in a raw text format without a structure, without a schema, but we need to be able from an application perspective as a log processor be able to handle this message. Actually, and when the applications start generating messages, all these messages go most of the time to the file system. Sometimes you can shape it over the network, but normally in production, you just write to this and then you just let the log processor collect all these messages. And then the whole goal of this is not just to deal with the messages. As I said in the beginning, the goal of this is to deal with data analysis and data analysis will happen after you have to be able to collect all the information and centralize the information back to a database or any kind of storage cloud provider like Amazon S3, Elastic, there are many options in the market, right? So the whole purpose of logging is to be able to connect point A to point B, but also in the middle be able to do some kind of data processing, right? It's not just a read and write. Actually, if you read the data and you want to perform some data modifications on the data, for example, if you are in a Kubernetes cluster, the ideal case will be when you get the data, be able to talk to the API server and enrich your logs with labels or annotations depending on the case. Now, who use FluentBib? Actually, it's been used by all the major cloud providers in the world. And as part of the statistics, a FluentBib is deployed more than 2 million times per day, just from the public information that we have in our public repositories. But also, we have some insights that there are many VMs, there are many Merb Battle servers that are running FluentBib. And of course, we don't have metrics from that perspective. But if you are attending this session, you can be pretty confident that you are getting into the right choice. Now, as part of FluentBib, one of the important things is the rollback, right? It's not just about to solve the problem that we have now, but what are other problems that we're getting reported by the community. And these problems are not just technically tied to FluentBib, but also to market trends or different needs that we have in the infrastructure, right? So we're happy to know that as part of this year, we started the journey where FluentBib will handle native metric support, right? What does it mean? Usually, you used to have your own agents for logs and separate agents for metrics. But from a community perspective, from the Fluent ecosystem, we always got asked many times, and I'm talking maybe more than 10, 15 times, why you don't implement the metrics support? Why you don't unify these needs in a single agent, which is FluentBib, which is pretty lightweight, so we can get rid of some extra agents that we have. It's not because of the other agents are bad, actually they're really good, but since we are using the Fluent ecosystem, we would like to have a more unified experience. And when we talk about metrics, there are many corner cases, right? The first one is that some application ship log messages, but these log messages sometimes are log messages, right? Be back, error, info warning, but sometimes they also ship metrics as logs. For example, they just ship a JSON map and they have a bunch of key values where they say, hey, this is a counter, this is a gauge, or an aggregation for a different value. And if we handle that information as logging, maybe we are losing the opportunity to do more processing on that data and do more smart things. Also, there's the other corner case that says, where since FluentBib is running on every single node of our cluster, why you don't collect the host metrics, disk, swap, CPU, memory, and so on. Also, we have application that ship metrics, right? Natively. So if FluentBib is there, why we cannot handle that, right? And if we talk about what is being used in the market right now, most I would say 80, 90% of the industry is aligned nowadays within the cloud native space with Prometheus and OpenMetrics, right? So as a FluentBib project and as a project maintainer, we decided that we're going to go on that way. Right now, we're going to support native metrics, supporting FluentBib, and backwards integration with the Prometheus ecosystem, right? Which is on top of OpenMetrics. Now, on a side, we know that OpenTelemetry, which is a sandbox project in the CNC, is being developed to try to solve all this complexity of logs, traces, and metrics, right? Right now, that break is growing, but we are in to integrate with Prometheus first, and once OpenTelemetry get a more maturity level, we're going to integrate from a protocol perspective, right? FluentBib, for us, is like a non-stick software, and we need to communicate with all vendors, all protocols possible, right? So if I can share here pretty quickly, I'm just going to share it in my screen. We have a POC of how we are building our own node exported in the FluentBib. Actually, it's a stripped down version of the one that you find in Prometheus, but we're trying to solve the needs for our own users. So I'm just going to run from the command line. So it would be node exporter. We're going to send data to the standard Open. And as you can see, we will able to collect a bunch of information from the host, but actually, this is my personal laptop. We are collecting data from CPU scaling, frequency, memory information, the total bytes, you can see they have 16 gigabytes on this laptop, the available memory, and so on. But what we did here is to try to replicate, collect natively all the metrics from the system, but implement a full layer where we can translate these metrics to different payloads expected by the users, by you. For example, you would expect to have this information in Prometheus format, right? And we are doing that at the moment. Now, as a second big thing that we have as part of the roadmap for this year, is the integration with WebAssembly, right? There's a bunch of information about WebAssembly in the markets right now. And the whole code of WebAssembly is that you can create, you can program any kind of function, right? And originally in any language and make it run on the brosk. But right now that WebAssembly engine is being translated into backend services. So one of the problematics that we have in Fluent Bed is that from a community perspective, people said, I have struggled to write SQL, right? Writing C sometimes is not that easy. You need a bunch of best practices, experience, because a managed memory, you can make it really good, sometimes generate a lot of problems. But they say, you know what, my team has experience with Go, has a scheduling experience with REST, but also we do some JavaScript. So we work towards WebAssembly, where now Fluent Bed, the back one, we continue reading the C language, but we're going to have a thin layer where you can write and integrate your own Fluent Bed plugins in WebAssembly choosing your own language. And of course, the compiler will do it all magic. And from a Fluent Bed perspective, using WebAssembly, we will be able to take your code and run it as an input plugin, as a filter or as an output to deliver data to different destinations. I would say that this is one of the huge things that are coming up this year. And we are really, really excited because we see that the project started as a small forwarder, now it's doing stream processing, distributed stream processing actually. We have SQL on it, but also we are going to provide and give a bit more flexibility to our developers from the user's perspective. Okay, thank you so much for watching this presentation. I know this was just a few minutes of a how Fluent Bed works, what are the next things that are coming up. So please stay tuned. We have many news. I know that during the conference, we're going to see more news about the status of metrics and the status of WebAssembly. So if you have any question or also you would like to share more use cases with us, so we can work together on those challenges. Just let us know. My personal email is there, Eduardo at Calita.com. So thank you so much for coming. And if you have questions, I will be in the chat. Thank you.