 Okay, hello everyone, name is Gabriel Freitas, and I'm going to tell you a little bit about my experience in the open source by integrating a project like RabbitMQ to Knative. And this is path to GA, how to guide a RabbitMQ story. Basically for this talk, I want to learn a little bit why it's Knative and why isn't it for this integration with RabbitMQ. So I will explain the big picture of Knative and how to connect these pieces into your project. The process for bringing your integration to Knative, this talk is not going to be a deep dive on how to integrate a project to Knative, but it's going to be a summary and hopefully a useful roadmap for people that want to integrate, who have thought about integrating something to Knative, so you can look at the steps that I had to follow by helping with integration. I didn't start it from the beginning, so there's a lot of context that I have to learn and I have to ask, but hopefully this will avoid some of the mistakes that I personally made and make your life easier when getting an integration done with Knative. I'm also going to tell you how and who defines quality gates for GA and how to prioritize them, how to make your components available through the Knative release process and documentation, how to overcome all mistakes and strategies to solve them, and how to gather qualitative and quantitative feedback from metrics and users. So let's get started. First of all, if you're going to integrate something to Knative, it's worth knowing what is Knative for. So to answer this question, you have to go a little bit back in the time and why is Kubernetes something that we use today? So Kubernetes was aiming to solve an issue with past time apps that they were hard to develop. Distributed systems have been hard all along the time, so with Kubernetes, you look to put an abstraction later on top of building distributed systems so you could manage less complexity than to manually manage containers like cycles, containers, crashes, errors, and all of the changes that you have to do on a distributed system. Kubernetes gives you an opinionated way of doing this. You have to do it by YAML, you have to do the Kubernetes API, but it's a light opinionated framework to do this because you can extend the framework, you can use tools that you use on your past project that are integrated with the Kubernetes ecosystem. So it has to be opinionated because it has to give you a way to do this simply and more easy than it was before, but it lets you extend a lot of things, a logs, metrics, and all of this stuff that are normal in the distributed system world. On top of this, Kubernetes, it's still hard. You can end up with YAML Forest, a lot of YAMLs, a lot of configuration, or it could be complex and complex even when using Kubernetes. So Knative is again an abstraction layer that offered you some building blocks or some tools to make your system a little bit easy in the Kubernetes world. So it is opinionated. Yes, it is opinionated, but again, it tells you you have to use this, but you can extend it using custom resources. You can use existing integration like the one that I'm going to explain today. So let's explain the Knative pieces and how they managed to get a pretty, a fairly big picture of the cloud-native state into the pieces that they separated. Knative divides in some pieces. Knative serving that offered you a building block to manage your serverless workflows in the cloud. Knative eventing, Knative eventing lets you give you the building blocks to build your event-driven architecture using the Knative way. There is broker, streamer, channel subscriptions. We will talk about that later. Knative function, it's again another abstraction layer that you hide from the user functions. Reduce the complexity that serving and eventing have because this is hard. System can get a lot of complex, can get complex and complex from time to time. So functions is aiming for simplify even the Knative way. And Knative build, that was a, Tecton was a part of Knative build. Like you have that CI-CD system that is independent for any cloud provider. So Knative is open source at the end of the day. So it has its pros and it comes. And the community is driven where it's heading. Function is a clear example of that function. Didn't exist like a year before now. And it was driven by Lance Mauricio. And I will talk a little bit about it because it's definitely a successful case of how to drive something on the OSS world. So Knative has a lot of pros and cons, as I said. One that you can see as any of them is community driven adoption and development is key. Communication and consensus is necessary. We are a big community. We are looking to thrive and to get bigger and bigger. So you cannot pull the project to a direction that you want. You have to communicate your ideas and look for people that are wanting to support and help you by implementing them. OSS is fun but hard. Open source is fun because you get to know people from different companies. You get to know a lot of brilliant people that it's going to help you. That in other way, working just for your enterprise, you are not going to get that opportunity. But these people is also rotating. They get tired of the project. They get involved in other projects. So the hard part is to being able to get in the role of the people that left and keep going with the community as you do that. Knative is big and complex. You have the docs and as a community, Knative is by no means a small project. It has a lot of moving parts and it's always evolving. So docs and community is going to be the key for you to start understanding the project and use the working groups to align and get the right context. Knative have these working group concepts where people join groups to work in a specific area of Knative. It's impossible for one people to maintain the whole Knative ecosystem. So the best way to start at least is joining the working group that you are more interested in. So if you are interested in eventing and serving, they have their own working groups. The function teams just got its own working group. So you are seeing how to start to get involved and driving the creation of your part of the project in the community. So now that we know what is Knative, what it is for and how to start your journey in the open source, then where to start when you want to drive your integration or even start your implementation in the Knative world. Basically, it's all about the spec or the type. The specification dictates what your implementation has to conform to or what your implementation has to look like for other people to see. Basically, in Knative, I'm going to focus on Knative eventing in this talk because I think it was integrated as the messaging protocol behind the Knative eventing implementation. So the specification is basically a document that tells you as a developer what your integration should have for you to implement, basically. And for a user perspective, this document tells you, okay, this is what you have to expect when using that implementation in your project. So Knative eventing has detailed specs on brokers, triggers, and all of these customer resources that it defines. The serving also have them, but let's move that aside for now for this presentation. Also, there are parts of the code that don't have specs that you're not going to define everything in a project as a spec. So if you don't see a spec and you want to implement something that is missing or something that maybe is not functioning like the flagged example that one of the folks told us in one bus talk, then you have to look at the type. The type is going to tell you, okay, your object should look like this, should implement these methods. This is a way that you should start your implementation on this and maybe change it in the way. That's something that you just have to start doing it and then you will know if you're in the right path. And sometimes it just does not exist on Knative eventing, so in Knative in general. So if it does not exist, go for it, write your specs and write your own types for you to start implementing your integration. Now that we know how to start, let's go to integration. You must have a reason behind your integration with Knative. The way a working group can stand behind your integration. Always communicate what you are doing in Knative. You can think that you have the new great idea for Knative, but maybe a seasoned maintainer is going to tell you, okay, this can be done in another way or this is already something that we discussed that we don't want, so communicate. So a working group can stand behind you or even create a new one as a function example. Understand the pieces involved, not the whole picture, just what you need. Basically, when you're going to start your implementation, there's a lot of things to look at. In RabbitMQ, we just had to look at eventing, but even then, we didn't look at the whole picture of eventing. Basically, we ignored the implementation of channels and prescriptions in favor of higher level abstractions that were broker-triggered sources. So basically, eventing RabbitMQ does not have yet channel subscriptions because we wanted to offer the users that we had in mind, the filters and the capabilities that the broker-triggered sources were already covering. So in that way, we went to an initial state to a GA state more quicker than it was implementing channels and subscriptions. Basically, now that you have your picture, you know where to implement it and what to implement, you need to create a Sandbox repo. Sandbox is a part of Knative that gives you the tools to create a repo that is not getting the Knative core repo. So everyone in the community can start seeing your code and checking out what you're doing and start contributing if they are interested. So this is a great way to start implementing. A lot of things are covered in the readme, so I'm not going to go into details, but this is like the roadmap that you have to follow to get your implementation going and the community to start talking about it. So after you create your Sandbox repo, you're going to start with the testing. How do you tell another software engineer that your project works and not just by your words is by telling, okay, I have test all of this code and the bar is high, it's always passing. So there are three types of tests that I found that they were key on the Knative environment. One of them are unit tests, the usual unit tests that you do to your code, E2E tests. These are ways to test scenarios, real scenarios by emulking all the dependencies, but you need to have a lot of resources on how to write E2E tests in different, based on GitHub actions using different cloud providers or cloud environments. And finally, the conformance tests. If you remember, I told you about specification or specs. Conformance tests are the way to tell Knative that your conformance, your specification is being respected by your implementation. So basically if you have an specification of a broker that you have a function that you have to implement and if the function needs to return something greater than zero or and it's correct or less than zero to be incorrect. Basically a conformance test to test this is okay. I have this function, my function is returning greater than zero, your conformance is passing. In the other case, you are not respecting the specification of that specific implementation. And this is what lead us to the whole picture. Basically this is a pipeline or multiple pipelines of the Knative and ending site where you can see where all of these is getting in the picture. You see here that downstream tests is something that I haven't explained, but basically Knative is an upstream repo or upstream ecosystem basically. So Knative is the repo that other repos that are the integrations or other external repos depends on. Downstream tests are the ones that based on the unit test the E2E and the conformance test are going to tell Knative, okay something is changing in the upstream or repo is breaking. Now you have to tell us what to do because the ecosystem is kind of breaking up. So this is a way of automatizing the whole environment for everyone to know like when it's on your release, there are new changes. Your project is still stable or your project needs to adapt to the new changes. This is something that is maybe it's not for the normal projects, you don't have to have an ecosystem to start your integration or all that. But having this is key for an ecosystem to be stable and to be usable around multiple integrations. Now let's go, now that we know a little bit how are things handled by Knative and where are we in our cycle to integrate something with Knative let's see event in RabbitMQ. So example, basically what we wanted to do is to integrate RabbitMQ that is a messaging multi-protocol machine system to Eventim, because Eventim underneath uses messaging protocol to make sure that the events are getting from the sources to the things. So we started going over the Knative rep material levels, we have clear that we have an initial state of projects and stable state of projects and usable state of projects. RGA was our goal to make the project usable for new users. So we started defining what we needed to implement. As I already told you, Eventim have channels, subscriptions, source brokers and triggers and also flows. So we basically in noted the channels and the subscriptions and went for the broker, trigger and source abstraction that when we're higher level for our use case and start documenting, we started documenting as soon as we were implementing this but it was not enough. Then we realized okay, documentation is key but we are not getting the right kind of documentation for our users so we started building POC demos starting getting RabbitMQ to the Knative.docs talks, blocks and at this point we started to get users to use them and to give us feedback and that was key with the user feedback, we documented even more and at that point we were happy on how things were going with RabbitMQ but we were not quite ready for declare GA for RabbitMQ. We declared usable when we integrate the repo into the release project, basically it's adding your repo to a YAML, having a way to release your repo in a GitHub action and integrating the tests with the Knative release process. At this point this was our architecture. Basically we have the broker and trigger that are the ones that receives cloud events and forwards them to a sync and we have a source. A source receives a RabbitMQ message and forward it as a cloud event. This is important because this is the only way that you have to forward events in the RabbitMQ architecture, of course not but this is the one that made more sense for us to integrate with Knative and the beautiful thing with Knative eventing is that you can use this as a RabbitMQ broker but if you change that type and use Kafka broker implementation it's going to look different but for the user it's going to be the same experience. You're going to get, if you're using broker trigger you're going to get a cloud event and forward it to a sync and if you use a source you're going to get that Kafka message or Kafka payload and you're going to forward it as a cloud event so it hides everything for the user. This was our responsibility as RabbitMQ experts to define this architecture. The orange bits are the one that are managed by RabbitMQ so this is not a problem in Knative. This has the wrong way of handling errors and handling the messages and the green bits are the one that we had to implement in Knative. The basically all of these receipt of adapter, dispatcher, the DLQ dispatcher, consumers and all that so I will talk a little bit about them in the next slide. One of the things that we had to do was define and quality gates. If you recall what I just said, this is usable. This is not yet GA. GA means general availability for the user to use your project so we needed to have these quality gates that we need to achieve for users to start trusting our product for them to integrate in the project so what is a quality gate? It's basically that a mail zone that you need to reach to get your project into a usable state. So, at this point we were implementing things as they were coming. If a user told us like, okay, we need 1,000 messages per second, we implemented. This is failing when I do this and this. We were fixing that but we didn't have a concrete set of goals that led our implementation so when we defined the quality gates was really a game changer for us. The first quality gate was, we need to achieve the 1,000 messages per second. Second one was, we need to maintain the at least once delivery guarantee and we, I'm saving two for the end or one for the end that was a user of us told us, okay, we need this and we saw that, okay, this need to be ready for GA because in our case we are not ready yet. Focus on the essential, setting concrete goals, let you focus on the essentials and errors and errors are also scenarios. We were, as you saw in the past slide, a lot of things were green, a lot of things were our own work to integrate with Knative and these things did not have any error recovery system so you have to be careful on how you do the retries, how you handle when one part of your system is failing because that is key on an event-driven system for it to be usable in other projects. And basically, user issues are critical. I told you that set concrete goals and focus on the essential is also critical but always a user that is using the project is in open source at least, is the keystone of your implementation. So when users start telling us, okay, this is failing, we need TLS, we need the MTLS also in our implementation that modified our concrete goals that we have, they weren't but it was all worth it because at the end we have already a useful system with all of these points that we may be missed because when you're implementing you have your own context of what you're doing but when someone starts using it from the outside, this is going to change the way you see your project. So basically when you set your goals, now you need to gather valuable metrics that allow you to measure, okay, I'm getting into those goals. How do you set those valuable metrics? You have to have in mind that not every quantitative piece of data is available metric. We have a lot of data available to us thanks to the revenue implementation but some of them were not key for our GA announcement like for example, how many bytes are we writing per second and are getting forward by the revenue system. That was something that if we follow that metric we will be in the wrong path and this probably will take us three or four months more than it took us. So you need to read exactly what you need and you need to integrate that into your development process or cycle for you to get a successful project. So K-Native provides a good set of metrics to consider and start there. Basically if K-Native is measuring that, probably all of the projects that are integrating with K-Native are measuring them so there's some good clues of how or what metric you have to use to start an integration with K-Native. Also use your quality gates, one that I explained before as a guide for your metrics. We wanted to reach 1,000 messages per second. That's a point where you can start thinking, okay, if I need to get to that goal I need to measure something that is relevant for that goal and that give us the clue to set two or three more measurements that we had in the metrics. Have a clear goal when benchmarking basically or benchmarks are adapted with things that we can measure and show to the users, okay, this is working. This was what we promised and in every release we run the benchmarks and update the images for the users to know we are still getting into that goal or we surpass it for them to be sure like, okay, this is key. I want to use RabbitMQ because they have these X throughput and they are maintaining in each release. So it's important for the user to see and to be attracted to your product, like, okay, this is your benchmarks and these are the guarantees that I'm offering to you. And also log certain metrics. We were logging a lot of stuff and we were thinking at some point that, okay, this could be used like a metric but no, logs is an event that is happening on your system and a metric is like a snapshot of your system in a certain instance. So be careful to take one as the other because at the end we realize that logs were throwing us off in the wrong path. So after that avoid common mistakes. Basically all of them is a list that is worth repeating of things that went wrong in the integration with RabbitMQ but also where we learned a lot about each one of them. I hope that leaving this here can avoid this pain point for any other future integration. So don't start implementing before talking about your idea, focus when in doubt, use the community and other channels. Go one step at a time. Don't buy it more than you get you. When you are going to implement something, go right into that path. You don't have to learn the whole project to implement something in eventing on functional and serving. Learn to prioritize about focusing on more important stuff and roadmaps and goals can be more important than the implementation. Hopefully this talk gets you a little bit excited or change your mind about integrating some stuff with K-Native. And I hope you all enjoyed. If you have any questions, go ahead. Thank you.