 Hi everybody, welcome for this new Jenkins infrastructure meeting. So yeah, in terms of activity, it was a pretty calm week. So the first topic that I want to cover is the Amazon account that we are using in a Jenkins project. As a reminder, we have two main costs there, the machine package Jenkins.io. And the second one is the CI environment for the agent for CI Jenkins.io. With the work that Damien has been doing on CI Jenkins.io, we transferred part of the cost from the Azure account to Amazon. And we are now investigating ways to reduce the cost because the last month was $50,000 for this month. And obviously, we have to put down that invoice. So we are currently looking for ways to reduce the cost. And yeah, basically that's it. If you have ideas, want to contribute, we are either, sorry, I need to catch up there. So last month's spend, we wrapped a $50,000 spend. Yes. Wow. Okay. And that's on AWS. Yes, exactly. Okay. We saw an increase on CI Jenkins.io mainly since the beginning of the summer. The issue in Fra 3097 has a bunch of details. I've extracted a lot of the costs to cover different, there are different rules proposed to go. It's just an information. If you're interested or if you want details, that should be a specific topic. But it means that on top priority, the issues associated to that epic will be taken on the most priority item for the three upcoming weeks, the goal being trying to decrease the cost as soon as possible. Okay. But, and I had misinterpreted the number, you said 15 as in 10 plus 5, not 50 as in 5 times 10. No, no, 15. Sorry. 15. Perfect. Okay. I just, I just heard incorrectly. I thought, oh my God. No. 5x increased our spend. 50% increased our spend at least and that's still too much. Okay. Yeah, exactly. So we, as I mentioned, most of the cost is related to CI, the Jenkins.io because we are using Amazon accounts to run jobs there. And we are now trying to investigate the best way to reduce the cost, either reducing costs for other services such as packages, or better use agents. I'm sorry. That's really the beginning of the fourth. Sorry. Mark. And should ideas go into that ticket? Where would you like ideas discussed? I think we prefer to have the tickets. Okay, great. Since Damien already did a lot of work to explain everything there, it's just better to continue there. And a special thing for Jesse Glick to help us. He gave us reminders, ideas, and he double, he tend to double check most of the changes we do on the CI port. So that's really cool. Because here the purpose is either to use the right size of the infrastructure or maybe to optimize the way we trigger jobs. I mean, every idea is. Yeah. We already have a set of seven issues that can be taken in the coming weeks that will for sure decrease the cost. And we already worked during the past 10 days on elements that will have a direct impact. Why do we are a good last question regarding the costs? While we are discussing Amazon costs, I would like to continue the discussion about packages that are your machine. Because, yeah, because of muscle packages that are you is the machine which hosts to the service updates touching in the package that you know, and we hope the Jenkins yet or which is deprecated. As a reminder, this machine is not synchronized with puppet anymore. So it became a snowflake. And because we are looking at the cost on Amazon. We also know that that machine is just oversized. So we can either reduce the size of the machine or just move it to a different cloud provider. So we are looking at the costs. The cost of the machine is not that much. The cost of the outbound transfer from that machine is a lot. That machine emits 50 terabyte. So I said 50, like 50 terabyte per month of outbound data. Which is almost 4k per month in the AWS. So it does not represent the major cost the Amazon accounts, but the cost is significant to find a better solution for that service. So we started looking at it with Damien. Sorry, yeah. So, so this machine is today a snowflake. And in the future we'd like it to be. We could consider separating the services out. That's one of the idea. Oh, that's one of the idea. The reason for that is because that machine is really critical for the Jenkins project because that's where we package artifacts. We are currently maintaining mostly by hands, which is, I mean, that's really comfortable. And it's represented costs. So just to give an idea because of the because of the network bandwidth that's happened on that machine. We have to use a pretty big machine. Like 62, 64. She get better from a country member. But in the end it's many an Apache machine. So the machine is just way oversized for what it's doing, except it's the network bandwidth. So we definitely have some optimization to do there. On top of, yeah, better managing it. So that's something that will start looking the coming weeks want to see how we can improve. But it's always, it's all related to the medicine costs. Next topic that I think Damien put in the agenda, which is traffic ingress controller. So we had some recent activities and that were just if Peterson was interested to contribute to the increased resources that we use on the Jenkins infrastructure projects. As Damien mentioned, I think we'd be better to start with a private ingress controller. I think it's, it's something that would be nice because we have to update ingress resources anyway for the next major upgrade. Damien, Damien, if I'm wrong. But so yeah, that's something that you have to do now because it's, it's mandatory for the next. I mean, maybe not yet for the month, it's not yet mandatory, but it will be needed to for the 20s version 1.22, which is coming. That's a topic that have been proposed by Olivia and Joseph at the beginning of the year. But since Joseph had a lot of a lot of bandwidth or already taken and because that change does not map to direct feature to our end users, it will be more. It's a kind of investment in the future to make us more portable in Kubernetes area because traffic has less specific annotation, even with the new ingress model, you still have specific items by using traffic will have something that costs less in terms of resources that should ease our way of managing let's encrypt certificate and should help us if we have to switch Kubernetes implementation. But it's only investment and it does not provide something really useful. That's why we never prioritized and if Joseph, so Joseph was unable to attend the meeting today, but if Joseph is willing to work on that, that could be a cool contribution. So the idea is to let him work and see if we have a nice outcome and we are able to switch from engineering to traffic at least for the private ingress, meaning every services behind the VPN for us. That will be the next milestone and we can do a retrospective once we are there. So as a reminder, I just want to show what we mean by updating the engineering resource resources. So let's take this one. I think we have ingress templates here and ingress. So it's not really useful. Not as much as I wanted to show you, but we are still using the old annotation for the ingress resource and we have to update to the new one anyway. So we can take that opportunity to switch to traffic. That's the idea. We are currently running communities 1.19. We have an upgrade to 1.20 coming pretty soon. The thing is this annotation, this structure, this ingress resource is deprecated in version 1.22. So that's what I mean by, I mean we have to work on it sooner than later. Yeah, it's not completely deprecated as I understand. It's the version of the API that allows additional objects next to the ingress. But the ingress rule still exists. It's just it might change on its structure and it's a breaking type change. Okay. Next topic that I put to the agenda. I'm not sure if it's something that should be discussed here or during the governance meeting, but considering the recent priority change, I'm just wondering if we should just update the roadmap to highlight the work that we need to do on the Amazon accounts. And also something that we thought I forgot to mention regarding the Amazon accounts. We need to reduce the cost. That's one of the things that we need to do, but there is a second thing which is transfer that Amazon account to the CDF because right now that account is maintained by CloudBees. So CloudBees is paying the bill, which means that in order to access that account, you need to be a CloudBees employee. So it's not really convenient to collaborate with non-CloudBees employees on that account. And CloudBees is interested to not be responsible for that account anymore as well. So that's why it will become a priority for us. And a special thanks to RV and Mark for updating the roadmap with items that have been done in the past months or being done. So thanks because that's an important way of communicating outside. And we thought that you forgot about doing that in the past months. So thanks for that. That's really important. Next topic is about trading issues. So we have quite a lot of issues in the JIRA projects. We started this week. I mean, we started several weeks ago, but more this week closing all tickets. So if you have some time, that's really useful to just go to the open ticket and just close those that are not relevant anymore. Yeah, we are closing as much as we can. But we still have a lot to do. And still the issue, the big, we got access to the GitHub issues. So we have access to the beta GitHub issues. So we cannot have a project across a week. I mean, that was a really good case in the past, but we just have access to more feature. I'm not really familiar with the new features. So I still have to play a little bit with that. But yeah, if you are comfortable with it, some feedback would be greatly appreciated. And also under issue 3-H something that Olivier and I discussed this morning is that we saw we use and we still have a lot of support like issue from end user that ends up on the infra projects in JIRA, mostly because we don't really provide on JIRA to a user opening a new issue. An easy pass that's clearly underlined that infra project is only aimed at Jenkins infrastructure. That's why most of the users might end up here, for instance, with the recent project being put on the top of the drop-down when creating an issue. If a user visited a few hours ago an infra issue, then the next creates form will direct them to infra. So if anyone has the skills or knowledge on JIRA to help us on something we could do on that, that will help a lot the issue 3-H and that will help a lot people opening issues because most of the time they have to wait for us to pick the issue and see it. So there could be an improvement there for our end users that report issues there. Yeah, most of the time we have a lot of people opening issues that are not, I mean, which have that have nothing to do with infra. And from time to time we see someone who's, I mean, knowledgeable about the Jenkins project and still open issues in the wrong project. So that's why we are thinking that maybe it's not, I mean, clear enough. The last topic that I have to the agenda, so which is I saw a lot of discussions about Algolia and Docksearch. There is something that I don't understand because Algolia is sponsoring the Jenkins project for plugins that Jenkins.io. So we have like 1000 credits that we can use and it's using the plugins that Jenkins.io. But how does that work for Jenkins.io? Yeah, so they're two very different approaches and for very, for what I think are valid reasons that they are different. Algolia offers a product to open source projects called Docksearch. And that product will index a site and then all the open source project does is inserts a JavaScript snippet into their pages and that makes the search accessible. So we didn't even have to have an account with Algolia in order to use their Docksearch. And in fact, we don't have any control over their Docksearch configuration directly. They decide how it's done and they decide what they index it set to. What we did today was open a support ticket to see if they're willing to extend what it could, what it indexes. Okay, that makes sense. Now plugins.jankins.io is different and they recommended actually that it remain different. They recommended it be different because plugins.jankins.io is largely data driven and because it's data driven, a text-based indexing system isn't as effective. We need the ability to tune it. And so Gavin maintains that and does a very good job of watching the search results when we get a high hit rate on missed searches, he'll tune it to assure that people who ask that question get an answer. Okay, thank you. So while I was in the Algolia accounts, I just invited Tim and Elvin this morning. So we have more people there. Well, I'd love to have more people looking at the search results. I've been reading them each month when Gavin sends them out and it's good insights that are offered by their search engine. What kind of information do we have there? I must admit that I don't often go to that interface. Any last topic that you want to discuss? The Algolia site will only show us plugins, right? So you will see this here, but it only shows plugins and already on plugins, we easily use half of our allowed quota every month. So it's actually, we're making very good use of it. Which is great. It is. It's really, it's been an excellent result for us and I think for Jenkins users. The user to the search. Yeah, so that allows us to know, yeah, it's pretty cool. Well, it reminds us things like how we should keep investing in documentation on those, some of those key top items, Docker and Kubernetes. And so it's worth it. Absolutely. Awesome. Thanks for your time. Then I propose to stop the meeting here. Have a great weekend. Bye.