 Welcome everyone, this is Jane King's platform, Signiting will earn April 9th, 2024. Today, that just the two of us, Mark, Wade, and myself. Hello, Mark. So for today, we have quite a few items. We'll talk about the agenda, Java 21, Docker Hub, the release work on the agent and control images, the current work in progress on images. If we have time, Docker base, quick start tutorials. And that's all, Mark. You have something to add to the agenda. Oh, that's great. OK, thank you. Then first of all, agenda-related items. With the government messing with all clocks, so it has changed. At least for me, I don't know for you, Mark, but I guess so. So it's now one hour later than it used to be during the winter time. So would you mind if we schedule that one hour earlier or would that collide with other meetings? That sounds great to me. Let's move it one hour earlier. Thank you, Mark. I'm afraid the action item is on you because I can't modify the kind. And I'm going to do it right now, so it will be scheduled. Now, time zone right now is UTC. Do you want me to switch it to Paris time so it doesn't change for you? Yes. I'm not a master in time zones and so, but I hope that will also work for you. We may have one of two weeks of mayhem because I think you changed time before I do. But yeah, I think it's it's very reasonable for us to say that we're going to set the time based on the leader of the SIG. So why not? So that's Western European time. Oh, dear Western European time. I see not UK, not Madera, not Lisbon, Berlin. I think it's the same one. Paris, there we go. Central European time. All right, got it. So setting it to and now I've got to be sure that I get the correct clock time. What time is it right now in Paris? It's seven, three past seven. OK, so seven o'clock PM. All right, so seven PM Paris time. Save this and following events. OK, done. So you said you wanted it. Six, you wanted it. Six. Yes, please fix that. My mistake, six PM. So I successfully retained the exact same time. That's not what we want. OK, this following events. Got it. Thanks a lot, Mark. And now I have a very bold request now that you have changed the time not available two weeks from now. So should we cancel or would you like to host this meeting? I should be available as far as I know. So I'll host it. Oh, thanks a lot, Mark. Lazy French people taking some time off. Well, Mark, we'll host a meeting. Thank you, Mark. Now, on to the next subject, is there anything new regarding the Java 21 support plan or? Yeah, yes, yes, sort of. So I've I've involved I've the spring project announcement is getting an awful lot of my attention and trying to understand what it will mean for us to transition Jenkins from Jakarta, eight to Jakarta, nine. Jakarta, eight to Jakarta, nine. And Spring Security, five to Spring Security, six. And Spring Framework, five to Spring Framework, six. And Jetty, 10 to Jetty, 11. There there's a really great epic in the Jenkins Jira that I will put into this link into this. Thank you. That Basel created about 18 months ago that outlines the many different things that have to be touched by this. So 18 months ago, right. It was August of twenty twenty two when he did the initial research and and but I'll link it into the into the meeting notes so that we've got it. OK. Now, the next subject Damien wanted us to address is to have a look at the Docker hub download statistics for twenty twenty four. So I had a quick look earlier today and I don't think I said that lots of people are using the LTS tag, which is a good thing. And probably lots of them are also using latest for the agent. And as Damien always said, friends don't let friends use latest. So people out there listening to us, you should use a pin version. They use something like depend about a VT and I or anything. You would like to keep things up to date and make a PR if you can when you. OK, that's just if you have infrastructure as code. Of course, if you do everything by hand, well, you're on your own. But yes, using latest, you could have some bad surprises. That's not always a good idea. But I've been there. I've done that. Well, now that I try to use pin version just about everywhere. Anyhow, so it's the same pattern for February. And even for March. So that's a good thing that we know that people are using the LTS and not really the weekly, except people who like adventures. Yeah, I don't know what they mean wanted me to say about that. Frankly, I'm not. Big static six fans. I don't know what kind of inside I could get from these. OK, so well, so are you OK if I talk to it for a minute? Oh, yeah, thank you, Mark. OK, so so let's go back to the genuine let's see is go to the switch to another tab, one that has there we go, this one. All right. So there are some key things here for us. Worry wise for me. Oh, row row number four, Jenkins inbound agent version four dot nine dash one is very old, very, very old inbound agent. Long ago, switch to a much, much more verbose version numbering scheme. But there are three hundred and thirty thousand poles of that thing. However, only from less than a hundred unique IPs, which is is OK, one, why aren't they caching them if they're why are they pulling so often instead of caching and that may indicate somebody's got a misconfiguration or we're shipping something that's misconfigured. And so there's a there's a danger, for instance, if one of our Helm charts encodes that version number and we're not updating it, we should. So that that was a concern for me of that one on the top of the list. The the other thing that caused me concern was looking in if you look, go to the March the March tab. And here, right. So here, look at row 10. Two two hundred sixty three dot one is a very old release, yet it's been pulled. It was pulled eighty eight thousand times by forty six IP addresses. So so again, somebody is using an awful lot of Docker bandwidth that is a relatively small pool of users. If we compare the the numbers of downloads, same thing at row three there, right, where we've got three hundred thousand downloads of the four dot nine dash one agent image, but only sixty IP addresses. So it's what is that three orders of magnitude different than the controller unique IPs? Yeah. So for me, there's there are some hints here that we may need ways to tell people we can't tell people stop doing this, but we may want to check for possible misconfiguration in any of our own samples that might still be using these outdated container images. Now, you mean the documentation of samples is something we should search for, for example, the four dot nine dash one, right? To see that, hey, we're not not somehow recommending it. And the places we might be doing that it might be in helm charts. It might be in in Jenkins operator. There are all sorts of places where we could reference that. And the other is they may just have bad practices and they're never updating. And if they're never updating, then we can't we certainly can't stop them. They are free to stay on that old version if they wish. Yes, that version won't work with modern Jenkins versions, by the way. That inbound agent is rejected completely when you attempt to connect it, if I remember correctly, to. Current versions that we support, it will just refuse. It says, I'm sorry, a remoting version is too old. Yes, and I also saw some jdk8 references in January, I think. Right. So, yes, we can't do anything about that. We won't rebuild, you know, just to add a warning message or whatever. We can't and we want to do that. So whenever if we never find any reference to these old versions in our own repositories, then we just can't do anything about that. Right. And and that's that's we are grateful that Docker hub is that Docker is willing to continue hosting, right? That's that's very and and we're probably not the heaviest consumer of Docker hub, right? There are there are certainly heavier weight consumers than we are. But being aware that this is is at least for me, interesting. Thanks a lot for coming to my rescue, Mark. I was kind of lost with the statistics. The only thing that I wanted to see that I was not able to find in these statistics were about the platforms. I wanted to know if some people were using arm 30 do, for example, or as 319 X and see the ratio between AMD 64 and 64 and the rest of the architectures. But unfortunately, I don't think Docker hub gives this kind of information. Yeah, I don't know. I don't recognize it. No. Whatever. Thank you, Mark. And this is linked to another subject, which is would we have too many Docker images? So most of the time, we try to help people when they are requesting, you know, I'd like to have another image with that tool installed when it makes sense for the infra for the whole Jenkins community. We do it. But from time to time, we have to refuse because we just can't make a custom image for each and every end user that who needs it. So this week, we had an end user asking for a specific image for easy to because it doesn't the latest version of what we supply doesn't work for them anymore. But it just happened to work in the past without any specific action on our side and now it doesn't work anymore. So the user was asking if we could supply another image that could work for him. But the answer was, unfortunately, no, we can't. That's not reasonable. That's not something we want to maintain. So you're on your own, but that's not a bad thing. Yes, sometimes users have to build their own images on top of ours, and that's not a bad thing. And to tell the truth, I'm doing that on a daily basis because it's not always customized to your own needs. So you have to do it by yourself. This is also linked to another subject, which is the Docker hub that keeps sending HTTP for 29 hours. I think Damian and the rest of the infrared team has already digged and know some things about why it's failing. I think it's partly linked to the fact that we are only using one IP to push to Docker hub. And it's not a weight limiting thing, but. I'm where it is. I don't know. It's kind of complicated. But the thing is, the more images we have, the more bandwidth we use for Docker hub, the most different these images are the most layers we are using, the more bandwidth we are using with Docker hub to build and push and even pull back the first layers we need to build. So all of this may help if we shrink the number of images, if we have a common path to build these images. So one of the subjects that we will work on in the coming weeks or months is that we should try to build our Docker images starting from the Tamarin binaries and not from the Tamarin Docker images starting from Ubuntu. I'm not mistaken, Jamie these days, so that we have fewer layers and the same layer for lots of different images. So I don't say you and I should do that, Mark, but we have to talk about that within the Jenkins platform sign meeting. And if I ever have the time to, I'd be glad to do that kind of things. We have already done so when we were experimenting with Java 21 preview images, because at that time Tamarin was not supplying on a regular basis their Docker images, but they were supplying on their GitHub repo the binary releases of the Tamarin JDK. So we were starting from the JDK and then importing that into an Alpine, a Debian and all the operating system images. So we may go this way. Once again, I know this has been a discussion for quite some months now. I was pushing for this solution using the binary and not the Docker images for whatever reason. And I'm not happy that we are going this way because I would have liked to convince people that this was the way to go. We are kind of forced to do that now. So I'm not as proud as we could have been. I realize I talked too much, Mark. We'd have any comments or questions. I feel no shame whatsoever. I think we chose that path and now we'll switch paths. We're going to... Damian has noted how much value it will be for us to switch to using binaries. I think it is a nice thing to switch to use binaries. And so let's do it quickly and get it done. Yeah. Okay, that's cool. And we already have example that used to work, but that shouldn't be too much of a big work. Cool. Next. The controller weekly have been released, hopefully, when I... Yeah. So there are tags and not releases. As far as I know, yes, there is. I have not been featured yet. I know Mark. Shame on me. No, it's okay. You are the only one to do that manually. And I don't know if we should do that via a script or something. I don't want to replace you by a script, but you've got already so much on your plate. So the releases don't exist yet, but I guess they will exist in the following minutes. Well, and more embarrassing, we didn't get one recorded last week either for the Docker container. So we did get it on the Jenkins side, but not on the Core side or repository, but on the Container side. So let me do that. Thank you, Mark. On the Docker agent, while Mark is working, we have just one version bump that led to one new release, and it was just that, you know, what is used for the test that was moved to 1.11.0. And for a CCH agent, my oh my, we had lots of version bumps and refactoring, which led to five new releases. The thing is, we did not see that we were not following the JDK new releases. Somehow it got lost in the translation. So we were several weeks or months without updating the JDK versions. Yes, that happened. So that's why now we have seen lots of version bumps because we put it back and we also had some refactoring. So as for Windows, the LTSC-29, it was in 19 images now use LTSC-2019 instead of the 1809. The Windows images are now multi-stages. We also burned the Babs version. We added the JDK-17 as default for Windows. That was not the case beforehand. And we added the JDK-21, which was not available, but that's OK. We still have time before Jenkins moves to JDK-21. But anyway, we're happy with that move. We also added the Windows LTSC-2020 to support. And this one, the correct Git LFS installation was a long-running draft PR. But it finally managed to get it to work. So it bumped its version to the 3.5.1. The major one I was talking about a few seconds ago was the rename element of the JDK update because, yes, we were not updating JDK anymore. We also brought back the Debian JDK-17 images. And of course, we bumped the various JDK version to the latest available. What else? Yes, we used the official tamarin build for Linux S3-19x because beforehand we were only using the JDK preview for the S3-19x platform. Now, for the work in progress on images, we still have the adapted DTLI JDK-11 and 17 manifests for Windows. That's a long-running draft PR, but it will take some time. That's OK. And for the Docker agents, we still have three PRs in review, or at least work in progress. I would say I'm not so sure they are in review. So I think this one, the automatic PR by Update TLI, could be closed because we are not working with the JDK preview version anymore except for the ARM32, except that tamarin doesn't supply any JDK-21 or the access version for ARM22 anymore, 32 anymore. So I guess we could close this one. This one, that was just a proof of concept, the Liberica ARM32 version. So close it or just leave it as is. And this one, the Traxubin 2-ADMN LTS releases, as I said, two weeks ago. In fact, it was a failed attempt. So it's a much bigger problem that I will have to address in a way or another. I think I will make three different PRs for this one so we could also close it. It's not really relevant anymore. And as for Docker SSH agent, we still have one which is the bump of NSSH29.5 and so on. But it fails for several Windows Docker images. So we will have to investigate before going any further. Now, next subject is about the Docker base quick start tutorials. So there was a midi drama last week. Now, just two users, two newcomers were kind enough to let us know that the tutorials weren't working for them anymore. At the very beginning, I was kind of skeptical because it worked on the CI. It worked on my machine. It worked on various machines I have access to. But the truth is my Docker versions and Docker Compose versions were not the most updated versions. So people having recent versions of Docker, Docker 26.0, I think, and Docker Compose 25 or anything newer than 24.4.27, I guess, were experiencing problems because we were using some kind of edge case in our Docker Compose. We were using the same name for the various container representing the agents. And that was not such a good idea. It was really handy for us because it allowed us to use one big Docker Compose file and having one agent defined in a jcask. How would the guy call that? Manifest, config file. So it used to work. But when Docker decided it was nice to push a fix to this, you should not have several containers having the same name in the same Docker Compose file. Well, our QuickStart tutorials weren't working anymore. But we found a way to correct that. And we have pushed it to the QuickStart tutorials repo. And we have also updated existing tutorials on Jenkins.io. And Mark, you were kind enough to test it up to the end because, of course, I tested it, but it's better on another mission by another person. So it worked for you. And we don't have any news from the people that created the issue yet. But I'm hoping that that will work for them from now on. And I did, I would say, a quick hack, a quick and dirty hack. I have now an update CLI manifest that checks the Docker and the Docker version, the Docker Compose version that are used within the Git of Action CI. And that's right then into this Docker version.txt. And this is put into the readme.md, which is part of the repo. So now when you read the readme.md, in the end, you can see the version that these tutorials have been tested with. And as you can see, these are older versions, but that's what GitHub gives us. So that makes no guarantee that that will work for you. But that's just an indication in case you would like to open an issue in this repo. You know that the version that were tested are these ones, and maybe yours are different. So you could just tell us within the issue you're having. Mark, any common question about that? Thank you for doing it. Oh, you're welcome. It was frightening at the beginning because the tutorials aren't working anymore. Wow. But then in the end, I was happy. And I discovered some nice things about Docker I didn't know. And I thought I had understood profiles before that problem. And frankly, no, I had no idea how that was working. Now I think I know a little bit better about what's going under the hood. So yes, it's working now. I'm not so sure we have to make another subject about the emperor server. It's still running, right? It is. It's been relocated physically and successfully relocated and is now sitting in a different part of the house doing what it does. OK. No, not but. And still working. Yes. Correct. OK, thank you, Mark. Any other subject you would like to address before we wrap it up? I think that covers us. OK, thank you, Dan. The recording should be available from 24 to 48 hours and see you four weeks from now. And of course, Mark, you'll host the next meeting two weeks from now. Thanks a lot. Thanks. Bye bye.