 Hello and welcome! This is Jenkins Platform's SIG meeting for November 18, 2022. Today we are lucky enough to have Damien DuPortal, Stéphane Merle and myself. We've got quite a big agenda today with some open-action items, some news on Docker regions, Java 19, web container support, container image application for Blue Ocean, of course, container repository management for Jenkins agents, Java 11, Java 17 and ChromeTip Resource Summit Remix. Unfortunately, Mark is not there today because he has open-action items. He will someday write a support page for web containers similar to Windows and Linux support page because there are tons of web containers that people use in front of Jenkins and we maybe need documentation somewhere saying if it's supported or not and give some pointers, so Mark has to decide one of these days. The next subject now is for Damien. Thanks for being here. Docker agents. So it's still something in the making. The paint is not dry yet, but you have something you want to address regarding the three repositories that we have for Docker agents. Do you want maybe to start with a status update on what we did on the actual images or do you want me to focus on the merge? No, why not? It's a good idea. Thank you, Damien. Go ahead. So status updates. First, Docker slash agent and Docker. So the image is agent and the image inbound agent, which is built on top of agent. Both images have been updated recently with the latest Chideka 11 and 17 version in bound dash agent. So both updated this week. So we feature now curl is back inside these images. So it is your life when you want as an user to customize from one of these images and then you run commands to get elements. Curl was highly requested by users. It was added back to the SSH agent a few weeks or months ago thanks to Bruno. And there was a huge debate about adding it or not. Should it be or not? My proposal was to include it again because the added size to the image on the worst case was for Debian and in the case of Debian, it's five megabyte more on the layer, which is acceptable. However, we don't install W gates. Please, if you really, really need to use W gates instead of curl, that means that you are able to decide to build your own image and update the gate in style W gates. But still we provide curl for a reason because curl is used to retrieve elements already. So let's keep it inside the image. There has been some issues fixed as well on the images. You can look at the issues recently changed. And the new thing is now we have update CLI which try to keep track of the GDK remoting images. And the Jenkins inbound agents now receive once a week an update if then agent has been updated. So that should ensure that we deliver faster when we have a new version available unless we're relying on the availability of maintainer. Okay, that explains your last message on Docker agent saying that in boot agent should be with a new version in a few days also. So because it's updated once a week. Okay, exactly. So let's see how it behave. But right now we did the first cycle. We still have an issue on these images. Regularly, the system in charge of deploying to the Docker hub tries to build an old tag which is two years old for all the free images. And that tag override the latest tag, which is a nightmare for a lot of users. So quite regularly, suddenly you go back in time with unsafe GDK version and missing features. And everything has been set up since one year at least to tell the Jenkins instance to not build tags that are more than six days old. So it shouldn't. And right now I've tried to debug directly in production at instant with the person in charge of the plugins on the core. And we still don't understand why it's being rebuilt. So in the upcoming weeks, we will try to directly disable this project and create brand new multi branch pipeline to see if behave differently. And if we keep having that issue, then we will migrate that deployment job from the current instance to the new one that we used to release Jenkins. Okay. So deployment error. So we are sorry for all the users suffering from that. We are trying our best, but it's weird. And the thing is Docker hub doesn't provide a way to lock or to constrain some of the latest tags. Either we allow a user to write it, all the tags of a given image, or we don't. We don't have a fine grain that will say that branch can only build that image. I see. And every time I see the same message of a user saying, how come there's a new old version appearing? No way. I can't believe it. It's amazing. That drive us crazy. So we really need the most probable thing is that that job is so hold and has received so much updates that we are sure there is something weird in its setup that we cannot reproduce on the brand new Jenkins modern instance. So creating a new job from scratch with the updated plugin should help on that area. Cool. Good luck with that. Sometimes randomly. That is our next focus. But yeah, that's the that's the status. So thanks to all the users. Some users helped us a lot by raising nice issues about IRM v7 not working for JDK 17. So we had to do a lot of back and forth changes. But now it's working very well. So thanks for this user. That means that people are using JDK 17, which is a good new. Yes, basically gives us the numbers of people migrating from 11 to 17 from time to time. And it's pretty encouraging. I don't have the most recent numbers. But at the beginning of November, it was really, really nice. Cool. One last thing. There is a discussion triggered about Arc Linux image. So Arc Linux image has been there for almost two years for the Jenkins base agent. It's not for the inbound agent. So I opened the topic, a draft pull request to create that new declination of operating system for the inbound agents team. One of the other maintainers said, I mean, why having so much different operating systems if we don't have the inbound agent requirement, maybe we should drop it at all. The thing is, right after that, I discovered that a lot of people are using Jenkins slash agents because we weren't good enough in communicating what is the goal of Jenkins agent. Jenkins agent today has the bare minimum for running an agent and the remote in jar file. And inbound agent only add a shell script. That's all. So shall we merge them or shall we move the agent remoting into inbound agent so Jenkins agent could be used as the same base for every kind of agent inbound and SSH. But in that process, that means we might have people using Arc Linux Jenkins agent since it exists. So we have to check the statistics on the Docker of the download statistics and see how much it represents on the total downloads before deciding if we should drop it or add it. So that's also my next step. Okay, speaking of statistics, this is not a trap, but the other day, you and I were trying to find some statistics about different architectures, you know, for the multi architecture images. Have you been able to find any input on which platform uses which image or is just a global thing and we will never, ever know which platform is using which image? No, I haven't had time to check the details, the statistics. Let me write it. I will check it on that part. Oh, so an action item for you. Thank you. Open action item. Oh, yeah. Okay, let me check Docker. So that's all for me on the status updates. For the rest, the proposal to merge the free agent repo, I have an action item opening an issue and putting everything I wrote everywhere on a single place. And I already have two users ready to help us testing that on their setup. So that's good. Yeah, indeed. And this week, it's minor, but we put back Git dependencies in order to be able to Git clone with Git protocol once again, because it had been removed along the way. So I was not trying to put a shameless plug. That was not the goal, just to share good news. Who doesn't love good news? And already merged. Thanks to you. For Git is back to the menu. Okay. Yeah, that's all for that topic. Thanks a lot. Next one. Unfortunately, we don't have any Jenkins core developers with us today, but it looks like they are still exploring Java 19. Does any one of you have any input on that subject? No, no, I don't either. Can we have an L desk issue for the Jenkins infra team to propose installation of Chideka 19 on the machines or the CI Jenkins IO so that the developer can start trying it on CI Jenkins IO? To add Java 19? Yes, right now. Thank you. Can I ask you, Stefan, to open the issue? And then right now, I'm opening it. Not to forget. Thanks a lot. Maybe at the end, Stefan, you could modify the documentation and put the link in it for the help desk ticket. I will try. Yeah, if you don't, that's okay. Thanks a lot. I like Damien to check out my English first, but with pleasure. Okay. This one has already been done. Come on. As you can guess, don't master Google Docs. Next subject, it was something for Mark and Kevin. Unfortunately, they're not there today. So we still have to do the container image deprecation for the Blue Sheen container documentation. Of course, everyone is supposed to know Blue Sheen is deprecated, but the container is not really deprecated. It's still supported, but it won't see any updates, any new features. And we also have old Blue Sheen containers, which are not maintained as far as I know. So we have to write somewhere that's the truth. It's not updated anymore. It's not secure to use them anymore. So just forget about it. But the documentation doesn't exist yet. So we have to do that. Oh, no, Damien. I've got a proposal for this one. Let me know. Is there someone against the idea of updating it one last time with a slip 10 seconds on the entry points, then a warning message, please don't use that image anymore pointing to the post and then slip 10 just for the sake of waiting a bit more. I know it's a bit rough, but at least people will listen. At least the users of that image. Very informative message giving status. That's pretty cruel, but efficient. I guess. Why not? Do you know who is supposed to maintain or use to maintain these containers or could anyone just make a pull request and see what happens? So I have access as administrator of the Docker hub to the registry. So we can always add images and push new references. But I'm sure there is Docker file hidden somewhere. I don't know, but I can try to search it. Got you. Most of the time when I ask such a question, the answer is go do your homework and you'll find out. Make a PR, please. That's the spirit of open source. Now, Damien, another round. I'm happy you're there for you. Container reposition management for Jenkins agents. Any progress on that? So I still need to write an issue now to get started. The progress is... So a reminder, the goal is to have a single location when we update JDK, remoting whatever base it's deployed on all images at the same time. We don't have to do a multiple step process. So we don't have to wait. Right now, the main challenge that I identified is how to split the different contents and that can be seen as what version pattern to use. We currently have a semantic versioning pattern for the SSH image. Because the SSH image does not depend on the remoting version. When you use the SSH image, Jenkins connect to the SSH agent and upload the remoting version that it currently uses with the correct version. If you upgrade Jenkins, it will upgrade automatically the remoting. So we don't need on the SSH agent any dependency to the remoting. The thing is the Jenkins agent, which is the base image, is currently bound with its version pattern to the remoting version. So the core of the proposal is that first we have to advertise user using Jenkins slash agent to start as soon as possible to move to inbound agent instead. Because both images do the same role, except on the feedback I got, people using the base image, it's because they use the Docker cloud plugin, which spawn agent based on container. And the Docker cloud plugin, like the Kubernetes plugin, they override the default entry point. Since inbound agent today only had a shell script and define it as the default entry point, I mean, that's why it worked, but it shouldn't. So the core of the proposal is that I want to propose a semantic versioning pattern for all images. So that means inbound agent will stop having the remoting version inside the version pattern. Or eventually, it could have it as a semantic version compliant post metadata. That will be 1.0.0 plus the remoting version. But since it's a really long version, most of the user don't even check. Because the main problem is given, the latest version of the remoting is 3071.git hash. Do you know with which version of Jenkins it's compliant with? There is a range of versions that work with that version, but which one? There is no such information. So the value of saying, hey, use that version dash four, because it's the fourth build of that image with that remoting version, that makes no sense. That's why using semantic versioning, since we have released drafter, that will gain more information for the consumer of the image. And that could help us saying, don't use the latest, use the semantic version. Always keep the patch version if you want the latest bug fixes that you have for your current line. And if you go to a version with the minor, then you can expect some breaking. And if you use the major version, then it will break everything. So that will be a change in the way it's handled. But I haven't met people right now really using that. They use the latest. So either they use latest tag, either like we do on the infra, they get the latest version, which is fixed on their Docker file. And then they use that fixed version. But it's just the latest and they try it like this. So proposal is have the same version pattern for every images. Each time we have a new release, it releases all the images, SSH and inbound. So you will have 1.0.0, 1.1.0, etc. And that's all that should simplify. Avoid the issues with the remoting and will allow us to have a single build. That's the core proposal. Okay, there is much more to this proposal than I had envisioned. In fact, it's much more complicated and it's not devastating, but it's a new concept. In fact, it's not just grabbing the three repos and grouping everything together. Okay. I hadn't understood that before. That's the long road. Just a note. I tried to give you a strategic view, but I will proceed step by step. First, we will keep this version pattern as we have today. We will start merging the source code because the goal is initially is to deliver more often and more efficiently. Then the version pattern will just be a way for us to automate the continuous delivery of these images. We merge a pull request. If we are in full summary, it will be clearly easier to say you merge a pull request and your release is automatically deployed and tagged. But that's the long road. That will be a step by step process. That's why I'm sharing the vision here. Then I will try to write on that vision on the issue and I will give a plan of the different steps. Some steps will be done, mandatory, some won't. Some will be discussed and refused. That's why first the vision and then the step by step plan. Got it. Thank you for this much needed explanation. So, it's not linked subject that we already discussed about Java 19 and required Java 11 on newer is still a thing for Jenkins core. But of course, Java 17 is progressing in Jenkins. And recently, I haven't seen any new issue appearing with JDK 17, because most of the problems that appeared with Java 11 were already solved by JDK 17. So, nothing bad to say about the progress of JDK 17 with Jenkins. All is fine for my point of view. I don't know if you have a different kind of input on that. We don't have yet. One of our next step is to move the Jenkins Infra controllers to JDK 17. And if it works, the plan is trying to move CI Jenkins IO to JDK 17 in January. That will be a nice goal demonstrating that we can do it with a size of this instance. Most of the people who worked on the JDK 17, including Basil, are pretty optimistic about that. So, for me, it's a good step. So, we are going to go step by step first infrastructure and then CI Jenkins IO if everyone is okay. The only bad thing I heard about JDK 17 and Jenkins was about using Jenkins and JDK 17 in a Docker image for ARM v7. Most of our users won't be bothered by that. And I think we already solved the problem by not using Jelink anymore. So, all is fine. And that will probably save money too because it's faster. So, we will save money on the engines that we spawn, especially on CI. Absolutely. So, more gifts at Christmas time. Which could be a machine, by the way. Anyhow, next subject is contributor submit remix. So, I haven't written the date yet, but it's on the 13th and 14th of December and it will three segments. I don't know how we will make three segments in two days, but anyhow, I think it's just two to three hours per segment. Because you can't just replicate two a whole day in a physical venue into something online. One full day online won't make it, especially because we are all over the world. So, that won't work. That's why I think Alisa Tong and Mark Wade shows a time slot that would fit for USA and EMEA and maybe the rest of the world. So, stay tuned. I think we'll already have something in the community Jenkins.io about that. And of course, there will be some blog posts on Jenkins.io too. The other subject, but I haven't progressed yet, is RISC 5. The machine is still there on my desk. It hasn't been booted yet. But I have found that there are several distro available, Ubuntu in 22.10, but also the latest Armbian is available for this world. So, as I'm an Armbian user, I think I will use Armbian. And we have several different OpenGDK versions from the 11, 0, 16 to 20. That was not something I was expecting. GDK's 24 RISC 5 architecture. Why not? Of course, I won't try Jenkins with such a recent version of Java, but even with 17, that would be pretty nice. So, my first goal is to run a Jenkins agent. And that's no darker, just a Jenkins agent with Java. And that's all. And hopefully, I will keep you posted. That would be a nice Christmas gift for me, having a Jenkins agent running on an unknown to us architecture. We'll see. And unless you have any comments, feedback, questions, I think we're done. Bruno, I put the link for the new issue that you can add in the... Oh, where did you put it? In the chat, maybe? In the chat. In the chat. Oh, thank you so much. You should be able to edit the notes, Stephen. I don't want to not see you. I don't want to miss anything that Bruno is saying. Oh, too kind of yours. When you... the date for the Contributor Semix remix, I don't remember. Yeah, sorry. I will write it down. 13th and 14th of December. It's our door. It's, you know, in a few weeks from now. Cool. Anything else, folks? So it's Tuesday and Wednesday. Yeah. Never on Monday. Anyhow, I was really happy to have you on this call. The recording should be available in 24 to 48 hours. And see you in two weeks from now. Thank you. See you tonight. Bye-bye.