 Hello, this is Jenkins Platform Sieg meeting, December 16, 2022. Today, we have Mark Waite, Damian Dupoteau, and myself, Bruno Verstein. We have quite a big agenda today, and that's the last meeting of the year 2022. In the Open Actions items, we have the platform support containers, and Damian, the floor is yours, tell us more about Windows 2020 and Windows 2019. Okay. Windows 2022, that's the new LTS platform for Windows containers and virtual machines. The Jenkins infrastructure team is working on adding Windows 2022. Results are really promising. The thing is, we will have to maintain both at the same time, because you cannot run a 2019 container and a 2022 machines, and neither the other way around, unless you use virtualized container, which means an Hyper-V virtualized process for a container. That works, but that's quite complicated to run and it has a cost penalty. So we will have to maintain both architecture, which some is not really an issue, but that's why it took us some time. It's not just replacing one by the other. So considering the challenge there, is Windows container support worth our effort? I know that's a terrible question to ask, but tell me your perception of it, just because there are so many good things we've gained from container support on Linux. Really, I love container support on Linux. It's great, but container support on Windows has just felt like one challenge after another. Do you see it continuing to evolve? Is Microsoft actively doing work to promote it, or is it rather it's going to disappear like Windows Vista did someday? So the Windows container support is not going to disappear. It's clearly used, not as much as what we have on the other platforms. The question is really good for a Jenkins controller, and that's worth asking the community. Are there persons running Jenkins controller inside Windows container? Because clearly we could benefit in maintenance at not maintaining a Windows controller. However, for the build agents, for sure it's required, and both of them are required. That's a sweet spot, and that for me is a helpful partitioning. If we were to actively plan to drop controller support for Windows containers, we may reduce our burden and not dramatically affect Jenkins users. Whereas agents, I know my world, I have one controller and 50 agents, right? So I'm much more likely to need a Windows agent than I am a Windows controller. Good, that makes sense. Thanks, OK. But for sure we have persons complaining for both platforms, more for the agents, but we have people who noticed that the Windows controller are dead since a few months now, and they build their own. But it took them a long time to notice that, and they took some work around themselves. Yes, but we have to be careful of the bias there, is that the persons were ready to face the challenge of running Jenkins controller inside a Docker Windows image. They know automation. So they are absolutely able to build their own custom image by forking our repository. It's not an effort for them because they need that kind and that order of magnitude of efforts. So the thing is that it's not because they don't complain that they don't exist, that's the thing. OK. I'm not sure, besides the download metrics from the Docker Hub that we call a retrieve, I'm not sure if there is a way for us to ask the community on different channels. I think asking the question is worth the effort, but I'm not sure which channel or set of channels, like having, I don't know, a QCM, Google, whatever form. Google form, I think that's the name. And we can push, can say on different communication channels, hey, we need your advice if you're using Jenkins with Windows. Well, but OK, so I think we may have some of that data, at least to present a subset. One question that I believe the stat system could already answer is what fraction of controllers are running on Windows at all? There was two. And if we could answer that, and if we found that only 20% of controllers are running on Windows at all, then it might say, therefore, cannot be more than 20% that are running on a Windows container. And that may lobby, all right, therefore, we may be able to justify non-support of controllers on Windows containers simply by the ratio of non-Windows to Windows controllers. Absolutely. But now I don't think we actually publish that data anywhere that I've seen. And it may be that I'm wrong, but the hosting platform may actually not be available in the data that's published that's sent by controllers back to our statistics gatherer. OK, I'm not sure where those statistics are. Exactly. Is it stats Jenkins.io with the plugin installations? Well, the statistics are actually shipped to a different location, and then Andrew Bayer runs a reporting thing that updates stats.jankins.io. So yeah, the collector is somewhere else, and the collector has very intentionally limited access to the data that's collected because there's a fear that some of the data might inadvertently be more sensitive, and we wouldn't want to risk exposing the raw data to anyone. OK. Yeah. Yep. Go ahead. Sorry. Go ahead. I guess if we had the form, everyone will answer, yes, we want it. I want the second question to be, but are you really sure you want to maintain it, just to see that the oak we can correlate the yes to both questions? And this question was about the statistics from Docker Hub. Would you be able to differentiate the use, the pools from the Jenkins infra, from the pools for from elsewhere, the real users, not ourselves? Not right now, easily. I don't know if the IPs are available. If the source IPs are, then yes, we can. Otherwise, we won't be able. Not so sure about that. Yeah. But yeah, that's a good point. Let me think aloud. No, for the controller, that's sure it will be easy. We don't use Windows controller at all, so you're sure it's not us. That's a good thing. And we have no compelling reason to use Windows controllers, right? Yeah, they're just I understand there are businesses that do. There are organizations that really have compelling reasons to use a Windows controller, but the Jenkins project does not. Exactly. But yes, for the statistics, so that because that's also the next topic, following discussion with Adrien Lechard-Pentier and then Jean-Marc Messon. It sounds like that there is a compelling topic for us to be able to store metrics that we could consult and retrieve. So I think we, that's a generic we, let's say the first person to experiment should be able to help the other, to test, oh, it sounds like Grafana could be used, connected to a PostgreSQL database. Which means if we have a Prometheus on the PostgreSQL database, which access requires authentication, but it's public data, which means anyone on the contributor could get an access on the infrastructure. We could have a kind of process where we use CSV. In my case, as a Docker admin one per month, I get the CSV export. I insert it in one of these two database and then we can get this information with Grafana. Why Grafana? Because anyone can authenticate with the LDAP account and then create their own dashboard to try to see things based on the data. That's the third compelling case in less than one month. One is the public, plugins, public metrics for trying to score them or see tendency. We have the, Jean-Marc is working on a subject that Olivier Verna and Kosuke back in time try to also work around. These are the community numbers to see how much pull request issues help this kind of metrics. And now we have the Docker hub metrics to justify whether we continue maintaining. All of these metrics are really valuable and could help us take decisions. So it sounds like there is an infrastructure topic for 2023 on trying to build that kind of platform. Before that on my side, specifically for the topic today, with the area of Christmas, if I'm a bit bored, I will try to get a CSV and see if I can put it on a Prometheus and Grafana database. So if it's that easy, then I could provide access easily. So anyone can take decision without only me being the best factor here. Good. So we've got one more data sample on your Grafana and database topic. Basil Crow created a tool that will read the Jenkins update center and convert it into a SQL lite database. And so we've got an, and I tell you it's impressive how easy it is to access update center when I can sit inside a SQL query statement and do from, where, et cetera, group by all of the things that SQL can do. So that one is also another example of the same kind of, if Grafana works well with SQL databases as a reporting dashboard, then we might want to push update center into that same database as a separate set of tables, et cetera. And now we can do even more things in terms of what information do we already know about those kinds of, about all the plugins, about Jenkins core releases, et cetera. Absolutely. So that means I have two information to get from my experiment, except justifying it's because it's fun. If it worked, it could be interesting to see how much request we have from Arclinux and for the Windows platform for controllers and agents. Good. Thank you. Wow. Nice. Thank you. Yeah, that's quite a subject. It looks like it's important for lots of people and that's really interesting. And I guess you'll, we have time between the Turkey and the cake at Christmas time. So that you will implement all of that in a heartbeat. I always get excuses to run away from the family during Christmas. No, I'm not joking. But yeah, and for the rest of the tasks that were my action item, I didn't have time to run them, so we can jump. To be done, not done. So that we get it, of course. But we knew before whiting them down that that's subject that we will address. We'll talk about them each and every month. Not maybe we won't be able to drag them, but yeah, that's okay. There was so many open action items, we can't do them all. Now, JDK support for Jenkins. So we created Java 11 or new world for Jenkins Core is progressing. We had, yeah, that's quite a milestone. The build tool changes arrived in parent.pom 4.52. So now we have to use Jenkins 2.361.x.4 when I'm talking to, that's the minimum version we need to use in order to use the parent.pom and Mark, Jean-Marc and I are trying these days on a set of plugins to upgrade them and see what's happening. And it's mostly going fine. Mark, do you have anything to share about that? So we love that Bazelkrow created a build tool chain, build tool chain changes block codes. Posted yesterday, nice popularity on the tweet was sent that mentions it and some nice feedback in the Jenkins developer list related to it with good involvement from people thanks to an article that John Mark Mason posted on community.jankins.io on his experience. So it feels like, yes, we're progressing and yes, we're alerting people that the Java 11 transition is a multi-step transition. Jenkins Core has now required Java 11 for multiple months. Now it's transition, next transition is plugins start making the transition to require Java 11 and that means they have to be based on the version of Jenkins that also requires Java 11. So, and now a secret to share with the both of you the Git plugin and the Git client plugin are both in progress of evaluating their transition to require Java 11. And as part of that, the Git client plugin has to make a change in its major version of JGIT. So whereas it was using JGIT five previously it will now be using JGIT six and therefore there's some healthy concern there to be sure that there are no surprises in that transition. Yes, and several other plugins also had we had discoveries with them regarding for example the JGIT version which has to be changed or various dependencies which have to update in order to be compatible with Java 11. But for a lot of them it just painless goes pretty well. For some of them it's not the same stories but most of the time it's because the plugins were vastly outdated. So that's why it's a problem. But anyhow, the blog post is really informative well written and I used it yesterday night and it was one of the plugins. So yeah, that's fantastic team work. Now, yes, we have a story of WMI Windows agent plugin and imply dependencies to say the truth I had some trouble getting rid of WMI Windows agent plugin but Damian cleared the pass for me. It was just because I was using Docker with Jackask and so that's not always easy but you should get rid of that pretty soon. So if you can open that implied dependency sheet that's in the third, yeah, that one just this one the reason I wanted you to open it is to show that there are, this is a report and if you were to count the total rows it's a hundred or more if I remember right. Yeah, so there are many plugins that have an implied dependency on WMI Windows agent. However, jump back to the top. You'll see that none of them have more than 10,000 installations. And in a world with 300,000 controllers this is a small installation count. Okay, and in the old world where I was doing commercial software we eight for 10,000 installations of anything but this is Jenkins and 300,000 is the number. So, but what I hope to do here is steadily work through this list to ease the pain for some users that may be dependent on some of these plugins and really dependent on them not just uninstall them. For instance, row three backup, you should not depend on it. That plugin is so old and so you should just uninstall it. And so I'm not going to actually worry about releasing a new version of it. Others though, the SBT plugin that one I think is easy to live without but port allocator for instance, row 10 that's a very interesting and useful capability for a certain class of users. And so that one, I will probably get to it eventually and release a new version of it. Oh, in fact, I'm using it when I try to update the GitLab plugin I think that's the one which is used by the GitLab plugins. And I was amazed when I saw it was so old in fact. Well, I think what you'll see is that the actual consumer in your world is the Android, one of the Android emulator plugins relies on port allocator and that makes sense because Android emulator needs to get a port to communicate. So just be aware that this thing is just gonna get some steady progress. There will be some where they just won't be fixed or where someone else could do it anyone could help with this effort. It's an easy effort. I'm just following the tutorial on improve a plugin and doing a fraction of it and then releasing a version. Gotcha. Thank you, Mark. Anything else you would like me to click on? No, no, that covered it. But I think the issue you had with the uninstallation of the plugin in your case could be worth adding a sentence on the readme. I saw we had a proposal contribution for updating it by a contributor yesterday. A contributor proposed a documentation update of the repository readme, which helped to fix a lot of issues showing a Docker compose example on the Jenkins CI slash Docker image. I think removing a plugin is worth adding a few sentences because it, yep, sorry. Go ahead, Damian, go ahead. It's only because there are a lot of confusion between the plugins that you embed on user share Jenkins ref, which is inside the image and added through the plugins TXT and Jenkins plugins CLI and the plugins that you have on the Jenkins data which is a different life cycle than your images. There is no way we should change that behavior but helping connecting the dots for people because when I told you Bruno, you say, oh yeah, that makes sense. So if you can do the same in a written manner, for sure that would help. So if anyone is interested and I think we could ask the contributor if they are interested in sending another poll request, then we could drive that person to showcase the example. They show, oh, I want to remove the Windows plugin, then I do this, this, this, et cetera. You're right. And I got confused stupidly by the icon. I thought I would see a trash can or something and I think it was a wrongs something. No, that was- Yeah, well, I thought there was also a problem in that domain with, I think Damian, you mentioned it of multiple directories. I operate in one directory, but Jenkins uses another and the plugins get copied from one to another in the container startup process. So that's a healthy thing for controller maintenance, but deletion becomes extra complexity then. Exactly. It's easier to update and add. It's really easy and makes out of the box, but makes the deletion a bit more complicated. So yeah, I don't mind opening an issue and mentioning the contributor and you, Bruno, if it's okay for you- Yeah, of course. Describe the pointers. Even if it's only the contributor, copy and pasting, I think given that person is highly motivated to help on the documentation, I really want them to continue on that area because there seems to understand clearly that documentation is also first-class citizens. Great. Thank you, Damian. Now, you again, Damian, Java 17 supporting you. I know Bazel does an awful lot of work regarding GDK 17 support in Jenkins, but Jenkins infra was supposed to do something in December, but it was already the 16th of December, so I think it may happen in January and not in December. You have to watch for the Christmas tree. That's the only thing I can say on that topic. Okay, no promise, but you never know. Secret. Yeah, you never know, but you have so many other things to take care of. That's perfectly normal. Oh, so GDK 19 now support in Jenkins, so it has been installed by Stefan on CI and there has been interesting and very alive discussion on the mailing list about Edge and future LTS version, because of course, GDK 19 is not an LTS version. It will leave for some months and then it will disappear because Java, where will it be, by the way, the LTS, the 20 or the 21? The 20, I guess. It's only the- 21. 21, I was wrong. I thought it was only for the odd versions. Anyhow, so it's not fixed. We don't know yet what we're gonna do. I think it's still in discussion, am I right? So the infrastructure is going to write down a proposal of life and support of the GDK proposed on the infrastructure. As I understand the latest consensus is we support all the LTS GDK until Jenkins core drop the support. So right now, GDK 8 is still on Jenkins core and one day, there will be discussion to say we drop it for everyone and then we move it. For the non-LTS, it's the first time. We will support it until it's end of life plus one month. By support, we mean it's on the platform and it's installed with the latest available version. That's the limit of the support. The second point is we are thinking about providing a way for developer to say, hey, I want to build my plugin on DH, which means next to the two standard platform that you show on the BasilBlock post as for today, like Linux 17 and Windows 11, then that will add a third platform that will be Linux with the latest GDK, that will be 19 and then when it will be 20, it will be 20. By default, failing that age won't fail your build only the stage on the platform. So you have a way to know my plugin doesn't work as expected or work as expected with that platform. That will be the default. And then the infrastructure and our automatic updates will take care of saying, okay, the age is 19, it will be 20 January, et cetera, until the new LTS is available. So that's a kind of common ground to fit every interlocutor of the conversation. Maybe we will have to change that support policy, but I mean, we provide the tool and then people decide to use them or not. Yeah, but that's pretty cool. We will be on the edge of the GDK world. And that's pretty cool because we used to work with an aging version of Java. So yeah, so much has been done this year. It's amazing. Yeah, what a great community. Mark, anything to say? Yeah, go ahead, Yodimian. But you have to know that right now, if you want to use JDK19, just define JDK19 like you would do with C1 and then it just work. Yeah, go ahead, Mauro. That's, there's still lots to do for Java. Of course. And certainly the Jenkins project I don't expect will actually ever support JDK19 officially in the sense of you're allowed to run the controller on it because JDK19 has a very short life, right? It will live for what? Maybe six months or 12 months total and then it's replaced by 20, then replaced by 21. And we just don't have the capacity to support those kinds of things officially. But we certainly want to use them to prepare for the eventual JDK LTS. I think you said probably 21, right? They've already selected that 21 will be the base with the next LTS. I think so. I saw that, but I'm not entirely sure. But next LTS will be installed as soon as it will be available. Great. Great. I think we've done, we're done with the agenda. I just wanted to add a small thing. It's Christmas time almost and I love to give gifts. And I also like to give gifts that nobody asked for. Like the ugly sweater, Christmas sweater or uncle brought you last Christmas, for example. And what I wanted to offer the community was the first test with a risk five for a Jenkins agent. So I tested, I just put some glue here and there and we now have a first risk five agent working on Linux. And it's working on my machine. I have a job working and I'm pretty happy about that. That's cool. That's nothing, but that's cool, not the rest, not the rest. And it's just this little machine I don't see myself. So I don't know if you can see it, but yes, it's that little pink machine that does work for me. And I do hope to do more with risk five in the coming years. That was my gift nobody asked for. Okay, so if we're giving gifts, I have to give my gift. I have a Windows, I've got a Windows on ARM64 computer that I'm going to be using. And it will be an agent for me running Windows as part of my cluster. So I've got an ARM64 Windows and intend to do again over the holidays, some experimenting with it, but it looks very promising. It runs quite well, it's reasonably quick. Okay, it's not an M1, but it's reasonably quick and whatnot. It's a famous Volteura, am I right? It is, it is Project Volteura, very good. Super cool, and you won't convert it to Linux. You will keep Windows on this machine. I would, well, actually they offer a way to boot other operating systems and I might, but right now I'm much more interested in Windows on ARM64 than I am in Linux on ARM64 because we've been doing Linux on ARM64 for multiple years, right? So yeah, that's fantastic. Damian, you can start to scratch your heads No, no, no, no Windows on ARM64 for thinking. Yes, this will happen. Thank you a lot, Marc. You are trying me, but you have to know that that nice desktop station is initially with Windows for gaming, but it's also Hackintosh because I'm using Hackintosh since a decade. So if you see where I'm going, I'll go and work on ARM. Could it be possible to run it? So the answer is I already tried and it doesn't work very well, but I can boot on the single user mode on an ARM machine. But yeah, don't try me on that kind of thing. OK. We are still little boys or three of us. That's cool. I love it. Absolutely. Thanks a lot. Do you have anything to add? No meeting in two weeks. Oh, right. You've already got it there. Yeah, no, no, no. It's on the making notes, but yes, we're not meeting. So the next time we'll meet will be in the new year 2023. Got you. Thanks a lot. Happy holidays, everyone. Everyone for your time. Thanks for coming. The recording should be available from 24 to 48 hours and see you at the beginning of next year. Bye bye. Bye bye.