 Hi, everyone. Welcome to the Jenkins platform special interest group. We're delighted to have you here. It is April the 23rd of 2020. Agenda wise, we've got open action item review. Then I had put core release project and I think given I proposed the following adjustment and count in agenda. We're going to put the Java upcoming topic with Matt sicker top of the list because Matt's joined us this morning. Thanks very much, Matt. Then I had core release project windows installer. Again installation manager. Alex, are you okay on that topic that it's on the agenda. Did you want to stay there. Sure. Yeah, I can talk over what, what I'm working on right now with that. Excellent. Okay. Platform roadmap and Jira epics. Series three, the system 390 and power PC infrastructure status and Docker PR status were the topics that I put on the list. Anything else that needs to be added to our agenda. All right, then let's go through the action items. So I've still got the open action item to open a jet for Docker operating system and platform support selection rules. Oh, like you've already opened the jet for windows support policy. Anything you want to describe there. I submitted a proposal to the developer my increased after some consideration. I decided that maybe Jep is not needed. There is a consensus. Right now there is a developer. This threat linked below. Each documents. The policy proposal. And I guess our current situation that we would go with multi-level windows support policy. I will say that recent platforms. These lines are actually fully supported and tested. The rest of the platforms are supported, but not tested. And they will be also platforms which we explicitly do not support. So there is a summary in the main increase and any feedback will be appreciated. Great. Thank you. Thanks very much. All right, and then we had one more to do item, which was the Docker build rework PR that Alex Oleg and I have got a review. I apologize. I am, I have not reviewed it and probably won't review it for at least a week. I think I ran a build on one of my systems and it built correctly. I haven't done any testing or looked at the script updates yet. I need to. Okay. Thanks. Anything else on action items. Okay. All right, then next topic is Matt sicker with Java Java upcoming and I'm going to turn off my screen sharing so that we can see Matt. All right. Hello, everyone. I'm Matt. I'm from the Jenkins security team, but I'm also big into Java platform itself because I use it a lot at Apache as well. So essentially been following development of it for a while since try to since some at some at the patch projects that I like to work with. We like to at least try to test some things out with newer versions to see if anything broke, which is how we, for example, found out in Java nine long ago that they had switched how localization had worked and things like that. So that was kind of interesting. But anyway, so the, so the major change that Java has made to the release train at least a couple years now, and has been doing this, and it's almost coming up to the next LTS is essentially that starting with Java nine. You can almost imagine that they've adopted Jenkins is release schedule is essentially the way I've kind of thought of it lately, because as you know with Jenkins how we have weekly releases where new features can always just go in there. But we don't support old weekly releases because you just get the latest week weekly lease right. Those are what that would be how Java is when it comes to versions like nine 1012 1415 and 1615 and 16 in the future. Now there's those special LTS releases now, kind of like how Jenkins has LTS releases where we'll just take a version every so often. Tag that as an LTS and then backport security fixes and maybe some bug fixes or any actual important things. Now you'll note that the interesting comparison here too is well there's a slight difference comparison there I guess is because Jenkins versus open JDK Jenkins does distribute binaries of Jenkins I mean we kind of need to do especially for on the Java Web start old stuff and everything anyway, whereas open JDK only release the source code still. While they do coordinate strong directly with all the various distributions that are creating binaries from it like adopt open JDK oracles JDK Zulu and millions of others now. Essentially these all just come from open JDK so with that in mind, the new LTS schedule is to essentially stabilize around every three years or every six releases basically of those updates. So kind of similar to where Jenkins where we'll take like every X amount of weekly releases becomes a new LTS release they are fairly it is a fairly stable cadence like that except we're a little more fine grain compared to the to Java but I mean Java is a programming language it has to release a little slower than us. So, in that regard, there's a couple strategies to actually staying up to date with Java. So for one you want to make sure of course that you're at least supporting LTS release because that's what people may be running in production more likely. However, if you're not actually using the LTS release which you always want to be doing is using the latest version just like you would be in a Jenkins weekly release, because if you were using two weeks ago release and you had a bug that was fixed last week. I don't care right and neither does just neither does open JDK if you found a bug in Java 13 they'll be like sorry Java 14 is the current version. You know it's like you can either report it to Java 11 or the latest version basically and then when Java 17 comes out you know they'll start slowly. downplaying support or whatever but here's the fun part to the second aspect of this is that now that Oracle has tried to deemphasize their own sensual role in Java. It basically kind of while they still are the primary supporter and primary funders of development and that sort of thing. That still opens the possibility to all the other companies that work on the JDK and maintain their own distributions to make their own LTS releases if they want to. So this is where things could potentially get complicated in the future but they aren't yet. But the gist of the LTS thing is there's a so there's a couple things to keep in mind about the LTS schedule. So kind of kind of like in Jenkins new features pop up in an LTS release based on whatever things were sort of in there at the time right and that's kind of how Java will be now. So in that in that mind you know how upgrading from Java 8 to 9 kind of sucked well going from 9 to 10 10 to 11 and so forth should actually be trivial. However if you keep waiting again or if you start or especially if you get if you get stuck on one of those intermediary releases you might get really bad. So look so there these are more hypothetical issues that can come up if you start trying to use some of the newer things. So let's say for example you downloaded Java 14 today right and you want to try out the new records feature. And let's say Java 15 comes out in a few months and that feature was still a preview feature and they want to fix and they changed something about it backwards incompatibly. But the thing it was it was a preview feature that you had to feature flag in. Now if you had relied on that and kind of ignored that update then the next you know basically the following update might completely remove it. For example I mean like well in it. Okay. I'm mixing two things one they might remove the next release or change it because it was a preview feature. So there's that on the other hand there's a new deprecation policy. I think this is probably more important to consider is up until recently Java has never removed anything that was deprecated. I don't think ever at least from the standard JDK. However they finally want to start changing that after they modularized Java. So there is actually I mean besides the fact that they've stopped bundling all everything besides the base by default which is something we already had to deal with. Not you know now they're actually finally slowly trying to get rid of some of the old deprecated classes and the way this works is they'll essentially mark something for deprecation in one release. And it will tell and basically it'll be and there can be like another release or two between its removal. But let's say for example something was marked. Now I don't know if a specific example but this is something that could theoretically happen is that Java 16 comes out. They mark something for removal Java 17 comes out it's still in there you use it and then Java 18 it's gone and let's say you never test any version of Java between 17 and 23 and then 23 comes out and you're like. Hey where's my shit. It's like well you should. You're supposed to keep trying it out on the new versions to see where you're where your shit went. So essentially it like the big change of the versioning number is like as they try to explain and various blog posts. This is kind of similar to what they used to do in like Java 8 when they the number after the U would increase by 20 every time they had like a major update like this. So it's like 8 U 20 8 U 40 8 U 60 and those are now equivalently 9 10 11 and so forth except now they're now they're less strict about what's allowed in the release though so imagine so I guess you can imagine if Jenkins weekly releases before didn't allow new features imagine if we had like some sort of master branch that was collecting all the new features and weekly releases were only like bug fixes and stuff and then the LTS releases were the ones that came out with the big feature bang big the big bang feature release. That's how Java used to work and as and as anyone who's been working in Java for long enough knows that has been terrible as big bang releases of Java has been horrible. You know like I wasn't programming professionally back when the Java 4 to 5 or 1 to 4 to 5 with with generics or the Java 6 to 7 or Java 7 to 8 or you know some of these have been around for but you know they've they've always been a huge hassle for various legacy software and and this whole release cadence is supposed to actually help prevent that issue. So, in that mind, what I'd like to see hopefully from our point of view is of course we we make sure that we keep Java 11 as a first class citizen, because that's still like the main LTS release and Java 8 support is still is is still kind of there, but it's mostly I mean it's almost like paid support at this point. I mean there's still the open JDK things going on with it but you know that's basically only getting security updates at best and that's because of companies like red hat and other and other people like continuing the backport updates. What I'd like but I'd like to make sure that we do is continue to not only with Java 11 but at least try to run out continue integrating with the latest JDK each time that comes out basically. To make sure that we are continuing on that. And I think that about covers it does anyone have any questions. The proposal on how we might do that. You know, do we want to do we need additional agents on our CI infrastructure with those JDK installed. How often do we need to test, do we need to test all the plugins what was kind of your recommendation there. So I'm thinking some sort of integrate like some full on integration tests and like a periodic basis would probably be good enough for now, because it's mostly the traffic. I would imagine a lot of the issues if they were to come up would be caught in compile time because it'd be like hey this thing doesn't exist anymore or hey you know this thing changed. I know I've been bit by issues of the bite buffer API has changed a little bit. For example between Java 8 and newer versions like they they changed into long to make it actually support, you know more than two gigabytes memory for example. And things like that can trip you up. If you're not actually cross compiling, I guess you could say. And then I guess there's also I don't think Jenkins has had to resort to using this yet but one other tool that we've used like in log for j for example, is the Maven tool change plugin which allows you to configure multiple JDKs to run in a single Maven build. We basically had to do that in the past so that we could continue supporting Java 8 as our baseline. We also have some specific Java 9 or even Java 11 now I guess specific version specific code that gets bundled in using the new version specific code feature of Java. Now I will note that the IDE support is fairly lacking in a Maven tool change thing I mean it mostly works it's just there's no, it's not convenient I guess. Like, I've seen one of the workarounds we've done here is someone, I forget who but somebody had written a small little library like a fields accessor type of library for the Java 11 migration. So it to avoid developer issues to if you ever need to make Java 9 plus specific code to pull in you could try and make it into a multi version jar, basically. And instead of having it directly in your project just to just to make it a little easier to edit but then again I mean it depends on what you're editing with editing it with I mean it's possible if you're using NetBeans it works perfectly because you know that's basically a GUI for Maven it seems like but you know. Any other questions. Oh yeah go ahead. One problem is supporting the Java versions is because just testing of them comes at cost. It's not an infrastructure. It's also ensuring that at least it's doing some classes basic builds. Because what we did with Java 9, Java 11 that Jenkins wouldn't even build on these versions and I'm pretty sure if we try building it on Java JDK 14 it won't build as well. At least on JDK 12 it didn't build it did run but not build. So supporting new Java versions is quite expensive. And if you want to do that, we actually need a team behind that. So for example, Java 11 support we had almost 50 contributors at different stages. We were contributing that obviously if somebody was working on that let's say just one day per week we could pull it off but still it's quite difficult. What we have on the Jenkins roadmap, we have Java 14 plus support in the future and also cloud needs with Java like Quarkus or RIVM which is a completely separate topic and a completely separate challenge. So I don't agree we need to do that but we need contributors. How about the girl VM ones and that's another challenge I've been facing like for Jay as well. Mostly around reflection because they don't fully support the reflection. No, they don't fully support the new reflection API as I should say they support each standard old ones but the you know the high performance method handles var handles all the other handles. Those are all too low too low level for girl VM to understand at the moment. Jenkins will be a very interesting project to get working in that system. Yeah, right. But I think that would be long term. Is there a way that we can, so you said it, you said you've at least been able to run Jenkins with the newer version of Java in the past, right? So is there a way that we can kind of run some basic tests with Jenkins on the latest JDK just running it not necessarily building it? Yes, and actually it's this is what we were doing, for example, when we were working on Java 11 support. So I can, this certainly the presentation that it summarizes major obstacles we hit when we were moving, why it wasn't building to issues we hit. So if you want to you can go through this presentation. But yeah, there is just a lot of different hidden stones. So for example, we started from building on JDK eight and running on JDK 11. And actually it didn't quite work because you have a lot of magic, for example, again, management of extensions management of annotations. And then the JDK 11 flow just got enabled in version 2.164. And before that you wouldn't be able to build the JDK 11 at all because it wouldn't start. Same for plugins. Oh yeah, I know I got I had that I had that issue pop up enough times I've noticed the error message that I recognize it right away. You cannot find whatever crack the class and whatever. It could find like some yeah some meta in file. Yeah, so in principle it's possible. But the problem just comes to cost. Somebody wants to contribute to maintain this flow. Let's do that. But we cannot just add an agent at another target and get running because it won't run. Okay, so it seems so it would probably I would hope that we can try to look at this before Java 17. I mean that's a while from now but you know a while creep up pretty fast, you know, to at least make sure that to get an idea of what kind of work meet may or may not be necessary for building on Java 17 in the future. Yeah. So some sort of period just smoke testing, just running it. Yes. What, what, what was it that you did then for this for the for this migration originally for running. It was pretty much the same as you described. So, yeah. Jenkins historically supported Java 8 for several years before Java 9 was released. And once Java 9 was released, we started getting some issues from users because Java 9 is completely different from Java 8 and nothing could really work. And at Jenkins for 2017 had first that there was a first experiment by Baptiste Matos and Mark who tried running it and reported a number of issues. So we started the year that is just a kind of warms. We agree that we don't want to open it yet. And the sometime after that several months later we started discussing it in the community. And finally, we had a certain distance in that only in May 2018 when there was a lot of the Java 10 released. We discovered that there is a lot more issues we discussed it in the community. And finally we ran online hackathon in June 2018. Where we worked on Java 10 support, we had almost 30 contributors at this hackathon. I remember that one. That was around when that was not long after I joined. Yeah, I believe you participated also. So this is was our main resource investment where we pushed all common obstacles and where we fixed the issues on the table. So we made pipeline working and we made bloat on the working and all the common components in five days. But again, it's required a lot of contributors. But it was just Java 10, another blog post, etc. So you can find this information on the Jenkins website. Then we had actually Java 11 release candidate. And Jenkins was running well under this candidate until the very last moments because they introduced some changes and actually broke our update center logic. So when we had Java 11 hackfest after Jenkins fold, before Jenkins fold. So Daniel Beck, me, Mark and several other contributors spent more time to expand testing and to close the areas. And actually we've got a prototype working on Java 11, just three days before GA version. So GA was on 25th of September. But it wasn't the end because after that preview in a week and general availability and general availability in LCS. And these took us six months. And actually, yeah, it took us six months to get working prototype to G because of because of fixing cold CI CD problems in Jenkins because of fixing developer to issues. We also discovered some regressions, for example, Java XML bind removal, which broke particular configurations, even if we removed him from Jenkins core. And over these six months, we actually had a number of contributors working and thanks to companies, we had a team working on that. Just to get it over the line, but it was a massive effort. So and basically what you propose on March is assumption that this massive effort can be done easily. I'm hoping that after I'm hoping that Java has opened JDK has stayed true to their word and since Java 11 has made the updates easier, or has made them less breaking. But you know, you, I think you raise a fairly interesting point here. This is something that I've noticed to my own lack of keeping up to date sometimes is that one of the nice advantages though, up to this. The essay of this release process now that they have is if you're actually trying out the early access releases like that or release candidates. If you notice regressions or changes like that, and you can report them upstream early enough they might actually fix it. Whereas if you don't ever test it until after it's released and you go and complain they'll be like well where were you when we were developing that feature we don't know anymore already five releases past that you know. We reported a few issues. Moreover, Jenkins is officially a part of the quality outreach program. Okay, the program by open JDK team, which bases it communicates and new releases and common changes. Oh, for all the early access releases. Yes. Okay, I see. Yeah, I've seen that same. I've seen that same subscription on. Well, I mean, of course we had done a lot for Jay, but we have. I used to be used to be on the ant mailing list to and of course they were, I mean they were one of the, they were usually the project that would at least try to integrate the newest Java stuff first because they're like well if we want to make a come want to make things compile want to support the new compiler features we got to add some definitions or whatever for it and then work from there, kind of bootstrapping your way up into the Java ecosystem. Yeah, and then you wait six years and then Maven finally supports it. I think there's a member of this project. We are supposed to test these new Java versions. And this is something we probably need to discuss at the Jenkins board level. So at some point, I kick you basically eliminated us and we've been added. But if you want to really deliver on this process. It's actually a good thing to do in principle. But again, if we have contributors working on that. Immigration just didn't just fix Java 11 support helped us to be the entire tool chain and it helped us to introduce new features. For example, the panda boat in our developer tools because many issues came from upstream and we actually removed a lot of technical depth in the project by doing Java 11 support. So this might be another technical debt opportunity. But yeah, I can see that. Thanks. Thanks for the info. I think that this doesn't make sense as a super immediate project. At least the full on support like JDK 11, like we did in the past. That kind of a tough decision to make though. I mean, I depend like how often to do that because following the LTS schedule of Java sort of makes sense because it's every three years. But it also depends on how much changes each time. So, in principle, it would be best if we can do some minor testing. So I had a comment. I agree in principle. In practice, there is one issue. Extremely slow adoption of Java 11. And then you think beyond the Java 8. So our current installation rate for Java 11 is less than 1% for Jenkins. And it actually reflects the industry state. So many tools, etc. They still run on Java 8 and the user is basically fine. Even if we know in principle that Java 11 is better, it shows better benchmarks. So until we let's say switch Docker images by default to Java 11, you shouldn't expect a massive adoption. And investing in supporting new Java versions, again, is good in principle. But I'm not sure how wide is our target audience for that and whether we could invest the effort better in other areas. So, for example, expanding platform support. Oh, like that I'm fascinated by that low adoption rate and you said that's industry wide, not just how where did where did you find that cool data that really we've got a very small adoption rate for Java 11. Yes, I can find some new statistics. So my statistics actually come from November last year. So maybe the situation significantly improved over six months, though I don't think so. But November of last year we had lots of we had we had spent that was already six or nine months after we'd released general support so so that yeah so I would not expect it to have improved dramatically than either. That's fascinating things. Go ahead Matt. I've been trying to use Java 11 more more. More frequently, I should say, you know, trying to make that my default Java home or whatever the latest version of Java is. But you know, of course, there's a there's a thousand plugins that still have an old pre 2.164 or whatever 160 whichever of Jenkins so that you know you always have to build issue and all that. Yeah, I mean, like I haven't, at least while running Jenkins like straight, you know, Java dash jar Jenkins and dot ward types things that work fairly well for me at least with whatever newer version of Java I've had at the time. I mean, I don't think this computer has like practically have a version of Java installed like I do on my other one but you know like it. And also part of this too is I also recently realized that home brews, the brew casks actually are updatable to nowadays so I'm actually able to keep my JDK up to date automatically. So that's also helping me try to stay a little more up to date on seeing things are incompatible, even if I'm not using it in production. Oh, and it does remind me one of the other interesting one of the I guess harder things, which is something that we came that we see in like for J but would be the same exact problem here is that of course we want to continue supporting Java 8 because that is the main, you know, platform for the foreseeable future. You know, I mean, even as it even as the newer version Java come out, you know, who knows how long we want to keep Java 8 as a baseline based on our users. On the other hand, if you know as you're continuing to support newer versions, basically you have to doing them both concurrently. The only the only way to do do that over time is through the like multi version support of things and try and basically designing things in such a way that you know like you might have here's the here's the default implementation and then here is the swap in that implementation with this one around a newer version and it starts to become a little more like C programming or something where you have all these if the equivalent of if depths and platform specifics. Well, this is Jenkins. So we have some of that in there too, but you know, it'll be interesting. Yeah, we already have multi release jobs inside the Jenkins package. It was one of the main challenges for developer to updates. Yeah, so we had that same issue. The first time we made a log for J release with a multi version thing we had complaints from people like oh my Android tooling is blowing up and it's like well it's not supposed to be scanning in that folder. It's the same story for like 800 other tools. And I'm sure you came across like half of the same ones as we did, or probably almost all the same ones. Yeah, when we were working on Java 10 plus support into 718, even maybe in compiler plugin, it didn't really support multi release job. And it was one year since the Java 9 release. Yeah, so now it's so much better. I remember the initial the initial stage of trying to support multiple releases of Java and stuff was basically. Oh, are you using and here's an easy way to do it. Are you not. Oh, sorry. Well, here's it. Here's how to do it. Here's how to call it from Maven. What. So it got super complicated. Yeah. So I think that I think that covers the topic though. Hopefully, I'll let you know in the future of my own personal running on newer versions of Java if I come across anything. But yeah, I can see this being a longer term discussion. Great. Thank you. I keep discussing this topic. Yeah, what exactly we do with Java 14 really depends on contributions. Because if somebody has a bandwidth and interest, we can do that. We fixed our infrastructure we can easily add a new Java versions to testing if needed. And we may need it for the purposes for example, new platforms or just switching distributions to adopt open JDK. Our agent can say you're actually all the difference on Java 11. And we mostly use adopt open JDK across the list of but still there might be cases where you can test multiple Java versions. You know, if there's a one little feature maybe to help get anybody excited about trying to have a 14 I just got me excited from a weird point of view but basically in Java 14 that they have a I don't remember if it's a preview feature or if it's fully released but basically they've updated a bite buffer API so that you can you you can actually directly access NVMe memory straight from Java so that you can do high performance off heat type of stuff and what not through and not just from Ram but through NVMe so To me that would be useful from a logging super high throughput logging or type of message queue system but Jenkins is low level that might find something handy there if anyone in the community likes that kind of thing. Thank you. Thanks very much Matt. I think we need to we need to proceed on to our other topics. Thank you very much for the Java status. Yeah, thanks for listening everyone. Thank you. Thank you. So, next topic we had on the agenda was the core release project. Oh, like did you want to take a minute and summarize where we're at that you want me to please feel free to summarize it. I was talking too much already. So, Alex you can you can chime in if I make some mistake Jenkins to 232 and to 233 have both released from the core release automation project to 32 had some packaging bumps and bruises that were resolved in 233. We've still got more items that will need to be resolved. But it's making good progress. We're pleased with the progress. Next stops will be working on how to do the workflow for security releases and how to do the workflow for long term support releases. Really pleased with it. Be aware this is a crucial time for testing of weekly. If you find a surprise and weekly here that's related to this build transition. It's it's much more interesting now than it would have been four weeks ago. Any questions with regard to core release project. Okay, next topic windows installer Alex. One of the items from the core release is that the newer windows installer is being built. There's still some issues issues in having the final artifacts uploaded during the release process. I think Olivier is looking at that to determine what's what's wrong. I'm just working through that. I'm adding some tests for windows installer to your install tests in the packaging repository. So just working through some issues there. Thank you. And so everybody's clear. Thanks to Alex's changes, the windows installer shrunk by almost 50%. Right, Alex, if I remember right, we went down from over 100 megabytes to on the order of 60 megabytes as a windows installer. Yeah, now that now the biggest artifact is just the door file. Which is great. We can shrink. Yeah, and that's usually bigger than the program. Because we used to bundle JDK and now we don't. Well, and that that that change now allows us to run 64 bit out of the natural and out of the installer out of the windows installer if the user chooses a 64 bit JDK so the windows experience has improved. Thank you. Before that it was inherently broken because we were installing 32 bit Java. So we were running in a 64 mode basically Jenkins and we know that there are some limitations for example for windows process management if you want to terminate processes. And yeah, more of the things. So great to see it finally landing. Excellent. Next topic, plugin installation manager. So I, we've talked about it for a while of integrating the plugin installation manager tool into Docker images. Since I have a, my main Docker platform right now is Windows. I have a Jenkins master on Windows PR, and I'm incorporating the plugin installation manager tool with good success so far. So I'm just working through some issues and then I'll push that as part of my PR for the Windows master image. And then I'll look at, I do have some Linux Docker platforms, I can then go look at Linux Docker images, incorporating that tool. Any questions for Alex with regard to plugin installation manager and its integration into the Docker images. I fully support that I would be interested to discuss that allowed plan because there are basically two ways. One is just replace install plugins shell script. Another way is to add a plugin manager to imperial change the documentation and also this experimental feature but install plugins SH only when we are fully confident that it behaves similarly. Because if I recall correctly, there are some minor issues, for example, how dependencies are picked up. And I'm totally willing to get rid of install plugins SH. But yeah, maybe we should have it as a two stage process. We also have plugins that SH, which is an even older method that is still in the Docker images. So my plan was to get rid of that but that just FYI that that's still there. Yeah, so I'm not sure what we could do about it. So obviously, we can just announce it is well it was announced as duplicate long ago. So I think that just remove it is a good approach. Though I know for sure that some companies still use this. Not in Jenkins infrastructure action but you know the use cases. So maybe just marking it as deprecated for example in the change log and saying the announcement would be a good stuff to go there. Thank you. Thanks very much. Any questions for Alex or for Oleg on plugin installation manager. Next topic then was just a reminder that the platform roadmap has been published, or has been through the governance board is still in Oleg would you call it still in draft state it's still evolving and growing and developing. But we've got a number of entries for from the platform SIG on the on the roadmap. Thanks very much to Oleg for leading that effort. The roadmap looks beautiful. You can see it. And the most of five teams actually aggregated here. So it's where there's docker support also multiple docker images which still need to get over the line with these flows. Open G9 and you will do some stuff which currently can be moved to list stage. I will probably update it this week. Also, we have some items suggested for the future. For example, custom Jenkins distribution service. This is a common project. Rick is working on. Then there are also some packages for docker images which we may include but the roadmap is mostly there. And plugin management is actually also included but it's referenced in management and administration site right now. So if you click here, you still get to the platform SIG holder. And if somebody wants to provide better documentation and description for this project, please do so. Excellent. Thank you. Yep. That's very cool. Like, like that guy said on Twitter, it's cool to see a software roadmap again. Oh, there's direction for a project. That's cool. You know, most, most, most software is just kind of, you know, winging it nowadays. It is, it is delightful. All right, next topic. Next topic is the series 390 the system 390 X and power PC infrastructure progress. This is Jim Crowley anything you wanted to ask there we've got we've, I can give my report. Go ahead. Okay, so we've, we've, we've delayed the use of the S 390 X and power PC infrastructure that IBM has kindly provided for us on CI Jenkins.io but it continues to be used in my test environment. Jenkins 2.2 222.2 release candidate was spent the last week or more on my test environment using the power PC and series 390 system 390 agents. So, now I'm assuming Jim that a first preference is agents as first target. And someday in the future that they'll likely want to run Jenkins masters. I know that there are Jenkins masters running on those classes of equipment in other places because I see bug reports in Jira on them. I think actually priority from the interesting, like, from people like in my team is the master and then agents. I actually was, I was last Alex this today. There was interest in the, the agent, the other day, a lot of got pains, asking if there was one. I looked at the evals, the, you know, the Docker hub eval user and I didn't see any multi arch containers for the Jenkins agent is that I just saw like windows I think was being pushed there. Those would be probably being pushed to a different area, not the agent, but they would be in. I'll find it and send the info. Yes, that's fine. Yeah, I just wondering that because I was on try to point and I was like, hey, you probably can just build them. I imagine they work fine but Oh, we're the reason there's only windows there is because we have to use our CI infrastructure to build the windows Docker images and push them for the Linux images we those are built on Docker hub builds right now. So that's why they're not being pushed there right now. Okay. We only have the for multi-arch. We only have stuff for the master right now on the Jenkins for eval. Yeah. Also, Alex, did you get a chance to look at the PR have open. I ran a build locally. I'm going to review the scripts a little just a little bit more. Okay. And but things are looking fine to me. And I would like to get that through for you. So, I think that was the next topic that you were just just discussing Docker PR status. Excellent. Anything else on s390 or Docker PR. No, I mean just let me know if you need any help testing anything. I'm happy to lend a hand. So, great. Thank you. Oleg agent images have been renamed. Do you want to give us a summary there. Thank you very much the word slave is gone in one more place. Well, actually multiple more places because we were able to remove around 100 occurrences from the documentation. But there is a catch. So one thing with it. Yeah, we didn't need three images. So it's now Docker agent Docker and about engine and dog. SSH agent. And with Docker SSH agent, we still have an open question how to name that. So that's why I postponed announcement a bit, but I think it's time to proceed. Because I reached out. There is a developer meeting this thread where basically asked people okay, would it be SSH agent or SSH built agent because both options have their own advantages and disadvantages. There was no consensus there. I offered ways to vote. Nobody really stepped up. And since all maintainers including colleagues and you mark agree with SSH agent, at least in principle, I suggest to just take an option. Unless somebody shall be keeping disease. And I'm going to write a blog post. We should just assess a bit that the current name is this agent slash SSH agent, and it is a potential subject for another naming but feel free to contribute. Thank you. Yeah, I agree with the logic that was offered in that in that developer discussion. My fixation on SSH agent as a command I execute from the command line is probably uniquely my experience. No problem. Thank you. Yeah, there is a problem because there is apparently a Docker image for SSH agent. Oh, yeah, not an official one. And it's called something slash SSH agent. I don't think that it's a big deal. But yeah, basically, it's contributed. Somebody wants to rename it to SSH built agent or come up with a new name which would actually get mobiles just to do that. Right. There was also SSH the agent, but basically it's exactly the same like SSH agent. Got it. Thank you. Thanks very much. We are nearing our end of time. Alex, was it you who added Windows support for agents as a topic. Oh, like, go ahead, Oleg. So actually, my question is, is it a good time to announce GE for them? Because historically Windows agent images were in experimental status. We have never announced that now they're fully supported. But in fact, we ship them, we test them. So anything else left to do before we announce that we support them. I think we can announce it. So it would be great to have a kind of blog post or whatever. Okay, I can work on something like that. If you could, it would be great because we can always use more content. Okay, fully supporting Windows agents is a good milestone. Sounds good. And when we're saying just for my clarity, when we're saying Windows support for a Docker agent, this means the agent that's executing is running on a Windows nano server image, or it's running on it. Or Windows server core. Okay, but it's running on a Windows kernel, not on a Linux kernel. Correct. Nice. Okay. You have to have a Windows server that supports Docker for that. But there are people out there who who actually do that. So is it, do people already do this with zones and jails to like the old school versions of containers? Well, so I do, I run an agent in a jail, but I don't think that it's very, very common. Whereas Dockerized Windows Docker agents, I would say become very, very common because of how popular the Windows Docker and Microsoft is pushing it pretty hard as far as I can tell. Yeah. Plus on things like Amazon and Azure, you can use Docker images. So the cloud providers are allowing you to use Dockerized container containers for build agents and stuff like that with their various plugins and so forth. So it's, and Kubernetes has support for Windows container pods as well. So it's becoming more prevalent. This is good for me as a user who just needs to build on Windows, not use it. So thanks for actually improving that Docker situation because I don't really like downloading 8000 gigabytes of things every time I want to test Windows. Yeah, you do have to have so Windows containers don't run on Linux. So you have to have a Windows system that supports Docker to run those but by getting a VM for that is has been fairly simple for me. It's just all the other setup involved with a normal Windows server is obnoxious but Dockerizing it would make it as easy as it is on Linux. Yeah, and we have so we have containers with they have get installed already in the container. We have ones that will have Maven and stuff like that. So they're different options. So one comment, probably something we need to address before Jay, if we talk about cloud providers you cannot use the current agents on GKE. So it hasn't been reported in the Jenkins bug tracker yet. I shared it in the meeting notes, but it didn't read if you want to use GKE we need 1909 core instead of 1809. Question whether it makes sense to migrate before we announce G. So the only thing we have to figure out is if the windows 2019 because you can't build a Docker image for Windows on a higher release version or for a higher release version than what you have. So if the agents that we have on Seattle Jenkins.io are not 1909, then we can't build a 1909 image. You can only build the current or less version. So I could do a test on that and see if I can build a 1909 image and then we can switch and it shouldn't be a problem. Yeah, so it would be just good to do it first because GKE is quite a popular scheme. All right. I think that covers all our topics. Any other topics before we close our meeting. Yeah, one topic, maybe for CIP members that we plan to participate in a Google season of docs, at least we are doing a best effort attempt. And if there are any project ideas which are related to platform support, and especially if there are mentors who are interested to help with such projects, then please submit your project ideas. I have just filed a pull request which creates a landing page. I haven't really filed it. I will file it in one minute. But we're looking for documentation specific project ideas. So thank you. And Doc Sig will meet tomorrow. Yeah. Anything else before we conclude our meeting. Okay, thanks everybody recording will be posted. Thanks, thanks, thanks. Thanks everybody.