 Okay, it seems we are live. Hi everyone, welcome to the platform SIG meeting. Today we have a regular SYNCUP about ongoing projects in a platform special interest group. In particular, there is a plan to talk about Windows service, I'm sorry, Windows Installer's reward could be done by Alex and also about the Java 11 support project. And since we have a do it with us on the call, maybe we could also talk about Docker images. Okay, would you like to talk about that? Okay, yeah, we can return to it later. So on the call we had Baptiste, David, Dolvidos, Mark and Oliver. Oh, we don't have Alex actually, but okay, we can press it. So I'll just share my screen. Okay, this is my screen? Yes. Okay, so we have several topics. Since Alex hasn't joined yet, let's just move Windows Installer's to the bottom and also conditionally at Docker images and multi-architecture ones. Does anybody want to add any additional topics to the agenda? Do we need an action items recap item, agenda item like we've used in the past? Well, let's do that. So yeah, we have a bunch of... Yeah, embarrassing. I believe that everybody was... So we had many activities ongoing in Jenkins project now. So we were closing down Java 11 thing. We were preparing JSOC, Mark was on vacation. So I do not think we have many action items completed, but let's double check. Okay, so Mark open JEP to Docker operating system support. Not completed, I'll try to get it done by our next meeting in two weeks. Sorry about that. Okay, thank you. Project owner for Jenkins Chocolatey, I believe it's done. At least there will be some discussions about that in the emails. Yeah, let's wait for Alex. I agree, it's looking like it's at least more than ongoing and probably completed, but I agree that we need to have confirmation from Alex. Yeah, so my action item still to do. Code sign and construct, I believe also to do, but here we can discuss it later. And yeah, are we missing any action items? I don't think so. I think it's not the case here. Yeah, so maybe there are some minor ones. So probably we could take a look at Java 11 status. For those ones who participated yesterday in the governance meeting, it will be pretty much recap of what we discussed there. Oh, we have a new weekly release 2164. And this weekly release includes this particular keynote entry, sorry, change log entry. Now you can run this Java 11 without setting enable future Java flag. It means that if somebody runs this Jenkins weekly with Java 11, there will be no warning because there will be no whatever and you will just get it running. So it depends on the, yeah. No, just I wanted to say, maybe you want to explain or you want, I can take over if you want, but because even we do a meeting only two, every two weeks, the second release is also important with regard to Java 11 support, I suppose. Yeah, right. Yeah, so I'll summarize what changed over the last two weeks. And you want to take over at some point, just. No, no, that's great. I should have let you go because maybe you were going to do it, we're going to do it, sorry. Yeah, it's exactly what I was going to do. So just to summarize what changed in the core, there are a few major changes. One is a Jax Bjars. So it was our main problem and main blocker for Java 11 availability. So, yeah, you remember this running with Java 11, I don't really know which, yeah. So these things that you need to download Jax, you need to insert enable future Java. Now nothing is required. So you can just say Java minus Jax work and everything will work because of this patch delivered by Batist. So, yeah, it's just a detached plugin and this is a plugin which is considered as a detached one only on Java 11. And if you want Batist, you can clarify how it works. Yeah, so it's using for people not really knowing how it works. So for historical reason and to maintain background compatibility when basically the Jenkins core decides to try and remove some API, for instance, given how much Jenkins cares about background compatibility, something called detached plugin has been introduced some years ago, trying to make kind of allowing basically to extract the core into a given plugin. So that then plugins that will be relying on some core feature that was basically removed would keep working. So how did it work? We, the core automatically adds implied dependencies which is also known as detached plugins to the plugins that are being installed. We reuse the same feature for Jaxby. So by packaging Jaxby jars as so-called Jaxby API plugin, the only difference and the only feature that got introduced is that it's going to do exactly what I described, basically adding that plugin as an implied dependency when we install a plugin, but only if the Java, the current Java runtime environment is 11 or more. So that's mean that if you're basically running on Java 8, you won't be seeing that plugin get installed. But if you're running on Java 11, you should be seeing Jaxby automatically get installed and that should work automatically. So one case, one one, we tested a few ones, but the main one I've been working on to try and test that thing is the slot count plugin which is using the Jaxby API to serialize its data model, as far as I understand. And if you install that plugin on Java 11 before that patch, basically it would fail. But if you install it on Java on the 2164, it should work just out of the box as any other plugin to see Jaxby because then it will basically add to the class pass the missing Jaxby jars. So thank you for explanation. And yeah, everything was released. So the plugin is in the main update center as well as all other dependencies we need for Java 11. So another patch we had is actually reverting all custom changes we had in Docker images and Jenkins core. So for example, here Jenkins now uses standard update center by default when writing with Java 11. And theoretically it means that everything is done. Yesterday we had a discussion at the governance meeting. We firstly, the agreement that 2.164 will be the new LTS baseline. And it means that in the new LTS baseline, we will have a certain Java 11 support out of the box unless we discovered something really serious during the testing. And if needed, we can proceed at any moment and say that the increment LTS baseline as well as any few the weekly release have J support of Java 11. So yeah, we actually completed all main coding tasks. And if you take a look at the epic for Java 11.j in wiki, you may see that there is some open tasks, but they're mostly about the documentation and about the testing. Because we still need to complete a full test we run for ATH and for PCT. So similarly for what we were doing for experiment for preview availability in December. So we will need to repeat that. And yeah, there are also some open tasks which we will need to discuss later. But the most of the epic is completed. And yeah, it's looking really good. So yeah, there are also some test tool updates which I needed for the test automation. That's it. So unless we discovered something really bad, Java 11.j will be within one month. You have a question? I did just, we had the enable future flag to safeguard us against enable future Java flag as a safeguard against users making the incorrect choice of a Java version. Does Ubuntu 18 still deliver Java 10 as its default Java version? And if it does, is that going to be a problem? Because now someone on Ubuntu 18 may attempt to run with Java 10 when it absolutely is unsupported. It won't be a problem because how the patch was implemented not now it supports explicitly only Java 8 and Java 11. If you want to run on Java 10, you will still need to set enable future Java. Yeah, but there's a really, I'm not sure. I don't think that's what Maria is saying, but basically there's a crop on Ubuntu which basically is installing Java 10 and identifying at Java 11, right? So I'm not sure how it identifies at runtime. So maybe- So we check Java specification version and it means that if it starts with Java 10, if Java specification version is 10, we will still require enable future Java flag. Yeah, excellent. Good, that's perfect, like thank you. That was my only concern to be sure that somebody naively installing on Ubuntu 18 with the Java that they packaged by default would have a bad experience. So what you're saying is no, they'll be told directly. No, you may not run this without putting in that special flag. Thanks. If you're able to still produce that issue mark that would be interesting to double check. I suppose there didn't go to like crapping so much that they would redefine the Java specification version system property. So if it's not the case, the DS or leg is fully right. This is just going to forbid you through stuff. Great. Yeah, so it sounds like 2.164 is a place where we should intensify our advocacy and promotion of testing of that release because it's the baseline for the long-term support and because it's Java 11. So there's lots of motivation for us to social media promote this to promote it all sorts of places. Test 2.164 heavily. Yeah, it's not something we discussed yet, but it would totally make sense. So the ETA for RC release is two weeks. So we need two weeks to back port to the stuff, but we have Oliver on the call. So maybe he's open to starting RC testing earlier. Oh, and I'm not even worried about starting RC testing earlier. I just think we ought to inspire people to test the weekly. No, yeah, weekly, that's for sure. Right. I mean, for now, there's not much of a difference, right? Because we choose the latest weekly to become the LTS. And I reviewed all the candidates for this morning and I realized that basically all the candidates are already imported in because of the choice of a baseline. So provided it won't be any LTS candidate that would be released in 65 version, I presume that the one might be just the same as the weekly. So I guess, you know, it makes perfect sense to encourage people to test the weekly on Java 11. Which would be great. Okay, so it seems to be lost to Durgos again. Okay, any other questions about Java 11 status? So any, no problem then with me to start promoting, you said, we've done a silent launch of Java 11 2.164. Is it okay to actively promote it? And should we change from silent to non-silent launch? To do non-silent launch, I believe that we need to complete testing. I mean, at least tickets. So, so far it looks good. But yeah, speaking of testing, it's one of the next items in the topic about the ETH status sync up. By the way, there is no Ramon on the call, but maybe Mark, would you have a chance to ping him again? I will ping him, you bet. Yeah, thanks a lot. Just because it's better to do it in Slack and probably the screenshot in Slack is not the best idea. Okay, so yeah, well, you're waiting for Ramon. So regarding Java 11 status, there are some other interesting things just to summarize. So two months ago, Adrien and Baptiste were presenting Java 11 in Evergreen and effectively it's something which is still on the table. If I recall correctly, Baptiste has recently updated the Evergreen baseline. So maybe we will be able to ship with Java 11 a little bit quickly as well. It's going to be easier. As I already switched to another base Docker image instead of Alpine to CentOS, but basically anything else than Alpine is now it is easy. So there's a link ticket for the whole explanation, but basically, so for instance, there was that tweet and explanations at length by Mark. I don't remember the architecture, Java platform architect, but basically explaining that they are not releasing Alpine packages and it's unlikely they will ever do it for Java 11 and maybe they will do it for Java 12. So given that, we decided to, well, okay, say, if you tell by nowadays is not the right platform for Java centric applications. So that's why I decided to just move away from that base image for Evergreen at least. And I guess we should probably do those kind of things for Jenkins in general, because I'm not sure this is a major and stable structure for Java applications to run on Alpine basically, given that. Yeah, that's for sure. And yeah, thanks a lot for this migration. So given or so the cadence, the release cadence acceleration that's ongoing as we know, Java 13 is already started. Java 14 might be starting soon. So yeah, we need to be able to basically upgrade JDKs like easily and not like a very real pain all the time. Yeah, that's right. And speaking of another project, the Jenkins file runner, there were also some updates mostly by Everista. He actually updated the Jenkins file runner internals to support running with Java 11. There are some issues with the class loader but now they are resolved. And yeah, if you want to try a Jenkins file runner, there is a demo for JDK 11. It works out of the box. So yeah, you can now run Jenkins file runner there. And this is a first foundation step to enable all cloud native Jenkins flavors to run with Java 11, including Jenkins X at some point. So yeah, there is a good progress today. And same custom word packages. Originally we had a plan to apply some fixes in order to support that. But starting from 2163, thanks to Baptiste for this patch. We didn't need anything specific to run with Java 11. And hence comes some word packages that builds images as is. So this flow is also updated to support Java 11. And when we announce a general availability, probably we can announce it. The garbage is evergreen. Or maybe we could just separate announcements depending on the timing. Okay, so that's what I wanted to tell about Java 11 status. So yeah, very green. So Baptiste, are the things that you learn in making the transition to Sentos captured in commit messages in the evergreen repo? Or is that one that we should consider a blog post on or something else to share how your Sentos experience was? To be honest, I don't think it's that complicated or anything special. The only thing that brings to mind, and I think that's mainly like this, the only thing that's, I think the witch command is not in the path, it's not installed by default, which I learned, I was very surprised. But well, that's probably the only thing because everything else was just working out of the box correctly. And in general, we don't really depend on the OS and everything. So it's not like how to fight anything. It was just kind of very straightforward and like the only blocker again was like trying to, because I had the tests failing because I was looking for if some, you know, common was in the path, so I was using witch, that command, and it was failing, but that was basically the only failure. So yeah, one question which Java is actually being used in the Sentos image now? Unconfirmed, let me have a look. I don't remember all the top of my head. I did that already two or three weeks ago, so I heard out, I'm checking right now, three seconds, wait, blub, blub, blub, blub, blub, blub, blub. I assume it's Java 8201 or Java 8192? Yeah, let me see, the change is going to be big. Yeah, Java one, so I'm not even sure I should probably pin it, but right now what I use it is the official package that's called Java-180-OpenGDK. I use the standard package for Sentos. That's one of the things, but yeah, I should probably pin it a bit more. Yeah, maybe since we have Oliver on the call, what would be advice for Sentos? Keep using OpenGDK or maybe switching to Red Hats built of OpenGDK? Well, I presume that when you're installing OpenGDK on Sentos, the bits are already coming from Red Hat. I mean, from, you know, the wider Sentos community, but I don't presume that would be all that different. Yeah, maybe we need to get the extended version to make sure, because yeah, it may be critical for some stories, and for example, for Java 11 support documentation. We say that we support any Java, but if we at some point decide to specify which particular Java builds we support, then it would be nice to know what we actually use. Right, I mean, I cannot think of a better choice than using Sentos and whatever OpenGDK they would distribute. So yeah, I guess the best thing I can do is to refer you to the upstream Sentos documentation. Okay. I just posted the version that it's using. So yeah, it is not specifically clear to me. I don't know what I could do to check that. Maybe Oliver knows, but yeah, it's 18, oh, 191, but there's no specificity as far as I know. Maybe full version will tell me who built that, but I'm not sure if I can track it down instead of, you know, using the build, like a SHA or whatever, somewhere. I'm not sure that who built it is particularly crucial, but that version number already and the fact that it's bundled in Sentos feels like the right choice to me. Sure, but that is just to kind of completeness. I totally agree. It should be kind of the same normally, but like for clarity or for like just making sure, like we know the full picture, but I agree with you. Okay. So yeah, maybe we can take it offline and see what's actually being used. So Alex cannot join, Durgadas has dropped. So I believe these topics we just passed on to the next meeting, but regarding acceptance as harness, we also still don't have Ramon on the call. So probably I'll quickly summarize what has changed there. So since we reversed that patches in 2.63 and in 2.64, we need to update the definition so that, yeah, actually this pull request. So we needed to update the build in order to actually test what we actually run in Jenkins. So without custom libraries and yeah, this patch was about that. So since it's already integrated, I believe the next steps for us would be to wait till master build completes and investigate what are the results there. Is it right? Or should we do something else? That sounds good to me. That's the best one. I guess having this thing released is something master PR build would benefit from right. We probably want to consume it from essentials in Jenkins core. So once we are okay with how the build goes, should I release it right away? Sorry? Are we interested in me cutting a release as soon as we are confident about it? For acceptance test harness? Yes. For acceptance test harness, we actually can take a release by commit ID. So there is no specific need to cut a release. Although if it feels good to me, it would be much appreciated. Yeah, I think that what would be really interesting for us as a next step is to actually compare what fails because yeah, there is a significant number. I'm writing a commutation version of the browser, but yeah, two, eight, seven tests failed. There is a pretty long list. And yeah, for some packages, it's obvious that the tests are the same. So Java eight and Java 11 tests failed. But yeah, we need to compare that everything is the same. Just to make sure, Oleg, you put that at the sub-item of every winner. I think it's just a small mistake. We should just put it on the left. Yes. Sorry. Done. Yeah, actually it should be here. Okay. Yeah. So unfortunately, we don't have Ramon on the call, but yeah, I think that for the next step would be to just compare this run result. Does it run at least two, one, 64? I didn't check to be honest. Oliver, do you know of the top of your head which version it's using? Is it spinned? Is it using a red master? Yeah, this thing is supposed to be using a master branch or the light is released. Surely it should be consuming quickly due to the full requests. So anyway, I have cut, well, I started preparing a new branch for LTS this morning and the tests are running against that, against that I get at this point. So we would have to go through the list of failures and see how serious they are. So I presume that we would have a decent release about the latest release. In this case, both weekly and LTS to report where we are. Yeah. By the way, just to kind of the side, does incremental work on non-master branches? I think it should or would, like for instance, for the upcoming LTS, we can deploy step-by-step. Yeah, it's what can fall CS. Yeah, I mean, yeah, for the ongoing branches, that would be your tool, I guess. Well, if we push a snapshot, that's probably fine enough, but yeah. So stable, oh yeah, here's our new branch. And here commits for this new branch. And yeah, so thanks Oliver for creating that. How I've done. Definitely clicked a wrong button, but if you click here, you may see that there are actually incrementals produced. It's wrong, but it works, yeah. Yeah, it's wrong because we still haven't fixed the decision. It's reported to the incremental tools, but it really depends when there's some availability or somebody else fixes that. Yeah. But yeah, this is the correct link. That's great because then Oliver doesn't have to do manually all the time. Pushing, deploying last latest snapshots. Well, we have that and we also had a discussion for Docker images. So currently we ship Docker image for the master branch automatically. I'm happy to enable it for stable version as well. But the thing is that we need to do something about Java 11 because there is one task here, probably in this epic or in another epic that we adjust the Java 11 build flow because it was based on Docker Hub. It was using feature branch. And now since we want to produce two images from the same branch, we actually need to use Jenkins to ship these images. So likely I will be adopting JEP 217 in the core at some point. I mean, yeah, there is an experimental Jenkins organization and we can write pipeline. Actually, I'm not sure what you're saying is that we need two branches or something. I don't think it's the case. You can point now since sometime in the automatically built branches in Docker Hub you can just specify different Docker files. Different folders. And the problem that the relative source is still the same, unless I'm wrong. I don't remember how the top of my head. I'm not sure. I thought you were, when we were able to point to a specific Docker file in order time, each time, but yeah, that's fine. We can check it. Okay, so yeah, what is a follow up actually shipping Docker images for stable? But should it be? Yeah, I see what you mean. Because I'm actually wondering if we should, is this your meaning? Okay, experimental will be the other one. My natural history. Yeah, if you want, you can just take over and screen share. Because for now, I don't want, I don't think there's anything private or whatever, but I'm looking at the configuration right now. So I don't want to expose without willing it the secrets or whatever. Okay. I'm wondering about the idea to publish Docker images for stable branch. So this is, you know, new image published every comment, right? It depends every pipeline run. So it means that each pipeline run, it can produce Docker images. Right, which would definitely be useful for both LTS testing and perhaps even supplement image we push during release. So that's definitely an interesting addition. Now I'm wondering, is there a way to prevent customers to be confused? Because right now it's just, you know, it would mean that as I do back porting and as I pushing it for testing, it can publish images that are not yet to the stable. Right, so people are not confused that this is the actual release. Yeah, here's the advantage of JAPT-17 because we create new experimental Jenkins organization on Docker. So it means that these releases will not go to the official organization. Now we call it Jenkins for eval, but yeah, we plan to rename it according to the discussion. The ball is on my side, I believe. But yeah, what we have here, we already have some images. For example, this one being produced by Durgadar, said this. And for example, there is experimental Glotion support. This is Java 11 image for Glotion. There are experimental remoting builds, et cetera, et cetera. And we can actually create more experimental flows as a reference implementations for this job. It hasn't been completed, but from user perspective, it will be safe until we know about this organization. Yes, that would be great. Mm-hmm. Okay, so yeah, this job is under review, but it's in the scope of the platform seek. We just didn't discuss it since the last meeting. But yeah, it's okay. Thank you. Okay. So we have a few minutes left. Do we have any other topics to discuss? I don't have a question. Go ahead. I had a question on the tags that we will use to describe Java 11 in the weekly. Today there's slim and Alpine, we've said no, but there's a tag for slim and one for LTS and one for latest. How will JDK 11 be represented in that? So it's also reflected in JET 211. Oh, I can just read the JET. Yeah, you can read the JET, but effectively our main branch is called JDK 11. So Jenkins JDK 11 is a baseline and we have other images. For example, if we decide to have sentos here, it would be sentos-jdk11. LTS-sentos-jdk11, which may become a bit complex, but yeah, this is what we would have. Thank you. Okay. Yeah, that's really part we'll start when we decide to switch the default image to Java 11 from Java 8, then we will have some fun, but we haven't reached this point yet. And actually we will need to talk about when we want to do that, but I believe not in the next few months. Yeah, I am actually considering to do this for Evergreen. It might be a way to do that first and have a subset of test because in Evergreen as people might not know or may know, we have a set, exact set, fixed set of plugins that are installed, so it's quite easier to know what we want to test and then make sure things are still working. But yeah, we can probably bump it there first because it's also designed by design, supposed to be always up to date for everything and then kind of ramp it up for the Jenkins Core itself. Yeah. I would imagine that what you're saying is something we will need, must do like in like one year from now or something because at some point we're going to have to go, you know, above the JVTK8. Not saying we will disubport JVTK8, but at least yes, the default JVTK install with the Docker image at some point might be that, yeah. Now, how we are going to communicate that to users so that they don't have an automated bump and screw up internal things or custom plugins, that's another story, but yeah. Yeah, it may be fun and there are still some open questions. For example, yeah, we've had serious issues with DBM, Shopify 1588. Yeah. So we might want to change the default image in Java 11. Absolutely. And it would be better to do it before we make Java 11 default in Docker. Yeah, I have expressed that length on the notes. I linked the issue on Evergreen side, but basically it probably applies to Jenkins too. Basically, I tried to explain the full reasoning behind the switch to CentOS and basically I said there that that's my opinion, that that's not really shared, but yeah, like DBM or Ubuntu have not shown. And again, that's coming from originally DBM fine, fine boy. Nowadays they didn't show really a lot of empathy towards running Java applications basically. So Java short fire 1588 as, and the other one on Ubuntu as Mark was saying, basically installing a JDK 10 when you're installing a JDK 11, what could go wrong? So yeah, a lot of things this way that show that I'm not sure and unfortunately that DBM and Ubuntu are platforms where there's enough developers that actually care about Java in general. So yeah. Yeah. So maybe we could even talk about submission with default image before we go to GE. I think in that we have no test automation for Docker images at all. Yeah, maybe we could just change, but I don't have strong opinions about it. I guess for now it's fine until we have a new backboard that goes wrong on DBM. Or the alternative might be to just say, okay, we don't switch the OS right now, but instead of using the distribution packaging system, we basically W get directly the open JDK build from the adopt open JDK server or from somewhere else and decompress on the disk instead of using the base package system. That might be an intermediate way to do things, but yeah. I think it will be interesting to have CentOS as an alternative image. So like we provide Sleem and Alpine for Java 8, we could also start providing CentOS for both Java 8 and Java 11. And then we can migrate at some point. That might sound indeed like a kind of almost a required path to try and ramp up things progressively instead of saying, okay, we switch everything and nobody had ever a simple way to test things. Yeah. So I proposed to just a topic about CentOS images at the next meeting. So yeah, maybe we could talk about it and maybe just do it. Yeah, maybe in the meantime, I think we could have an action item to surface and have that idea discussed like publicly on the dev list maybe because yes, I know we have a mailing list for the SIG, but I'm like, well, this is maybe something we want to socialize more than just in the internal, kind of internal, even if it's public, but there's not the same level of not the same amount of people there basically. I can handle that action item if you think people hear that it makes some sense. We try to socialize that idea that we would during the long term to switch the base image basically. Or do you want to, do you think it's better to just for now wait until the next meeting? No, I mean that if you start discussion in advance, it would be a good advantage for us. Yeah, that's why I'm proposing that, yeah, because then on the next meeting, we have more data to chew and actually possibly, maybe not, but feedback from people saying good arguments why it makes sense or why it doesn't or why it's dangerous or whatever. Okay, and yeah, you have one minute left. Okay, so I'll do that then. Yeah, thanks a lot. Do we have any other topics to discuss? I got 10 seconds to be on time. That's absolutely great. Bye-bye. Okay. Cheers. Cheers. I'll see you in two weeks.