 Okay, welcome everyone, this is the Jenkins platform special interest group. I'm going to start sharing my screen and let's look at our agenda. First, we're going to review the review the agenda and the topics on the agenda. Be sure that we've got all the topics and then we'll work through the topics. So you should see my screen. At the top, the November 7th agenda and we'll spend some time reviewing open action items. Then we'd like to talk about adopt open JDK as the base JDK for Jenkins image images instead of open JDK. Adding adopt open JDK J9 as a new image for Jenkins. Then we'll talk status on the Windows installer. We've had a plan to talk about October fest results, but Oleg is out today on vacation so we'll defer this in two weeks. Natasha added the plugin installation manager tool as a topic for this agenda. Are there any other agenda items that people would like to add before we start working through the agenda. Okay, I'll take silence to mean mean that we can proceed then so let's look at open action items on the mark is embarrassed we've got I still have to open the Jenkins enhancement proposal for Docker operating system support and platform selection rules. The issue there is we really need as a project a set of well defined criteria we use to decide when we will add a new Docker image. In particular because they are sticky once we add them they tend to remain forever. And as an example the alpine non support with Java 11 has been an interesting win with a sticky image. Oleg has the action item to open a jet for Windows support policy. Alex you've got the action item on the Windows installer code signing with only the evening anything you want to report there. The ticket for on the CDF is moving forward. They created a new legal entity in order to have an entity they can purchase the code signing certificates under so that's where we're at they have established that LLC. And now we're just waiting to see what the process will be for getting the code signing that's good done. Great yeah that's that is that is wonderful progress there. That was the sticking point for us getting that code signing certificate. And as noted here we also need that for the Jenkins release automation project that Olivier is leading. Thank you. Alright, so in terms of next item adopt open JDK as the base JDK for Jenkins images. So the thinking here was that adopt has better platform support has has active platform support for some interesting platforms that we might need. And the question was what do we need to do next. So we need an integrate we will fundamentally we need a pull request which proposes this right. And I have not seen one yet to propose change that change from adopt from open JDK to adopt open JDK. Is there anybody on the call who is intending to take the lead on creating that pull request. Are we talking across the board, or are we just starting with the master images, because there are also the Docker slave, the Docker JNLP slave and the Docker SSH slave. The images we provide as well. That's a that's a good question so it would need pull request you said for master for the Docker slave. And isn't there an image that is the basis for those two. That's Docker slave. Oh okay so then it's SSH what's what's the third Docker SSH slave. Thank you Docker SSH slave. So there's, it's not singular pull request it's multiple leading several pull requests. I think Mark for right now. Due to adopts limited support of like base images. Right now these, you know we talked about in the last call that I'm working with the adopt team to update their testing so they can do a PR to official Docker library to add support for a lot of more, you know, bases like Santa was Debbie and stretch buster, and all that good stuff. Right now I think they only have you bun two 1804 and I'm blanking on the other one. Maybe you be I think they have official ones. I think starting off with a small PR would be a good idea. I do like a proof concepts. So maybe focusing on that master and seeing how we get you know some reception and see how it goes via idea. Okay well and so my thought was that as I've looked at the Jenkins Docker image. It seems to have a base operating system. Oh no that's right it does use a it uses something provided by open JDK. Good okay so Jim your point is that since adopt doesn't have some of those other platforms. That would be a really significant change. Thanks. Good point. Okay. Yeah. And that's that's that was my like first kind of idea was just target. One of the bases they do have. I know they have and Jordan I've been using the Ubuntu 1804 base for adopt on S390. We haven't hit any issues is it been actually really really stable and recommend really well and plug and support surprisingly hasn't been an issue either. I know that's something we talked about last time like oh well, you know, do we know of all the plugins, you know work on S390 or something like bugs, but we haven't hit any. And we're using like Ansible Docker and a couple other major, at least I think major plugins. I think I think targeting just the master or just a kind of proof of concept. You know showing hey here would be the here's how we would do the switch over to adopt to be a good idea. And then I'm going to continue putting a little more pressure on adopt team to support and get that PR underway to get the needed base images so that we can you know cover all our images over to adopt. Okay. But by the way, I landed a PR recently with the adopt team for nano server images. I don't know. I don't know what their, their, how they're going to roll that into an official, rather than a non official. Because the build environment stuff that they have is all Linux based. So I may do another PR to get get have actions involved because they have Docker on one of the windows agents for get have actions. Oh, it's awesome. I didn't know about that. I can I can reach back out to them I was on the talk. I think the other week with them. And I know they just got more Linux server capacity. So that's why they're more open to supporting many more base images. But I didn't hear anything about windows. I know they do windows, I think core not nano. So it would be interesting to see if they would be able to support now. Yeah, it's it's really easy to build the nano images if you can support windows images. But for my understanding, there's someone who is actually like building them manually and pushing them to Docker Hub. Yeah, I could leave that for those. Yeah, an automated version would be better. Okay. That's something I have not touched in my experience with Docker, working with windows images. It's always been Linux. I'd love to take your brain a little bit more on that. Because whether you do a PR or I kind of prod internally with adopt team, we should be able to get that through. Cool. Sounds good. Alex, it seems like the windows nano server images would be very interesting for the agent for me at least for the agent side for the Docker slave or the Docker SSH slave. Yeah, definitely they are much smaller. I think the normal images like two point something gig for a server core and the nano server images like 500 bag. So it is a pretty significant savings in space and download time. Would you be willing to provide you can do it after the meeting provide a link to the PR so that I could embed it into the meeting notes Alex. Yeah, I'll do that. Thank you. They are there other items so Alpine support status. Jim, I think you had some information on this one that I didn't capture in the notes. Could you tell us what is adopt open JDK's position with regard to Alpine I think you described at last meeting and I didn't capture it. Yeah, so the digging through and I maybe I'll give the links for you guys to digging through the last official PR that open JDK open against Docker library, which Docker library for those who don't know is the official way to add official images to Docker hub. They had Alpine supported and if you look at their unofficial images, there's a lot of Alpine images out there, but the official Docker library staff was a little concerned with how they were implementing Alpine since Alpine uses muscle versus live GC. And, you know, pretty much everyone uses live GC on, you know, their base images except Alpine. And the problem was the dot team, all they did was disabled muscle and then pull down live GC, and put on top Alpine and then just, you know, make adopt, you know work on top of that. They were a little concerned about having that happen, they were they were looking more for like official support of muscle with adopt on top. So, I don't, I just I talked to the devs for the top team and they look like they're going to try Alpine again in the official PR and I'm hoping it goes through because I think I think they've done a lot more testing a lot more vetting. But my concern is that the official Docker library staff are still don't say no on that, which would be a big loss at least for, I like Alpine at least, and I know some of the community likes Alpine too. So, they, they, the, the, the team definitely wants to support it but I don't know whether it'll be blocked by the official Docker library staff so. Okay. Are there open JDK images for Alpine. I thought I saw them. Yeah, there are but the, the open open JDK. But how are they implementing it is my question. It's a good question. I don't know. My recollection was that there isn't a an open JDK image for for an Alpine open JDK image for Java eight. But if I recall correctly, not for Java 11. And that the open JDK project bluntly declared that they would not support muscle. Because my my that's the poor. So for recollection I'll double check I can do some. I think that that that open JDK refused to support Alpine with Java 11. And I think there's a long thread on it. But it does support Java eight. It's just if it only supports Java eight and Java eight is approaching an eventual end of life. It feels like Java 11 non support makes Alpine at least to me less interesting. Yeah, definitely does getting to Java 11 would be really nice for everything. I think you definitely bring up a good point if adopt got Alpine through to the official image. Maybe they did it in some way that please the Docker staff. It'd be interesting to kind of dive a little deeper into that and see how they actually got a supportive. At least for my understanding was that the Docker library staff didn't want to do it because they thought it was not. I guess well 100% okay with you know switching out the C compiler on top. But I mean if open JDK you got through then I don't see why adopt wouldn't be able to. Right, we don't yet have pull requests on it so I'm going to delete those two items. Any other items with regard to adopt open JDK as the base JDK. Okay, and let's talk about J9. So Jim, could you give us just remind everybody what is open J9. And you can keep it short and simple but tell us what open J9 is then we'll get into the details. So open J9 is a kind of different implementation of hotspot. You know what hotspot being more of oracles implementation. J9 has a different approach. It was originally I think made by IBM but then since open source date. So the advantages at least from what I've seen is a huge ramp up huge speed boost and ramp up. So open J9 has a much faster time ramping up to I guess full capacity or full or peak performance if you want to say that. And then we had Erwin on the call last time and he's talking about how hotspot usually edges out or sorry. Open J9 usually edges out hotspot in performance. There's just a couple of edge cases where hotspot, you know, you might want to go with hotspot for, you know, certain use cases. I forget what he said. I'm blanking here Mark. I don't know if you remember. No, I don't remember. You gave a great summary. That's wonderful. So thank you very much. And we've got a pull request open. We've actually got two pull requests open. I think the top one listed here 890 is more active right now. And what we need right now as far as I can tell is more review and feedback. One of the things I thought of is I think I'm going to, I'm at a good point personally where I could take the pull request and apply it to my tooling and see what breaks. My tooling involves windows agents, many different Linux agents. I'll host it off of a Docker image on Linux. And so with that tooling that should give us a little bit more evaluation. Hey, what are, are there any surprises in running open J9? I was doing my PR for adopt for nano server images. I did find that the open J9 crashed. When it was when I was trying to run a simple like Java to exe dash B for version. I haven't reported it to anyone because I just have time yet, but that shouldn't impact for master because we don't windows master images, but just an FYI. Okay, so, and I'm expecting I was expecting all sorts of problems just related to surprises around garbage collection or around the startup sequencing because Jenkins is huge. There's an awful lot of Java code in Jenkins. So, so for me that I assume we're going to see some interesting failures as we run it at larger scale. Yeah, Mark, I would love to hear about that if you guys do run into issues. Thanks, I can definitely kind of asked, you know, ask the internal open J9 team and see what they're doing. I know they do a lot of stress tests. That's actually part of the test we're actually pulling in for the adopt images. We're actually running through our the open J9 test suite. Because, you know, we can use it for non open J9 things to just bunch of Java tests. It's interesting to see the results over there. But from Jordan's eye implementation of it is I mentioned this last time to you Mark, but it was kind of funny we we spun up hotspot on that 390. And it took like a good amount of time for the Jenkins master to come up and get you know get to that web web UI when I did with open J9 it was instantaneous which was really nice to see and interesting to see. I actually thought I did the open J9 first and thought that was like the normal Jenkins kind of startup time and then I did a hotspot. I was like, did I break something? You know, is it not working on S390? But it just took a little extra time with hotspot, which is interesting to see. Excellent. Thank you. Thanks for thanks for doing the end. Is that that your notes on it running on open J9. Could you post those also to that pull requests that 890 pull request. Are you using that Docker image as your test on S390. I don't think I am what I did. And there's probably some performance tweaks I noticed they had a couple of Java flags. What Jordan and I did was we just actually pulled down the the master image for Jenkins and then just swapped out the from up top. So we were probably running, you know, running on open J9, but we're probably not optimized or anything like that. Oh, yeah, it'd be even more interesting to see if we can fine tune it even more. I'm not a Java wizard by any means. I'll leave that to the open J9 team. Great. Well, yeah, because I know that 890 definitely has some what looked to me like ahead of time compile, or maybe it's not ahead of time, but some caching operations where they invoke something in order to get it pre cached into the Docker image. So I think that may be even more interesting. Great. Thanks, Jim. Yeah, I'll ping that thread and see if we can move along. I'll be on the PR. Well, that's that's very encouraging and anything else on adopt open J9. I had a, so Jim, you have contact with people internally that work on the open J19. Yeah, Alex, I'm over IBM Jordan and I. The last call I was on with the adopt team, we pulled in a couple of people from the adopt internal team and a couple of people from the internal open J9 team. Yeah. Is, is there a place to file issues online for open J9 or adopt open J9 is that for the nano server crash that I saw I was just wondering if I could report that somewhere. I don't, I haven't bugs. I haven't looked around on my GitHub. I was being mostly on just the adopt GitHub. I can ask and I'll take the report back to you. Yeah. Okay, cool. Thanks. Jim, I'm gonna put an action item for you to, excuse me, find a location to report bugs to report. Yeah. Open J9 bugs. Great. Thanks. Any other topics on open J on open JDK or adopt open JDK open J9 as an additional image for weekends. All right, next topic then Windows installer Alex. So the windows installer is currently being built on the release instance that all of you is set up. So that flow is working. We just need the code signing certificate at this point. And then we can start releasing. So that'll be nice. So really the parts and pieces that you need except code signing are all functional. Good. Good. Can anyone grab those bits and run tests against them if they'd like. I do have a build on ci.jankins.io. I will send the link to the latest artifact that people can look at. Thank you. Thanks very much. Anything else there Alex that you needed to report. No, that's all right now. Okay. The next topic. I actually want to talk to Hacktoberfest results briefly just to let everybody know we had an amazing Hacktoberfest. The month of October is a sponsored month where digital ocean and GitHub create this promotional event called Hacktoberfest, encouraging people to contribute to open source projects. We had, by the midpoint of the month, we had over 50 new contributors and 50 new pull requests coming in. Really, really great results. Oleg will schedule a retrospective exercise so that we can see what we learn from it, what we need to do better and differently. One of the key things he noted was the amazing power of having newbie friendly or first time user friendly issues and in particular having them as GitHub issues more than having them as issues in the Jenkins Jira. Because GitHub has provided some easy facilitation to find friendly issues. It's made me think, gee, maybe I want to shift some of the plugins that I maintain to track their issues in GitHub rather than in the Jenkins Jira. So it was very interesting results. I suspect he'll be with us in two weeks and we can talk about it then. Now, has Natasha joined us? Natasha, are you with us? Oh, yes. There she is. I see her. Stopa has the plugin installation manager tool. Natasha, you want to give us a status report? Yeah, sure. So after our conversation two weeks ago, I tried doing the first official release, but I got an error. So basically I couldn't deploy the artifact. So we re-looked at some of the permissions files and I think there were some maybe naming discrepancies that we fixed those and then I tried doing it again yesterday and still I'm having some issues. I was just going to see if you guys had any ideas and actually overnight Tim had replied back to me, but I don't know. So one of the things that we had done over the summer was like we had renamed one of the JAR files after I guess like using Maven and I think I'm like a little bit confused about, so I think that had to do with like the order in which things got sent to Artifactory or something. I can't remember exactly why we did the renaming, but I know that like without it that had caused problems, but yeah, I don't know. So I don't know if you guys have any ideas or if that made any sense. The Maven release process is always like a black box to me. I never know why it's going wrong. I'm still having problems. Maybe we can try looping in someone like Jake Lick, Jesse, or something because he's like an expert and everything like that. So he might be a good one to maybe ask on the dev mailing list and see if anybody has any input there. Usually Daniel Beck and Oleg are able to help us diagnose those kind of things, but yeah if we had to, we could certainly appeal to Jesse. I'm hesitant to pull him in any earlier than absolutely necessary. Yeah, I would say post the dev mailing list. Then people, you know, when if they have cycles, they can respond instead of, you know, going directly to them for now. Okay. And now Natasha, you said Tim and I assume that was Tim Jack home. Yeah, correct. Okay, great. Yeah, so the dev mailing list feels like that was the place that I had to go to anyway when I had really had problems doing release and in my case the problems were related to to getting the right, right bits and pieces into the file system locally. Getting the right bits and pieces permission granted on the remote side. Okay. Okay, yeah, I can do that. Anything else that you wanted to report there Natasha. No, I think that's it. So, I think the action still stands then that the goal Natasha is to get plug in installation manager 1.0 released. And thanks. Any other topics that we need to cover today before we we conclude our meeting. I had one thing I wanted to bring up that we kind of briefly mentioned last talk was, I thought it was interesting, or at least you, I think I thought you thought it was interesting was, we were talking about reclaiming the official Docker image from the Docker library the official Jenkins one. And the interesting part I know you guys had some trouble in the past with them in terms of availability and like you know quickly pushing out images when you guys need it. One thing I think you found interesting was their access, their build pipeline which has access to pretty much every single arch they support so whether it's arm power as 390 x86, which could be helpful for us when we start, you know, say we do do the switch to adopt and we get on something that has support for many different architectures, would it be worthwhile, at least trying to reclaim that and utilizing some of their resources and then one of the things we also talked about was kind of modeling after how adopt does it, or adopt has their unofficial repository, which they play out nightly builds they push out, you know, pretty much anything to in everything to their night unofficial Docker Hub repository, but then they push out long term support major releases to their official so we're not swapping the Docker library staff, but we're just pushing out our best images over there. I just thought it was interesting point and something that doesn't need to happen now or anytime soon, just something to get the ball rolling and get people's thoughts on. Yeah, so the hardware access thing for me is is interesting that's that's does does the Docker official image actually have access for instance to s390. Yes, they do. I once tried to ask how they have it. And I still need to trace it down. Not that it's a security concern anything that I'm sure we gave it to them once upon a time but yeah they have access to power and s390 The one negative thing is they do not have windows auto build. Really interesting. Yeah, there's a there's a question a forum from 2017 that they said it's in their roadmap but there's been no update for a year and a half. Yeah, that would put a damper on some of the stuff we do, but that's interesting how does how do official images actually go out then for windows. Automated just so many only have to go do it. Well I'm pretty sure that like windows, they have their own pub instance, right they don't publish images to Docker hub they have their own servers that you pull from. And I for the like for the maven images I did recently I set up get have actions for that. But I'm not sure how other people do it. Yeah, because looking just looking at adopt right they have windows server core on support up on their official image. So they got through somehow. I think they push those manually, but I'm not sure. Interesting. I'm going to definitely track down, because I know the Docker library staff do a very thorough inspection I would imagine they want to do a thorough inspection of the windows images but they don't have something that be interesting to where they test it or how they test it. Thank you. Okay, so, so Alex did I capture that correctly that's currently as far as you know, Docker official images don't have an auto build, whereas these other architectures like s390 arm and power do. But you noted that get have get have actions delivers windows images can get have actions deliver on arm power or s390 I mean do we have. Do we consider some hybrid or that I'm not sure I haven't I when you specify the OS, you know, get have action. I don't think there's this place to specify architecture but I'll have to look again. Okay. So the suggestion that we use internally and then a lot of open source communities I've been working with has been Travis for builds at least where they support arm. I think they support arm. It might be in their beta. Same thing with power they sport beta and then I think just recently as 390 got into their public beta. It would be interesting to see if we could possibly use the Travis public images or public resources to build some of our images. I don't know if they have windows support. I think they do have windows. They do but I believe it's only windows 2016. And so, and most people are because of the huge increase in support and windows server 2019 have moved over to 2019. Interesting. Okay, so it's still roadblock day. Well, at least for me I'm very interested in windows server 2019 because of the, the addition of Microsoft supported open SSH server. It makes managing windows agents much, much simpler for me thanks to that open SSH server. They've given me much better results thanks to that serve new server that's added in windows 10, 1803 and later and in windows server 2019. Alex is the GitHub actions is that the new like tool GitHub just pushed out where like they have that the new tab up there that the only thing I've seen is like lets you like okay if I do a PR, it can automatically build something. Is that what you're talking about when you're talking about GitHub actions. So, I, I, and if you want to look at something I've done. If you look at C Sanchez's Maven repo. I recently added a PR for building the windows images on GitHub actions. Okay, yeah, I definitely take a look at that because I haven't, I heard about GitHub actions but haven't done deep. That's deep. Sorry. So, Alex, would you be willing in a future meeting to possibly present a tutorial for us on GitHub actions and what you'd seen what what what your experience has been. Sure, I don't know that I'm an expert but I've, I've banged my head against the wall a few times. Well and for me the banging your head against the wall makes you an expert by virtue of the banging so I think I put it on next. Next agenda. Alex review you'd have actions I think Oleg has also done some things so it'd be a good. Good topic to have next time that we meet with you presenting and you and Oleg together reviewing and discussing. Sure sounds good. Okay. Any other topics we need to cover today. Sorry to keep you interested. Mark is there. I know I know you mentioned and I think we talked about last time to getting access to s3 90 hardware and power. Also, I can I can we talked about this briefly last time is the Oregon State has a power and s3 90. You can do but we also can get or at least I can get you access to just a raw server for at least s3 90 I would have to go trace down some power people but I've done in the past to get power access. That'd be helpful for you guys as we got to venture off into the dock and you being able to run your test cases and stuff like that. It may I would hold that at least for two weeks because I think they're enough. I think we've got enough action items that will keep many of us occupied the amount that we can for at least two weeks. Keep that keep bringing it to us because it's a good reminder that hey there there are compute resources available that we may need to negotiate with the for me until we've been through some of these earlier action items. Yes, okay s3 90 isn't excuse me isn't as interesting to me yet because I haven't even done the basics on open J9. I had you know I think that I think I need to do with with equipment that I already have available that I'm conveniently using and then I'm quite experienced with. So when problems occur I recognize them quickly as problems the s3 90 stuff I would see problems and they would be complete surprise I'm sure. Yeah. Okay, I see what I mean. Great. Thank you. Any other topics we should include. All right well thanks everybody. Thank you very very much for the session today I proposed that we go rats, I may have forgotten to turn on the recording well we'll hope that I have the recording on. Thanks everybody. Thanks very much for your time today let's go ahead and end our session. Now I've got to stop sharing. I'm going to record it. Thanks everybody.