 Welcome, everyone. This is the Jenkins platform special interest group. Let's look at the agenda and go through our topics for the day. So we've got open action items. Google summer of code 2020 custom Jenkins build service discussion. Yeah, this is an action. This is a topic I added yesterday. After the discussion, I'll be sliding. I believe because sliding asked for another session. And I suggest that if he's fine, we will just do it in the policy meeting. Great. And so there is no slide in here is also we can just keep it and I'll probably organize a separate session next week. All right. Okay. Yeah, for whom I was interested, the results. Another project related to Windows service rapper and the support there. We want to have a dedicated discussion next week for that. There is pending Google. So the, sorry, said the other one again was Windows service rapper. Is that right? Yes. Okay. All right. I'm just trying to find an email. Yes. It's in the JSOC. It's not on the platform. Okay. I'll probably forward the invitation. Super. Thank you. All right. Then we had in addition. HTTPS and HTT to support with jetty and Jenkins. I like that you had supported yesterday. We might discuss here. I put it on the agenda. Not sure what the, the topics are, but propose we have it on the agenda. Let's use this, this time, capture some notes and discuss. So why put that. Historically, we didn't use HTTPS support from jetty. Because our default recommendation in the Jenkins project was to use external service to do HTTPS sync. But now the fact that many users use features provided by jetty, because they exposed in Jenkins command line by Winston. So anybody is able to enable HTTPS provide certificates and get them working. And it was one of the major regressions we had in a 2.2 or 4.3. Because we missed that built card certificates for not longer supported. And since we had exactly zero test automation for HTTPS mode and basically one test straight in the LCS. There is a developer, my name is with retrospective. The situation with the HTTP 2 is basically similar. Jetty provides a disruption. You can enable it in Jenkins. But we have no test automation at the moment and we have no official documentation which will stay with the state with what it supports or not. So for me, for most of the options, the strategy would be that. Yes, we would like to make this support official. But in order to do so, we need to do some investment so that we can really provide some coverage for these options. Okay, so the, so the, and you've described my preference as well, then the challenges we have to find someone who can help with we have to find people who are willing to contribute tests to cover that for HTTPS and HTTP 2 grade. That's right. So this test should be really implemented on a Winston level. So that we discover issues when the update happened. Because one of the major issue in one of the major causes of recent preparations was that we got them from Jetty. And we didn't really test the techniques on the Winston level. And then we didn't test them on the Jenkins core level. But it would be better to have tests earlier so that we know that Jetty updates break something we need to adjust. That makes sense. Okay, better. So better to test in Winston. With some form of checks there. Yeah. In addition to test coverage, it would require some documentation for users. Maybe some support in Docker images. Because you need to go through additional hoops if you want to enable the options. And if we decide that it's officially supported more than we should probably have an easy way to enable them in Docker. Right. And that was a kind of joke, which is probably deserves a serious discussion later. It's about using HTTPS and Jenkins by default. So discouraging use of HTTP there. But it's a complex problem. And it's definitely not in the scope for the initial steps with HTTPS support. But it is, I could see how it seems at least to me to be worth considering. Because pay we ship secure by default at the password level. Why should we allow sniffing of sniffing of content. Good. Okay. Well, the problem there certificates, right, right. Yeah, certificates you get a bunch of warnings in browsers and stuff like that. So that's, that's the harder thing about doing HTTPS by default. Yeah, I was assuming we do attempt something with let's encrypt and ACME to try to talk them and talk to them ask them to grant us a certificate but I guess that's probably dreaming isn't it. So the problem that it would be users who have to set up their own certificates. Right. It's not us. And I can imagine how we could do it automatic in an automatic way with let's encrypt for connected instances, but since Jenkins has been installed in isolated environments. Yeah, I'm not sure. Anyway, it's a subject for long term discussion. And definitely there is no way we disable and remove HTTP mode support. There are external services which provide HTTP to many users and moving this case isn't something for you would like to consider. Great. Okay. Anything else on HTTPS and HTTP to Alex, what would be your perspective. I think the biggest thing we're looking at is increasing our testing on these isn't it right now. So I think I need to look at the issues that came out and see if I can understand a little bit better what happened fully so I can have a little bit better idea of what to recommend but I think our test cases should at least be increased to catch something like what we had happen. I think this retains at a bare minimum. Yeah, JC has already added a smoke test for HTTPS. Definitely one test is not enough. And so that smoke test has been added to Winston or to to Jenkins. Winston. Yeah, testing it on the Jenkins level is lesser than trivial. Because if you want to test HTTPS you either create mock certificates and using HTML unit for that is probably not appropriate. So it would go to a set of test harness to proper integrations there to running Jenkins in the container and connect to that. All of that is doable. All of that requires some kind of investment. Right. Okay. So ideally, I would like to run all existing web UI tests and both HTTPS and HTTPS mode so that we get the initial test coverage. It's a wishful thinking right now. See for me that that seems like that would that almost feels excessive to me just because the seems like those things should be protocol independent whether you're HTTPS or HTTPS are there in cases where they haven't been protocol independent. So, yeah, practically everything should be a protocol independent. But theoretically it doesn't mean that JT works that way with HTTPS. So when you start forum submissions you may discover for example that for HTT you get one forum size limit, HTTPS you'll get another one. For HTTPS it's even more risky because HTTPS is implemented by additional library in JT. So these are the things you need to keep in mind. We are well equipped to run such kind of tests because when you're working on customer package or when you're working on session and session, sorry, essential test framework, which is basically ever been a test framework. So we included opportunity to run existing tests with customized Jenkins versions. It was the whole point of customer package in the first implementations where you can package everything you can include configuration as code and system properties which will enable HTTPS by default. So you can run, for example, Git plugin test with HTTPS using this framework. I guess it's not that easy when you try to implement that, but it should be possible right now just to implement these pipelines. Okay. Anything else? Alex or Oleg? Okay. Last topic on my list was PowerPC64 and Series 390 status report. I've actually been running the both of them agents on PowerPC64 and on Series 390 from my CI server at home. And it works great. They work great. They run tests. They've run the tests of the Git client, the Git plugin, the platform labeler, general purpose test jobs, all seem to be holding up well. I did learn that it is quite important for performance to run OpenJ9 on S390. It performs much more reasonably than if you run the OpenJDK that is bundled with the operating system. The operating system bundled JDK is interpretive mode only, no JIT. Operating system OpenJDK has no JIT. It's an interpreted only and it is slow. Any questions there or Alex, have you been using doing anything with PowerPC or Series 390? No, I have not, but I was just wondering. So, so it's only OpenJ9 basically that we can use on there. The gym had said that adopt OpenJDK said that adopt OpenJDK has JIT enabled. I didn't spend a lot of time checking that. Keep in mind that there are two distributions of adopt OpenJDK because adopt OpenJDK includes hotspot and OpenJ9 engines. These are separate distributions you can download. So when Jim said that, I believe it's related to the OpenJ9 engine. Ah, okay. See, that was the spot that I must have misunderstood. I thought he had said that, okay, the adopt OpenJDK definitely has no JIT and that I've observed. And I've seen that OpenJ9 performs quite reasonably, so I think it must have JIT involved. So it's not a hotspot in your line. It's OpenJ9, at least I believe so. Okay, all right. We still need, in terms of other things, we still need the final PowerPC configuration. Yeah, in fact, oh, like so on that one, I intentionally, oh, you're putting that in as a question to investigate. Yeah, because if you assume it was hotspot, yeah, I'm not sure about that. And nor am I, because my observation was, and that I think is correct. I know that the adopt OpenJDK, OpenJ9 does, and I know that OpenJDK does not. And the test is for hotspots, I agree, absolutely. Then we still need the final PowerPC64 LE configuration. It's currently using an SSH jump post, which is, it's rather complicated to manage, manages a Jenkins agent. I can give you the instructions on how it's done. It was, I found them on a page. We launched the agent with a command on the master, rather than using the SSH agent plug-in. And that means then that we have to must approve a system script through script security. And that approval is not supported by JCASC. You mean, so JCASC now supports script security. It doesn't, yeah, but unfortunately it doesn't support that aspect of script security. At least I haven't found a way to make it work. It definitely works for a medicine vacation at least. I have never checked system script because system script implementation is, script security is quite weird because it basically approves hashes of scripts. So I believe that as long as you provide the proper hashing JCASC, it will work. But you will need to somehow get this proper hash. Well, and I've certainly approved pipeline scripts with a hash and those do work in JCASC. This thing is somehow different. And I can't even find the keyword necessary to tell it to approve the script. Okay, so maybe wasn't supported. Yeah, so but I'm not sure it actually should be supported because I don't think we really want long term to use to require a system level script to run an SS to run an agent. This thing should be supported through an SSH agent, just like all the other agents. Today is also as far as I understand it working on the steps to include in include on ci.jankins.io. Any questions there. All right, Alex, any other topics from you. Just from the arm 64 front least. So we're getting AWS we have AWS credits. They have their graviton instances which are arm 64. So I'll be looking into arm 64 there. Ah, great. All right. I wonder whether sponsorship includes F1. What was that. F1 is FPGAs. I can barely provide any reasonable use case for ci.jankins.io. But if you get the sponsorship, I will definitely try to come up with something. Right, right. But your hardware, your heart of heart of hardware suddenly gets pumping and the blood starts flowing. Oh, that could be interesting. Oh, yeah, cool. I didn't know about FPGAs a service that sounds that sounds interesting. We had a student in a FOSI foundation this year who implemented some delivery pipelines for projects using F1. Well, it wasn't powered by Jenkins but the concept can be adjusted to Jenkins as well. So at least it could be good as a case study and maybe as a demo. Yeah, definitely not in the top of my power list. All right, any, anything else on arm 64, Alex or Oleg. Nothing from you. No, nothing else. All right, I think we have concluded all of the topics that we had on the agenda. If, if there's nothing else I propose we call an end to this meeting thanks everybody. Thanks. Thank you. Thanks and I'll host them. I'll save the save the recording and talk to you in two weeks.