 to the Open Jenkins Essentials Planning Meeting. It is Tuesday the 7th. I think we're still waiting for Mandy to show up, but we can get started anyways. There's really only a couple of things that I wanted to talk about in this meeting. It should be fairly straightforward and I think we'll wrap up early. The first one that I wanted to chat with you, specifically Batiste, around was some of the AWS work. I mean, I feel as I'm actually not sharing my screen so you can't see what I'm referring to. I was waiting for the rest of your question then. So there's a lot of these AMI and AWS related tasks that I think are important before we give an alpha user a launch stack button that they can work with. And I understand you've got some vacation coming up. So I'm curious, is this something that you think you'll wrap up before vacation or I should try to take on or figure out some of that? This is a good question. So well, it depends. So definitely if you're referring to using a marketplace and so on, I have no idea where to start from. No real idea to start from. I mean, it would take much more time that is left. But I already started yesterday for the removal of AMIs and so on. The one you see that should actually be, I think you scrolled down a bit, it's in progress because yesterday I was having a look at how to tackle that and so to give a context to people possibly listening to what I'm talking about. We have an issue because we didn't want to really use an AMI. We would actually bake ourselves. We want to try and find some official public ones so that we don't have to maintain, you'll republish when there are security issues and so on. So that's for the context. So I looked a bit more deeply in the AWS marketplace and the official images and so on yesterday. And so there's, from what I can tell, no image that provides, for instance, Docker out of the box. I mean, already installed and available, you know, able to run Docker, run when you're starting up. But so that leads to my kind of question. I guess the only way we have to make this work for given our requirements is actually to enrich, to use even more user data, to do what I was previously doing, you know, baked in the image itself, to actually, you know, yum update and yum install Docker and configure the, I think, resource or user or whatever is needed to actually start Docker. And actually it's kind of a bit more a waste for agents. It would be similar. But there we would install Java and I think I'm forgetting something. We were needing, yeah, we were assuming that we would like to have Docker and Java. I'm assuming that you're talking about enriching this by providing an init script in the cloud formation template. Absolutely. There's already one for, anyway, we already have a user data because we do, you know, Docker run to actually, you know, pool and run the essential AWS cloud flavor image. So that's already existing anyway on the master side. So, but it was kind of, you know, add some time and I would hope that on the AWS it's very, very fast, but you know, we would have add maybe like 30 seconds or one minute or maybe worse. So people would be waiting for it and it could be slightly more flaky or something. You know, that's what I wanted. That's why I wanted to kind of avoid, you know, kind of blockers or by experience possible experience for users. But I guess it's even more important that we don't indeed have to maintain an AMI ourselves. So yeah, that's the gist of what I wanted to talk about. Yeah, I think that's a reasonable approach. The user data approach does give us some amount of flexibility to change this as we move along. I'm very surprised that Dockerfink doesn't maintain a Docker-enabled Ubuntu. Yeah, I didn't find, I found, for instance, on the AWS side some ECS-enabled images, but those images seem to have some kind of ECS client or something which we really wrote and even some kind of runtime. So we don't really want to have that running and even really not even present for abuse reasons. So on Docker side, I didn't really find something. Maybe I just missed it, but maybe it's kind of consistent with their move towards Docker Enterprise and so on. So maybe this is only available somewhere else or I don't know. Maybe I should just ask someone else to have a look that that's possible I missed it. Isn't there a, I swear, there's a French contributor to the Jenkins project who's very, very in love with Docker. Can't you ask him? Yeah, I can do that. I can do that indeed. I believe he's a doc. We're chicken. We do have many people. Yeah, I can even reach out to some people who were previously a colleague and they're so very deep into Docker and you know that guy too, but anyway. I'm talking about Damia if we stop trying to be hinting anyway. Yeah, that's just something. I've been testing out the Docker cloud image and it works beautifully. I'm really excited about it. I can't wait to have a lot stack button that we can put in a blog post or on the GitHub read me so we can start getting people testing. That's why I'm really excited to get this. Yeah. Get the AMI stuff done. Yeah, by the way, I think it might be getting off the pick slightly too much. Well, maybe not. I think at some point we might be needing some Jenkins owned but maybe there is going to be Spawn Sword ship and so on issues with this but I think some kind of AWS official Jenkins account to host and to store I think things like cloud formation template maybe or maybe not or maybe the provider of that marketplace owner or something. I think I didn't dig into that part very deep but I guess you would be needing some kind of account that would be providing some thing in the marketplace. So can you create a ticket and just describe the requirements of what we need on an AWS site of things? Yeah. I guess I cannot really do that but I hope because I don't know if I didn't dig into these how to create something made available in the marketplace but I guess it would make enough sense. So I don't think we need to make something available in the AWS marketplace. This is different from just providing a cloud formation template with a launch stack button from my understanding. So this is my understanding right now and this is based off of your possible information. Is that if you have... Yeah, possibly because I... I never used the AWS CLA locally triggering the reading the cloud formation file locally so I don't know if this is there. There's some things standard to be able to do that in the cloud fully. Maybe that's... You probably know already about that or... My understanding was that you... A person is able to create a launch stack button that references a publicly available cloud formation template which is something that could be embedded in a GitHub repository on a third-party website and that will launch... Maybe I just... into a flow. I really thought it was something really to marketplace but maybe I just was rifting into something that was seemingly obvious but is not. So yeah. And anyway, so getting back to the kind of issues that are still existing I think if we agree on the fact that I'm going to use user data then the kind of the issue is already solved because it won't be really hard because I know what to be done. The only issue with that is well for instance it's not really an initial master because it's a one-shot thing when you're pursuing your master it's slightly more pity for agents because this is going to be losing time each time an agent is going to be provisioned and I hope it's not going to create flakiness on the agent side provisioning agents each time and installing each time Docker, Java and blah blah blah before it even becomes an agent connected to the master. Right now the installing of Docker and of Java and Git actually happens on every single Azure VM that gets provisioned for CID it's very inefficient I acknowledge that but it is fairly reliable I don't think I've we've had agents fail to provision but they've failed to provision typically because Azure failed to create the VM not because it's failed to install the dependencies. Right. Yeah it makes sense I mean and I'm not really worried about like that in the US and people are provisioning VMs all the time and calling a pity gate or YAM or whatever and there's local caches and so on so it should work 99, 5, 9% of the times but yes anyway it's more like adding 30 seconds or maybe one or two minutes to just have the agents coming online it's going to be mostly an issue for demos Right. Maybe we'll hack that around for demos So since Jesse's here I want to make use of his time because there's this conversation that we started to have on this pull request that I created is one of the topics I wanted to discuss in this video Maybe pull up the thing specifically maybe while we're chatting about this Mandy will get back from coffee or wherever she's run off to She needs to try and pick her I'm really excited about that other pull request by the way, Batiste hiding the freestyle in Maven jobs I think that's cool That was not even created before and on the milestone one that's why I can sometimes when I'm kidding about the difference between the things I'm doing on work time and personal time when I get to choose I know the feeling So this is By the way with as far as hiding Maven jobs what we actually want to do is not have the Maven plugin installed and that basically means making sure that the plugins that we do have installed are built against a suitably new version of Jenkins Core and all of that that's entirely doable and as far as not having freestyle job types there's a long standing issue with the split plugins from Core, Label and Jira about perhaps moving at least the freestyle project freestyle build associated descriptors and some of that stuff into a plugin moving other stuff like project or even abstract project into the plugin would be a much more serious task because there's some of the things that have been proper dependencies on it for various historical reasons it would take us a long time to do something So the task that the task that Patisse picked up on one of the things that I wanted to try out with Jenkins Essentials was I wanted to hide that stuff from the new item view and I wanted to discourage the creation of new freestyle and maintenance jobs I agree, like generally I would like to see them disappear from Jenkins at some point or at least become fully optional in Jenkins but we're definitely not there yet unfortunately Yeah, because I mean this is not an opposition to what Jesse was saying I think because they were cutting off a bit at the beginning what I did is just kind of can be maybe seen just as a kind of a shim in the intermediate time where we actually extract things and we remove that shim and we actually do not even install the GT freestyle plugin or whatever that will be named Yeah, I mean to be sure as a short term solution we can use something do you know about the mask extensions plugin or hide extensions plugin or something like that? Extension filter plugin? Extension filter plugin, yeah so we can use things like that as a short term technique too I'm not sure we're talking about the same thing here because actually the task is already done and the freestyle and maintenance jobs already filters out Yeah, I just meant as a more general way of doing what you did you can use the extension plugin and then there's just a matter of turning off different kinds of UI elements Yeah This is getting into a tangent that is attracting from the really big topic that we need to discuss Yeah, right So the dependency resolution work that I had done last week and I'm just going to set up some context for the video or for anybody that's watching One of the things that we need to do in Jenkins Essentials is take the Essentials YAML which is a description of the plugins and versions thereof that should be distributed with the Jenkins Essentials and generate an update level or an update manifest in back end So basically we have some files let me just go to the Essentials YAML in my pull request here So we have some files that describe certain versions that a developer has said this is ready to go into Jenkins Essentials say there's Essentials plugin with an incrementals build at RC20 and then through some process we need to determine here's the full manifest of all the plugins and their dependencies that will create a legitimate Jenkins distribution and so that's what this plugin this pull request number 169 is really getting into and I wanted to start discussing some of the behavior which was implemented here and the gist of the behavior is for plugins which we do not have a explicit declaration in Essentials YAML of what version we're relying on the first version of this code is consulting the update center when we resolve a dependency to say workflow aggregator for example we say we need 2.0 a workflow aggregator but that's all that exists in Essentials.yaml with 2.0 we're going to the update center and saying give me workflow aggregator says I depend on workflow job and so we go to the update center and we say give me workflow job and that's the version that we're pulling there so Jesse there's a few terse comments in here and I'd appreciate it if you could expand on your thinking here terse comments from yesterday or what I was able to type on a phone keyboard gotcha apparently my attempt at telepathy didn't work yeah well I mean I think there are 3 possible policies 2 of which are acceptable yeah I mean what I would really recommend is that Essentials.yaml just has to list every plugin because I think in practice with the possible exception of Blue Ocean which which is done via monolithic mega release everything else runs on an independent release cycle and it would be perfectly normal to just update one component when that update is there and you want it to go in which case you would just have the script it would just be generating exactly what you listed Essentials.yaml and it would still be parsing manifest to make sure you didn't make a mistake and it would fail if you attempted to process a YAML file that had an inconsistent set of plugins so either a plugin is missing a dependency or its dependency is too old or it requires a version of Jenkins and what you're specifying like basically all the stuff that Jenkins Core does at startup you would be pre-checking that seems like the most straightforward option to me if you really want to keep it artificially short then you can have it load in transitive dependencies but the only way to make that deterministic is to pick particular versions of each of those transitive dependencies as a function of the versions that are specified in the YAML file in the associated actual manifests and in practice what that means is that you take the transitive closure of all of the plugins for each plugin that's not listed in the YAML file you collect all of the versions that are requested from dependencies and you pick the highest such version if you do that remember there's no need to ever load the update center JSON because it wouldn't matter you only care about the YAML file and the manifests when you pick up versions according to whatever is published on the update center today then you basically break the ability to know for sure what Essentials is publishing today and it just causes it basically destroys the idea that Essentials YAML is a manifest of what goes into the product and as you saw with Workflow Aggregator it's meaningless because it's been like two years or something since we've got a release of that and in the meantime basically you're getting a random version of each of the pipeline plugins whatever happened to be published to the public update center and sooner or later you would want to publish a particular version say an incremental version so you would wind up filling in Essentials.YAML sooner or later anyway so why not just start off by having them all written down My concern, my primary concern with having them all written down option one as you described in the Essentials.YAML is I don't think that that's going to be practically manageable in the case of all the pipeline developers preparing a release or having an incremental release and going to the Evergreen repository and updating Essentials.YAML then I think it would work but if it's Batiste and I keeping track of the versions that are available and sort of keeping an eye on things and trying to rev them or do an automatic change of some form then I don't think really curating that list is going to be I don't see that happening like I imagine that we would just have stale Jenkins Essentials instances out there as a result of us not being able to keep up with releases that are hitting the update center Well I have two points to that one is that in certain cases not the normal case you do actually want to have developers explicitly push a particular version out there or more specifically a particular combination of versions because we deployed some sort of complex multi plugin change and you want to push it out there and start getting feedback from telemetry and so on and you want everybody to have a consistent snapshot of the whole system so you want to file a pull request to do that as far as the more common case that developers are not really engaged with Essentials in particular and they're just dumping stuff out there and occasionally running Maven release perform status quo status quo situation then I think you should be doing what Jenkins X does have an automated tool that creates a new explicit revision of Essentials.yml according to what stuff has come in and I think you should do that because it makes it explicit that you're pushing a new version of the world to users it means that you can if you want to roll back or something it's a matter of get revert you can also do things like in the future you can imagine doing something like having a set of automated tests acceptance test harness smoke tests some other stuff which would run as a multi branch project against evergreen and it would reproducibly consume a particular version of Essentials.yml so that the automated process rather than just dumping stuff in master would be creating a proposed update to a set of plugins it doesn't update it once and before any of that goes live you have a complete test run that might take a few hours but it goes through and verifies that things are not completely broken before it actually emerges. The JenkinsX approach that you described I'm struggling to see the difference between consulting the ingest.json is what is going to be processed by the evergreen backend anyways if there's an automated process that's running this script that I've written here that generates the ingest.json with no no no that patches Essentials.yml what's the difference Essentials.yml is what is versioned and so as part of that change you have validation that a particular proposed change to this Essentials.yml is in fact something that meets all automated quality criteria whatever they are that plugins dependencies are satisfied to integration test paths so in the case of workflow aggregator I think that's a good one to pick on which is something that shouldn't exist and you have no actual reason to include in Essentials at all workflow aggregate? Yeah it does nothing it lists all needs it's very helpful the only purpose for that plugin is basically in the Jenkins 1.x days so that people could have a single checkbox that they could click in the available app so it's basically totally irrelevant for Essentials where you're assuming that this gets pre-installed anyway So I'm thinking that Batiste feel free to jump in if you've got thoughts on this as well I'm thinking some combination of both of these actually makes a lot of sense for the pipeline suite of plugins listing all of those in Essentials.yml I think is really valuable but I don't want to manually maintain an entire dependency graph with all the dependent versions in our source tree right now but this process makes sense to me Batiste had to go, he did mention that he had to leave it at a specific time this process makes sense to me all of these together make sense to me something mutually exclusive I don't think is going to be a good solution but combining these makes a lot of sense to me I mean you have various options for tuning in I'm just trying to say how I think things ought to work and I don't think I mean I'm not proposing manual curation by you and Batiste as a proper solution there are various ways to automatically propose updates to Essentials.yml both incrementals versions and update center versions you can decide at some point whether you want to do automatic update of the latest incrementals in master that's an extension of the existing tool or if you only want to do automated updates to official releases and leave it up to developers to explicitly push incremental patches or something like that but my point is just that however you choose to prepare those updates that the actual fact of the update is recorded as an explicit get commit yeah I agree that makes sense to me I'm going to add this as a comment and I think I have an idea of where I need to go with this script and it's not just one script I think there's some other tooling that I'm going to need to write yeah it's a script instead of workflows and policies and so on well that's been very helpful to have that clarification Jesse thank you I couldn't express that in writing more clearly so okay I definitely understand the terseness of phone keywords so it looks like we won't have Mandy today and Baptiste had to leave early so yeah thank you for jumping on and participating and I'll see you and everybody else later