 Well, today we have a short discussion about the Jenkins distribution packaging service. This is a desktop project ID, which has been proposed by Rick, who is on the call. So we will just use this opportunity to think up about the project ID and about our expectations. Because in fact, there were two parallel discussions, one in the mailing list and in JSOC channels. Another track was at the contributor summit in Brussels, where we discussed pretty much the same idea but from different angle. So I wanted to use this meeting as an opportunity to align with regards to the presentation and maybe to build a more detailed proposal or document which would include expectations from sides. Does it make sense? Yeah, perfectly. Yeah, so did everyone have a chance to read this meeting from Brussels? I didn't have a ready to because due to my understanding, there was not that much documentation here anyway, but we were good some and Chris, did you have a chance to go through this proposal? I might have briefly looked at it. So since I participated in both stories, so my understanding that here we try to build the service with some examples of what would be included. The discussion at the contributor summit was about how to define the packages and what should be inside there. So yeah, these stories are definitely complement each other and I was wondering whether we could just set some expectations here so that students reading through this project basically follow the expectations and also define the contributor summit. So basically concept of meta packages for different technologies and purposes and also we discussed here. I have a question about the meta packages and I think I don't have fully understand what the meta packages are. So it's like imagine a package that doesn't have any code of its own but basically it's just a manifest and that manifest specifies all the packages it will depend on. So if you install the meta package, then it installs all of the dependent packages. Like a project for just a container.com.exe, that would be a good example. Yeah, so in Jenkins we have some examples of that. For example, there is pipeline aggregator plugin. Basically, I guess it's called just pipeline. Yeah, it's just called pipeline. So this plugin doesn't include any functionality on its own but it pulls in dependencies which are pipeline and basically if you install this let's say meta package you get all the pipeline plugins for core functionality. So above this idea, maybe different use cases like AWS or GitHub will have several meta data package projects, right? Yeah, conceptually there would be one meta package for each cloud provider and then for stuff like VCS I guess you could bundle up the Wifi branch thing and the GitHub kind of check out thing and that would be a GitHub meta package if that makes sense. Okay, from my point, we need to consider the database of course GitHub can be a database or you can use like a Mexico or other circle database. Because if we have a lot of meta packages projects, maybe I don't know, I don't know if it's good or bad. Just want to hear what you think. It would be reasonable to shortlist some of the popular ones like a lot of people will want the AWS thing because they're running Jenkins on AWS and a lot of people want the Azure equivalent because the Jenkins is on Azure. So you could and indeed I think there is already a mechanism when you first start Jenkins that it suggests packages that you should have. I think Jesse sent that to us in the email for it. Yeah, there is installation wizard. This installation wizard is a bit outdated. Yeah, this recently started doing the documentation seek. So we are working the plug-ins side and just a few days ago we added technology labels. So for example, now you can search for AWS plugins for Asia plugins or Google Cloud or several other technologies and we started grouping these plugins. So for example, here you can see a bunch of plugins for Asia. This is just the first step and hopefully we will expand that to the installation wizard and to the plugin manager. But it's yet to be done. So it was made possible because now we support GitHub topics as a source of labels. So we can pull in a lot of information and now plugin maintainers can easily manage that. So hopefully in a few months we will get a good database which we can use for other stories. Yeah, I think I just announced it yesterday. It's making your plugins discoverable with labels and GitHub topics. But yeah, I hope that you could use some results of these stories. Rick, your concern was about backup functionality, etc, right? Yes. For me, one of the difference of what is documented here. So yeah, should Jenkins bundles with Shafantin for coming with cases. So it includes Jenkins plugins and extra configurations. So my understanding of the meta package originally was that it would also include configurations so that we could get, okay, how I call them ready to fly Jenkins images. I think I've like in my head, at least I've scaled that back a bit to make the meta packages more sort of single responsibility. I suppose, I mean, meta packages could exist without the distribution customizer service. Providing there was a way to version pin which there is not currently in a standard plugin. Yeah, so I've kind of cut their definition down a bit, at least in my view. And then the thing that bundles sort of conflict, like ready to fly Jenkins is, I mean, we've got Jenkins X for Kubernetes, and that is a distro if you like, ready to fly on that platform. So generalizing what that's doing to allow people to prepack, like configurations and stuff, that would certainly be a component of the distribution system as a service. Right. So basically, what's recommended here, so common configuration and other things. Would it include a sort of a builder section where users could build their own custom Jenkins with a maybe simple user interface that allows a normal who are not much familiar with Jenkins configuration as code and sort of configuration management as such to be able to configure their own custom Jenkins package, for instance. I think there is definitely an idea to have like a user interface for doing it. Yeah. So it's rather a service. But yeah, if you can run this service on your own, then I guess it addresses the tooling use case as well. Okay. So basically, if I understand correctly, we would want the service to be able to sort of have, I mean, make certain Jenkins distributions that are most commonly used like the AWS one or the GitHub one, more accessible to users, or is it just to build their own Jenkins service, I mean, service, for instance, with their custom package? It could do both, right? If it can allow anyone to specify their own Jenkins setup, then presumably, the outcome of that can be saved and then stored somewhere. Okay. Okay, let me just write that down. It was listed somewhere, just a second. Yeah, it was the trade. So here, there is a message from Ramon about a way to share configurations, et cetera. So yeah, share. I think we can have two parts of the solution. One of it comes from maybe like officials. Another part is comes from users. We do not guarantee the quality or risks. Just we can store their solution. Maybe other users can find one of them and just reuse it. So I think we need a database to store the history recorded. Could that be done via GitHub repository is much like the way Helm charts are defined for Kubernetes? It could be. So yeah, I can show you example how it's done now for some projects. Okay, probably not the best. Start from the Spring Boot documentation. You mean Jenkins for Spring Boot or? No, the Quick Start section as a link. Yeah, so you mean Spring Boot builder? Just a second, I'll find it. That could be an example. Is it on this? Yeah, if you just type the Spring Boot, Quick Start, you'll probably get it. Yeah, that's the amount of this service. Yeah. So this is the service which is usually presented by Josh Long when he says built a jar of smart words or something like that. But yeah. Yeah, so here you can select languages you want to use some technologies. I guess you can request some different dances like. Yeah. Why not? Yeah, not sure what I'm doing, but after that you can generate your setup. So for example, for Java 13, why not? You can click button and you get this demo which is generated with all these specifications. This is what this service does. And if I understand correctly, the idea by Rick was to have something like that, but which would generate specifications for custom word packages, right? We can imagine that we use Jenkins like we go to a supermarket, we just pick up what we want, maybe we don't need too much about the details of the technology. We just, for example, we want to use Jenkins under Java level and then we want to, we want the Jenkins wrong in Kubernetes cluster and then the user just clink the button and get what you want. Okay. There was a line that you highlighted in the email chain that the share your Jenkins configuration, I think it was. So that almost is what our central tooling team does workplace. So we have like this team that works on producing a standard Jenkins template, which is the product teams because currently we don't have a centralized Jenkins in the understanding as the teams, the product teams run their own, they all from this template. They see it was ready to fly Jenkins. They plum in sort of, you know, they get a API key to let it start cloning things and off they go, or at least that's the theory. And because we run our stuff on AWS, not Kubernetes yet. That tends to be, that's a big pile of Terraform and the Jamel thing, the configuration plugins are pre-installed. I don't know if the team's using custom or package here or if they're just shoving them on a Docker image. But yeah, that's kind of what we're at the moment. That's our results. Share your Jenkins, if you like. And it's, of course, massively laborious to do. Yeah, there are dozens of different shared configurations in the net, which are based on Docker packaging. So for example, this is just one my old to demo, which basically extends Jenkins Docker image in standard base injects YAML there and you get everything working. Yeah, custom work package it was one step forward because custom work package are intended to switch all these custom scripts. So you can see that there is shells there, but there is Docker file and other things. So the intention of custom work package was to provide a single YAML based definition, which would define what you want to get in your Jenkins. So here you can see that you specify a word file, you can specify plugins, configuration is called like that, or just by including text, and then you get your instance running defined by a single config. And since it's YAML, it would be more compatible with Helm charts and these other things like let's say one could create a special operator and Kubernetes for Jenkins or special Helm charts with using case and other configuration. So that's why we were heading with custom work package as a definition. Well, a tool right now is a new alpha because I started doing major work and I put it in alpha just because I want to change the format, but the tool is there and it's usable. Then there are some examples in public which demonstrate using it for package and Jenkins. But yeah, in this case, there is still a lot of files on the top of it, but still there is a config. Which actually looks like that because I use pomeximal as an input and using dependable to bumper my plug-in dependencies and same for other files. So it's probably not really cool as an example. But yeah, the service is there. So if somebody wants to just explore how it could be done by a single configuration. Yeah, you can take a look at that. Yeah, that is a bunch of them for different cases. So I was working on a proof of concept with the custom warpackager could, I mean, could we, so basically, maybe we could take in from the user interface at the front end, what sort of plugins and configurations that the user needs and then generate the YML file. And then the custom warpackager could just take that YML file and generate the instance. That would be a good proof of concept, I mean, for this project. Yeah, it could be possible. Okay. Moreover, we have some examples of tools doing that. For example, there is support core plugin. Plugins, you can say your support core. So this plugin generates support bundles. For example, we use them for support purposes, but it's generally handy for different kinds of analysis. And it also generates a docker file which allows to partially create the instance. Without full configuration, but it dumps to docker files, plug in lists and other things. So you can partially recreate the user's instance. Yeah, if you want to create something more sophisticated, which exports the Jenkins definition, it's totally possible. Okay. Okay. So studying is definitely experienced with the JCASK YAML export by now. Yeah, definitely. It's a project, but it's a separate site of an export. Yeah, definitely. So I was, maybe I could try building a proof of concept for this project while making my proposal, yeah. I'm definitely interested in this project. So yeah, any sort of point as helpers would be really helpful. It will definitely make the life of our tool team easier. Cause while the YAML is quite easily human readable, guessing what the hell goes in it is a bit of a game. So the nice thing with a UI for generating it is it takes the guesswork away. So I guess your is, what is actually? Well, this proposal includes your and frontend part. But yeah, I think that it would be nice to have a boss eventually. Well, personally, I'm not a frontend developer at all. So somebody else could provide insights how it could be built, but as the link said, spring examples and the other configurator examples could be a good source of inspiration. Okay, one question I had. Could we use the installation wizard for using the provider sort of frontend or taking in all of the common plugins? Because when I install the instance, I guess the installation wizard does provide us with the most commonly used plugins, right? It provides most commonly used plugins, but it doesn't provide plugins sets per se. So for example, yeah, I can just launch it properly. Do you have a book from Yank? I have no idea which version I'm going to run. Okay, I'll clean it up later. Sorry, yeah, I can just show it, but yeah, the main problem with the current step wizard that it provides you suggestions, but after you see these suggestions, it's still not, there is no meta packages, which Chris was referring to. So if somebody wanted to implement that, it's still needed on the installation wizard. So it might be an interesting feature. The key sticky thing with the meta packages is the versioning, the ability to pin the versions to ensure the stuff works together. And as Jesse pointed out, that doesn't currently work on community Jenkins because everything else. Thank you. There are some examples, for example, Jesse was working on a bill of materials for plugins. So these bill materials mostly focuses pipeline ecosystem because test the dependency for pipelines is an item I had to manage. So Jesse was mostly addressing this use case, but technically something like that could apply it on the user side. So there is a repository Jenkins SEI bomb. So somebody could implement the same pipeline for testing and automatic dependency updates, let's say for AWS plugins. Well, if you have good SEI for that, then it could be run in a mostly non-traumatic way. I wonder if we might end up with a case if there are two separate bombs, so this one and the SEI one for AWS, that if there are any shared dependencies between them and there are incompatibilities, then the two bombs will sort of conflict with each other. I don't have a good answer to this question because, yes, they will. It's pretty annoying, isn't it? Well, CloudBees results that by having a super package of something like 200 plus plugins, which we provide as CloudBees assurance program. So we integrate integration testing of everything together. But if you wanted to have meta packages, it would be a separate story. Yeah. And I kind of wonder how much value there is in the meta packages if we can't pin versions because the idea is that they should improve the reliability of the experience. And if someone in the stores and they go, oh, hey, AWS plugin X doesn't work with AWS plugin Y, then, yes, not great development experience. Yeah, I totally agree with you. But at the same time, something is better than nothing because right now you have to figure out dependencies between all plugins. And actually with meta packages, at least you would have some domains so that you would know where the intersection with the problem. And it could be problem mitigated by proper organization of meta packages. So for example, pipeline, core pipeline plugins could be a single meta package, like what is pipeline aggregator doing, more or less. And the technology packages like Kubernetes, Docker could be built on the top on that. But yeah, it's yet to be implemented. Your starting point, yes, there is installation wizard. But yeah, I would argue that it doesn't get you where you would like to get. So you can see that it's basically just a set of recommended plugins. And since we didn't really spend much time on keeping it up to date. So let's say one wants to use Jenkins with Docker, we don't have a way to do it here. Okay. So for me, that's why I started from labeling and grouping plugins on the plugin side and update center side. Because eventually it could at least provide data at least for maintenance action decisions. Right now there is no. Okay. Okay, I'm not sure what's going on. I will just kill it. So wrapping the custom WarPackager using a REST API would probably serve some of our purposes while at least downloading, I mean, at least providing users with an interface to at least install the most used plugins using a sort of simple user interface than a YAML file. Yeah. It could solve the issue. So one thing you would need to keep in mind, it's for your information reek that custom WarPackager, even if it provides Docker file. So technically in your service you can spin a container which builds an image for user. The problem is Docker packages because right now it doesn't support to docker in Docker and custom WarPackager just generates a Docker file. So if you wanted to run it inside Docker, you would need to have something like panic and whatever, which is totally possible, but it's not how it's working now. So right now it's just generates Docker file if you run it inside container. Does GitHub still have issues today? Yeah, I'm not sure. Yesterday's shields were just crazy because of GitHub, REST APIs. Okay, so could custom instead of just, I mean, when you run certain scripts or run CLI, it does download a file. So could this service just run on a, I mean separately, where it does allow you to download a sort of file or something? I don't know if I'm making sense, but. I'm not following right now. I mean, I mean, I don't know how to put it, but. If you mean it will save the generator demo file just as a download in the browser rather than generating the war itself? Yeah, I mean, how would we obtain the, I mean, how would we obtain the war file, right? That's the, I mean, we could, how would you, like just like this spring initializer gives us an entire zip file to download just on the same lines. How would the, how would we actually download? Maybe have a downloadable file for the. So there are two parts. One part that you can, so here you don't generate a project for yourself. It doesn't provide you jars. It just provides your project definition. Same on the service could generate, for example, what I have here is just a configuration which includes a bunch of stuff. But basically this is configuration which you can take and build on your own, but it's ready to go. So it's click a button. You can run and make file and get the image. Okay. So one of the ways for the service is to not provide war files or Docker files or Docker images. But just provide definition so you can build it on your own. If you want to provide a build service which actually builds the packages, then you would need to run customer packages and set your service. Yes, exactly. That's what I was saying. Yeah, it's doable, but it opens a lot of questions like spending additional containers in the service, all kinds of security questions if you wish. Yeah, you could deploy this artifacts somewhere, but this artifacts, just look at the word packages, takes a while to complete. Yeah. So we, well, not that long, but it may take 10, 20 minutes sometimes, depending on your configuration complexity. So yeah, it would be a more challenging part. So if somebody was working on this project, I would recommend to split it to phases. One phase just generates configuration and only after that you could work on the build service. Okay, thank you so much. That helps a lot. Yeah. Rick, what do you think about that? You mean the service just provided the configuration, right? Yes, the first step. I think it's okay, but we can, I think we can also provide the binary file. I mean, you just can't get a Dickens dot WER, file or doping measure. And I know that we all that need a lot of resources to build the file, maybe a amount of storage, but maybe we just do not provide the service from the public service. But if the project have the ability to build them, maybe the users can like deep load this customer service in their environment to build their own Dickens dot WER. That's very, very helpful. Do you understand my point? Yeah, you're right. It may be difficult. Well, we can have another goal for packaging service. That's for sure. It's just quite difficult to implement it as a first phase because there is a lot of hidden stones. And second phase, for example, we look at quite different skills because first step, you just build this generator. It can be written to your JavaScript and if you go, whatever you like, but when it comes to the service itself, you will need to likely create a health chart for that. You will need to resolve security questions. If you want to host it within Jenkins IO infrastructure, then we will need to talk to infrastructure team about that. And yeah, you know, right? We were resolving a lot of budgeting issues recently. So I'm putting new services while it's possible, it will require significant discussion. So if you put the build services as a first step, it may just require a lot of extra effort. Okay. I agree with you that it has value. It has a lot of value actually, but it's difficult to eat this pie in one piece. I would also say like for this, like our tooling team can run the build steps for like the compiling the Jenkins images and that in their sleep. But what they spend most of the time on is guessing what's meant to go in the YAML file. So stage one, at least for us, is where most of the value would come from. And also we probably have some requirements around like the actual build step because we've got stuff like Play for a Gexray and all of that coming in. So like our enterprise wants to track, you know, put your code where I can see it please. Rather than kind of getting a remote service to compile the war and then handing it to us and it's kind of a black box. So for transparency, we probably only use the step one bit anyway, I think. Okay. We're a large enterprise, so others have different needs. Okay, yeah, the phase one requirement by Oleg sounds like a plan because then you just have to maybe parse or sort of have a front end and then maybe generate a YAML file or a Docker image and then as a phase two, you can probably focus on the build service. That just shot up the required skills of the project tenfold. For Brownie points and possibly to increase the work again. A lot of the time is spent guessing what plug-in configuration is meant to go in. So not the configuration of the core itself because you can go and read one set of docs pretty much and work that out. It's say like, if I want to use like even my own plug in the AWS credentials provider thing, well, what configuration do I have to go in to Jenkins YAML? Okay, I've got to go off and read the plug-in, read me and hopefully the plug-in also remembered to document that. So if there was some way to kind of inspect the plug-in configuration classes automatically and work out what is meant to go in there as a kind of scraper or harvester process that would certainly help users though it would be quite ambitious. We'll have a separate project idea for that too. Yeah. Well, it could be probably split to two project ideas if we can find mentors because some bits could be detached but yeah, at least doing some parts now would be interesting. So in a GSOC project, you set up a project idea and this project idea may involve various directions, various goals and basically we expect students to come up with their proposal based on this list of ideas. So even if you think that the project is bigger than what can be done in GSOC, it's not a blocker, it just needs to be explicit in the project definition. And yeah, from what I see, there are a lot of people in the thread who are interested to work on that even outside GSOC, so yeah, who knows. Again, the student could be participating, can it be a team as long as they're first by the student isolated and can be isolated separately, it's not a problem. Yeah, sounds cool, I guess. I guess that sort of ends the questions from me but do we want to edit this project idea or is this fine? I mean, since we're splitting it in two phases, would we want that to be reflected on the side? Well, if somebody could comment on that, it would be a good improvement. Cool, maybe I could do that. If you'll feel free to coordinate with Rick and if Rick is on board with such changes, then we can just update the project idea. Yeah, sure, sounds cool then. Yeah, I mean I could get a draft idea for this proposal because I was just working on my proposal, I think I put it in the mailing list but after this discussion I guess there are significant changes to be made to that so yeah, I'll submit it probably by the end of next week or something. Yeah, well you can just work on your proposal draft because yeah, we can publish recording of this meeting so that it can provide additional information to potential students. So yeah, just to be clear, Sladyan, it's not your responsibility to update a project idea. It would be much appreciated because it's also a contribution to the project but your main goal is to create your proposal. Keeping all the information public is important. Yeah, yeah, definitely. And yeah, thanks a lot for exploring this area. Yeah. Is there anything I can help with with regards to additional information or maybe some alignment increases? If I somehow can help with meta-package definition please let me know. But yeah, I also think that we could just keep discussing that and I hope to send an email by Monday with summary. So maybe we'll just have this discussion in the thread started by Rick. So I will express the point of the day so we don't create duplication. It's already read it referenced here in the top. So yeah, we can find more collaborators, I hope. I'm sure they're out there because so many people have to deal with the stuff either manually or whatever but focusing the amount of their hiding pulses. Yeah, you can see my demo repository was for 45 times. Yeah, some demos but if you navigate to some repositories you can see that people just fork it and start hacking. And yeah, if we had more supported an official way to do that, it would be an improvement because it was just a proof of concept by me at some point, which kind of works. Relating to the Helm charts that are packaged up with Jenkins X, I'm trying to delineate in my mind like what the AWS equivalent of that looks like because obviously because like you could imagine a Kubernetes meta package in the registry like aggregates the Kubernetes credentials provider and the Kubernetes like the old agents and all of that but it wouldn't be enough to get going with and that's why we have a full next distribution for Kubernetes. I'm wondering if like for the AWS equivalent, okay, we can have an AWS meta package but it's not quite enough to get you started because you still have to have all the surrounding terraform and whatnot to scaffold the server into existence. Yeah, at some point there was a project, sorry. Would it warrant a kind of a full if I can Jenkins X for AWS? If so, what would we put in it above and beyond just the meta package of AWS plugins? Because Kubernetes at least has the benefit that everybody uses Helm charts or whatever and it's sort of one standard but not everybody uses terraform to put stuff on Amazon. Some people use cloud formation or whatever. Yeah, so one of the projects that you could take a look at is Jenkins server green. So it's a bit obsolete and it's archived because we shut down the infrastructure in late January but it was exactly an attempt to provide a solution for AWS with some goals like having continuous delivery to these instances but if you deep dive you can find that there are Docker files which package Jenkins is a set of plugins and then there are some verification flows which prepare this list of plugins. So they don't use customer packages internally instead of that they use their own scripting. Well, customer packages was created later than ever green but a final result is basically the same. They have some dependency management, some CI, they prepare package specific to AWS. They also pre-configure things like dependency on S3 buckets and other things. So out of the box you get some kind of web environment and you can run it. So the repository is still around so some needs could be reused. It definitely does sound a lot of it would be useful to us. Probably the thing that would have knocked it on the head is the continuous always up to date on the bleeding edge of plugins which is, I mean, that would have made it unusable for us. We would have couldn't have that. Although the idea is cute to kind of, I suppose it's like the arch Linux of Jenkins always up to date and the downside is always up to date but if the kind of the evergreen bit was removed then I suppose it would be packaging Jenkins for AWS and that would be handy. I agree with you. So we can just explore these options. I'm not sure. So evergreen folks are still around in the community so they could probably provide some insights or if somebody wants to detach AWS, let's say meta package or whatever story to a separate project, it's totally doable. Yeah, understand that continuous delivery is really complicated but we can get some bits of that maybe it's dependency management. If I can help you with that, yeah, I'll do my best. So, Chris, how would you like to organize this work for AWS? Do you want to keep it independently from a GSOC or would you prefer to somehow get alignment and to have it as a reference implementation? Well, no one will ever say no to a Flurry-Australian army of GSOC students to do work for them. The question is, how would alignment involve? So one of the choke points is about production readiness because what you need is definitely a production ready template by production ready definition and your teams already got a load towards this way. So I wouldn't expect our first parks to be really ready but maybe what we could do is, for example, if your team is interested to open source some bits of your shared configurations and maybe see whether we could convert them to templates gradually. It could be- I was about to suggest that actually we could open source the kind of share your config thing that we've got and sanitize it so it might just be a static snapshot of it. And then you can kind of go through that and go, okay, we can reuse this and that and know that bit looks else a bit specific. So we can do that. And yeah, I think the tooling team will be happy to do that. That was great. And even in the beginning, it would be a great case study because I really can deep dive and see what are the issues in the field. Because yeah, I have some pet projects which use customer packages, et cetera, but I have to admit that I don't have big production and it's powered by that flow. So some information would be definitely useful. Yeah, it sounds like a good plan. Yeah. So Rick, what do you think? I think just the last couple of minutes. So did you get enough information from this meeting or would you address some other topics? Would you like to address some other topics? I think it's enough for me. Okay, so then, yeah, thanks to everybody. If you want, I can publish at this meeting, let's say on the JSOC channel or I can just publish it as unlisted. So just share a link so we can have more information. Yeah. This meeting was really helpful. Thank you so much for your time, Oleg, Chris and Rick. Yeah, and thank you too. Yeah, it was a bit chaotic and sorry for the late proposal about the meeting. But yeah, maybe we could meet again, I would say in a couple of weeks or months. Once we have more details, once the slide, for example, has a proposal draft, so we can review that and maybe map other stories. Yeah, definitely. I will do that as soon as possible. Thank you so much, Raki. Yeah, thanks all. Bye, everyone. Bye. Bye.