 Hello, this is the February meeting of the Jenkins Cloud Native Sieg. And we have, oops, just dropped. We have three things in the agenda, four things in the agenda. And we will start with Craig from Google. We cannot hear you. Nothing, nothing yet? No. Maybe we can give you some time and Thomas, you can go first. Yes, I can go first. So let's share my screen. It's very strange. There it is. We see two of them. Two of them. I'll error in this class. Wait a second, please. It seems we have connectivity issues. Yes, you've been able to go there. We can hear you, but now there's a problem. Yeah, it happens sometimes in Hangouts. You would have to reconnect to the call to fix that. Yeah. Just have a few colleagues who experience it almost every meeting. I have to reconnect. So give me some time. We all need to cut this part of the video, I guess. And Craig, I don't think we can hear you yet. OK, so Tomik is rebooting his computer. Be right back in one minute. Sorry, guys. Thomas, maybe I should go, or Oleg, do you also have some things? Yeah, but I have a really quick update. So if you want, we can start from it. Sure. OK, full disclaimer, I just moved to a new laptop today. So if my screen hangs or whatever, well, I believe it's to this meeting. OK, does it work? Yeah. OK, so my topic and the agenda is actually to briefly summarize what we have in Google Summer of Code and what is related to Cloud Native SIG. Before that, we had a few project members joining the SIG meetings and other meetings to discuss what we have. But yeah, just to summarize, today there will be announcements about the Google Summer of Code in the Jenkins project. I can disclose what will be these announcements, but I can summarize what we have in terms of project ideas. So Google Summer of Code is just international and probably the biggest in the world program for open source contributors. Many students work on various projects for three months. And Jenkins participates in this activity. And we have a number of project proposals for this year created by Jenkins contributors. So you may see a pretty long list of projects. And some of the projects are actually directly related to Cloud Native Special Interest Group. For example, if you go to SIGS.jsoc, yeah. Can you guys hear me now? Yeah, we can. Yay, that's a good one, guys. So here in Plugable Storage, we have two links. It's proposed projects. We have a project for fingerprint storage. We have a project for workspaces. So Cloud features for external workspace manager plug-in, external fingerprint storage. These guys are here. So if somebody is interested, you can just join this project ID and discuss it with potential contributors. But in addition to that, we also have a number of other projects, which definitely in the SIGS interest area. For example, we have remote improvements, like remote in Kova Apache Kafka with advanced features for Docker and Kubernetes. We have a monitoring feature for better integration of networking and Prometheus. And yeah, there is a lot of various fine-grained proposals for other things. But yeah, for the Special Interest Group, there may be a number of projects which might be interesting. And of course, all projects like Kubernetes, a Periator, or various Jenkins-Fadrana-based flavors, they could also be a part of Google Summer of Code if somebody is interested in mentoring students or just working with them. So yeah, if you have any interest, let us know in GSOC-SIG. And we will help to prepare more project ID and to get it published. So this is a quick update. And again, I cannot say whether we are accepted to note the nonce one till we went into your house. But if we are accepted, we will start reaching to all Special Interest Groups and specifically to this one. So yeah, if you have any ideas in mind, just let us know. So this is a really quick update. Any questions? OK. So yeah, if you have any interest in Google Summer of Code, let us know. And yeah, if not, still be prepared to some cool projects if you're going to embark on. Are the project ideas closed already or? No, they are not closed. So we will be accepting project ideas for at least one or maybe two weeks. So if we get accepted, the deadline for student applications will be in late March. Until that time, it will be potentially possible to propose new project ideas. But yeah, we ask all interest people to propose ideas early. Because yeah, JSOC hasn't officially started now students. But we had students reaching out to us three months ago. And many of these students already did some contributions. They drafted some project ideas. So the fact that the project has been already started. How we can reach you? Oh, OK. Yeah, it's easy. Just a second, let's try my screen again. So if you see the screen, if you just go to the landing page, it's a project's JSOC. You may see that there is a number of contacts here. So for example, we have separate Gitter chart and separate mailing list. So these are two main communication channels. And both of these channels are pretty active. So you just send us a message. And we will start from there. Perfect, thank you. OK. Anything else? Thanks, Alec. Thank you, Tom. I guess we can continue with Craig now. Hi, can you guys hear me now? Yes. Hey, excellent. I apologize for the technical difficulties. But the old IT crowd adage, have you tried turning it off and on? Again, worked. So let me share my screen. And we will get this party started. OK, so basically what I'm going to do today is I'm just going to talk about what's going on with Jenkins on GCP. And then I'm going to give you a tour of some of the plugins that we support. I'll give you a demo of our latest one, which is our GKE plugin. And then I'm going to use this as an opportunity to get feedback from the community here, because we're interested in finding out what it is you guys would like us to work on next. So without further ado, just before I go any further, can you guys see my slides all right? Yeah. OK, fantastic. So just to give you a little bit of background for those who may not know, my team that I work on is called the Cloud Graphite Platforms Team. And we're a team underneath Google Cloud. And we're a pretty unique team industry-wide in that we are dedicated engineering resources for building out integrations with open source platforms like Jenkins. Our team supports some other platforms and tools. The most notable would be Terraform and Cloud Foundry. And basically, my sub-team is responsible for building out integrations with CI-CD platforms like Jenkins. And it is my team's job to look at the Jenkins on GCP experience as a holistic end-to-end experience. And we really treat that like a product. So we collaborate with our partners at companies like Cloud Bees so that we can cross-pollinate and we can sync on priorities. And we can work together to deliver a great user experience for our GCP customers running Jenkins on GCP. And as a part of that is we bring enterprise level support to the integrations that we provide. We bring the kind of engineering excellence that is expected with a Google product to our integrations and also bringing the documentation that has become the expectation when integrating with a Google Cloud product. So again, we're treating the entire holistic user experience of a Jenkins user on GCP as a Google Cloud product. And just to give you a bit of a contrast, my team is relatively new. We were founded, at least my sub-team, but the platform's sub-team, was founded relatively recently. And prior to our team's founding, this was kind of the old experience that Jenkins on GCP users would get. And for those of you not familiar, this is what's called a patchwork quilt. So what the previous experience was is various product teams throughout Google Cloud would invest a little bit of effort into building out a Jenkins integration for their product. And then they would check that checkbox and they would move on. And what that was left with was a smattering of Jenkins integrations with various Google Cloud products that didn't have very good user engagement, didn't have very good user level support, and had varying degrees of documentation and varying degrees of the amount of engineering effort that went into them. So what my team is doing now is we are now trying to change that story so that we focus on a core set of the plugins that we kind of think are the table stakes plugins for the Jenkins on GCP experience. And we have taken ownership of those, so we've recently taken ownership of the storage in the OAuth plugin. Our team published the GCE plugin and recently the GKE plugin. And we're bringing all of those core plugins up to a consistent level of user support, a consistent level of documentation, and a consistent level of user engagement. And so right now, these are the four core plugins that my team supports. And I'll share these slides. I'll add a link to the meeting notes so that you all can have access to these links. And another couple of things they wanted to call out is thanks to the Kubernetes Engine plugin that Carlos works on, we have a great story for running Jenkins on GKE and Vic, who is our solutions architect, who many of you may know. He actually wrote these two solutions guys, one aimed at the architecture that you would set up for running on GCE, and another aimed at setting up the architecture to run on GKE. So I highly recommend checking out these links if you haven't already. They provide some great getting started steps if you're interested in either of those scenarios. And then I provide links to each of the four plugins that our team currently supports. And stay tuned, we'll be releasing more as well. So really quickly, before we go on to the demo, I just wanted to give you an idea of what is on our roadmap right now. And hopefully coming out of this meeting we'll be adding to the roadmap given your all's feedback. So right now in the Jenkins 2 side of the story, we've heard from a number of partners and customers that getting our existing set of Jenkins 2 plugins to support declarative pipelines as a priority. So that's going to be priority number one for us going forward. And hopefully within a couple of months we'll be able to kick out some new releases that will support declarative pipelines for GKE and Google OAuth as well as Google Cloud Storage. GCE is more of a cloud setup. So we'll have to investigate if it makes sense to integrate those as declarative pipelines. But if in that investigation we find that there are convenience ads by doing that, we'll also integrate it there. And then another big one that customers seem very interested in is having support within the GKE plugin to be able to deploy to GKE on-crim. So we're also prioritizing that as well. And then a couple of new integrations that we have on our roadmap are a Stackdriver plugin for Jenkins 2 as well as providing a GCS Artifact Manager plugin. And then for Jenkins X, which we all want to support because it's the future of Jenkins and we want to go there with you, we're still formulating what we would like to prioritize as far as what Jenkins X integrations we want to work on. We're collaborating with our folks from CloudBees to figure out what they think is important. But also I'm hoping that another piece of information that we'll get coming out of this meeting is what you the community thinks is important and what you guys would like us to prioritize as far as Jenkins X integrations with Google Cloud services. OK, so for the demo, I'm going to give you a kind of a broad tour showing you what it all looks like to work with the four core plugins that we support. And then our newest plugin, GKE, I'll actually give you a live demo of how that works and show you it deploying build artifacts from the Jenkins pipeline into a cluster running on GKE. So without further ado, let's load up the Jenkins environment that I have set up for this demo and we'll kick things off. So I'll start off with probably the most table stakes of all the plugins, which is the Google OAuth plugin. And this kind of underlies all the other plugins is what enables you to be able to set up credentials within Jenkins using your Google service accounts. So when you go into your credentials interface and you click Add credentials within Jenkins, one of the options that you get when you've installed the Google OAuth plugin is to be able to install a Google service account from a private key. And this is an account that you can set up with the G Cloud CLI. And you can also grant it permissions to, if you're working with GCE, you can grant it permissions to GCE with the Google Cloud CLI. And then another thing you can do with the CLI is you can actually export a JSON key that has the credentials you can use to authenticate with. So basically what you need to do here, once you've selected this, as you enter what project name you want, basically it's the project that corresponds to the service account you've set up. And then you select the JSON key file that you've exported for that service account. I've already set up the service account for my project, so I don't need to do it again. But basically what this enables you to do is then downstream the GCE plugin will be able to utilize that service account for authenticating against the Compute API. The GKE plugin will be able to utilize that for the GKE API. And the Google Cloud Storage plugin will be able to use that to authenticate against the Google Cloud Storage API. And then another pretty fundamental plugin that we have is the GCE plugin. And that enables you to set up a cloud. Once you've installed it, you go down to underneath the system configuration for Jenkins, you can go down to cloud. Once you've installed the plugin, it gives you an option to, here, let me hide this really quick. It gives you the option to select Google Club Compute Engine as a cloud that you can configure. And this enables you to run your pipeline workload on agents running in VMs inside of GCE. And essentially what you need to do here is you need to specify what project you want to use. You then select your credentials you want to use. I've already set up my service account credentials via the OAuth plugin. So I've selected the ones I've already set up here. And some of the initial options that you have to configure up here is you can set an instance cap. So for example, if you don't want to use any more than 100 instances, you can set that up here. But the real meat of what you can do is actually in the instance configuration here. So you can set up quite a few different instance configuration options. So these are VM instances that you're configuring here. You can set up what labels you want to be associated with them so that when you go to set up your jobs, you can specify I want. And you can have multiple instance configurations. You can say I want jobs matching my Windows label to run against my Windows VM. You can say I want my jobs against my Linux label to run against my Linux VMs. We do support Windows. We allow you to specify Windows SSH credentials that you can use. And then here, you can say what region you want your instances deployed in. But the other really useful thing is advanced. So you can specify what kind of machine you want to use. So these are the GCE VM configuration skews. And you can look this up on the GCE documentation to get the details. But for example, if you want to use a 16-core CPU, you can do that. Or if you want to have an army of smaller machines, you can select the small one. But probably the biggest and most important option that you can configure here is preemptible. And I highly recommend that people consider using this preemptible option. Because what happens when you enable this checkbox is you get an 80% across-the-board discount on your VM core hour usage billing rate. And that's because you're basically telling Google Cloud that I'm allowing these VMs to be preempted by higher priority work. But if you're running a Jenkins pipeline, you don't notice that. I mean, if a VM gets preempted, meaning it gets rebooted, if it's in the middle of a workload, the Jenkins master can handle that scenario. I mean, we're working in an environment where workloads are recoverable and jobs can recover when one of the agents goes down and comes back up again. So in a workload like Jenkins, which is a batch sort of job workload, we highly recommend that people consider using the preemptible option here. And that has a significant savings to customers. And so this is the kind of the de facto recommendation that I give to customers who are wondering how they should configure their Jenkins to run on GCE is definitely run using preemptible. You can also configure what CPU architectures you want to use. You can configure startup scripts. If your workload requires GPUs, you can configure that here. You can configure what networks you want it to use. And also, if there's a particular boot disk image you want to use, you can select that here. So Google Cloud comes preinstalled with all these various OS images that you can use right off the bat. But if you want to use one of your own, you can upload that through the GCloud API, I'm sorry, the GCloud command line tool, and then it'll show up here in this dropdown. So lots of really cool ways that you can configure how to run your agents on GCE. And again, if you're looking for end-to-end solutions guide and how to set this up, I would check out this solutions guide here that we've written up that walks you through like setting up this entire architecture. So that's the GCE plugin. And then before I get into the demo, one more plugin I want to highlight for you, which is the GCS plugin. And this is what enables you to upload things and download things from buckets in Google Cloud storage as a part of your pipeline. So once you've installed the GCS plugin, you get two options that are added to your build steps here. One is called Google Cloud Storage Upload and the other is download. So with download, if you would like to download artifact that is in a Google Cloud Storage bucket and incorporate it into your workspace as a part of your pipeline, then this would enable you to do this. And as I mentioned before, getting support for this in the declarative pipeline, DSL is a P1 for us. So you'll be able to do this in a short amount of time using the syntax that you're familiar with in pipeline. And then additionally, if you want to at various stages throughout your build, upload artifacts that have been produced as a part of the pipeline, then you can select and upload build step as well. And it's the same thing. You specify a local file pattern. You specify where you want it to go in GCS and then it will upload it. And it uses the Google OAuth credentials. So it's automatically selected this one because it's the only Google OAuth credential I've configured. But if you have multiple service account credentials added to your credential store in Jenkins, they'll all show up here. So that's the GCS plugin. And then the one I want to show you an actual demo of because it's our newest one is the GKE plugin. And let me just quickly grab the URL. And this is my GitHub profile if anybody's interested. This is actually a picture of, I can't actually zoom in on it. This is a picture of me at a local cat cafe here in Seattle. So if you guys are ever in Seattle, I highly recommend going to Mealtrapolitan. It's a very fun experience. But anyways, I set up a repository that just has a manifest in it so that I can demonstrate deploying to a GKE cluster using this plugin. And this just uses an Nginx deployment. So nothing fancy here. So what we're gonna do is we're going to set up our Git repository here so that this repository will be pulled down into our workspace. And then we're going to select the Kubernetes engine build step. And this is what shows up after you've selected or after you've installed the GKE plugin. So what you're gonna see here is a few options to configure this deployment. So essentially the main value add that the GKE plugin provides for you is it does the negotiation against the GKE API so that first of all, it'll authenticate using the service account credits that you set up with Google OAuth. And then what it'll do is it will actually encapsulate the negotiation of the GKE API for you. So once you've entered what project you want to use, so this, if I had multiple projects associated with the service account, they would show up here. Then you select what zone you're operating in and then it will automatically talk to the GKE API for you and get a list of all the clusters that you've set up in that zone. And so we want to deploy to our test cluster. And then the other thing you specify is what the manifest is within your workspace. And so what it's doing behind the scenes is it's communicating with the GKE API when you actually execute this and it's grabbing the cluster information from the GKE API and it's getting the authentication tokens that you need and then it's actually creating a temporary Qt config file in your workspace so that it can then execute Qt Cuddle Apply against your manifest. And for those of you not familiar, as familiar with Kubernetes, essentially the Qt config file is how Kubernetes encapsulates authentication against the cluster that you're actually deploying your manifest to. So again, the main problem that this plugin solves is how do I, using a Google service account, automatically authenticate against the GKE API and then get the credentials for deploying to that cluster into my workspace so that I can then run my Qt Cuddle Apply locally. And it also just as a convenience for you will also run that Qt Cuddle Apply for you. So that's basically what this plugin provides. So when we save this and we run it, we'll take a look at the output for this so that you can see what's actually going on behind the scenes and then we'll show you what it's actually deployed. So it's successfully run. Let's take a look at the console output. So first thing it did is it grabbed our repo from Git and then it, after it communicated with the GKE API, it created a local Qt config file here, which is what we're seeing here. And it's basically telling Qt Cuddle to use this local generated Qt config file as the context that we wanna operate in. And then the final thing that it's doing is it's running Qt Cuddle Apply using that Qt config file that we generated against the manifest that was specified in the configuration of the plugin, which allows Kubernetes to then run your deployments using the build artifacts that you've generated. So theoretically speaking, the workflow would look something along the lines of my Jenkins pipeline runs the build against my project, publishes a container image from this build to either Docker Hub or GCR so that then as the final step, the actual deployment of it can reference that container image that was generated and just run out of Kubernetes deployment from that. So then if we take a look at Qt Cuddle here and we do just to verify which context we're operating in. So we're operating in the test cluster that we configured via the plugin and then we do Qt Cuddle services and see, let me actually verify that I'm, so container, clusters, Qt credentials, central, and C, and cool. So when we run Qt pods, we didn't actually create a service that's why it's not showing up. We just did replicas of this template. Okay, so we can see now that the nginx pods have been set up using this image that already exists on Docker Hub. But essentially, if you were using this as a part of your workload, you would swap this out with the image name for the container image that your pipeline had built. So simply, this has run this deployment for us and set up replicas of our app here inside of our test cluster and just to bring it back to the configuration. So if we go back to our demo project and back to the configuration, again, this is our test cluster that we had selected and this was our zone and this was our project. So that's the demo. And then the next thing I wanna go into is community feedback. So I wanna be greedy with this opportunity and basically ask you guys, well, first I'll do some Q and A if there's any questions about any of the plugins. But then finally, what I'm really interested in learning from you folks is those of you who currently use any of our core plugins, are there any burning features that you would like to see aside from the ones that are already on our development roadmap? Are there any other big holes that you as a customer or you via your customers know about that you would really like to see integrated in the Jenkins 2 space? And then the really top of mind thing for me is in Jenkins X, because this is all green field for us, you guys really have an opportunity here to let us, as the Google engineers working on these Jenkins X integrations, let us know what are the integrations with which services do you think are very important? And this will help inform what we said as our development roadmap going forward. So without further ado, I'll stop sharing my screen and open up the floor. First of all, the questions, are there any questions about the demo going once? So you were saying you're adding pipeline support, so? Yeah, so that's our P1 right now is adding pipeline support for GKE and GCS. And we're going to investigate if it makes sense in GCE as well. But for GKE and GCS, that's our P1 is adding pipeline support for those. Any other questions? Okay, so there's no other questions about the demo. I just want to open it up to the floor and ask you all, what either Jenkins 2 integrations would you really like to see us work on? And or what Jenkins X integrations do you think would be very helpful to the community and to Jenkins users in general? Yeah, one question about configuration as code. So you presented the configuration of the plugins in Jenkins Web UI. Yeah. But do you plan on integrating this configuration as code plugin? The pipeline, the declarative pipeline? No, there is another plugin, Jenkins CI slash configuration as code plugin. I've actually, it allows configuring Jenkins for specific YAML conflicts. Oh, I see, okay. Yeah, we'll put the link to the meeting notes. Perfect. No, that's good feedback to have. Yeah, we can definitely look into that. Do you know is there a significant portion of the Jenkins community that uses that plugin? Well, it's been a while ago, so it has almost 1,000 stars. I'm not sure how many installations, but it's definitely trending. Okay. Yeah, some build flows, for example, Carlos will be presenting the Jenkins file runner later. There are also some integrations between the Jenkins file runner and the configuration as code. Yeah, I believe that it will have much more adoption soon. Okay, fantastic. Yeah, that's great feedback to have for sure. And that's just to verify it's Jenkins configuration as code plugin. Yeah, I'll put the link to the meeting notes. Okay, yeah, we can definitely look into that. And is that, is an extension for that also available in as a Jenkins X extension, or is that just a Jenkins 2 plugin right now? Well, Jenkins X uses configurations called in the serverless mode. So serverless mode is used, so Jcast plugin is used to configure serverless mode in Jenkins X. Right. But yeah, I don't think it has other integrations in Jenkins X right now. Okay, all right. Probably will have more in the future. Okay. I believe so. Gotcha, okay. It's popular because everybody wants to configure everything as they are moved today. Right, yeah, that makes sense. Okay, so speaking of Jenkins X, like which services, if you had your dictator for a day hat on and you could just magically have, you know, these three Google Cloud services working natively in Jenkins X right now, which would those be? What do you think are like the prime targets for us to implement integrations for as far as Jenkins X and Google Cloud? For me, it would be interesting to have but Stackdriver integration. Stackdriver? Okay. Specifically, there was a specifically for pipeline. So there was research project for pluggable storage and currently pipeline supports setting of custom lock storages. And yeah, for example, we have AWS CloudWatch proof of concept implementation. And it might be helpful to have something like a bit for Google Cloud. Okay. Yeah, full disclaimer, there is some, so it's in the early access now. So there is no one to zero release, I believe. Yeah, there is no confirmed plan to have one to zero. But if it happens, I would really like to have Stackdriver in place. Gotcha. Okay, cool. Stackdriver got it on the list. Okay, I have shown with Stackdriver, I mean, it's worse today if you would just want to show like Jenkins X on serverless mode. You just show all the logs go to Stackdriver. So you can just look in there. The other one that will be interesting, especially for serverless is anything related to storage. So every time you do a build, you can store and retrieve data from a blog store. Okay, so a GCS integration would be important? Yeah, we've looked at that in the past with like a stash and an stash. So then you, I mean, I guess you would un-stash before starting your build and then stash the results somewhere, especially for serverless because I mean, if your containers are going away, do you need a place to store things permanently? Right, right. That makes sense. Yeah, another story which might be interesting, for example, this summer, here we have external workspace manager plugin. And one of the ideas for this Google summer of code is to have integration with cloud providers. So the idea of this plugin that you can provision the workspaces on demand, including size of this workspace, et cetera. So I'm not sure how it would map Google cloud, but it would be an interesting integration if this project starts. But yeah, right now there is plugin, there is extension points. But yeah, there is still a lot of work to do before it becomes production ready. Yeah, it's really handy in some pipelines. Gotcha. So being able to provision workspaces on demand. So the integration point with GCP, are you envisioning that the workspace would be provisioned within GCS or what are you envisioning as far as how could Google Cloud help realize that story? GCS would be reasonable. Yeah. So my use case is that I have a user on my Jenkins instance and this user needs, for example, 20 gigs of storage. Gotcha. Just implement his temporary workspace. So obviously these containers, there are some ways to do that, but if you want a permanent storage, which also remains available after, for example, the container determination. Right. It would be nice if a user would be able to just request a storage in his pipeline. And this storage gets provisioned and he maybe works with this provisioned workspace without doing anything else inside his pipeline. And then there may be retention policies, et cetera, configured for that. So yeah, we have part of the equation. We did such workspace provisioning for classic modes like NFS, but yeah, obviously we're interested to have cloud providers in this story. Gotcha. That's good to know, for sure. Yeah. All right, any other ideas from anyone else? Now's the time. All right. Well, thank you very much, everyone. I appreciate the opportunity to come and speak. And yeah, and I hope that you guys found this useful. So without further ado, I'll hand the reins back over to Carlos. Thanks, Greg. It's good to see the progress you're making. Okay, Thomas, are you ready? Yes, hopefully I'm ready. So I can share my screen. Can you see my screen? Yes. Yeah. So I would you like to introduce Jenkins Kubernetes operator? This new project for me to slap. There's some information about me. My name is Thomas Seng. I'm working at the virtual company. I'm DevOps engineer currently. I'm finding with AWS and Azure cloud providers. I'm maintainer of custom Kubernetes distribution and I'm writing tools in GoLang to automate my work and help users to use Kubernetes platform. And how to run Jenkins on top of Kubernetes? Basically running Jenkins on Kubernetes cluster is not trivial work. Kubernetes platform was released 10 years after the first version of Hudson project. It means that Jenkins couldn't be designed to run on top of it. And what problems you have to solve when you try to run Jenkins on Kubernetes? Basically, there is no visibility that Jenkins can't be configured as you want. There's no visibility that what happens to Jenkins. For example, pod can be not running for now or waiting for this quota was exceeded and can stop again. There is problem with plugins management generally in Jenkins and when you try to use PVC for Jenkins home data storage, then can be used that after the touching, this PVC, no new pod can start. We have this issue with AWS Kubernetes cluster. There's no security by default. There is a lot of people with which reinventing a wheel, they build some custom distribution, build some plugins baked in Docker images. There is lack of tests for this type of custom distributions. So this is some not production ready solution. This is our Jenkins operator logo, still work in progress. The blue guy in is the Goulang official mascot, the GoGoofer because operator is written in Goulang language as Kubernetes and in the right hand here, it has that cake with Kubernetes logo. So every thing is in one picture. So what is goals for Jenkins Kubernetes operator? So first one, run Jenkins on top of Kubernetes is the simplest one. Another one is put configuration of Jenkins in one place. We can use for this CRDA. I will talk about this later. And we have to keep Jenkins instance configured. Basically, the user can change configuration in any time and we have to make sure that everything is put at the Jenkins instance. So we try to make Jenkins immutable. What does it mean? Jenkins master instance should be restoreable to desired configured state at any point in time. Any change made to Jenkins configuration that are not in CR, this configuration file, should be removed after Jenkins master restart. In this particular solution, we can avoid snowflake problem when we have one source of true. And when, for example, PVC where every data file for Jenkins located goes crashed and you have to configure it again every piece. We should obviously scale Jenkins agents to spawn new workloads. And I want to make default configuration secure as much as possible because you can run Jenkins, but you don't know if it's secure or not. We try to make default configuration secure. Another one, we only went to backup build history, not all entire home Jenkins home. I think this is the most important thing to keep between, for example, restarts of Jenkins. And another one is to make errors more visible for end users. For example, you can configure Jenkins in these Ruby scripts and you have to dig into the Jenkins logs and you don't know it's pass or not. And you can have Jenkins that is not configured as you wish. So you have to some kind of inform user that's something is not okay. And we want to reduce Jenkins master unavailability because Kubernetes is very dynamic environment. So Jenkins pod can get restart or killed on very various reasons. So to solve these problems and try to reach these goals, we decided to run this as operator, as operator concept. This was introduced by CoreOS and basically an operator. It's an application specific controller that extends the Kubernetes API to create configure and manage instance of complex stateful application on behalf of Kubernetes user. It builds upon the basic Kubernetes resource and control concepts but includes domain of application specific knowledge to automate command tags. Basically, it can automate some DevOps user task and it's baked in basically code. The parts of operator, this is the custom resource definition. When we will use as configuration file for Jenkins, basically the custom resource definition API resource allow you to define custom resources. Defining a CRDI object creates a new custom resource with a name and scheme that you specify. The Kubernetes API service handles the storage of your custom resources. This is basically a storage for Jenkins configuration and we use controller pattern to be able to rack on every change made to, for example, CR or secrets or config maps and react to make sure that Jenkins is configured as we wish. So overall architecture is very simple. We have main reconciliation loop. It comes from operator pattern and this reconciliation loop is split into parts which is base and user in the base reconciliation loop. I think I can, yeah, this is more detail. The basis reconciliation loop, we ensure manifest, we create, update and delete Kubernetes manifest required to run Jenkins. For example, secrets, config maps, ports with Jenkins master and services, service accounts and all these Kubernetes resources. Another step, we create Jenkins pot and verify status of Jenkins Kubernetes master. We ensure Jenkins configuration. That means we configure Jenkins instance including hardening, initial configuration for plugins, et cetera. So we basically run some Ruby scripts to make sure that, for example, set executors on masters to zero or disable deprecated J9P protocols. The last step of the base phase is to create Jenkins API token and Jenkins will use that to communicate and configure Jenkins basically. And the base phase is decided to run clean Jenkins without any user changes. The next phase takes care of reconciling user, provide the configuration which includes restore backup for build history. And the next phase takes care of reconciling user provided configuration which includes restore backup can be done basically depends where your backup will be. For example, in PVC or I don't know, AWS S3 backup, it depends on this concept of sidecars which will take care about it. And you can basically write your own image to backup your essential information and put it everywhere if you want basically. Then we configure SID jobs via job DSL plugins to make sure that every jobs configuration is stored in Git, so the GitOps pattern. And then we ensure user configuration via groovy scripts or configurations like code plugin. And the last step is to configure backup top to ensure that we can backup build history basically. So I think it's all this very quick and basically introducing to Jenkins Kubernetes operator some questions. Yeah, it looks a bit complex from the current vision. Do you plan to provide a simplified mode or do you want to have Jenkins operator as a single implementation with all the configuration management, backup management, et cetera? Because it seems to be over complicated for some cases. Basically, you don't need to use all features. For example, you don't need, I don't know, use configuration as a code plugin. You can write groovy scripts to configure Jenkins or you can use only configurations that can't plug into configures. So it depends what you need. Basically, I don't think it's a good idea to write some simple, say, Jenkins Kubernetes operator with simple features. Basically, if you don't want to use this, just don't. And you should work with all the features basically. How far along are you? I mean, what's the status of the implementation? Yes, so after getting some feedbacks from Google groups, we have to change some architecture. For now, we use Jenkins jobs to apply groovy scripts and backup mechanism. And we want to instead use, basically, we will execute groovy scripts via Jenkins API because there is security concerns about setting executors on master. We should disable it for by design and let user use it if they want. Basically, there is one thing, another thing we should basically integrate configurations as a code. This is not done yet. And for now, this project is not, I think, well known, but the users post some issues, so we have to take care about it. So for now, it's like alpha stage, I think. So we have to wait to make sure that everything will be covered and CRDI scheme will be defined well. So this is a lot of work basically here. Thank you. That's all? Yeah, thanks. So thank you very much. Do you need any help process-wise? So there is a JEP 219 created for Jenkins creator? Yes, so this was submitted for draft status and yes, if you have time, it will be very thankful if you can help basically. So I really don't like to write the commentations. I like to write code, so. Yeah, but you said that now we have a video recording so you can just link it from the JEP and probably it can be a part of the description. Because yeah, I read this JEP carefully before the meeting and yeah, I watched the slides and it seems that there is significant difference in terms of the depth of the implementation. JEP seems to be a pretty top level and you'll go after many additional topics not referenced there. It's good, but yeah, maybe it's something to be mentioned. Yes, as you read there's overall for Jenkins operator but for specific architecture design I should route another JEP, I think because it's not too easy to write this in the one. You can understand the bell. Yeah, currently JEP doesn't have a different delegate. Have you already had some discussions with potential work candidates? No, no, I don't know which person is the best to delegate basically. Maybe Carlos. For what? Maybe they felt a delegate for JEP 219. Oh yeah, yeah, that could be it. Yeah, so if somebody meets KK maybe it makes sense to mention that so he officially assigns it. Or maybe Rick would be interested because yeah, Rick maybe participated in reviews on house. Okay, thanks Thomas. We ran out of time for what was scheduled with the issues at the beginning, but I'll move where I have some things about the initial runner on serverless. I'll move that to the next meeting that I'm trying to schedule for the first of our second week of March. So it's gonna be soon anyway. Your second week may be complicated. No, I'm sorry, I think it's the third week. Okay, before March 10th, I guess. Yeah, yeah, question about that. Is everybody in this seat familiar with Jenks for the runner? Because if not, since we plan a new meeting, it may make sense to have a short introduction to the presentation in the beginning. Yeah, I guess that would make sense. Take a note. Yeah, so I'll check. Maybe I'll find a presenter for that. Oh yeah, if you find somebody, even better. I think that's all. Thank you for attending. Thank you to organizing that.