 Okay, it's being recorded. So welcome everyone to the Jenkins Configurational Scopes, our projects at SYNCLUB, it's August 28th. Today we have only two people on the call, me and Sladin who is a community of rich internal working on magic as developer tools. So since there are not that many people, we will adjust our agenda to focus on the subproject. I will still do a quick update regarding new releases because there are some important changes. And after that, we will just proceed with community breach and all other topics we will move to the next meeting where hopefully we will have more people. Yeah, so Joseph isn't here to be. Okay, so I'm not sure that's great. Yep, yep, yep. Yeah, integration, that's good. Okay, so we had two releases since the previous meeting. One release, so yeah, last time we talked about security fixes, then there was some delay. So we had a release 1.28, this release was mostly about maintainability. You may see that there were many changes in developer tools. For example, now we have dependable enabled, we have some tweaks and change logs, et cetera. But yeah, really all of these things were about maintenance. What really changed in this release is removal of the vault secret management. So before this release, you'll be able to use vault as a secret source just by installing a Jenkins configuration as code plugin. Unfortunately, there were some issues with that, like client-library conflicts, this HashiCorp vault plugin, and basically it was quite odd to have a dependency on a vault functionality that engineered the configurations for the plugin anyway. So we made a decision to move the functionality to the plugin directly. So now if you use vault as a secret source for JCask, you should be using vault plugin directly. So yeah, there is a HashiCorp vault plugin. Joseph Patterson took ownership of this plugin. So basically it's kind of maintained by the JCask team at the moment. And here you can see that there was a new release to the 5.0. So yeah, there were some changes in addition to JCask, but basically you need to install this version to keep using vault as a secret source inside JCask. This is a major change. So keep it in mind. And after that, we also had another release related to my secret sources. It's 1.29. So we had one issue report submitted in the community that HashiC, that there are issues with password resolution. Actually it was a long-standing issue, but it was hidden because before version 1.24 we didn't have any kind of logging for unresolved secrets. So once we got this logging, some users started getting notifications and finally we ended up with the discovered bug, which is basically related to initialization of secret sources. So yeah, you can see the fix is pretty simple, but yeah, basically some secrets were resolved too late, which could have impacted all kinds of integration management. So yeah, it was a fix submitted in version of 1.29. It's also available to users. And you can see that now there are also dependable updates. So we use release automation nowadays and yeah, all dependencies will be kept up to date. So that's it. Regarding the incoming changes, well basically there are not that many incoming changes in the moment in terms of merged functionality, but there is some work-in-progress changes. One of the changes is about schema generation. So Sladin will be talking about it a bit later. And then there is another change here, which is rather about quality of life. So there is a lot of plugins using the JCask as a test dependency or as a configurator API dependency. So what is Joseph Patterson doing at the moment is adding Jenkins configuration as code to a plugin build of materials. It's a new project, which has been developed by JCBrick at the moment. So it's an evaluation phase, but it has dependencies and versions for many plugins. And now we also include configuration as code there. So it's here. Basically it's work-in-progress, but yeah, hopefully it will be integrated soon and it will simplify maintenance for all plugin developers. Oh, we actually have Joseph on the call. Joseph, would you like to add something about that? Yeah, there's some issues of course, but luckily we're plugin tester code those or plugin compatibility tester code those. So that's nice. Yeah, right. So what else do we have in the list? Yeah, so dependable updates have been integrated. So this part is completed. Do we have any other major activities inside the plugin right now? I guess we've finished up, or I finished up Hasher Kovvald migration. Yeah. Okay, you covered that already. Okay. And we also fixed a initialization issue. Thanks too. Yeah. Yeah, so this part we already talked about. Okay. Okay. So regarding the rest we have in the list, oh, we have four people on the call. So I guess we can just switch to the full agenda. Yeah, because originally when we started, we had only two people. So what else we have in the pull request? So let's see. It doesn't look like, so we still have this to don't the system configuration permissions, but I guess it's blocked on the Jenkins core changes. Yeah. And I also think Tim is on vacation or something, right? Yes. Yeah. But that would be super awesome. Read only. Yeah, so one thing about that is that the Jenkins core patch hasn't been integrated yet. So yeah, there is upstream patch. And yeah, we have a LCS baseline selection today. So it's safe to say that it won't get into the current LCS. And the ET in Jenkins LCS will be December. Wow. Yeah. That's how it works. So maybe it's a good idea to think about splitting a contributor API and other stuff. So yeah, these things will stay around for a while. Okay. Yeah. Hi. Hello. Yeah, would you like to introduce yourself? Because it's the first time you're at the call. Maybe it's just listening in. Yeah. No problem. Okay. So yeah, this one will get stuck for a while, but on the other hand, it gives us an opportunity to finalize this change because yeah, from what I see, there is still a lot of bits to integrate here. And yeah, let's see whether we can resource that somehow. I would be really happy if we have this fix for the next LCS and maybe also initialization milestones fix in the Jenkins core. Because then it will put the change of the stability of JCAST dramatic data users. Especially for job DSL. Yeah. That is actually something that's still blocking. There was the people came back and said it's still an issue with the job DSL. Yeah. Yeah. There was a recent report from James now about running to early in the life cycle. Oh, I missed this. It's loaded. Yeah. So I couldn't believe it's basically the same as the whole issue we've had about chicken and egg. So basically you need to do something about the milestones. We cannot move it after job's loaded because after that it will start configing provenient hooks. This is probably even worse. So I think we need to do something for job DSL to hook into as well. So it would come after. So yeah. We'll come in later in the next session steps. Yeah. So basically what would be my preference that we firstly have kind of pre initialization the step here. I mean milestones or pre initialization logic can run here and post initialization after job's loaded. So there would be two additional milestones. Nice. And maybe even two milestones after job's loaded because again, we still have issue with Groovy Hooks and JCask. All right. Yeah. Because whatever we do, we need to somehow prevent conflicts of them and retain the current behavior when Groovy Hooks run only after JCask. Otherwise it will be a massive regression for users. So it's yet to be seen how we do it but my understanding that we actually need three milestones with fancy names. Okay. So basically no problem. Can you hear me? Yes. Ah, okay. So short introduction for myself. My name is Daniel Estermani. I'm looking for a spoke. So yeah, I did speak. I was born on Crimea in Russia and now I live in Zurich in Switzerland. I work for TI in Dem. And for a DevOps team and we use as well Jenkins and GitLab runners. So I am much responsible for Jenkins. We shortly migrated for usage of configuration score plugin intensively. And I also supported by contributing for plugin compatibility and stuff. And I'm very interested in development of it. That's it for my side. So welcome. Yeah, thanks a lot for your interest. So yeah, one comment about Switzerland. So I planned to organize a hackathon in early October with JCask as a topic. So it might be something interesting to you. Yeah, I'll probably send heads up later once the date is set. But it will be like the beginning of October, most likely in Neuchatel. Though, yeah, maybe in Zurich. Yeah, to be seen. Okay. Sounds interesting and keep me informed. Yeah. Okay. And yeah, thanks a lot for your part and fixing the regression and loading. Okay. So anything else to discuss about ongoing development? Yeah, I started talking about the warning to get back to the deprecation of plugins. So yeah, that would be nice to get around to hopefully. And I'll see if I can, I'll probably start with a mailing list and see if I can get some feedback and then do a JEP if I can, yeah. Yeah, it will be appreciated. It would be really nice to have this feature actually because yeah, it's annoying to have plugins that just keep gaining in installs because people read old documentation. Documentation for what? All right, well, I've seen people read like some of the blog posts for JCask and then they come around asking, well, the support doesn't work, blah, blah, blah and all those things, right? So they install old plugins for some reason and that nothing works and they're confused. So I want to fix the deprecation issue and have the same as warnings, right? Or security warnings. Okay, so what it means that our LTS mission list but maybe this could just be start out as a plugin. Like it's, I hate having stuff in the Jenkins core but it could be a recommended or recommended install or something that it just decides, I'll put this in, could be a module, I don't know. I just, for older releases as well, it would be nice to just have it in working. So it's not locked to the latest LTS. Okay, return on the system configurations, milestones. So in this case, unfortunately, I think it will like to have to be in the Jenkins core because otherwise. Yeah, I understand, but let's see. Definitely it'll be nice to have, at least for sure, to deprecate plugins and get them uninstalled. So we're speaking of another side of this issue about documentation. So there is another project which probably works sharing. It's about publishing the documentation on PluginString.io. Because now if you want, if you go to configuration as code. So here, you can see, well, basically it. And apparently we see that not that many people go there and it's a case for many plugins. So there is ongoing activity for having PluginUp updates at the supporting GitHub as a source. So yeah, it's, this activity is a part of the documentation sub-project. But yeah, it's still something which is really related to jscask. So it's been in connection, it's one of the contributors. He basically proposed a patch for the plugin site which does that. And yeah, right now, yeah, we spent some time to finalize this patch. So it's technically ready to go. And with some patches, you will get, you'd have a documentation listed on plugin site. So yeah, it's basically this patch. Yeah, so yeah, basically after some patches and follow-ups, it looks like that. So this is a point of the folder-based authorization plugin created by Buda. But basically this information is picked from GitHub with all badges, with images. And yeah, we will have the same project task hopefully soon. So now we are just waiting for infrastructure team because we need the GitHub API tokens to make it really work because we need to pull the data from GitHub and yeah, you know, then you must move to, it's not going to end well. So yeah, it's a stage and yeah, once it's ready, it will be available for JCASC. And once it's done, there is a part of Big Epic which also includes stories like publishing change logs and other things on the plugin site. So yeah, I hope that all of these bits will be ready maybe in one month. So we will have JCASC documentation also republished on plugin sites. Hopefully it will help some users. And yeah, my end goal is actually to have the documentation also in Jenkins update center. So that, for example, features like removal of plugins, et cetera, they will be highlighted right inside the Jenkins. That would be nice. No, still something to do in order to make it possible. Yeah, most likely it will require some Jenkins core patches as well, but yeah, it's in my wish list for this integration for GitHub releases. Get to be seen how it will work, but yeah. Let's have just one patient releases on the plugin site. Okay. So yeah, I guess that's all is ongoing development. It looks like we have quite a list of, I think. Okay. Let's go into the next topic, I guess. Yeah. Yeah, so I just put it here and maybe it makes sense to put it to the developer mailing list to keep stuff in the discussion. So, but yeah, probably I'll do that. Yeah. If you want some additional text to some of the topics maybe just ping me. Okay, well, just put it here if you want something, send it to the dev list. Let's say on September 2nd or so. Okay. Yeah, well, if you learn for these features it would be brilliant. Yeah, we'll see. Okay. So, other topic I wanted to discuss is about moving to platform special interest group. So just to provide some context in Jenkins project we have a number of special interest groups which you can see some groups like advocacy outreach, cloud native, documentation, Google summer of code platform, et cetera. And the problem that not all of these special interest groups are really active nowadays. When we started cloud native special interest group we had some discussion with Evelina and other stakeholders about having Jenkins integration as code as a part of cloud native special interest group. So if you scroll down here you can see that basically, traditional code is here. One problem with the special interest group that well, basically it's not active. And another problem that Jcast is not only about cloud native is basically used everywhere as a part of the platform. So my thought was that what if we actually move it to the platform special interest group? Platforms. So this group is currently focused on packaging, on support of operating systems. But yeah, there is high interest in Jenkins configuration as code today as well. So for example, Mark White is a contributor but he's just heavily using Jcast in the projects and basically all of the people here somehow showed up in Jcast channels as well. So while we're thinking about moving it here, but yeah, so it's something for the discussion. So for in platform C we have regular meetings every two weeks on Thursdays. So I guess it would not change anything for the project specifically except maybe periodic updates at the platform C. So I just want to propose for opinions what do you think about it? Makes sense to move it. That's all I can tell you, really. Okay. So Oleg, are you the only one working on cloud native? Right now? Well, the proper answer is that I'm not working on cloud native. Okay. So yeah, I'm listed here, I guess. Yeah, I'm listed here. I was working on some things for plugable storage. So just up on the Gitter channel a while ago because yeah, Joseph referenced this page. I see that there is a number of activities which were originally planned, but yeah, it didn't happen. There are many reasons why, and I can attach all the reasons on this call, but yeah, basically what was happened we delivered artifact management. There is some interest in configuration sources. So it's not a configuration sources we were discussing with you, Slavin. It's rather generic configuration source. So yeah, obvious overlap between these stories. So there is a JEP which was submitted, but yeah, this story is full. So this JEP was submitted basically by me just to summarize the discovery results we had and yeah, there were some contributors interested like Alex Nordlund from Sweden, from Stockholm, yeah, and other people, but yeah, this JEP, our reference implementation at the moment. Do do cloud native meetings happen by weekly? They don't. Okay. That's why I said that this is it. So the last meeting, I mean, the scheduled meeting was at this, yeah, in March. Then there was a meeting at, well, yeah, at KubeCon in May, but it was face to face and let's see. Okay, good. Yeah, there was some meetings. I think about recovering to be special interest group, but even if I recover this special interest group, I, to be honest, I'm not sure that the configuration of school is well fit there. Maybe it's better for platform. Yeah. Good. Sounds interesting. Yeah, it's just thinking out loud for now, but maybe we, so if it helps configuration as code project to get more visibility, more contributors and more collaboration with other projects, because for example, there are obvious opportunities for integration in the Docker. So a part of what Natasha Stopa was working in your JSOC project. Yeah, yeah. I think there are more opportunities for that. In cloud native seek, yeah, for example, if we talk about official Jenkins health charts, then it's cloud native seek. And again, there are opportunities for collaboration there. Yeah, for me, so, JCASC is basically a part of every seek nowadays. So we, to see where it best reads. Good. Sounds good. Yeah. Maybe creating new configuration as code seek could be also an option, but yeah, I think it's overkill. Yeah. Okay. Yep, I think that's it. Yeah, so anything else about that? Okay. So we have something like 15 minutes left. So by this, I can just transfer the presentation to you, Slavin. So you can do a quick update on community bridge. Yeah, so I think in community bridge, most of the work that we have done till now dealt with, I think I could share my screen if you're not, Oleg? Yeah, please do so. Yeah, so I think where we are with community bridge is like we are handling nested-nested YML structures. So I think Joseph suggested that we could actually go ahead with the described structure, kind of a sort of function in every configurator, and that would make it much easier for us to be able to transfer the entire base configurator and attributes to JSON and then probably build the structure from there. So actually the problem that we are facing currently is, and I wanted to ask Joseph, that since we are generating the schema and we call described structure and iterate over the nodes, we would need to decide whether we add the required label to each one of them, because if we add the required label to the sub-attributes, then it would mean that the user has to compulsorily include them into the YML and if they aren't, then because otherwise if we don't include them, then there would be no checks done. So suppose if it is tool and we have git and we have installations and if we don't require installations in git and we just make a typo, then that doesn't get picked up by the schema. But if we actually include them, then the user has to compulsorily add them. So that was one point that maybe Joseph could elaborate on. So here it would most likely depend on how people have defined non-nulls and all those things. But for something like git and installations, that is probably the only property that can exist inside the git object as I recall. And then I guess that would be required or but it doesn't have to be required because it can just end up, when you do your, when you're inside your IDE, it will come up as a suggestion. So it doesn't have to be required. But any property that is a non-null value on like, for instance, under installer, I think a name is required or it could be, let's say name is required. Like it's a non-null, it can't be null. And it's in the data constructor, it's not in the, it's not a data bound set, then it has to be like, then it has to be there and it's required then. Yeah, so, so yeah, my question was, yeah, so since we would have to decide on what tools, what kind of sub attributes are required and what sub attributes aren't required. Another thing that actually we could go over. That's again, that's not actually as up to decide, it's depending on how the, the way we created the plugin, I guess. Yeah, true, agreed. Yeah, we could come to that later once we actually go over the node. So yeah, one question, since we're on the call, of course, is the described structure. So currently, we would want to return all of the descriptors from the described structure that include, that include the, that include the configurator, this configurator and the set of it. Yeah. Okay, cool. So, okay, and I think that's, I think about it. Could we use describe, yeah, as we were discussing in the chat. No, because, because what you actually have already done is I think you're filtering out non-nulls, right? And that's incorrect. You shouldn't, because you need the whole data structure. So you cannot filter out objects that might be null because they might, they might be, actually they might be just a class that says, like in the CM, I don't think, but there are some, I know there's some plugins that act like they have their class as like a behavior that makes something happen. So if you filter those out, then the data structure doesn't have or it will be missing from the possible schema, I guess. Okay, so if I, if I don't filter out non-nulls and I map the nodes, would I be able to return them as scalars? Like, just a, let's just be in. Then it would just be a scalar, yeah. That is like, just have a key name and no, no value maybe. But it's still, of course, in here you need the type. So it would be a type of some class object, I guess. Exactly, yeah. So maybe if we return the scalar, would that be beneficial? Because then we could just write over this and get the type. Yes. Okay, cool, sounds interesting. Another thing that, would the senior here currently return only the base, only the root objects for the hetero-describer configurator? No, because for each node you have to go and look or if each, you have to convert to node, right? So that would convert everything to a node and it goes over all the configurators and keeps doing that till it gets to like the end. So it will do through, so it will do the recursive look up for you. Okay, and once I get the node, the C node, I could iterate over each and every node in C node and get the attribute? Yes, you should be able to do that, yes. Okay, and what if they have like, no question I have, so the thing is the attribute is also a configurator. So that's why you will get them in the end. You will have attributes that are configurators, so you will be able to figure out what they are. Okay, cool, I think that answers most of my questions. I think, I don't need to test for Scala, right? I could just do a return. Yeah. And how do I call describe structure in my schema generation? Or do I need to pass it a hetero-describer configurator? So I think inside the schema generator you will do as you did before, but instead you will just reduce it to only call, like for all the root configurators, you will just call describe structure. And then describe structure will ensure that you will get it down into hetero-configurator because that will also have a describe structure function. So it will just go through all of them. Yeah, cool, got it, yeah, that sounds cool too. It will do exactly the same way as how we do YAML descriptions, but instead just for getting all the types and all the type data. Yeah, cool, cool, cool, thanks a lot, Joseph. Yeah, yeah, I think apart from that, yeah, one more, the next point that I wanted to discuss, I think I've included in here is the document updation. I think, yeah, you suggested Oleg, that we are going to have a look at this one. Yeah, this one. We discussed this, I think today, yeah, you said that we would have as a beta feature instead of production ready. Yeah, what you probably were doing is that you would like generate schema as beta and you would probably could like describe structure as beta and then, yeah. Yeah, so yeah, doing any kind of soft breaking changes until we can see the finalized and taking the fact that the current schema doesn't work at all anyway. All right, you know, big deal with it. Yeah, that's true, it doesn't work anyway, so. So do you want me to make any changes in the documentation and submit a PR or is that, is that like no priority? No, I think Oleg, you do it at the bottom. Cool, cool, I think, yeah. Any, the last point that I think needed to discuss is, yeah, it's local for end-of-facel. The last one. Yeah, once we have schema generation working somewhat. Yeah, cool. It was nice to, et cetera, it definitely makes sense to have a blog post for that. Yeah, done. Yeah, that's it for me, I think. Thanks a lot. Yeah, so just to add some bits, yeah, slightly ahead of presentation this Monday at the Jenkins Online Attack. So we've been doing some project demos for multiple projects. And yeah, if you want to watch this demo, you can just go to the link to YouTube channel. Yeah, we will actually have another demo next week, maybe on Thursday or Friday. So if you have some progress there to show. Yeah, definitely, definitely. I would love to be a part of that meeting. That would be cool. Because at least if we can get kind of the described structure working for every configurator, we might just have to tweak it a bit here and there. And we might have the schema ready. So, yeah, the testing framework would be an issue. Yeah, that would take maybe by the end of the phase one, exactly, but I think, yeah, maybe I could present something for the demo. Do include me in the list, actually. I would. Okay. Yeah, cool. Thanks a lot. Okay. Anything else to discuss today? No. I think we're good. I think, yeah, we're good. Okay. Then just keep working and sliding. Yeah, yeah. We have some time until the end of phase one. Yeah, yeah. And we will need to work with Joseph and Tim to understand how we do the evaluation. Because yeah, so I guess it would be a kind of a JSOC way. Yeah, yeah. But yeah, let's see. Yeah, another problem. Absolutely. Yeah. But now just focus on getting the cool request over the line and if you have opportunity to ship smaller scale changes like this, documentation update or whatever just to do that. Yeah, cool, cool. Thanks a lot. Maybe the simplest documentation could be by the way, schema doesn't work. And we're working on it. Cool. Okay. Yeah. Yeah, thanks everybody for participating. So this time we're doing this in Zoom so that I will need to post it manually. But I'll try to do it within a couple of hours. Sure thing. Yeah, cool. Sounds good. Thanks. Okay. So yeah, thanks everybody. I will stop the recording and the Jerrigan Jalsz-Polsky is still on it. Yeah, I was just telling him that he stops for the second half of the meeting. Okay. Okay, another problem. Okay. Yeah. Stop. I'm stopping.