 Hello, welcome to the Jenkins configuration as code project meeting today is the June 17th, and we have team Adrienne Joseph me on the call. So we'll just discuss News related to configuration is called in Jenkins and then focus on ongoing development and other topics. I Also wanted to discuss a jkask Roadmap status today just to check whether we have for all option items on the roadmap and whether we need to have these statuses and I guess so that's it Okay, so regarding news one is a kind of ongoing thing we have did only Jenkins configuration which needs a lot more testing and feedback though generally it works It's here So team with any new developments regarding to don their configuration Nothing's the last one who last year it was done during the UX hackathon which we talked about last session Okay Yeah, so one thing Regarding that I finished the demo Jenkins configuration as code while it kind of works and now it has recent plugins and It supports a read-on remote which can be used for testing Of this configuration though, I still need to migrate the demo to jkask because now the demo is largely configured by Groovy scripts, but if anyone wants to test the only permissions it should be basically the same and Yep, I think that we can also Something else Are there any other demos which have been already to configuration as code? I mean to redone the Okay, so other news that the bill of materials is finally up to date So it means that Homa we use as Jenkins plug-in bomb now. There is a recent version of jkask embedded in version 10, right? So, yeah, along with all other plugins we find this in the light of that So thanks to everyone as a special team Getting quit over the line. So now everyone Their plugins with recent jkask without a lot of hustle Okay, any other news. They're missing Well, maybe it's a relatively minor Yes, they released plug-in installation manager Basically, it's a part of our jkask stories and then we released We released a few patches. So firstly and now there is a caching of downloaded plugins which can be used for Utilized for example to speed up docker bills or just common instances, then we have option to skip Downloads if something fails but implementation of retry and I also fixed the issue with transient dependency resolution because actually before this patch The tool was installing plugins, which we're going to decide on the instance And we hit a buck with your themes being installed through a transient dependency in lotion So, yeah, now it should be fixed though. It's just a partial fix the behavior of Dependency resolution is still not optional Reported bugs for example for optional dependencies Not resolved properly in terms of version control and also collisions between trans and dependencies in lower levels It won't be working reliably Still, it's a kind of improvement being compared to the previous state Yeah, as always contributions are welcome okay So I'm going development Joseph you want to to talk about secret resolver, right? yeah, um The only thing that concerns me in the the problem first again is the Introducing something that you might remove or might be confusing to users that exist It might be hidden somewhere. Yeah Yeah um Again, I put a lot of discussions with myself in to actually for anyone reviewing it to hopefully assist with their thoughts, but uh Yeah No The thing you could do is you could go down to like just have it one thing So there's only one key would which would I guess would be base 64 and then it will figure out If it's a file if it's a file it will do the right thing um Or if it's a a string it will try and do that But the problem is that you would get weird behaviors. Let's say the file doesn't exist and then it goes into the other logic and then But then you have to check if it's a path and it gets quite messy Yeah, maybe we should just uh A bit the prefix something like base 64 file and that's one of the other options This this is really a thing where you can combine them actually um Because you might have a file you want to read in and then again Maybe we should just support two and then don't support nested Uh, but then you want to support nested because you might want to have a variable That's not obvious if the file path if you don't want to show where it's it All my experience with a variable handling conjunctions, especially with an object plugins is that let's avoid nested variables But yeah, if there is no real use case for that I would rather prefer to keep it simple Uh, but if you see a use case for that, okay Let's try to implement that could be a use for it where you would have a secret that you want to just base 64 like a string So that would be a secret that you wouldn't have to or a variable you have to resolve So you'd need the nested thing Of course, we could try and prevent the nested thing like you could I could write some logic that would prevent the nested thing Of these two together like the base 64 and the file 64 Yeah, so yeah Maybe we should think about a bit more intellectual macro engine for resolution Because I hit similar issue. So there is a pull request which basically is very related to that about encrypted passwords The one which I still cannot finish though Yeah, so for the decrypting credentials and the So yeah, this is the current implementation. So basically I have strings like that Which is also a kind of weird macro So maybe we could standardize a macro handling It's really You could write a with the current one you can write another Replacer that researches for those things with the current thing that I have in my pull request You can create your own and you can create your own replacers And it could look for those things Um, it is a bit it's another common library, but it has a lot of features for like For like doing string replacements and having multiples and stuff like that So actually I needed to create a secondary replacer for having the luck when people didn't have a secret that was filled So But yeah, it wasn't that hard to to get around that Okay, so anyway, these things are slightly isolated They're slightly rid, yeah Yeah So our resolution logic becomes more and more complex That's why I think that at some point we should think about either a formal macro engine Use extension points, etc. So similarly to token macro plugin Oh Yeah, actually So you want extension points? Okay. Yeah, that that could make sense Well, it would help to break down logic a bit because Yeah, I already hit the number of issues in tests when I was implementing this pull request I'm not sure what I Even submitted the pull request, but yeah, it was mostly stuck because of edge cases. So talking about the plugin, it basically implements logic like that And the full is an extension point. So you can create your own ones We can Yeah, we can we are almost there to have extensions points, but then at some point Why not just use the token thing? The token plugin token micro plugin to to do that work for us If it's supposed to same with stuff Well In principle, yes in practice This plugin had no support even for pipeline for a while So yeah, you can see the extension point right now So I set this macro name, etc. And then for example, it will lead for Abstract project. So it's what we had for Evans And now we also have it will lead for pipeline. Okay. It doesn't really work for our use case Well, yeah, technically we could Introduce it on the configuration as code plugin past configuration as code context But yeah, the thing is that we were almost there actually on the other end and in my pull request I I think it would take like one or two lines to add the extension point to like Resolve those extensions and then because as I said, you can add any replacers And you can then just search for those replacers and then add them in Yeah Plus one for new extension point That would be nice. Yeah It also would enable other stuff That we have had people asking for like instead of the current way of like Creating secret resolvers that way we do now that it would be nicer with more syntax like if you if you could like Divert on that part instead Yeah, like if you have a vault sub path or something Exactly, I would much rather have vault sub path and then like Don't worry vault colon and then something string stuff That would be a lot nicer. Yeah, so something like that So it's the clarity. Yeah, so it's the clarity of like that. It's happening Yeah, kind of like kind of the answer will look up plugins for vault where you kind of embed all of that in line Ah Also for so it's easier to anyone who is a Configurator is an administrator to say, ah, this is current for vault I should probably go look at vault instead of like, where do I look if you're using Kubernetes and vault at the same time and some of them are coming from both Yeah, because yeah, right now it's so easy to fall into whatever false resolution Yes The current logic and really start enforcing something like that That's kind of nice that you can't have multiple strategies though. So like locally you can load it from a file system whereas Um in your actual production system, you can load it from body of a vault system Uh failure points and that is nice. Yeah, that is true so And I'm not in favor of removing old stuff that works because then we would have people that have workflows that gets destroyed Yeah, so Yeah, uh, basically generic, uh, it is a Lucian and it's Prot uh, definitely And we we've definitely been using that For our use case as well. Like yeah, we we use it as well We we have a in our case We have a dug a compose that just have the environment variables loaded in and then yeah off you go Yeah So I think that uh, this approach is don't conflict with each other but yeah Or maybe Introduce another syntax for one off because unlikely you have three sources Well, theoretically if you deploy the same instance, let's say in kubernetes and docker and locally for testing Maybe but yeah anyway I think that we need to review this full request Yeah, this one Well, I think that it's kind of reviewed already but yeah For me I think do you want to implement this extension point? So do you want to just keep it as is and we can build on the top of that? I I don't want to have a rather extend this pr because what it does right now is again it has yeah Yeah Yeah, I don't necessarily like the way it's doing it Okay Good And again, I want to try to have this little less Not not so much confusion like for the users that are using it or would be using it Okay So basically another iteration before we merge with it. Yeah, okay But yeah, thanks for working on that It's a useful feature Yep Okay, system read permission. So do we have anything? Stitched for reviews, etc No I think yeah, the only semi-related one is the log recorder in core for configurations code That's currently the only Thing that we use to take into your life for is not recorders well Makes total sense to support that Yeah, the only problem with it Follow-up recorder we system did permission. Are you able to configure your recorders or not because both approaches kind of make sense Uh with system read can you notice the system read can't configure recorders. It can just view previously configured ones Okay I was just wondering how much it helps because for example, if you want to diagnose something usually you're at a new log recorder and principally Still a key for the don't leave permission because uh, you don't add new functionality You just capture existing folks at the same time No idea. Yeah. I mean one of ours is that we use it. We lose our log recorder configuration as soon as it reboots And it gets reverted whenever patches can apply automatically in any reason Okay, well, anyway, we can figure it out later, but yeah, we only use a safe choice Maybe not great for user experience, but Yeah, at least it's definitely good from permission standpoint Okay, and did we get enough issues Uh for plugins reported Not doing it to the monitor is probably plenty more that could have support the same that issues raised and Yeah, I started running my demos to verify a dark theme and Tables to divs migration But yeah, in principle, I want to use the same demo to do scrap or system with permission and maybe Reuse other demos to have more plugins for example Yeah, we still have this demo for cloud based jinkies distribution and j cask where we can spot check all plugins we have embedded something like 200 ones But yeah, you still need some work But just processing that and the report cases where we hit them will be definitely helpful Yeah Okay So, yeah, what is taking it? Okay, anything else on system rate? Yeah, so for plugin installation management. Yeah, just a quick update. I will be submitting more patches because I really want to get rid of custom plugin installation manager implementation and jinkies file runner And apparently I still heat issues when trying to adopt it So I will be submitting a few patches But I can confirm that yaml configuration etc worked pretty well I also did some experiments with depender bot for yaml configurations using the tool created by olivier for infrastructure And actually it works So maybe I will create such update flow because I'm really keen to use pomeximal as a source for common users It works pretty well for customer packages, but customer packages takes ounce to build And with plugin installation manager now we support for caches. We can actually speed up the bills a lot So I would rather prefer to try implementing automated dependency management for plugin installation manager not for custom work packages So yet to be seen Yeah, hopefully we'll get it over the line Yeah, so depender bot like I think I think it will be just embedded in the plugin installation manager So there will be no new tool for that And yeah, let's see You're not sure does anybody else use plugin installation manager in production now We tried several times So yeah, we tried and didn't work I think you fixed one of the bugs though the unwanted transparent. Yeah, the unwanted plugins I'm sure it would work for us. I don't think we're doing anything that would cause it not to just to see whether it's faster and If it was in the document, we probably just use it. It's because it's not the default in the document Um, and the main reason that would get us to move there would be automatic updates If we could get if we could get that Well, uh, there is already a command which shows updates. Um, I spent some time To get it working and for example, uh, the recent patch for Demaging is configs code was basically produced by the tool So this one is a machine is a machine processable or is it human? Well, it's uh it dumps a text file and then simple regular expression and you get this format So the next step is to put this simple regular expression right inside of the tool So I created An option to generate a new updates file In available updates. So basically The plugin manager will be able to generate the list on its own without all this magic in the yaml or plugins 60 Uh, well technically it might be difficult if we merge Uh Plug-in management yaml and jcask yaml in the same file But yeah, as long as they separate it's not a problem. You can always mention separately like Yeah, one thing I noticed that it still has some well, it still has glitches with trans independence management. So It will get patches before it becomes fully reliable But yeah, at least command for check updates. Well, it's maybe 30 minutes to implement it Yeah, I think I think we just specify everything anyway, just so that we don't get out of date plugins Yeah, nice if you didn't have to but So let's see Anyway, I plan to spend some time because I needed for doing his father honor and for new edition of customer package I want to have Two less and limitations of plugin management. I'm keen. So This tool is a way to go for me Okay, any comments about this story Other things The only thing would be optional to tendencies those would Would be probably another blocker for adopting I would say Yeah, optional dependencies. Yeah, right here Yeah, I come from that. I want to resolve it So yeah, the problem that It needs a complete redesign of how dependency resolution works. So that's why I started from my poor man's patch just Change the top level dependencies Even there I hit some issues. So for example, if you take a look at this pull request You can see that for example If you use maven If you're familiar with maven enforcer, then whatever you do find on the top This is what you get And if you use enforcer if you have trans independence of high version, then you get never And I tried to implement such logic new plugin Sorry in this tool and actually I just Got an error in existing tests after that Because the logic of this tool is quite different. It still tries to pull the recent version from trans independences And I think we should stop doing that instead of that just fail the build And yeah, that's just the beginning because basically we need to redefine completely how this logic works Because There are so many pitfalls in the current resolution So if somebody still remembers graph theory from the university is probably I think to apply it Yeah, I think I will just take a quote from maven enforcer or whatever and embed it in the plugin because Yeah, I think that it's better to not implement Another resolution on our Or maybe junk is called codebase, let's see Okay, but yeah, thanks for the feedback So I definitely want to spend some time on that So any other ongoing development? Nope Okay Let's just take a look at the roadmap a bit. So I wanted to check with you whether we have all stories There So there is definitely no story for plugin installation manager Probably I will call it plugin installation manager to the zero taking the break and changes in how dependencies are resolved And yeah, right, whatever manifesto for this version. I put it on the roadmap But the rest so what do we have now a system read permission then? A pluggable configuration sources, but I guess that's all we have specifically for jcask Yeah Do we want to add something else? Okay, does anybody have something big in mind that you want to implement in the next six months? probably so Yeah, I suggest to spend some time at the next meeting But yeah, I really integrate plugin installation manager and customer packages stories in the roadmap And yeah, basically the same for any other plugin not For jcask So my personal plan is to have a roadmap officially approved at the next governance meeting So not today but in two weeks So if there are big stories you're working whatever Integrations with a vault or whatever we can put it there Because more items we have special specifically user facing items. It would be helpful for the project Anything else for today? Because thanks everyone for your time Thanks Thanks everyone. Thank you. Bye. Okay. Bye. See you in two weeks