 Welcome to Jenkins documentation office hours, it's the 25th of February 2022 in Asia where we're focusing this time zone for some of us it's not quite the 25th yet but it's the 25th of February 2022 in Asia. Thanks for being here topics I've got on the agenda today include news. Africa contribute on adding a troubleshooting section, the switch from system five in it to system D. Whoops, wait a sec how did my other things not get on. Oh yes that's right and that was one that this is one that needs some guidance because there's I wanted to have a discussion here in the group about about that specific one because it's it's a structural question. What other topics need to be on our list for today. Should we. Yeah, we've got. I had put on my list. The 2.2.332.1 change log and upgrade guide. So let's be sure we at least have a brief talk about that one. Any other topics. Yes. One thing I am curious about yesterday is does Jenkins has any tests that it forms like a normal project does. And I think I know the answer just wanted to ask it anyways. Yeah, like, there's a product and I see it currently in the product I'm working with. So there's like some manual progression tests that are written. So this Jenkins has it or not because Yeah, because when we release something new it. It introduces some bugs. So I was thinking like why can we decrease those bugs. Just want to discuss on that. The tests that run for CI. No, no, I think he was he was asking a different well, just to be sure so dear I think you're asking about human beings running tests, not about not about automation tests. There's a lot of tests on the UI, just to see make sure that all after the new icons are introduced, I'll do some tests and see whether they are not breaking anywhere or not. Before releasing it to Jenkins. I mean to the public. Yeah, good question. Okay. Just. Yes. And I think it's a good, good topic. So added any other topics that we need to include. Okay, then let's go ahead. So let's let's take these on news. Jenkins 2.336 released it had several fixes for image and icon issues which may in fact be what's inspiring. Dear I just question. And the Jenkins 2.332.1 release candidate is available and ready for testing. It's got an open question about which Linux installer should we use. The discussions are happening in the Jenkins developers mailing list. So by all means read the list we'll have a brief discussion about it later here because there's a documentation impact that I want your input on. Okay, change login upgrade guide is needed for 2.332.1 I'm not ready for that yet. I'm just going to take more work. The next opportunity we'll have for review it will be next Thursday I'll probably send you all of you a request to review the content before then, so that we can, I can get your feedback asynchronously. That's it for news. Now dirage onto your question, are there interactive tests that Jenkins performs. And I think you were saying specifically prior to a weekly release. Yeah, so I'll try to put paint the picture better this time. So there's a product, let's say, and it has some test cases redefined on Polarian. And those are like the manual tests as per the web UI that it has similar to Jenkins. It's like click on this, make sure this is seen. It's not possible. I mean, type this type that submit. There's like lots of test cases. Now coming to Jenkins. Does it have these test cases which are, I think I know the answer but does it have all these test cases which are performed before we release any new releases of Jenkins so that we can see some recurring issues from the Jenkins users and we tackle them before releasing it from our end. So, yeah, so my I got this question because I was seeing some I can issues. So I thought, hey, if we had these test cases in place we would have seen them from our end before releasing it. And does that explain it because I just woke up a few minutes ago. I think I think it does. It does explain it and it's a, it's a, this is a really fun topic. So, so you've, you've sort of touched on a hot button for me dirage but maybe not in the way you might expect. So this is great. So, no, we don't have those types of descriptions. And the reason we don't have those types of descriptions is most of the time those types of descriptions don't actually help us find bugs. In many cases, a scripted style of test, which is what you're describing here. And if you refer to refer to James Bach, Michael Bolton. And who are some others Cam Caner for the surprisingly high cost of using that style of testing compared to other styles of testing and the relatively low, low benefit that's received. We do have regular explorers who use the UI. We do have report issues. And we do have regular poll request reviewers who check that poll requests are okay. Now, you may say, okay, but why didn't we detect two dot three three fives regressions right why how did those get past our regular explorers. How do they get past the poll review process. Yes. So, so let's let's talk. Or is that sort of what you're asking or is there something different you wanted to guide on icons are breaking could we have prevented that from our end before. Right, exactly. Yeah, so so and so let's, if we. The fundamental question, could we have, what should we do differently what should or could we do differently to detect issues sooner. Right. I think is what you're asking so two dot three three five had several bugs in visual elements. And it's that's a sort of a first for us it's been a long time since we had that many bugs in visual elements and if we look at the list of bugs and the changelog. You'll see that hey, this set here gives us a hint of the kinds of bugs we had like those. Right, so if we're lucky that will actually turn those in the hypermix still he does. Oh, that's good. So those, those kind of reports. What could we do to, to avoid them. And now there are several of them where we'd say, ah, well let's see what could we do. And, and should we, let's see, oh sorry one of those is bogus will delete that one. It's nonsense. And so if we look at those, we can see all right. Let's look at this one. The organization folder scan when it is not run was missing an icon, but only when it was not run. If it had been run the icon was correct. So in order to detect this thing, you had to define an organization folder, look at the organization folder, not have run the organization folder and detect that so you had to define the organization folder and have it initially disabled. It's a relatively obscure case that that would would take some relatively sophisticated exploring and yet human beings detected it quite rapidly, because there are so many of us. And, and so we said, Oh, whoops, that's just wrong. So you're saying like it's, it was a very tricky edge case. It does and there are several like this in that are that are really interesting edge cases but but when we have 300,000 installations, edge cases are everywhere. Right. So the reality is calling it an edge case is not very satisfying because everyone sees edge cases because there are so many installations. So calling it an edge case doesn't excuse it, it just says, Ah, okay, that's a that's a tough one to decide how we would attack that generally. Right. Now, now maybe let's let's look at several more just to get a sampling so visually right there's a visual problem there. Let's look at another one. Let's get another visual thing and then let's talk maybe. Okay, so GitHub organization folders were not rendered correctly. Now this one was very visible on if you had a, let's see what was the specific condition it was that in order to see it you had to. I forget the details but there was again, some rather obscure thing however it showed up for get lab giddy and get and get hub. And then we got an additional report after 336 was fixed that said oh and there's still a problem in one more case with bit bucket. The bit bucket one is really novel. So, so the bit bucket one required that you have a very specific reverse proxy configuration order to see it. So, so most interesting. Now back to the question, what could we have done to detect those. I wonder, could we so. Could we open every web page. For, for broken images with that have helped us detect these problems, much more rapidly. And by that I mean having a machine do it not having a person. As you said in some cases we need to have some configuration in place just to see whether it's breaking or not. So, we need to also tell machine that hey do this configuration then go on and check for broken images. And that will go ahead. That would be a little difficult as well right to well that one so so in many cases. We need specific plugins in order to see the issue right. Yes, exactly. I think that's what you're saying. And so, for that one. So we here we would need to find a broken a broken image detector. We would need to have interesting configurations that are easy to start and stop. And as it turns out, we have some interesting configurations that are easy to start and stop. Gavin Morgan has his interesting configurations. Mark weight has his interesting configuration. And I think you're as you've actually had access to my interesting configuration. It's this thing right here. Right with several thousand jobs of various forms that are testing various things. Yes, yes, yes, right. Something. Yes, so this is this is my Jenkins server but yes there's also, it's also connected to a giddy server and to get hub and a bit bucket and all sorts of others. Yeah. Make sense. So if we could find a program that would walk through a a an authenticated website and report to us all the broken images. That might have been a really dramatic improvement for this. I don't yet know of such a program but I suspect they exist. Makes sense. Would there be a way to characterize the plugins that I mean what the background was of this because I'm thinking if we have to check against every possible combination of plugins that you could install that that's a very large number. And it certainly is so, so I'll go ahead. Yeah, I was just thinking like I can imagine that when people were reviewing this they just took common things and they were like oh it works for what they're used to seeing. And so it's probably another reason why because it's like even if you were visually checking as a human. Unless you knew to do certain types of cases you were probably, you just look at what you have or like look at things that you know about and it rendered. People immediately disable the folder after they create it so it's like I can understand like having that, you know that situation or. Yeah, so just. Right right and but about very good point. So my thought was, if we if we get something like this thing a check for broken images, anybody could run that against their own Jenkins controller, and maybe able to help us do these kinds of combinations. And it would just use whatever configuration they have and okay it's not every possible combination, but it's probably a bunch of very interesting combinations that we get. Sure, I think I still see like the people that probably end up using this the most or like you and I think it'd be I'm not sure how often that would be run by people as they open a PR. Right right and agreed wholeheartedly. I don't I don't know that this would be a viable. I mean, I don't know that it, I guess it could be viable but I have a hard time imagining that we would spend the money to host these that kind of test on every pull request because many pull requests don't change you I many pull pull requests don't make dramatic changes and this would be probably very expensive. It takes takes quite a while to load every web page. Yes. So did did that address your question any any further questions. No, no, nothing as of now I got the idea and the full picture and it makes sense to me. But one last question just remember it now. You were saying I pressed the hot button so what was that about. Oh, sorry that's that's there is a there is a multi port the association for software testing software testing as T has a multiple course series that includes foundations of let's see foundations. I think it's the first course bug advocacy is the second course. So foundations of software testing bug advocacy. Let's see what else test design I believe is their third third course. And each of these is a six week intensive course just to give you a hint I flunked bug advocacy my first attempt. And that was after 15 or 20 years in the business I still failed the course the first try. So, so they're very serious about this particular these courses and, and they are really cool there's a brilliant book, the bug advocacy workbook is five millimeters thick. Right 10 millimeters thick. So it's a thick book filled with instructions on how to do a good job of submitting good bug reports. And that's just about how to write a good bug report. And when you report a bug and why and why you don't report a bug. So, so it's fascinating and this the whole. The whole thing is, is just brilliant and of course, Dr. Caner and James Bach Michael Bolton and several others were involved in creating it. It's it's marvelous material. So yes I'm deeply attached to it. Awesome. And a little embarrassed at having failed the first time. Right. I mean, really truly. Good good stuff. Okay, for sure. Yes. Anything else on that topic. I could, unfortunately I could bore you for hours on that topic so let's be careful we don't spend too much time on it. In fact, you said something about we wouldn't run this against every PR but we always talk about some tests that you don't run against but maybe once a week. Everything that's in master at least, at least we might get a heads up on it before me say we run at the night before we release but then that opens up a rather large possibility of getting like this would have been a mess what we held the release then and how to explain it etc. And I don't know that we would have but we certainly would have could have informed people that hey, this this detected it so that's, I think it's worth considering could consider more layers of tests. So, do some things only weekly. Right, because because it's there there are there are values to some tests, some tests are more valuable than others. And, and we want to do the more valuable things more frequently the less valuable things are the things that are less likely to detect a problem. We may do them less frequently, or the things that are a greater effort and a greater cost. Right, right. Exactly. Yeah. Because well in this case I'm wondering if we had say been running this a few weeks ahead on master, would we have started to see some of these and would that have alerted us to be watching for them. Not really because because the changes these bugs did not exist in the previous weekly. So they arrived within a period of a seven day of a seven days. Okay. So I'm just also wondering, like, we can write these tests and stuff but will we even have written the test to catch these bugs. Because if they're on like such. I mean, we, yes, we have a lot of users and stuff and it has like, you know, the edge cases, you know, going to have hit more people but on a day it's still an edge case. And would we have written the test to even do this in the first place. And then now we'll, and so we might not even caught it anyway. And there's right that's that's a very real risk that escapes are going to happen. Right. Some bugs are going to get past us. And my thought with this technique was that it at least exercises the pages available from an arbitrary Jenkins server from a controller. And if that controller has enough interesting configuration, it, it will probably tell us something useful. If there's not interesting configuration then when we find a new bug we add to its configuration, so that next time that one won't would be detected. The challenge I'm having right now is I'm not sure I know a program that can do this because web page image loading in web browsers now is deeply attached to CSS apparently. I don't know if you can get it without using the real web browser so that's that's the part I've, I haven't explored enough yet to see if this is really feasible as an idea. Did that address your question Kristen, did you. I guess like I'm not 1000% convinced it's worth the resources. But I'm like, but yeah, if it's caught catching enough people's stuff because I mean how often are we making these changes. Like I think a lot of this is because we're doing a lot of visual updates currently right. So it's like if we're doing it now like these things would have, you know, hindsight 2020 it would have been a lot better to have it in place before we did a lot of these updates. And I'm quite frankly not sure when we're going to do another overhaul for something like this where we would run into these problems. Maybe, maybe if we're planning on doing any type of visual change like coming up soon, like and that maybe is something that gets developed ahead of that. So maybe we just this is a learning opportunity like a learning that we take out of this and then do it the next time we decide to do UI updates but I'm just like me I just keep looking at this like if it's going to be relatively stable. We're putting a lot of effort into running something that's just going to keep returning. That's a good point and so every for everything and it's just also a lot of effort on everyone's part for something that might not actually be worth it. Right, so it, I wonder to address yours, it may be that it'd be more valuable just to admit explorers guided by what they know has been changed in pull requests, maybe much faster than any automation we could do. Okay they're not as repeatable but we don't actually care about repeatability we care about detecting problems. Right. Right. Yeah, so I just like I'm like oh man we're taking a lot of time and it's like that time be used for something else or if like we do know that there's a massive you like another UI effort or something coming up with icons or anything else. Then maybe I would see a little bit more but to me it's like now we're just creating a whole bunch of tests that will always be right right and that's and it's just a lot of resources. A lot of resources to spin everything up a lot of a lot of time, people's time to write all this stuff, and then to maintain it. So, yeah, and that's sort of why I was thinking this thing would just look at everything. It would admit to not being not being specific to anything if any pages detected with broken image that's a, that's questionable. And then humans reviewing pull requests, again is is nice and focused but you're right I, I can't see investing writing specific test cases, even for these individual bugs. I really, I look at those bugs and I think you know what, I don't get enough benefit out of trying to check for the existence of that icon. Human beings were founded very very rapidly very rapidly. Yes, yeah. So it's just one, one possibility that's the other thing is, I mean they're, we still know there is some validation that's best done by a human that it's just hard to read. Right, yeah, like I remember the Jenkins to upgrade like that was a lot of manual testing and that because it was a lot of what are people doing. Yeah, it's like that's a lot can be a lot faster than any type of like test. But yeah, it's a requirement that was just a massive effort on the whole but it's just kind of like a, maybe that is something if we know that we're having UX changes it might be. I don't know if the UX SIG is still active. It is very active. Because it's like maybe that's something that their SIG should look at too. And right now we're very much in UI revisions. There are rapid changes happening. But, but again, I'm not sure I want to risk slowing down those rapid improvements. They're they're really breathtaking they're doing such a good job that I think this kind of breakage for me is an acceptable result from from the fast movement that we're making. Yes, it's it's awkward saying wow that was a bunch of stuff that was broken but one week later it's fixed. Right. And that's a cool story about these these bugs is almost all of these are fixed already in 336. So they didn't spend a long time broken, and it was total time broken we detected most of them two days after the within two days after the release, and they were fixed two days later. And they were a little bit of ugliness but they weren't anything serious like they destroyed. You couldn't get into something anymore. Frustrating to users, but not damaging to functionality. The ability to do work. Sorry, and the only users that would have seen these are users who are on edge. Right. Right. It's not like I got into an LTS where it was like, that's correct. In fact, they these were specifically introduced at this point in the cycle because they are probably eight weeks or more away from an LTS. They won't be eligible for LTS for at least to least June. Right. So the users who are seeing this might have a little bit more for understanding and flexibility and maybe a little bit more forgiveness because they know that they're using something that is. So actually we could say that the system is working because weeklies are known to be vulnerable to certain things. Yeah, there's certainly Daniel Beck would remind me that weekly is still expected to be production ready. Right. Right. And he's right. We don't, we don't get it. We don't get a pass because we because it's weekly right but exactly. Yeah, especially in terms of security and stuff but yeah it's like it's just kind of like maybe. Maybe they're, they're saying should look into some of this stuff too. Like if they want to do the testing but I'm not sure it's worth like standing up as a. Yeah, well everyone's like time resources but now I've said it might set my piece. All right. What actually strikes me is how many pages I mean if you told me to go and look at every web page. I'm not sure I would do it. No, no, no, I'm not talking about I'm saying that I wouldn't have a linear path through that I could be sure I'd hit everyone. Right, right exactly. And so one thing I mean one thing we could have is a list of these somewhere so that you could have 10 people. And they each did 10% of the or something like you know. My problem with that kind of thing is it's that's very human intensive and this open every web page is is very much robot right it's a machine can do should be able to conceptually do this. And therefore, it's just it's expensive to run but it's still cheaper than putting 10 human beings to do it. We're saying that we do have people who are looking at this so. Well we have people who explore. Yeah. All right, yes, ready to go on to the next topic. Okay, so I think I want to shift gears just a little if you're okay with it I'd like to take on this topic next because I need some guidance. Are you okay if I if I metal with the order of the agenda to get your guidance. Okay, so, so here's what we've got the Linux installers in 2.335 switch from using system five in it using system D as their way to manage the service. And I think it's brilliant because whereas previously, how do you configure red hat was different than how do you configure Debian. And that's really messy and documentation, it's also more prone to bugs. There were eight or 10 bug fixes that came as a result of this switch so nice improvement in general, however, it's a new thing. System D is not the same thing we used to use to manage services, stopping them starting them configuring them modifying their configuration reading their logs etc. This is new and during this transition I learned a bunch of things that I wasn't aware of and I think I'm a pretty experienced Unix administrator. The experience was good now the question to to all of us is where would a page on managing system D services belong in the Jenkins documentation. So now I'm opening up the documentation site, Meg I think you're usually are our first and best thinker about these kind of things suggestions recommendations. Okay, let's look at well it seems like that would be system administration. Okay so system administration. Let's see if most of this. Well, a lot are there going to be a lot of these changes that they have to make right away. No, these are this is, this is material for so. Oh, a good point here we got viewing logs. So this one already will need a change, and it's actually got a change now for system. So, so I think that lobbies in favor of your argument it should be in system administration. Good. Okay. Others. Meg I interrupted. No, no, I'm still reading in thinking school and go on. Even for me first guess was system administration, and that is coming from the fact that I have no much background. So that is where I would go as well. Okay, good. All right. I think we had that discussion to what's the difference between the managing and system administration. The managing was supposed to be more routine stuff and then administration was more sophisticated. What about the Kubernetes chapter is that going to take a hit from this. Does this affect Kubernetes also. It does not. This one this one is unrelated to Kubernetes because, because this system D thing is not used for Docker containers. It's only used when you're actually on an operating system that is doing more than just managing a single process at in a container. So, so this is I installed on red hat, or I installed on Ubuntu or I installed on Debian or I installed on Susie. And in those so. So almost at that point do we need to have some references inside of installing Jenkins to. We do and, and there will be a section there is already a section here in installing that talks about how you use system control to do some of these things. So those things are described here. How do you start it. How do you stop it. What I was thinking though is that there's this is linear flow the starting and stopping right and not so much reference material whereas I was assuming this this other page is more reference material. Okay, okay, here's how you do this thing and here's how you do this thing, not linear flow. Right and we but and we want some place where we actually explain what this is system be, you know have sort of that background, and then other places you can say you have to do this because of that, go over there and read it. It could be like another high level because you see our system administration we have the back of restoring wandering that administering Jenkins on Kubernetes with this be a administering Jenkins on Linux, and then a page there. Oh, okay, go for it. I was thinking it would be something more like system service managing system services. Okay. Yes, good, good question. I think back to your title. So as could you say that again Kristen, you know I was just copying what this is administering administering Jenkins on Kubernetes. And it's then it turns into is like is this kind of administering. Yeah, right there is like is this administrating Jenkins on Linux, or is this, or is this kind of more like would it fall under a page like this is like a section on system D. So, so now I'm going to I'm going to attempt to describe why I think it shouldn't and let's test that. Okay, that's that's that's excellent because thank you for asking the question because I had not even considered the question. So, a Linux administrator could choose to use the Jenkins war file and manage it themselves, either with a Docker container or with their own their own setup. I'm not using system D at all. So my thought was, this thing needs to be very specific to system D, and it's relatively involved because well because here all the different topics that I identified we would need to describe in it. Just at first attempt. And this is this is by someone who's relatively inexperienced with with this thing. So I found 12345678 eight topics. For me, I thought, you know what it's, it's not Linux, it is actually Linux specific, but it's not the only way to do things on Linux, there are, there are other ways so for instance, there are some Linux operating system still that use system five in it, and don't they've intentionally not chosen system D. And so for them, this section won't matter they won't read it. So did that does this does my does my argument persuade you or not so much. Okay, not so much not so much as a funny I just think like it would be a system, or it would be a section within a page of Linux. So if you had the option, you would be able to see it like, I'm almost like, I'm actually kind of interested out like, Oh, we don't have one for windows. I'm like, Oh, I wonder if that's like, because I know the windows is a whole process thing as well. But, or at least like maybe since there's not a lot of there's no, it's just a little under construction triangle for the Kubernetes one I didn't know if like this is something that is like a, okay, I'm on Linux, I click here and then here's the different ways I can use it on Linux. Does that make sense. Or. Yeah, well, you can always say no. It's a good, it's a good question. I don't want to get into the into the discussion of all the variants of Linux that they could do because this one already is has enough interesting things in it that that need to be described I'd like to take one path and recommend it and tell those people who are doing exotic stuff like, like this, who are staying on system five they're relatively small group and tell them, sorry, you're going to figure that out on your own. Sure. Okay, but if we were going to, if we were going to have documentation about running Jenkins on Linux, there would be a lot of subjects beyond system D in it right. So there'd be things like, how do you configure it to do mail or how do you I mean, you can imagine all sorts of things that an operating system. There's been a long one that you may need to modify some of you know the number of slots number files you can have open or something like right right yeah. And, but I guess the thing is like is is system D as a topic enough alone to be a high like a standalone page or is it like information that would be on another page. Yeah, my argument was like I think I would see it personally as something that's on it, like on a page. But I don't, I don't care. I'm not strong enough feelings like I'm like that's I'm not going to. This is not the hill I want to die. It's like, no, we will have it. It will not be on its own page like that type of deal. So if we, if other people think of you get on a page then I'm fine. So let me see if I can, if I can persuade you why I think it's, it's large enough there's three examples here. So here's one from Digital Ocean. So this is just the size of this scroll bar for the description of system control. So, so this is system D on on Digital Ocean, and they're not even an operating system provider. Now if we look at red hats, description page, there's is even bigger on how you do system D. So my, my thinking was, wow, this is, and this thing, its goal really is to be the one and complete total management solution for processes and services on Linux systems. And that makes it very, very large. But our part of it is, is not as large as that but it's, it's I think still got lots of interesting topics. Good. All right, thank you. Thanks for the guidance. Okay. By the way, on a related subject being obnoxious, under managing Jenkins, we have a section called Jenkins features controlled with system properties. Under this definition of system administration, I think that's something that might be long in system administration. Okay, so I mean I isn't again that's another little bit that we get into all the. Yes, right, stuff you put on your job with command line. Right. And that, and that's, I think that's consistent with our picture that we think managing Jenkins is really a description of things that are visible and direct accessible under the manage Jenkins page. So, so configuring the system configuration is code tools, whereas this one is relatively niche, right at this is, and, and probably and so what we need is a redirect that points from this location into system administration and we move the page. Yeah. Yeah. Good. Okay. Thanks. Everybody okay with. So Kristen, I'm, I'm noted. I'm okay with putting it as a top level page. Yeah, it's just kind of. Okay, great. All right, so we've got about 15 minutes left here are the three topics I had any of these that are particularly hot for others one versus another. Not for me they're all interesting. Do you have any questions on Google summer of code, or it's documentation related topics that we need to do address. You mean this can you show an automation idea. Any, any topics related any topics you want to discuss. Yes, I do have one. And so it's related to pipeline steps documentation generator. Okay. So I spent some like a good amount of time to understand the current code base in order to see how things are working before thinking about proposing some UI related changes. So, so to do that in greater detail, I wanted to run it via debugger because I just got introduced to debugger and I think it's so cool. I wanted to run it via that and see what are the values that are there in the this this variable that variable, what I what is getting fetched. How is it getting used how is it getting stored. So that's the aim from my side. So, after that I'll be able to think about hey I can use this data to propose this UI change. The question is, how can can we run it. Can we run this project on debugger mode on any ID, let's say eclipse, because I tried that I see saw, saw an error. So, I thought, let's first discuss maybe. Yeah, and I think, I think that I was using it now I'm, I, I've used net beans so mark uses, and I think that was what I was using the last time I was working on that code base. Kristen what, what IDED you use typically, IntelliJ. No, VSC Visual Studio code. Yeah, Kristen uses Visual Studio code. So, you might try the one of those two. Yeah, I'm going to say like I've never tried running it with a debugger. Oh, okay, Andy console, or logging or just kind of to see what and I think that I did run it in the debugger but because there was there were some places where I was confused and usually when I'm confused the first thing I do is bring up the debugger. So, it's been a little bit since I've tried running it. I have to look at it again, look at it again and try running it again but yeah like a a lot of it to like comes from a lot of there's some areas where it's like there's the trust that the plugin manager stuff ends up kind of worked or like taking lifted directly from the plugin manager code. I think that that stuff is like all right well it works a little Jenkins stuff. Some of the information in there was a little, a little hairy, like it was a lot of digging through Jenkins core to figure out what's going on. So, so if you'd like, do you Raj we could we could consider next week we might do some exploring to see hey could we get it ready and show you a tour in a debugger. For me it's, I probably spend 30 minutes and then if I didn't have it wasn't successful in 30 minutes I'd give up. Just tell you, I couldn't invest more than that. Who actually wrote that. Exactly. That software. So, I need to look at it again. You wrote the step generator. She did. Yes. Well, no, I wrote this the one that does for the late I didn't write like the actual like the annotations and all this stuff that stuff I just wrote the thing that pulls all the documentation with the website. Right. So yeah, yeah. So yeah, I got to look at that again but it's been, it's been a couple, it's been several years. Did you use visual studio code to develop that. Yeah. How did you debug it then. A lot of console log. Yeah, so I run it like basically I stood it up a whole bunch of times and then just kind of ran like you're trusting that yeah I did a lot of console blog debugging. Because I was trying to make output into a file or into like ASCII doc anyway so a lot of it was just kind of like, what am I printing an ASCII like is this actually showing up an ASCII dog so yeah like a lot of that type of situation but yeah I didn't use the debugger. I guess maybe not the best world's best developer but it worked for me. You know Mark didn't ask me what I do I use because he knows I use vi so. So we're good vi is my favorite gluey it's my favorite ID. I like vi too. Very simple things but So Chris can you use the system.audit.printl and statements right. There's like a logger whatever the logger statements so like yeah you can use line or you can use like if there's, I think there's a logger in there, you can use the logger. Okay, right. Okay, but Just to be able to figure out what's going on. Yeah and dirage as far as I understand it should be usable in a debugger. It is not the most commonly used debugger. I think IntelliJ is the most common right now in the Jenkins community visual studio code is a strong player. Yeah, and then there are a few few holdouts that still use Apache net beans. They're, they're somewhat high visibility holdouts, but there are holdouts. You still love net beans. Much more than eclipse. Yeah, I like how I liked how lightweight visual studio code was, especially many, many years ago it was incredibly lightweight. Yeah, absolutely. Did did that address your question, at least as well as we can in the time we've got you Raj. Yes, because I see that VS code is an option so I'll, I have, I do work with it so I'll try to run it on VS code and see how it goes and then I'll ask my question directly on Gator channel on GSOC channel, right. Great. Excellent. Okay. Sure, thanks. All right. Okay, I think last topic then is, well let's let's in the few minutes we've got left. How about let's talk about contribute on one of the questions I have is I'm looking for mentors during the Africa daylight hours to help with some of these project ideas. If you're available, send me email, or let me know if you're willing to assist there. I put several of your names here in danger mode saying okay. I know Meg you I think Kristen you as well had mentored last year. And so if you're willing again, please let me know, and we'll, we'll think how we approach it. What what our African daylight hours. Oh, let's see. So that means it's when it's 11 o'clock my time it's just ending their working day. 11 o'clock at night here is no am 11 o'clock. Think think you're you're up working times. So when you think about your overlap with Damien as a good example of African working Africa working with my entire department now. And then mostly going to want to do stuff after working hours or is it going to be during work. Don't know that's that's part of the complexity right because if they're students, they may be available right within their, their working day. Right, and then we've got Angelique Jard who's agreed to be a mentor, and she's right there in the in European time zone. So, yeah. So we've got we've got some good candidates there but just looking for more. Okay. Yeah, I'm, I would be willing to do it. As long as they're doing projects where they need a writer and I'm not doing a meeting where every issue is we wish mark was here. Well, well and if, if the issues are coding issues then then we need to be sure we've got people there to support them in coding questions. Absolutely. So thinking of which do we have. Once again, we don't have any projects in our proposal that are really just writing projects do we. We do not. Well, well, I don't know you might. Some of the some of the inclusive naming things are awfully close to writing projects. Right. So what I was thinking about is whether this troubleshooting section for the user handle handbook is a potential project that could actually be done by a team of P of participants. Because I mean it would be a lot of research and trying and, you know, talking to people, you know, it would be a chance to learn a lot of good documentation skills. And there's so much there. And it, I did get the impression from our participants last year that a lot of them might have been more comfortable working in a team. And it's always the same goal. It's, it's something else. We could try it and if it worked we could eventually propose it to G sock, but everything we do now is individual things and that might be a, you know, and then of course the problem with them is what is one of one of we have three people on it one of them bales out to the other two get hurt but but they get anything they get done is positive right we don't exactly yeah that's that's that's akin to this same story test the tutorials right if we if we did troubleshooting as a team. Yeah, interesting idea. Good suggestion thanks. I'm sitting there who is going to write this and when and it's, I'm thinking of something like a year from next show Tuesday you might have resources for it. Right, exactly. And because, because last year, the participants I don't think any of them were particularly interested in growing up to be technical writers but sign up indicated that there were some candidates out there that were very interested in this so that I haven't had the sense of that one way or the other, but she's when she when she said in a recent meeting that project management may also be I think we can consider broader broader cases than just software development. Right. So we could ask I mean if nobody applies for it then it's a moot point but right, but if we put it out there as a possibility. And nobody applies for it we have we're not any farther behind than we are now with it. Right. Exactly. Yep. Good. Okay. Okay I'm done now back to she code in general. Actually, that's covered all the topics that I had and I confess it's been a very long day for me so I'm at the edge of saying shall we call this done for today. And that gives you a four minute break to your meeting. Oh no no J suck meeting tonight. Today was was Europe time for G suck so I'm actually done I'm going to go to bed and take a take a sleep. Can you stop the recording and I have one question for you after I can stop the recording. I'm just want to know how clean is.