 Okay, welcome. It's the 12th of August 2022. This is Google Summer of Code automated cash, cash maintenance for the GITS plugin. Ruchakesh, what questions do you have? I had questions regarding, you know, storing the files. Like we discussed it before, you know, we spoke about the metrics plugin. Okay, on how we would store files, you know, regarding the maintenance caches, and how would we then display it back? I went through the metrics plugin, but I didn't find any way of how they are storing it. So, could you help me with that? And I'm not sure I can give great help for it, but let's look together and see what we can find. I may have to take an action item to go do some research to see. So the kinds of data that you're trying to store, what you're trying to store is a record of when the maintenance was run, how long it was run, or tell me more about what information you're trying to store. What maintenance task was run, when was it run, how much time did it take, did it succeed or fail, and when was it last run, you know, if that was needed. Okay, so what we need is, and the idea is the user will be able to click something from the UI that will show them a table of results where, hey, the last maintenance tasks. This task was performed, it was working on this cache. It took this long, it ran at this time, that sort of thing. Yeah, in a table of 10 entries, and then you have this, you know, page, pagination. Oh, okay. One, two, three, four. Right, all right. So, so how to present results. And, okay, so let's, let's go looking for some examples and see if we can find some I'm going to share my screen. And let's just see what we can find as possible candidates. So what you should see is a pretty background. And now you'll see Jenkins coming. All right, so, so what we're thinking well so okay so, so let's consider what are some alternatives one might be if we wanted to use a presentation like what what if we wanted to use a presentation like the way we present warnings. Let's let's let's grab one here from that so in the in this thing probably here. I can even look, let's do one better let's look on sees a place where you can see it without without having to get access to my machine. Let's go to see I dot Jenkins that I owe. And let's look at the platform labeler plug in, or even better let's look at the get plug in. And on this one. What we see are some there are some historical records here, right, like test results code coverage results and tabular results of those things for instance, we might want to look at the coverage report. Yeah, this this may be more data more a different kind of data because what what I think we you need to see is something that's sort of tabular like this one is where it says, here's a table of types of warnings. Okay, they wouldn't be warnings in our case but the type the total the new and the distribution right. Yeah. Okay, and so and then then they may want to click through and see something that tells them something more. And so this, this does that feel like a something like the user interface you think you might might like to use. Yeah, yeah, something like that. If you if you go back to the, you know, plugins, if you go but what can you click on, you know, dashboard plugins there. Sure. So dash something like this. Yeah, some table like this, you know where I have columns. Okay. Okay, to add, you know, this table also is fine. If you could add columns to it. Okay, and there are there are certainly tables like this. For instance, I had submitted a pull request. Let's see if I can find the one to Jenkins core that I submitted not long ago. And it was merged actually so let's find, find that one. Let's find pull requests author me closed. Use new layout for remote class loader stats. Okay, so this was the, this was the presentation of something and so I was changing it from what it used to look like this. And now in the new layout, it looks like this. And so that that gives us some hint. Now, how was it doing it, the change was it changed. All it did was change a, in this case a groovy file but it could also be a dot jelly file to say, is this syntax. Yeah, so, so with this. Now this is this is talking about presentation, your, I think question is as much about data, data, data storage right how do we remember the data. And for that maybe we want to find something that gives us some historical information. Find something that gives us historical but we can use as a reference like. Oh, like build history. Right with that that's probably yes that's that's probably very similar to what what we need to do if we look at, if I look at CI on the get plugin. And if we look at build history this one right here. This really is showing the kind of information you're describing right. It's something historical, how long ago it was, did it pass or fail. And this one shows me more information about it. Okay, and that is stored. I, when I think I read about it, if you go into the build folder of this of this, you know, project of this project, you will find a log that okay regarding this in each build folder you have a log regarding that execution. So, I'm not sure if they're using persistent objects I shared a link once in the Gitter channel. What you're saying is inside the workspace. And if I go to a workspace, let's choose okay now I'm looking at my own file system now so if I go to the get client plugin folder. And I find the path to that workspace easily. I might be able to workspace. Workspaces here we go. Okay, so. Well, as it was not on this machine. Okay, so that doesn't help me as much. Let's go looking here and see if we can find so what I think what you're saying is, for instance, if I look in this on give me something Oh I know, I know. So, continuing this looking. Alright so and the name was it dash client. So what you were saying is here's the workspace. Now that's just a workspace so that doesn't include any historical information this is exactly one workspace so that's not the right place. Wouldn't it be stored and you know the Jenkins Jenkins root directory or something. It is it should be stored there in the I think in the jobs. Okay, and now if we look at Jenkins plugins folders jobs get client. I would expect it to be stored here so if we look in get client pipeline get hub names. Yeah builds here we go. No actually I think we have to go in this case because it's multi branch we have to go to branches first. And then we go to master. And then in this one. There's a config and next build number. Let's see so there's the canal. Okay, where is the history stored because there definitely is a history it must exist somewhere right. Yeah. Is this folder built once at least you could check in that you know file build file. No, oh this one definitely has been built is absolutely it has been built. So, no, not all okay so let's try. There we go. Okay, I think I'm getting closer now. Well, okay this gives us one example. It's not a perfect exam. No, no. This is a different thing so that's not it. Hmm. Okay, continuing. I may have to do some separate investigating hush cash. Jenkins plugins folder jobs. Get client and let's take that client pipeline. And then in builds. So, what have we got here. Next build number indexing config files. Well, I apologize. I'm not finding it. Okay. What is the next bill number here like to tell us the next bill number right. It does that's correct so it would be so if we look at this one. So that is definitely stored data, and it is just stored as a plain file. The configuration of the job is stored in that config dot XML file. But where is the history. Okay, let's, let's take a different approach. We're going to look for something that has several iterations. Maybe is a poor choice. How about, and even these I just recently flushed this system so that. Okay, let's just do this then. Here we go. Yes. Okay, so I've, I think I've got a place where we can see historical information. It's in a folder named builds, not jobs and not workspace. So if we go into the folder named builds for my pipeline, this is the job name client pipeline and there is one and two, right and each of those contains a change log, a J unit result, the log file from it and more. So they are storing data as an XML file like the build XML contains the data rate. I think so yeah so let's let's look at that and see. So here it's saying here are the parameters that were passed. So here are the parameters. Here's what caused it. Here is how long it spent in queue, because I was measuring the time in queue. And here is the pipeline library that was used. And here are the credentials that were used. And here is the build data so this gives the historical information of which Shaw ones were used in this build. And here's some build history and it looks like a lot of build history. Oh yes this is this is this is that get forensics plug in storing its data. So yes, so it looks like these things are storing things in this build XML file. Okay, now you don't have a build. But I think I would think still there's a way to store. Data let's see if we can find other examples of data that's stored for historical. Oh yes, like slow requests. So here they're just stored as text, but something must be writing those and it's not a job. Right. And then what others for instance, monitoring and monitoring I think you'd already done the looking to see. But monitoring has all sorts of data that it's storing build queue length. And these things are even. What is an RRD file I thought that was some form of real time or some form of time series database, a reduced resolution data set. No, no time series data. Okay, here it is. Good. So our RRD tool is handles time series data like network bandwidth, temperature CPU load. It's a circular circular buffer database. We don't I don't think we need a circular buffer, necessarily right, although, yeah, maybe I guess the question is do we need a circular buffer. Well, I know what exactly is a circular buffer. I have no idea. So a circuit here let's let's bring back that that page. So what a, when a time series database. What it's doing is it's tracking history over time, right so this graph shows from left to right time earlier to time later. In doing that, we need a way to discard the oldest things and retain the newest things. And the time series database is using a round robin archive. So it remembers a fixed number of things and then discards things off the end as new things arrive. Okay. When buffer size is only 20 I can store 20 and when I'm storing the 21st ideally the first one. Exactly. That's that's the kind of thing and, and I wonder maybe we want something like that here that really we want to use a round robin thing because we don't want to, we don't want to accumulate things forever. So maybe, maybe it is something where, but again that's leading us, that's be leading us back to the monitoring thing as a way to okay how would we, how would we store this store. So, so are you okay maybe we look for this. Yeah, surely. Do we do that internally or is this RDD, RDD thing managed by us or is there. I think it's managed by Jenkins I assume it's managed by the Jenkins monitoring plugin. But let's let's look together and see. So, here's this and does it mention our okay so it mentions them at least. Okay, it already uses our RDD graphs. Okay so the so the met this monitoring plugin is in fact using our RDD. So it may be that if we look at it as an example, and we go slash RDD in this repository let's see what it knows. Okay it's mentioned in the read me. Where else could it be so. Okay, well let's see what if somewhere in this thing, main Java in the plugins nodes. Okay so I would assume the listener somehow must have a way to monitoring filter. Yeah, I don't know. Okay, let's keep looking. So, node monitoring action, which says, do we have monitoring permission I don't see where it's doing the recording of the data though so maybe that may just be that I'm not looking real well. Monitoring scripts. I'd never use this. Interesting. Okay, so there's. There's something for me to learn. Okay, so, but I don't it's not immediately clear to me how we do. Okay, so how does this thing right those are RDD files and present them or Java melody is that some package being used or something. It is that's that's the thing that does this presentation right so so when I if if we. To show you how it looks on my system. If I run Jenkins up and running so if I run this one and I go manage Jenkins monitoring monitoring of Jenkins controller you see these pretty charts up at the top here. Yeah, yeah, yeah. Those are coming from Java melody. Okay. So this one says hey I've, I had this steadily growing memory usage. Yesterday because I was heavily abusing this system is it was reset we bring itself back to life after I threw everything away. And then it stabilized. Okay. Now the oddity here is. Yeah, and, and it's pretty responsive all the time 120 milliseconds that's pretty good. But so the, the idea then is, but this, this is just Java melody so this is not anything specific to to Jenkins. And then if I want to go back to Jenkins, I go. How I forget how I returned to Jenkins from this UI. I think it's. Yeah, okay that at least got me back. So Java melody storing it internally or we are using Jenkins to store that I using that I apologize but I don't know. And so that may be something I have to just a task you to put a debugger on and watch it to see. Hey, what what happens when I click here, because this thing definitely is showing useful information. Managing. That's not. And likewise this monitoring of Jenkins agents I believe shows the same thing. Now I'm not sure how it's dealing with the fact that I have 40 or more agents. Oh, it gives me one per agent. Like this. Okay. That's fascinating. What is. Okay, once again I've got lots and lots of open things. It's something be 44 million milliseconds long. Yeah, I clearly I need I have more to learn here who should cash and I apologize but I've obviously got more to learn. Because when I looked into the metrics plugin I went through the code I kind of kind of started understanding it, but then I didn't find any way of how it's how they are writing the, you know, files, and then again reading it back. So that was something which I couldn't understand. And I think, I think it may be that what I need to do is find someone who's expert in that and ask for their help, because I think it makes sense that that we should, we should use a similar technique, or at least consider a similar technique to an RRD file. As a way to store the information about the about the cash clean up runs right that that RRD file seems like a very sensible thing to say, let's not keep more than a certain number of these and let's rotate them automatically make sense that would be efficient as well. That's right without us having to then clean up logs or tell people to go delete truncate the logs themselves. Yeah. So how about if I'll plan to take the action to go ask for help. Sure, yeah. Another thing I had was, you know, the UI, okay. Is there any way of, you know, pagination, you know pagination already like you know this thing. I've seen few plugins having that one, two, three, four down. So is there any way reference you give me so that I can go through that and you know, you know, get started with that as well. Let's let's see the alternative for pagination so there are maybe maybe there let's look at see if there's something in the design library already that would give us a hint. So what what we're thinking about is, how do we present a table. Okay so here's a table. No, that really isn't pagination yet isn't. But in the warnings plugin I know it has to be doing pagination. So that that maybe, well, let's see, let's ask a pagination jelly. Pagination in Jenkins range notation for pagination. Jelly form controls. Okay I'm not seeing a mention here so let's go looking in. Apparently I don't have stapler cloned. So let me go get stapler. Let's get the source code and go see what the source code says. So the source code for the stapler library I think is in a thing called stapler. There's a mention of pagination there. What would, is there another word that might be used to describe pagination. No. Okay, it's best six symbols synonyms for pagination. How about page number paginate tried that one already. Page number okay maybe page number. Nope. Okay, so, so I don't well so maybe let's try the warnings plugins certainly has pagination in it. So maybe let's look for it. Yeah. Okay. Okay, so what he's describing here is this pagination happens actually client side, and is controlled by this show number of entries field. That's what I think trigger something on the server. Yeah, so I, well let's see so let's look into the jelly file I guess. Right, right, let's see. Okay, and see how. Okay, number of issues is surviving. All right. Yeah, let's let's go look at it and see so. So let's see what jelly files there are here. So finding an example, going back to warnings and G. So let's take this one, this one, let's look at maven warnings. So here are a total of six and when I click it there and if I want to show that. All right, so here it is, and it's in a page called maven dash warnings. Now that probably means the maven part is it didn't find any commitments. All right. Now. Okay, so maybe let's look at this. Oh, here we go. Jelly source. I thought we could look into the warnings plugin itself. Oh, right. Sorry, I'm sorry I'm sitting in the stapler plugin. Yeah, very good. Thank you. Let's do that. Let's go to warnings and G. Okay, so let's look in the warnings plugin. And here we have. Okay, so for instance. Let's look at this now. Okay, so how about, let's take. Let's take a little simpler approach. Where could we find the jelly file so here for instance is charts. Open tasks. And here is the open tasks. Does this show the one. Oh, here we go. Let's. If we if I would assume one somewhere in this we will find one for the maven warnings wouldn't you expect. Yeah. So how about if we look for the word maven in this repository. Or we could you know look at some kind of paragraph present in that UI, you know some text. Very good suggestion. Yeah, so let's let's go back to the UI. And here what we see is details or something like that. Yeah, see the word details for me is how about thinking that details is probably not. Well, what if we look at the page source and see if there's something hiding inside of it. Details. Data tables API. No, I'm not seeing what I was hoping for something obviously easily searchable. Well that didn't work for me who she cash, because this feels like a reasonable presentation. It certainly gives you okay we've got access to details easy expand and contract age of it and more more information available. If I have if I, you know, get to know those at least I can build a basic UI, and then you know, research on how do I store the data. Well, and I've, yeah, I wonder if here again, I feel like the data storage thing should be should be something just weak. Yeah. I apologize that I can't can't answer that one for you should guys really sorry. We've been figured it out together. Yeah, so. All right, so how do well. How would you like to approach it further. I'll certainly ask about how do the data storage. And I wonder maybe what we should do is should we ask for consultation with Uli Hoffner on on the how do we present the data to see if he could give us some. Any ideas of more effective ways to present the data based on what he's learned from warnings plug in. Yeah, I think that would be a good idea. Now the the other side would be okay we could we could just ask somebody who knows the monitoring plug in or one like it and maybe we drive it from the data storage side without worrying as much about the presentation. Now it's mostly I've got some action items I need to ask tomorrow before I go on vacation to see what I can find for you. Yeah. And next week when would you be back again. I'll be back. I return probably next Saturday. Okay. Yeah so we're we we leave on Monday, and we're there through at least Friday. And so Saturday I may be back for sure I'll be back home by Sunday the 21st. So, I wonder should we should we bring this. Maybe what we should just do is bring this to the developer mailing list and say hey let's let's ask the question there look marks. Mark doesn't know how to do this help us and and and let people say okay hey how do we do it because they may be able to help us better than than me trying to find a specific person by asking the question more generally. Oh, one request from myself, I think it would be better if you put the question right and I think I think I'm going to go ahead and do it now if you're okay with me asking it while we're here that way. I'm going to start the question and you and I can phrase it together that way we make sure it gets done. Yeah, I assume if I put it in some words which are, you know, someone doesn't understand it. Well, and just the matter of people people know me and they're willing to tell me mark you've asked a very dumb question. Just do this and be done with it right and and that's great. Whereas for you they may have to be much more gentle and say oh well who should cash have you tried this and tried that. So data storing the. Okay, so here I am trying to can you see my screen okay. Yeah, yeah, yeah. Okay so storing the maintenance data or get caches maintenance results. Or storing the results. Or get cash maintenance tasks. Right so Russia cash. She cash. Roush has been implementing the get cash maintenance. In Google summer of code. We have working a working demo in the in call requests to the get client plug in and get plug in. The cash would like to would like to present results to show the administrator. The results of the most recent maintenance tasks in order to do that. He needs to store. Those results. He needs to keep a fixed number of those. Some form of time series, question mark, and needs to present them to the user in a comfortable user interface. Those are all areas that are outside of my skills can others offer suggestions of existing implementations. That Russia cash could use could could consult to see effective and reasonable ways to store the store a. Yeah, a set of store the most recent result results of a task. Present the most recent results of the task of tasks to the administrator. What do you think does is that stated well enough. Yeah, that was great. Alright, so, so here we go. We're going to post off we go. We don't promise that we'll get an immediate response but if we don't get a response we can certainly ask more questions or you can reply to it. When people say, what did you mean by this what did you mean by that how can I find your code, etc. Yeah. All right. What else Russia cash. Those were the only things that you know, I have to, I wanted to discuss. There are, there are a few more tests which I have to write. You know, there are a few failing tests again. I'll fix them. That's not nothing from small feelings, small spot bug issues and all. So I'll fix those and a few parts of the. Yeah, there was one thing regarding the hash, hash set. We are adding caches into the hashtag but whenever we delete a, you know, repository, we didn't add any logic to delete, or you know, the cash from the hashtag. The caches would be deleted if you restart the Jenkins, you know, software completely because we are using a static hashtag. And the static initialization blog. So, during that time we would read all the caches and then assume if you delete anything there's nothing which is, there's no way of deleting it from the hashtag. And I'm not aware of anything that actively attempts to delete caches on the Jenkins controller so I think that's a safe assumption. The technique you're using seems safe to me. Now, now I may be wrong and someone with an enormous controller may say, Oh, you should have. But enormous controllers have all sorts of other things that are using much more memory than this little bit of cash management. Anything else. No, that's it. Yeah. All right. Thanks for Shikesh. I'll try to get the copy of the video uploaded within the next 24 to 48 hours. If not, there's a danger it may not be uploaded until until I get back. I think the last recording I'm not sure was that uploaded or not so. I'll check. I'll have to double check to be sure. Thanks. I'll check. Yeah. All right. Anything else. Okay, thanks very much. Have a great day and good luck writing tests, etc.