 There we go, that way. I'll approve. So recording is on. Great. Great, welcome to today's GSOC meeting. We have a number of PRs to discuss and questions to answer, including by Sagar. Thank you all for being here. Should we address Sagar's question first? Actually, that's maybe the best way to go about it. And I will, I'll screen share. And we can look, is that all right if I screen share your question, Sagar? It's in the GSOC. That's good, yeah. Perfect. Good, and can you all see the GSOC? Yes. Great, excellent. So Sagar has a question on, well, I'll let you phrase it, actually. Would you like to say, Sagar, to your question? Sure. So I mean, I was that meeting again to understand, but Oleg said to me on 13 gen, where I asked the questions about rest AP specification generator. And the conclusion we had in that, he said to me, you have to try to extract the endpoints for the, just start with some small plug-in, which is, he said, code coverage plug-in. And when I tried on how can I extract that endpoint, even I don't know how to use that plug-in, first of all. I just first, I mean, to just hand my get dirty. I just, I came to know that this was a project from 2018 GSOC again, made by one student. I mean, and Xin Yang, something like that. And so I just posted my questions in that Gitter channel. And I just am currently still trying to figure out how it is working. And I mean, I mean, there, one person said to me, I'm just use this my GitHub link and try to run Jenkins using Docker. And so that, so first, I just learned Docker, which is actually quite amazing. And now I'm running Docker using Jenkins. I mean, yeah, running Jenkins containerized by Docker. So, but still when I looked into the source code of that plug-in, it's, I mean, I mean, I don't have any idea still what to do further on, even though I just, even though it's all like just said to me, OK, Sagar, you have to extract the endpoint, but I don't know actually, I mean, where to research, actually, I mean, it's maybe quite unclear to me. It seems, again, confusing, I mean. Yeah, so I think it seems like there would be, at least for me, if I were approaching, there'd be a challenge first. How do I identify a REST API endpoint inside the Java source code of a particular plug-in or of Jenkins? So that's one, because I think that if it's going to be an automatic generator, it's got to walk through all of the 1,700 plugins that are in Jenkins and read their source code and identify, oh, this is a REST API endpoint. And this is a REST API endpoint. And then it's got to identify. And these are the arguments to that endpoint. And in a perfect world, here's the Java doc for that REST API endpoint implementation. Now, I said perfect world, because many of them know Java doc, varied levels of quality of Java doc. So that was what I was assuming. Is that likewise what you were assuming, Sagar? Yeah, in the last meeting on Rettingen, you also said, I mean, we find out, I mean, some plugins even don't have exported bin. And currently, we are relying on exported bin to get the rest endpoints. And even you said, even my friend has a one plug-in. I mean, Mark, and even that plug-in don't even have exported bin. Right, right. So the challenge, and I think it's an interesting challenge. The challenge is, how do I identify a REST API endpoint? And I think that so there's certainly one level of REST API endpoint that we can identify from the Jenkins user interface. So would it be OK if I shared my screen and illustrate that one that you might use as a first exploration? OK, this is not even using a plug-in. This is just the Jenkins user interface. But for me, that helps me already see, oh, I might look here and here to see how is that API expressed in the Java source code that I can see from the Jenkins UI. So I'm going to share my screen and let's take a look at that. Oleg would certainly do a better job of this, but we're going to use my inadequate techniques here. So I'm going to go to my Jenkins server. And this is just the Jenkins installation that I use. And so if I look at, if I remember right, if I just put slash API on the end of any URL, I get this REST API endpoint. And so already I've got, and if I want to see the JSON data, there it is. So already at the root level, there is an API endpoint that returns. In this case, it's actually quite helpful. It returns the labels that are available on agents in my installation. And it returns the description of the controller. Oh, shame on me, it's using that word. And the description there and the list of the top level jobs. So already there's something now that the question then is, OK, what Java code did this execute? So you might say, Gar, turn on your debugger on Jenkins Core and exercise this page and watch to see which APIs are actually called to populate assigned labels and to populate jobs and numb executors. And then those things are being discovered somehow. So now the challenge is, how are they being discovered? And once we know how they're being discovered, that discovery process is the hint. Oh, that same technique is probably being used to discover things from plugins, like. So let's see, that was one example. Let's try. Did you have a question before I continue talking? Yeah, so you said in order to just know, I mean, how actually this thing is working from the Jenkins, I have to put a debugger. But how can I know where to put a debugger? There are thousands of lines of code. So good question. And in my case, I would go looking for, so again, this is, oh, like I'm sure would have better and better guidance, but we're going to use my feeble and inadequate skills. So I would look at this string assigned labels and look for that string in the Jenkins source code, thinking that, OK, it's probably something related to that and set three or four set breakpoints at various places around that label, trying to find it. So let's take that example and test to see if that would actually work. Let's see how many occurrences there are of exactly that string in the Jenkins core. OK, there are only a relatively few. So yeah, so search for string and setting breakpoints looks like it might work, at least in that case. Yeah, I mean, just for Jenkins core, maybe who am I? And maybe these root stuff, yeah. Oh, right, right. Who am I is a very good example. Yeah, that's a very good one, because when I go to who am I, it presents all sorts of information. Interesting, oops, did I type it wrong? Is it capitalized like that? There, yeah, so very good. OK, but even though if I understand from where that code is, I mean, which line is executing while running this endpoint, then where I have to put my code in order to extract that endpoint? I mean, that I think is the next question, right? So for me, the first question was, OK, what does an API endpoint look like inside the Jenkins code? Yeah, yeah. And so that was the first question. But that's certainly not a conclusion of the questions, because once what does the API endpoint look like? The next question is, how does Jenkins determine that that thing is an API endpoint? And it may be that we've already got the hint right here. It may be this line right here. See that index.jelly line that is in this? Yeah, this jelly is from Stapler. Is it from Stapler? It is, exactly. Very good. OK, so you already know the terminology yet. So that jelly is processed by Stapler. And now I'm not sure if that's the thing that provides the REST API or if it is that defines the REST API, not provides it, but defines the entry point. I would assume it's coming from the Java code, and so I would have assumed, therefore, it's coming from this abstract build class, but again, I could be wrong. And that's part of the exploration. Yeah, that's part of the exploration. Now, the other is we could probably just ask for Oleg's help on this, schedule some time with him to get some more one-on-one time so that he could assist with a quick tutorial. Oh, here's this, here's this. Yeah, yeah, that makes sense. I mean, the first question that we currently looked into and just, I mean, a possible answer may be that we looked into just looking into the files that when we are hitting that input and just put a debugger and explore those files, maybe, I mean, I get some hints and another possible ways to explore further. Right, yeah, so I think we're also open to it. I can ask to see if we can find some of Oleg's time specifically for this topic, just because it would be a good conversation to have. He might be able to give us in 15 minutes a quick tutorial we could record of, oh, this is how, okay, we saw, here's some examples of the endpoints and we found that in the source code, but then how does Jenkins decide something should be a REST API endpoint and he'll probably guide us and say, oh, here are the two or three different ways that Jenkins decides that. I see, I see, yeah, makes sense. Yeah, but I remember on the 13th and he also said, I mean, who am I, he's quite straightforward also. I mean, I don't know, even still, I didn't even explore it, but he just said, I mean, it's quite straightforward also. So let's take a look at it and let's use that one as an example. I think that's a good example. So here is, and now let's search the source code for it. It was spelled lowercase who, okay, it's who am I, okay. So they're a bunch of tests. Let's look just in source. I guess you have to share a screen. Oh, whoops, yeah, it would help if I shared my screen, wouldn't it? I'd talk about what I'm seeing and none of the rest of you can see it. Sorry, that was not very polite. There we go. Okay, thanks. So first, let's start from A. So I was thinking, let's look for interesting, okay, who am I? Okay, so this is proof that, oh, oh, here we go. Okay, so why did that, why did that earlier search? Okay, good. Huh, apparently I don't know how to use getGREP and that's embarrassing because that shouldn't have found what I was looking for. So this thing is the who am I implementation, right? So, and here we see, yeah, so now, so I think if we were to review that, we would see, yeah, okay. So this has the convenience that it is an exported bean, right? So what we had discussed earlier, this one's an exported bean. That is, if I understand correctly, definitely one way to detect a rest API. However, and maybe that's, I guess it depends on what's best for you in terms of exploration. You could conceivably say, hey, I'm going to do it just for exported beans initially and then try to extract, extract a sample for an exported bean, for exported beans and see if I can generate the swagger definition of that and look at that to see how does it look. That might be, you know, a small exploration of just exactly one thing. Now, you could then say, but exported bean is certainly necessary, but it's probably not sufficient. It's probably not all the work. There are likely other ways of defining a rest API endpoint. Yeah, I mean, yeah, that maybe Oleg is going to tell us in maybe a short tutorial, what are the possible ways to our Jenkins understanding, is it a rest API? Right. Yeah, so, so does, is that enough to help you continue, Sagar, or should you have other questions you'd like to ask? Yeah, just last one. If I, OK, so for now, if I just continue with who am I and started with exported beans, so I have to just look into the source code, right, to understand and where to put that source code in order to extract that endpoint and that I have to write. That's that's what I was assuming. My assumption was that that the the source code has some way of documenting itself. So for instance, here, this is this is the Java doc for the this this comment right here is the Java doc for who am I? And then I don't see any other Java doc. So that yeah, so that's the total Java doc. And then the other things that are here are the at exported annotation that tells us this is probably giving us an API endpoint that is for the name. And this is anonymous and is authenticated here. Let's are you OK if I check that? I think I think this would be this might be helpful to check. Let's see if I do slash API on the end of that and then look at the JSON API, it says yes. So here's an anonymous authenticated and name. So there's there's the there's the representation name for for the get name at exported annotation. And there's the one for authenticated. Yeah, so so that I think what we saw see there is who am I slash API in the UI maps to the the name field is this the authenticated is this the anonymous is this. Now I don't know. Oh, oh, interesting. OK, so and here's here's an example of one. The truth of session ID with same security. Oh, so it was probably exported at one point and is not exported to be longer. And if we look at that, it's definitely not in the output. Yeah, yeah. So I don't know what is session ID, but it seems little bit yeah, redirected security, so that's why maybe they remove. Yeah, yeah, I certainly we if we wanted to know why this was removed, there are marvelous tools like this one that blame this that will tell us why it was when it was removed and we could read the list. So but but for the rest API, you don't need you don't need to know why it was removed. You're just that if you were feeling curiosity or thought that you should become an archaeologist, it's the programmer then. OK, yeah, make sense. And I was thinking, OK, I believe on this, who am I class is being maybe processed by Stapler and then he's doing some magic behind the scene and just showing the output on the screen. I think so. Yeah, I think I think you're you're on you're on the right track and I have to I mean glue my code somewhere where the Stapler code is doing the processing and try to I mean from that and process that same class to extract the endpoints maybe I need I just need to check where this who am I call is calling in the Jenkins source code and maybe then I have to maybe a sconce or look some maybe look some endpoints I mean control clicking on control and just clicking on this class name and it will show me all the places where it is using and just seeing where how it that that that you certainly could that seems like I mean for the rest API specification generator, I would think you probably don't even care who this thing calls right you don't care because because for the rest API specification in terms of the specification, there is a there is a rest API endpoint. Let's go back here. So there's a rest API endpoint that looks like this or something like that, right? It looks like Jenkins URL slash who am I slash API and then there are arguments that that thing takes like, like this. Well, here we can see the examples that this talks about are things where we can say should it take a certain names of jobs or should it do a certain depth there are those kinds of arguments and then there are additionally the arguments that that may be passed to the to the function that's answering her to the method that's answering the question. Okay. So so back to I don't I don't know that you need to look at who's who this is calling it's calling somebody but the fact that this is exported and that there's a get name means that the rest API specification should say, yes, this thing returns a name, a field with name in it and the value of name and a field authenticated and the value of authenticated in it. Yeah, I was and go ahead. Yeah, and and the parameters is being that we entered is being again handled by Stapler. And yeah. So, um, so eventually, um, I mean, you know, so we so in order to extract endpoints, where I can put my source code, I mean, to call just, I mean, for the exploration to where I just I can actually log some of the endpoints in that. Yeah, um, I was just thinking about that. Yeah, yeah, right. And so, okay, so I think what you're describing is how do you detect more endpoints? Is that is that what how do you discover more of those endpoints? Yeah, okay. So I know, I mean, I in this file that contains exported been and some exported annotations that is that data is going to be output on the screen. But, um, so our main goal is to make to show a documentation to a user, a beautiful location, which is being generated by again, Swagger. And for that, we need open API specification before that even we have to extract rest API endpoint. So in order to extend, extract that endpoint where I have to, I mean, I should I have to, I mean, can you give me any helpful, I mean, tutorial or just like previously, like I made a plugin that was a great tutorial. I just made a scratch plugin. So if I put this code, that menu is being created. So I mean, where I can. So, so in order to make this thing happen, so where I can, I mean, put create files and put, I mean. So, so maybe what you're asking is how will the rest API specification generator be invoked by someone who wants to generate the specification? Is that okay? So, so, and that I think that I think is, I think the rest API specification generator should be an external program that, that is separate from Jenkins, but reads the Jenkins source code. So we've got, we've got something similar that we do with a thing called the pipeline steps documentation generator, where the pipeline steps documentation generator does what it does is it asks for the list, I think it asks for the list of all plugins and then reads those plugins looking for any of them that implement a pipeline step. And when it finds a pipeline step implementation inside one of those plugins, it goes, it reads inside that plugin source code to find the documentation that is related to that pipeline step. So it's a separate program that we run independent of Jenkins. So, and it's separate so that we can run it and, and show and extract the data and then use that on a website. And I think this open it, this rest API specification generator be the same thing that we would ultimately, you would create a program that is a separate program and that separate program would read the Jenkins source code, read the plugin source code and decide, oh, here I've discovered these rest APIs generate the definition of those APIs, the documentation for those APIs and write that somewhere that we can then feed to open API or to the swagger display utilities. Yeah, that makes sense. I believe. So, I mean, from now, I guess it seems I have to maybe explore maybe about pipeline step documentation. I mean, how maybe it is working. Right, that that might be a good, that's probably a good, that's a good place to see how does, how did someone else do the discovery of, of, of information from inside Jenkins plugin source code. Yeah, because pipelines, pipelines, pipelines depths. Yeah, I'm sorry. I'm not remembering the exact name, but let me find it because I was just working with it with the docs, docsig, because it was it. Okay, here we go. Oh, there it is. And just a minute. So it's is that generator is also is, is, I mean, make but made by Christine. I mean, I mean, is it? Yes, that's the one that Kristen Whetstone made. Yes. Yeah. So, so I am impressed. How do you know Kristen Whetstone's name in that context? That's very good. I mean, I did some research. Yeah. I mean, I mean, I'm while I was exploring, I just, I get different links. I'm just reading all that I'm making notes and yeah, that's. Yeah. So the, so here I'll paste into the chat. Here's the GitHub repository for the pipeline steps steps doc generator. Oh, okay. Oh, oh, so that's and what is this? I mean, maybe Jenkins infra is, is it the external programs? Yeah. So, so Jenkins Jenkins dash infra is the GitHub organization where the Jenkins project keeps its public infrastructure. And so, so that's where Kara and I and others do maintenance work to, to improve the Jenkins.io website. Or to fix the update center or to capture ratings of new releases or to publish, publish change logs, all sorts of things like that. Yeah. Okay. Okay. So, so it's completely external program, right? It is. Yes. It is an entirely external program. It's, you know, now it's, it's written in Java and I suspect that it would be good to write it in Java because that way you get the benefit of using Java to do the, the looking inside of class files if you need to, if you need to do Java bytecode manipulation for whatever reason, having it written in Java is a lot better than choosing to write it and go or Python or something else. So, so I love Java. Great. Yeah. So, and when the Kristen made this plug in, I mean, is there is anything that she read it before? I mean, which I can, like, there is a documentation on how to create plug-in. So the read me in this plug-in, this pipeline plug-in doc generator is a, is, is, has some good outline of, hey, this is how you use this thing. And when you use it, this is what happens. The other is that there are, there are at the moment in the doc SIG, people who are actually working on pipeline step generator to be sure they understand it. So if you have a question about that one specifically, go to the Gitter channel for the, for the doc SIG. And I will paste that into the chat here as well. If you have, now, again, the, the pipeline steps doc generator is not the REST API generator, but it's still a generator that uses Jenkins source code and generates an output. And this pipeline step generator is that when we have a different task, can we chain them and it will show something or graph? Is it that, is it that it is generating? Oh, here, actually, let me show you what it's generating. If that's okay, that makes it, makes it sort of real for us, right? So I'm going to, so here is the Jenkins documentation site. And if I look at Jenkins pipeline in the Jenkins documentation site, down here at the bottom left is this pipeline steps reference. So when I click that, it brings me to this huge page listing all of the pipeline steps. And here's a plugin that, that's familiar to me, the get plugin. And it has one step that it provides. That step is called get. Okay. I hope I did not cause your eardrums to explode. Forgive my sneeze. All right. So this get step is this, all the text that you see here has been extracted from the get plugins source code. So if I take that, that text, let's take this phrase preferred SEM checkout method. If I look for it here, we should find that in this code. And here it is. So in this particular example, but this documentation, the steps generator extracted it from this file. And, and so I don't know that that helps you at all with, with the REST API generation task. But, but it, the, the source code, the steps generator. You discovered that there is a plugin that's the get plugin discovered the source code for the get plugin, searched the source code for the get plugin and found this page that it could then embed in its output. And that's, that's so that, that gives a hint of what the pipeline docs step generator, pipeline steps doc generator is doing is it's search all plugins, find the source code for each of those plugins from that source code, extract relevant documentation collected into files on the disk and ready to be ready to publish to some final location in the case of this final location. It's Jenkins.io. Oh, OK. So this is that it is generated by pipelines, the output which is generated by pipeline step document, step documentation generator. Yes, right. This page that you're seeing here is generated by the pipeline steps. And if, if you wanted to see the what actually generated it, that's generated by ci. Jenkins.io. If we go here, there is a job that runs periodically that generates where it is. There it is. Here is the job that runs periodically that that actually publishes the its results and Jenkins.io uses it. So ultimately, your rest the rest API specification generator would someday be run in a ci. Jenkins.io job to do exactly this. It would do the reading of source code, writing of documentation files. Oh, OK. And yeah. And that entire documentation generator is running while runtime or it is all pre-processed. It's so it's it's doing if I remember correctly, it's generating its ultimate output is static pages that are then used in the Jenkins.io site. We don't want we don't want the Jenkins.io site. This thing to be anything but static pages. And so it I believe it the steps generator creates static pages or ultimately creates static pages that are displayed here as static webpages. I see it. It was was that your question or did I misunderstand your question? Currently, even I don't know much about that. I mean, I get a little glimpse about pipeline step documentation generator. Maybe I have I will look into it for this week. And then maybe maybe if I have questions, I will further on for next week. I mean, it's better to explore it. I mean, we are just I mean. OK, yeah. And that sounds good, too. That's that's great. The the for me, the the concept that pipeline steps doc generator is providing is an external program that week that somebody calls you from a development environment or Jenkins from a CI environment that reads the Jenkins core source code and the list of Jenkins plugins and then uses that information to extract use some useful content pipeline documentation or REST API documentation, writes it to a location that then it can be used to publish to users. Why we are naming it pipelines step documentation? I mean. Oh, because because it's describing the steps of a pipe of that are available in the Jenkins pipeline. Oh, oh, oh. Can you please repeat that story? Sure, sure. It's it this particular tool is named pipeline steps documentation generator because it's describing the the steps which can be used in a Jenkins pipeline domain specific language. So so maybe let me show you quickly what what I mean by that and and hopefully that that will be that it will be a little clearer with this picture. So if we look at let's take out here, let's take we're going to take an example that is comfortable for me. The get plugin. So oh, no, that won't work. That's not nearly straightforward enough because it uses all sorts of fancy. Sorry, let's use something very simple like this. We're going to pick one of the bugs that I've checked. So if we look at this and if I do a click here. And no, not Blue Ocean. That was not what I wanted. I click replay. This shows me. The Jenkins pipeline definition that this job will run. And so what you're seeing here is this is a Jenkins pipeline. And this Jenkins pipeline uses uses uses a domain specific language that is derived from Groovy and and it says, OK, we're going to on a note, we're going to create a pipeline stage named check out. In that we're going to execute the step check out. And this step check out would be one of the steps that the step generator documents. We're going now. I see, I mean, for writing this language, what are the things that we need to know before writing this? That's all being I mean, for a bunch of I mean, he just the accurate information is being generated by power planes. Right, right. Yes, you understood perfectly. That was exactly right. It's it's OK. This is what which which things can be called in this context. And so now if I if I click, if I remember right, if I go back here, there is this handy little tool, this thing called pipeline syntax that actually helps me generate valid syntax. And so, for example, you remember, we we looked at that get step. Here's the get step. And it says none of this is particularly relevant to the to to your project other than being aware that this exists. So so here's this tool that says and let's say special branch. Oh, no, no, no, no, no, no, because cars here and she lives in in in the UK. We use my six, of course. OK, so that just generated. The the the pipeline syntax, if I wanted to use this. Now, you don't care about this for for the rest API generation, but being aware that this exists won't do you any harm. It's this is a kind of a tool that is very helpful for people who are writing pipelines. I see. I mean, this all configuration has been written in this one line. Is it? Yeah, exactly. All of this, all of this all in the answer is expressed as code in that you got it exactly right. And this is being done by. I mean, yeah, yeah. And it. Yeah, so again, this this this is this is not this this piece is actually not related to the rest API. And so so I only show it to you so you know why pipeline what pipeline step generator is doing. It this has this has no direct relationship to the rest API. Yeah, yeah, yeah. I got it. Yeah. Okay. Okay. Okay. So I will look into the pipelines document documentation so how it is working. So that maybe that could be now I have idea because last time when Olex had to me discover what is code coverage plugin even I don't know first of what is code coverage then one day later I came to know it is just a testing practice which is just showing a measure. Yeah. Yeah. Right. Well, well and so exploring the code coverage plugin is interesting but but we failed we had failed to give you the general framework for hey this is an external program and it needs to read from Jenkins core and read from Jenkins plugins and you may find the pipeline steps doc generator closer to the final structure of the program you would then and just exploring code coverage plugin. Yeah. Yeah, that makes sense. I mean on 13th and you are you also written in that in the short notes I mean for more I'm explored these pipeline step documentation generator. Good. Okay. Well, so again you're seeing my poor bias that thank you very much for working with us. This is great. Yeah, my pleasure. Yeah, that's for today. I mean for from my side. Yeah. All right. So, Cara, I think you we had you had some PRs that were that needed discussion as well. Do we have time to do the PR reviews do we need to call it a done for today. It is the end of our meeting time and maybe for the PR reviews this is something that I just couldn't discuss with you Mark like I we can we can hop on a quick link or something I don't think we need to take any questions. Okay. Good. Great. That is all I have for this meeting. Yeah, any other questions ago. No, so should I text or lag on data I mean, or is it is it or just prepared for one week. Okay, okay. You are certainly welcome to it's probably a good thing to ask all like hey all like I have some next Google summer of code office hours that way you've got a week to prepare. And he's got a week to get it on to his calendar. Yeah. Yeah, that makes sense. Yeah. All right, thanks. Thank you very much. Thanks Mark for taking my even dumb questions. Those are brilliant questions and thank you very much for asking them we are delighted that you're asking it's and thank you for your patients with my giving inadequate explanations or imperfect and flawed explanation that's wonderful. We'll talk to you in a week. And I look forward to more questions. Yeah, just just post this on YouTube. I will eager to watch it again. Great. And yes, so it'll be about probably two or three hours it takes about an hour or two to process it before we Yeah, that's fine. Yeah. See ya.