 Welcome everyone to Google summer of code office hours. It's the 31st of March, 2021. So are there open questions, open topics that you have that you'd like to ask about. Grateful to have Gareth Evans with us. He's one of the agreed mentors or proposed mentors for several of the project ideas. Hi, everybody. Hi. Hi, I'm Akira cutie. I have a question. So I'm here. I'm, this is my first time to join this meeting and I'm very interested in the project code. Sorry. Jenkins remotely monitoring. I have several questions about this. It's okay. It, it is though Kiyuchi son, if, if you would prefer, well, why don't you go ahead and ask your questions. And we may some of them we may defer the likely mentor for that project will be a few minutes later arriving so go ahead and ask your questions and if we, if we cannot immediately answer them we may ask you to ask them again later on in the meeting when Oleg joins us. Okay. So, now, the project, the project goal is to correct the metrics from the agent nodes like CPU usage or memory consumption and expose the metrics in the four in the format, which can be used from it from this or graph. And I think the end point should be exposed on the master side, not in the agent side because it is a lot easier to access them. So, is this right? Is this correct? Is this correct? What, what, how do you think? Yeah. I think we are both pros and cons of this idea because exposing on the agent nodes can be, is very, is very good because when exposing them on the node side, we may correct the metrics even if the connection between master and node are closed. So, but I think it is not a good idea because the agent side is not designed to run the script while the master and node is the connection between the master node is done. So, what, what is the opinion of you, of your. I suspect that the reason Oleg originally described placing an endpoint on the agent was for greater diagnosis, not that typical users would actually have access to that endpoint on the agent. I assume that the endpoint on the agent would probably be consumed by the controller and use the controller would also use that to assist, rather than rather than assuming that an endpoint would be inaccessible for, for any generic user because in many installations kubernetes for instance, the endpoint, or the agent is completely dis is completely inaccessible to most users the only thing they can access is in fact the controller. I would guess that I think it's intentional when I, when I look at the description there, where it says, allow exposing monitoring endpoints on agents. And the reason the rationale is because on the controller side, there's not enough data so I assume what that would mean is the agent adds additional monitoring endpoints that are typically only accessible to the controller or to someone who's opened that network access to that agent. Then that would allow querying of more information about the agent as needed. Now, this is one really let's let's bring it to Oleg when he arrives because I think it's an excellent question I've given you my best guess. I'll let Oleg answer it when he, when he arrives, just to be sure, Gareth, do you have any, any other insights that you would offer on that. Yeah, I'm going to defer to Oleg on that one. Okay, I don't know enough about that area to comment. Shisan, do you have other questions about remoting monitoring as a project idea. Yes, I have. And yeah, I think, in my understanding, the, we cannot have a way to distribute the server road of the Jenkins master node, right. I mean, and we have a, we have, we can distribute the road on the agent side using criminal use or something like that. But the Jenkins master side is not, we cannot distribute the server road on the master side. Is that right. I think what you're asking is, is there a way for a Jenkins controller to or a master to be to be spread across more than one node more than one pod and Kubernetes or more than one computer. And the answer is no. So it was that did I answer your question or am I answering the wrong question. Yeah, that's what I want to ask. Yeah. And then the correcting the agents metrics on the master side can be near high. High road on the master on the control node and can, can it be the critical problem. Ah, so my assumption was that the choice to collect the metrics or not is something that would be made as needed to do diagnosis that it might not be an always collect the metrics kind of thing. Now I could be wrong though so that's that's again a good place where we'll ask will invite Oleg to answer when he arrives. Good, good, good question and yes you're right we know that the Jenkins controller is, as you noted, not a horizontally scalable thing right we cannot create multiple copies of it concurrently, working for the same on the same jobs. Okay, thank you very much. And the other questions on remoting monitoring. Now, I don't have. Great, well then let's, let's be sure that we, when when Oleg arrives, I'm going to bring you bring you back and ask you again to, to ask your questions again because I would love to have Oleg's corrections to my poor attempts at answers. Thank you very much. Other questions. Yeah, I have actually a question regarding the, like these are proposal submission process. Can I ask it over here? Sure. You're, you're welcome to ask. I can't promise that we'll do more than a great answer, but please ask. Okay. So like an application proposal period that the students can now submit the applications like for the proposal. So there's one thing which like, there's one document that they are asking Google, which is proof of enrollment for the Institute, like for Institute. So, like, the issue is, I like, add this like we are since online and we haven't like received any kind of when it was offline or college, we used to get like a signature like they want something which has signature from the authority of the Institute like that. So we used to get the identification card with the signature on that on it and the year of enrollment, like if I'm enrolling this year, I get that. But since it's online, so there's nothing like that, but we are like, is there an alternative? Like, can I send a bona fide certificate to Google for proof of enrollment? Like, if do you have any experience or idea about that? I don't have experience. That's a very good question. And I think it given COVID-19 it's a great question to ask in the in the general Google Summer of Code question line because I'm sure I'm sure if you have that question, there are probably thousands of people in the world who have the same question. What what is acceptable as a proof of enrollment, especially given your experience, you said that in the past when you were physically on at the university, you would receive official paperwork and it would have a signature on it. Whereas. Yeah, like, sorry, sorry to cut you, like, like identification card, like where it was written, we have enrolled in this year with the signature of the administrator, like, like that. But now it's not there. Right. So, which, which I think if, if your university is doing that, if your educational institute is doing that, I am confident there are others who are doing the same thing. And therefore, it's very much a good question. I'm not sure what they would accept or not accept. Himanshu, is that what we need to do when we are submitting on Google's website. Yeah, yeah, yeah, we need that. I haven't. And what we need, what we need any agreement, do we need signature from university? So sorry to interrupt you, but I'm not wrong. You need a proof of admission to show that you're a bona fide student of a registered university. So there has to be some documentation, either in the form of your mark sheets or transcripts, or an ID card, or a letter of admission to the university which says that you are a student as of 2021, may of 2021. So that's all which they need to validate your academic status. That's about it. Do you have any sample image like how it looks? Right, so, right, so actually I did it for GSOC the other day and they are going to load a few samples on the GSOC website soon. They haven't put up examples as of now. So maybe like wait for a day or two. Let them update it on the GSOC website and then you could look at the example. Yeah, thank you. Thank you so much. We can wait for a week. Yeah, save a lot of time. Okay, so, right. Mark, so I had a question. So the other day I was trying to create a multi branch freestyle project. But it seems that since 2017 it's been deprecated. Right, but I mean I've been very new to Jenkins right and I'm still someone who uses Jenkins and I'm not developing. So I would want to understand if it's possible because I really see a situation where I have I want to use freestyle projects but at the same time make them in a multi branch format and I do not have the expertise right now to write a Jenkins file for all of my repositories and branches separately. So maybe if you could help me understand that a little that would be really cool. Sure, happy to so the simple answer is no you can't use and you shouldn't use I know you don't like that answer but that's the simple answer. You can't use and you shouldn't use multi branch with freestyle, you will be much happier and much, much better off. If you spend the spend the minutes necessary and it truly is relatively simple if you're if you've already dealt with the complexities of creating freestyle projects, you'll you'll be surprised how easy it actually and actually you'll likely never go back. The reality is, most of us after we, we made the transition, particularly to declarative pipeline it's, it's pretty simple to express. So long as you're not doing. Well, if you're doing terribly exotic things in freestyle projects you really want pipeline. If you're doing simple things in freestyle projects you want pipeline because it's simple to understand. Thank you Mark. Yeah, and good luck that's having been through that transition and having, having grieved myself at the deprecation of that particular thing. It was, Oh, no, but I love my freestyle projects. That's what it's so much easier. Well, well, I mean, I was, I like my web pages right there just they were they were nice and click and point. Don't forget the snippet generator, be sure you use the pipeline syntax snippet generator I cannot tell tell that story nearly enough people need to use the snippet generator more and more and more. So don't miss out on the snippet generator. That's that's the magic for transition from freestyle the pipeline. Okay, let me look into it. Thanks again Mark. Any other questions. Actually, honestly, I did the point is I can't find that proposal anymore so few. So I have seen that there isn't so that there are two problems have actually come across and since nobody else seems to have any questions. We have a lot of plugins which are up for adoption, but they are never really claim. First of all, like I've seen a lot of plugins that which are which are not up for adoption in a second. So we don't really have good plugins for like gcloud and AWS and let's face it, even if somebody's not deploying these runners directly on Google cloud or AWS, and they're using their own infrastructure for some reason or the other maybe security practices, maybe ease of use or something. There are plugins which are broken, which are not maintained anymore, and there's no, there's no way out if somebody doesn't know Java or Hudson properly. They can't really, you know, even fix the plugin for, you know, on their own accord and then, you know, make an open source contribution at the same time. So maybe, you know, if we have some kind of a terminal based access allowed, I know execute shell works, but like for example in my organization itself Jenkins is installed via Kubernetes. It's installed as a head and shot, or as a directly as a Kubernetes part. I do not have the Jenkins password available with me which is something which is only available during installation. I could not install gcloud I had to ultimately go ahead and learn GitHub actions and set it up using GitHub actions since I could not do it using Jenkins with something I prefer myself. So I mean, do you guys not face such issues or is it like, like you guys already have other ways out to this. I mean I don't know this is a very weird experience but since these are office hours I mean why not ask and share. Fair, fair question so I think what you're asking is, if you don't have permission on the controller to install a plug in. How can you work around that is that what you're saying. No, a little bit of a change it's more like if I do not have access to the node directly, the node, the gen, the nodes on which Jenkins is running directly. So, while I can install plugins but if these plugins are broken. Then what do I do. Okay, so, so I'm not sure I'm understanding the question still so I'm going to keep asking you questions until we until we understand each other so. So it is that that there is the, you need some permissions on the computer that's executing the Jenkins controller that you don't have those permissions is that correct. I do not have permissions on the pod itself I do not have users on the pod I do not have permissions on the part is any way to get about it besides you know maybe running a dummy project and then using execute shell on that dummy project to do it. And that's, that's certainly actually a healthy way to do it if you, if an agent then can do the work for you and so have the agent do the work for you is very good. But I do not have to do this on it. So that's where the entire and it comes to full circle. But isn't that where then you need to negotiate for the capability, either inside the controller or at add on an agent that you can use to do your work. True. Okay, so like has joined us all like we and so sit down. I'm going to pause our questions on Jenkins administration for just a minute. Thank you for joining. I can hear Okiuchi had some questions on the remote and monitoring project idea. He asked them, I gave really poor answers, and we would much prefer that to allow you to give good answers than my really poor answers. Thank you to sound would you like to ask your questions. Yes. Hi, I'm a hero. Nice to meet you and I have a question about the remote monitoring. And yeah, first, I think the purpose of the nose should be corrected on the master side and expose it on on the end point at master node. And that's because the axis to the node is not available can be a cannot be is not available in general so even if we correct the matrix from node. Be a be a master node. We don't have to establish any correction direct connection to the agent node and very easily correct the matrix from master node. You can access matrix from the master from the controller only when the node is connected. I'm sure I'm sorry I can't hear you. Notable. Okay, thank you very much. So, yes, there is a connection between agent and the controller. And when this connection is active, you can connect the metrics from the controller side. But it's not always a case. So for example, when agent loses connection to the controller. And then, yeah, controller won't be able to provide any metrics. It's not reasonable in this case. Plus, we have some agent types, for example, a agent operating cover Apache Kafka, which is not continuously connect the controller is being connected on demand. Every time there is an instance, which is running continuously, which has remote in, and which could be exposed to many torrent. Why we did that is support multiple controllers. So, you mean, you should expose the metrics on nodes arise. So, exposing metrics on the controller side isn't enough for complex setups. In this case, what is the best practice to run the script on the node side agent side. Even if the connection between the master and agent controller and agent is lost. There are multiple ways. So the easiest way that Jenkins controller when connected can install additional many tools on the agent side. So basically launching additional threads to expose metrics there. This way is doable, but this way isn't very reliable. And it requires first connection. There are relative ways to just have a standalone agent or just a part of remoting. I mean, I mean Java agent, which basically exposes the remote instance and its metrics, even if it has never connected to the controller. Thank you. And I understand the advantages of exposing the metrics on the agent side but I think it is very valuable if we expose on the master side because very easy to set up for the system admins, I think. For example, when use the community, when we use the Kubernetes on the OTS, OTS carries the agent nodes. And when we want to correct the metrics from agents and using like Prometheus, we need to set up the service discoveries and set up the configuration on the Prometheus and it can be a hard task for system admins. And so, if we correct the master side, the system admins do not have to establish any extra connection between nodes and the monitoring servers. So just tell the master endpoint to the metrics to the Prometheus. I think, I understand the advantages of establishing the endpoint on the agent side but I want to, I love to create the plug-in to expose the metrics of agents on the master side and I think it's very high variable. And what do you think of this? Well, you could do that. At the same time, we all have metrics plug-in and this metrics plug-in already exposes considerable amount of data from agents. Oh, very. Yes. So if you explore Prometheus plug-in with permanent agents, then you should be able to see the data being returned. But it has many limitations, for example, for cloud APIs, because, well, I think that it would still expose agent metrics for ephemeral nodes, but I'm not sure about that. Sorry. Sorry. The metrics plug-in are now correcting some metrics like CPU usage on the node side. Yeah. There is also plug-in called support core plug-in. Ah, that one. Yeah. So if you look at the support plug-in, I saw the plug-in and the plug-in corrects the metrics at one not continuously, right? Just continuously. Well, there are only a few tools which correct metrics continuously. This is not really needed if you want to stop Prometheus, because Prometheus is a pool-based service. So once a while, it will connect to your instances, request the data, and you need to provide the data. And on your side, it has to be updated periodically, but not for any request. Oh, okay. Maybe for any request, but I don't think that all metrics would be collected in that way, because some metrics are really heavy when we talk about support core plug-in. Can I again ask my idea to correct the master, correct the agent metrics on the master side? Yeah. I think it's definitely something doable. My recommendation would be to explore what's already available, to see the gaps there, and to build your proposal based on this research. Because if you want to create a new engine for agent monitoring from scratch, then it will be like another head for the implementation. Thank you. Thank you very much. Oh, like I think I missed a phrase there. Okay, so it's, you were saying that if you were attempting to create an agent monitoring system from scratch, it would be overhead? Yeah, most likely there will be a lot of duplication, because how current metrics plug-in periods basically collects various data from the system using, for example, GMX and points, using already existing endpoints for collecting agent metrics, and then it exposes everything basically in a three structure where you can have a lot of information. And these three structure could be exposed to parameters. So if you look into the parameters plug-in, it basically wraps metrics to expose them up on request. And yeah, you could do the same for adding new metrics, but you wouldn't need to write a new engine from scratch. It's a bit more different for support core plug-in, because support core plug-in also uses metrics to collect some metadata, but for some other reports, it just does calculation by itself, and if there is something you would like to expose to monitoring systems, you would like me to move this code this way. Do I make any sense? Sorry, I cannot understand about the part of what you say. I do more further research. Thank you. Yeah, that's fine. And if you have any questions, please don't hesitate to ask in the chat, so no need to wait until the next meeting. Okay, thank you very much. Yeah, and the chat channels are quite effective as a way for us to use do question and answer when we're not when we're not in a in this meeting so that's good. Do you have, do you have the URL to the chat channels? Do you have the URL to the chat channels? I'll just post it here. That way you've you've got it. I don't need to. I didn't even ask the question. Let me just put links on each project page where you can find the channels which are specific to this project ID or jsoc6 channel. Right, good point. So let's let's point to the one for the for this specific remoting monitoring project. Let me go find that. So, I'm going to share my screen so that you can see the navigation. I don't want to do anyone any harm and other questions can proceed while while I'm on guitar. Other questions. Well, I have no one coming. So, hello, like, like, I submitted my draft for the proposal. I wanted to discuss about the add support for generating a new plugin file based on the current Jenkins for like I asked this same issue in the last meeting as well. Where you told me like the original plan was to get the list of the plugin files so as to provide it to the configuration for the configuration as code plugin. Right. That this is what I like thought or have understood. So, my question is basically I have also commented it so I thought maybe I go ahead. So, like, what I'm thinking is giving a giving out a YAML file but in the issue with Tim Tim created and the GitHub issue tracker. He said like he provided like all the three alternatives so standard output text file and as well as YAML. So I was thinking of like all the three, like, is it okay or is it fair to go ahead with that? Like, what do you say? Yeah, I don't think that complementing three outputs is a big overhead because once you can create a data model you can export this model in any format. It shouldn't be a big deal. Okay, okay. It's probably worth pointing out where those, so where those three different data formats would be used. So the standard out is probably going to be used for just debugging what plugins are there and what's what needs to be done. The text file format is to create a plugins.txt file that can be used for generating a like a custom .rimmage with all those plugins in there. And the YAML format is for the helm chart. So one of the things you can do is list plugins to be installed on startup inside the helm chart for Jenkins. That's the use case for those. So just to give you a bit more context on why there's three different formats. Thanks. And like, can I ask like for the, like if I have, I had posted my draft for the proposal, like I don't want to make this meeting about me, but I'll go ahead and say like ask this question. Like what do you think? Like is there any feedback from you guys? Like I saw, but there were not many comments as much as I expected for my proposal for my draft. Like any comments as such by the mentors and the community members? I think we should really take it offline and handle it synchronously, because in my case I just started looking in proposals. If you want, I can send a few dozen technical questions and clarification requests there. But keep in mind that there is no objective to create a full specification at this phase. So the idea is to have a proposal which can be related by mentors. But it means it has to have clear deliverables. It should be possible to estimate time for this deliverable. So to understand whether the proposal is feasible and mentors should be able to see basically what's your way of thinking and decide whether they want to work with you on this project or not. So that's the main objective. There is no objective to create a full description of what you will do. And most probably it's not possible at this stage anyway. But for the mentors to see what I'm thinking, I should describe it. It is kind of contradicting each other. If I want the mentors to understand what I'm thinking, I should probably describe it. What I understood from my research or whatever I did. So isn't it contradicting? You also said that it shouldn't be too much technical for the draft. So mentors make a decision based on overall impression. So they meet with you. Let's say this meeting is in the charts, etc. They got some permission from there. If you create some pull requests, etc. They also got information from there. And proposal is only a part of that. And your proposal will be also only a part of this decision making. If you have never talked to your mentors during the application period, of course, proposal is the only way to convey your idea. But for the majority of us, it's not the case. So you just focus on following the GSOC guidelines and make sure that you have all the required information there. You will like to get more feedback. We still have two weeks before the application deadline. And if you don't get enough feedback, if you're concerned about particular topics, don't hesitate to ask me in the chat. And yeah, okay. So now Oleg, you guided towards potentially a less precise definition. When we did the get plugin exercise last year, that project we felt in about middle of it, we felt like, oh, we should have thought about this before. So my experience was the other direction that gee, I should have, we should have had more asked more questions in the beginning. It's okay that the project during its execution changes its definition somewhat. But can you give some hints there? Yeah, there are three phases. First is application period. Second is community bonding and 30 is coding. So our expectation that by the time the coding begins, you have clear understanding what to code and how to basically deliver on the changes plan for the first phase. For that, it's not on the application period, there will be a community bonding phase. And community bonding phase is when you meet with your mentors discuss what would be the project deliverable and it's also an opportunity to adjust your project. So, yeah, because anyone has different starting points, sometimes you just need knowledge transfers, sometimes you need to study particular technologies, and you can write the proposal and specification without studying it. But you are not expected to study all the things before you get accepted. Yeah, it's understandable. So we basically expect mentors and students to work during the community bonding to scope the project, to do the evaluation, basically do some reality check and they, when we discover together you can make adjustments in the project. Okay, and I think that was one of the weaknesses. I'm glad you pointed that out that was one of the weaknesses we had is, I'm not sure we were as effective during the community bonding phase as we should have been in doing that kind of deep detailing of what the next phases would look like so we were getting familiar with each other but I'm not sure we did nearly enough specifying and final design during community bonding last year. Good insight. Yeah, to be final design, because for many projects, first phase is actually a discovery. Yeah, now it's a bit different because in the previous years, so in 2016, there were two phases, but there were two long phases, then there were three phases, and now we have two phases but they're also short. So probably it's, if you will dedicate the first phase to discovery, then you will have almost no time to do the whole thing. So please be cautious about that when you plan your work with mentors. But at the same time, I still find to firstly do prototyping and to see whether the ideas work or not and to adjust as you go. Okay, you just need to have a clear plan. So on June 7 or when the recording begins, you are not locked by anything. So you have the required infrastructure you need, you have the required knowledge, and you have a plan on what you would do. Because the worst thing which can happen in that coding period starts and you cannot really focus on implementing your project. This is what mentors and students expect to handle during community bonding. So when you were describing the difference between two phases and three phase in that case those are, they used to be three coding phases, and now it's two coding phases, did I understand correctly. Yes. Okay. There are two variations in this year. Got it. So, while in principle, it doesn't change too much because just subject for the planning, but before, basically, if even during the first coding phase, there was no final deliverable produce. So for example, yeah, there was a prototype, but nothing we should go to production. There were two phases to basically learn from the prototyping and to implement something final and many projects actually follow this way. Yeah, this year, it's unlikely an option. So just when you plan your project, if you see that you need some prototyping before you can do design and final implementation, it's perfectly fine. You can put it in the schedule. That's okay. So deliverables and deliverables, but yes, you cannot predict everything. Yeah, I don't know any JSOC project, which would go 100% smoothly in terms of delivering coverage in case it was planned. Actually, I have one more question. It's regarding the like office hours only. You had mentioned like you guys keep a meeting for configuration as code. I also put up a message regarding like this question. Is there any meeting but there was no response. Is there any meeting like that I'm not aware of because I went to see the events calendar and there was no mention of. Yeah, there is a configuration as code project meeting every two weeks. But we've been skipping it for the law since the new year because there was no much interest. So sometimes it's only me in team available and we're in a good sync without project. So we basically decided that if there are no other contributors interested to participate in, we want to host them for now. There is one time agreement that we would create the configuration as code infrastructure is called special interest group. We'll just make configuration as code again installation manager and other components are part of this seek and probably expand the scope so that there are more contributors invited. But yeah, right now we haven't done that. So for like for actually since the project is very much related with as you also mentioned with configuration as code and I would like like but since you are also saying that no, there are not many contributors, but I would like if it is possible to have these meetings like if you can or if the community can. We can do that. So, yeah, one of our responsibilities of mentors during the community bonding is to ensure that the community bonding happens. It means that the GSoc students get introduced the community to the stakeholders and that the connections with the community are not limited to mentors themselves. So there are special interest groups. Yes, some of them are rather dormant at the moment, but it's not like we don't have contributors working in this space. Because, for example, what had this configuration as code plugin, it reached a certain station, it addressed the most of the demands. It became stable enough. And basically the focus of the development switched to other areas. So for example, plugin installation manager, I wouldn't call it stable now. It definitely improved since we introduced into southern thinking, but there are still bugs that are still a major features that are many interested users and contributors in this space. And basically host meetings just for that. But for configurations code plugin development. Yes, at some point that there was just no interest as is to host it on a regular basis. Don't worry, we have a lot of contributors around. Yeah, I'm looking forward to those meetings. Thank you. Okay, so was there any confusion, possibly a separate SIG or like had the configurations code team considered just bundling into like the platform SIG. We were talking about three options. So currently, so configuration is called the plugin was started under the umbrella of cloud native SIG. The reason was mostly because of your configuration is called the essential part for that. The main time configuration is called isn't limited to cloud native for deployments. It's actually much more relevant for class configuration management use cases. And yeah, so there are many areas where it will be applied. We were discussing making a part of platform SIG, but at the same time platform SIG was pretty packed. So we decided that it would rather be better to have a separate SIG, but it's something which could be revisited. Option. Great. Thank you. So we've, we've reached our time. Are there other questions before we conclude. All right. Thanks very much. See us in the chat channels. Conversations are good there to start those and continue them looking forward to pull requests looking forward to yes we'll get more reviews out for for proposals that are out there. Thank you for your proposals. Much appreciated. Thank you so much. Thank you for the chance. Thanks.