 Well, if you're watching this recording, we are doing the first part of the public retrospective for Google Summer of Court 2020. Just to clarify, we are doing public retrospective for the organization. For projects, we recommend all teams to have their own retrospectives. Yeah, we know that many teams have already done that. And this feedback is most like private. But for organization level, we'd rather prefer to have a public retrospective. For that, we have sent a form where everyone could provide feedback. And if you haven't done it yet, please do so. It takes maybe 10 to 15 minutes. And it will be much appreciated. Today, the plan is not to finish the form, but just to go through feedback we have already received and get feedback from participants of this call. And mostly to discuss what would be improved and what we would like to change. So before we begin, I suggest that those who haven't submitted feedback yet speak a bit about key highlights. So we have a number of students who participated, but if you could, it would be appreciated if you just share what's your impression, what are the main advantages of JISO, what were the biggest changes for you? Photoshop, would you define if it starts from you? Yes, I'll make sure. So I think what I've written here, so if you see the last, the highly active community point, that is what I wrote as a key highlight for me. So one of the first impression I got was, which is very important for me personally, is that money is not the driving factor for someone to work or to contribute to something for a project for such a large scale. When I saw the amount of time my mentors in general, the Jenkins community, the people they give within this community, for my project, the technical advice I was being given and the time they've been provided, that personally is really motivating for me to do this in future. Because I personally was not aware of how open source community works, what is the motivation behind a person to contribute in an open source project. And I think I've learned a lot in terms of the reasons why someone would do that. And I think that's a great thing, that was a great thing for me. The second highlight would be the blogs. While I was doing the project, I was not, I would say I would be honest, I was for the first phase and the second phase, I didn't consider the blogs as important as the coding or the research tasks I had during the phase. But as I was ending the nearing the end of the project and I was writing my final blog, I realized that there is no point of developing two or three features which you send to the production. And if you're not able to present them in an easy or compact manner to people who do not have the time to go through your code or to the developments you've done throughout the three months. So what I want to say is writing a blog is a great way to reflect on whatever you've done in each phase. And it's a great way for other people to understand what you've done because what you're trying to do through writing that blog is to provide an abstract concept of what you've done throughout that phase. And actually, that's a difficult task when you've been trying to solve a complex problem and you start writing a blog. It's actually, it's a challenging task to make the complex concept simple enough so that people who do not have maybe more than a minute to understand what you're doing and be impressed about what you maybe impressed enough to actually look and use what you want to provide to the user. So I think it's a very what I as a suggestion for mice, it was the suggestion for myself and I think for the future students is that they should not start writing the blog before maybe two or three days of the deadline. They should do that before the week. The mentors should get a draft of it. They should review it first. And it should be as important as the coding task, although it's not required by Google. It's a great, I think it's a great addition by the Jenkins community. But it should be, the weight provided to the, to that task should be equal to the weight we provide to the coding tasks in each phase. Yeah. Yeah. And what were the biggest challenges for you? Okay, biggest challenge for me was to actually plan how to, so in my project, what was difficult for me was that it was not straightforward coding tasks we could design. It was not just designing a feature and then delivering it to production. Before that, we had a significant amount of research we had to do to understand, so my project was performance improvements in Git plugin. So to improve performance, we first needed to find areas where we could improve the performance. So now, doing that planning for the whole, this whole project, it was difficult for me because I could not gauge the amount of time it would take for me to research or to explore time I would take in the exploration to find an area within the Git plugin where I could improve the performance. So I would say I was not, I was not sure, at least during phase one, how much amount of time I should give to the spikes which we use in the agile methodology that we're basically researching the, if we do not have a time frame for those tasks, balancing that with the coding tasks was a difficult aspect for me within the project. And then the second thing which was challenging for me was to take the code from my personal ID to production. I did not, I think I did not estimate that the process would take much more time, especially for a plugin which is used, which has such a wide audience that it's planning to just release a single feature is, requires much more, I would say, the developer should be much more informed before even planning the feature that how much time it would take to deliver it to production. So knowing those two things, not knowing, but maybe I could have done better in those two frontiers planning and it's basically planning. Yes. So how would you expect to understand that before the project starts? So what would you be in anticipation for this information when should we discover the server heads and plan for them? Okay. So you're asking, so this is in particular to the application phase? Yes. So for you, it's application phase, right? Yes. Actually, when I was, whatever I'm saying, I was saying it in perspective of the whole in general GESOC project, not because of course, I think before while writing the proposal or before doing the project, it would be difficult to understand how much time it would take for a student to write a code and to take it to production. But maybe I would say, I'm not sure if it's something which is necessary, but if there was maybe like a cautionary advice from the mentor that okay, since the project you're about to start, it has a huge, we're responsible for a huge user base. Whatever you plan to do, just make sure that the features you're planning, if the amount of time it's going to take to write it to code it and then to test it and then to release it is is considerable. So count that time when you're thinking about doing ambitious, maybe thinking about doing ambitious features or it just would be, it would make sure that the proposal or the after the proposal planning is better. Yes, I'm not sure if this is something which could be generalized. I'm not sure if this was the experience for everyone within the project. This is my personal view for the project. It's definitely valid and it's definitely something to be in the process because here's student guidelines, presentations to students, inputs during phrases. Yeah, that's why we do retrospectively because even if we do act on that too much, probably it's something that needs to be documented explicitly. So that students see that when they prepare or when the teams see that when they do planning during community bonding. Because for me, during application phase it's great to know that but the only way to actually know that is to try out submitting some pull requests and looking at how it proceeds. And that's definitely during community bonding when all the team works together, mentors have experience about operating within a particular component. And yeah, doing adjustments for the plan during community bonding phase. I like your suggestion for community bonding, particularly before we show that probably would have been a very healthy place for us to assure that we did a good job as mentors of reviewing during community bonding. That was a time when I felt like I was not as effective at reviewing as I should have been. And so that would have been that would have been a place where we could have done a better job planning by more active reviews during community bonding of every change that's being proposed and getting it all the way to production. Yeah, so our expectation, yeah, so I just opened out the quotation. But yeah, this is something, well, so what we have, we continue to discuss and plan the project with community and mentors for the bank objectives, patients define the project plan, etc. But yeah, maybe something like adjust the plan to the project's goals of engagement or something like that would be reasonable here. Because there certainly are Google Summer of Code projects, which were in lower demand in terms of what it takes to get to production, right? So there's a varying thing, varying level of difficulty there. Yeah, so, yeah, so I just might suggest detection, maybe you can follow up on, but it's mostly to demonstrate how we would be processing that. Yeah, we have spent considerable time to review this particular item, but I think it's important to demonstrate how we usually handle it in retrospective. So next, you will be reviewing this action items based on feedback and see what exactly the schedule will create tasks and hopefully it will be reflected for the next year. Okay, I suggest to move on. So, Kezhe, would you like to share your feedback? Okay, yes, I'd love to. So, where should I start from the highlights? Yeah, okay. I think the first highlight is about the mentors. We have more than one mentors for each project. And I think from different mentors, I can learn different things like the team has taught me a lot about how to manage this project from a view as a manager of this project, not just a programmer. And he encourages me to do a lot of other rich things like doing a meetup and helps me deploy the plugin to the set.genkins.io. I think those things just helps me not only the whole process to manage a software during its life circle. And from Wooly, I just learned how to, he taught me many tools as a student in SS tools and taught me many principles on how to build a robust a bug software with less bug. And so I think mentors just taught me from different views. And I think the second part is in Jenkins during the Google Summer code, we can see that our product, our deliverable has rarely been used by other users. Like I have seen some users start using my plugins and I give some feedbacks as issues are in progress. And I can see there are some installations growing every month. So I think this encourages me to contribute more in open source and gives me more confidence. And I think that's important for a green hand at this in the industry. It's the highlights I can think about now. Thank you. And what are the main challenges for you? Well, the challenges, but I think the first one is the languages, but I think this more personal issue than the community. And so, and the second one is I think we have many, and we have many documents about the details. Like we have documents for our even extension point, how to write a plugin and such kind of things. But I think it would be helpful if we have some like top level design documents. And just like we did in community bonding and we wrote some top level architecture document. And I think if we can, if Jenkins could provide such documents, it would be helpful for the green hand in Jenkins. And I remember when I was first Jenkins, I read, I get to know those things from some YouTubers from third party pages, but not from the Jenkins official site. So I think the top level architecture document or even just some object graphs are useful. Thank you. Thanks for that. Any other obstacles you experienced? Currently, I don't have other challenges, but can we just add up the other challenges if I can came up with them after the meeting? Yeah, that's perfectly fine. Again, we are just starting. So this will be an eventful process. And when I say two weeks, it's a rather wishful thinking. Okay, moving on then. So meet, would you like to share something? Yeah, just to, okay, so key highlights with respect to my entire project or so like key highlights with respect to what Kulio, please? Well, we had a target on a feedback for your experience of interact with organization during JSOC. So again, the key highlights of my project basically, right? Okay. Of your experience with the project, let's say so. Okay. So if I start off with the community, I think one of the biggest highlights, one of the biggest advantages of Jenkins was I think a really great community. So that's been mentioned before, but I'd like to reiterate that. The team I got with respect to mentors and with respect to even other contributors who were helping on during the entire course of the project was awesome. So I was getting good code reviews. We were having very interesting design discussions during our bi-weekly meetings. So that was amazing. Understanding my basically getting to learn better coding practices and actually being convinced that, okay, this design is the best for our best solution for our problem. So that was really nice. We were able to follow our timeline really well. So that was also something, I think, that was another highlight. We were able to deliver the changes that we had proposed to the community on time. So that was a good highlight, I think. And I think I would be just repeating the things we discussed in our retrospective meeting. I think we mentioned everything over there also. The retrospective we had for our project. So yeah, I think only positive things from my side. I think a plus one on everything that others mentioned. So yeah. Yeah, absolutely. We also had a hackfest. That was nice. So hackfest was awesome. Yeah, writing, getting to it. So yeah. So I think with the blogs that I should mention, it actually, I think in the future also now I have something solid that I can go back to and see what my thought process was back then and have something for myself. So those blogs are amazing for the community also, for me also personally. And so blogs also and those videos that we made during our presentations. So that was really nice, I think that we did. So the org admins decided to take forward. So that was really nice. I think even GSOC office hours were being held. So I think I attended a few of them, but that was really nice to see if anything was going wrong or something. So I knew that there was a GSOC office hours happening every week. So that was also nice. I think I missed maybe a thousand things. But yeah, that's fine. Again, if you see something feel free to it. And yeah, of course, it's important. And what for any I can get. And what were the challenges for you? The biggest challenge is, I think okay, so I think we pretty much know that adoption was a challenge at least for my project particularly. And I think one and it was not a challenge, but I think much of a learning experience for me that I had to, you know, so we created the JEP also. So I had to, you know, get community support behind it. So our design decisions and documents had to be really well planned. So that's maybe a challenge, but also something I learned a lot from. So I'm not sure if by challenge, you mean something that we need to change in the future. So if that is the case, then maybe not at this point, but in a good way challenge, then of course this, yeah. Okay, let's note it a little bit. Okay. Yeah. So from your point of view, in the future years, does it make sense to do such projects which involve a lot of hardcore changes and hence create challenges for adoption? Would you advise to do them or would you advise to focus on something else instead? So, I mean, I think, so I don't think that's, yeah, I don't think that's a wrong thing. I mean, of course we should go ahead. I mean, it was something that was required and I think we should keep doing, I think yes, definitely. We should architecture focus projects, yes. So do you mean like changes to Jenkins core? So that's what you had in mind? So there is a case when a project has significant community value, especially strategically, like changing architecture, but at the same time, it is most likely to have challenges during adoption and during the feedback cycle during the project. So basically, for example, Kasia was working on GitHub checks and they were able to facilitate feedback really quickly, but an external fingerprint storage was quite opposite, mostly due to the project specifics, though it also has a quite high community value. I think so the two things you mentioned first was changes architecturally. I think that is a good thing because that gets people to understand Jenkins core as well. So I think we know that, getting contributors there also is one of the challenges. So I think that is a good thing. As far as adoption, at least my understanding is it may be a bit hard to predict it beforehand. I think that may be the case. So I'm not sure if you can really say at the start that how would an adoption you are going to get? So Well, in some cases, it's possible to predict that adoption will be relatively low. Right. Because the deliverables which can be shipped to users quickly and which can generate a lot of excitement like resolving a particular problem, say providing integration for machine learning or GitHub checks, and which are user-facing. There might be features which face administrators, for example, Windows services, and there might be features which actually are focused on architecture and it will take a while until it gets to users. So for example, we created external fingerprint storage, assuming that somebody creates a plugin which heavily relies on traceability and observability of items in Jenkins. It will be really useful to users, but it will take a while until it gets to users. It's not something which can happen in three months. All of that is still valid types of the project. So for me, I rather self-question whether such kind of project is ideal for JSOC. So I think I would say yes, but yeah. So I think you need to put proper disclaimers such as maybe something like that. Danger, it's hardcore project. Okay. Anything else from you, Sumit? That would be it. If anything, I'll add to the document later. Mark, would you like to add something for highlights and challenges? Or we can process mentor feedback later and focus more on student feedback? I like focusing on student feedback and my comments are actually already in the document from answering the survey. So yeah, let's focus on students. Okay. Then we could talk about application phase. So from your point of view, what could we improve and what changes would you suggest for the next year? So what we got as feedback that maybe we should do October first to get students familiar with the code base and projects and attachment for plugins to fit additional complexity. I'm not sure who provided this feedback. That was one from Rishabh and from me. I think I'm the one who actually wrote the text, but Rishabh had to change Git Client and Git Plugin both under control of the mentors, but also needed to submit pull requests to the GitHub branch source, the GitLab branch source, and that was a complexity we could have predicted from the beginning had we thought about it, but we just didn't think enough about it. So it was predictable. We just missed the fact that a multi-component project will be more challenging than a project that is changing only one component. It happens quite often. So sometimes we can call it collateral benefit. So for example, if we change something and we see an opportunity to improve something nearby, for example, submitting a patch to upstream library or upstream component or drinker score, it's definitely an advantage. So sometimes it complicates deliverables if your committed project deliverables depend on that. So we experienced it a few times, though we were largely able to predict such changes. But again, I think it's subject for community bonding. So for example, a good refinement scope of changes. Right. And I think that that is a good one that would have been fit well in the community bonding if we thought more carefully about, oh, this is going to happen this way and this way and this way. Had we done a better job of predicting the future, we probably would have had a better experience on actually arriving at the future. Okay. So we can add something like that. Okay. Any additional items we could improve, especially questions to students? So all students on the call contributed during the application phase. So if there is anything we could do to improve your experience, it would be nice to know about that. So I believe all of the students that were accepted were active participants in the Hackfest that happened in May. And for me, that is that really did matter a bunch. That was very important that gave us a phase, even before community bonding, where we could interact with people they could see and understand. Any other feedback? Comments? One point that I think I mentioned is before also in the project retrospective. But I think so, let's keep it here also because I think it's a big thing in what did go well before the application period was Jenkins was one of the first communities to have the ideas page live for the next year. So by I think November or December, Jenkins had a GSOC 2020 ideas page. And I could just Google this that GSOC 2020 ideas page. And so Jenkins was the top result there. So if I wanted to start something early, so that opportunity I had with Jenkins that I did not have with, you know, so other communities that maybe get their project ideas later. So that opportunity for me to start early was that idea page gave. So I think that's a great thing that we should definitely do. I think that's ingenious to consider, for instance, if I were to offer get plug and improvement ideas as part of Octoberfest and start touting them on on social media, you want to try Google summer of code, here's your test drive. I think that's, I don't know that anybody would be interested, but I think it's an intriguing idea to promote more actively. Hey, here's some ideas that we might pick for Google summer of code, Octoberfest your chance. No, I think you're onto something that sounds good. That's definitely. Yeah, I was actually tempted to introduce 2021 project ideas today. I started parking the website, but yeah, finally I decided to split it out to separate full request. But yeah, most likely it will happen in a few days. So if someone has any project ideas, you can already start thinking about them as I think you're going to remain in peace. Just just one last point and what did go well. Even the GSOC office hours, I think that were held, you know, I, those were happening since January or something. So that helped, you know, in community bonding. So, you know, you can join those meetings. And you can actually ask your questions verbally if that also helps, right? So that, that is also awesome. Yeah. And you think we could improve? I think I have an idea, but that's not for the application phase. That's one of the later phases. I think what Smith has said, I would reiterate it's, it was great that I could find a detailed explanation of what the project could be. And not only that, I could find how I could start contributing to get selected for that potential project. I could find newbie bugs, which I could solve, which was a great way for me to, you know, initiate or maybe establish a communication channel with the potential mentors, which really helped me in the later stages to, you know, get feedback for the proposal as well. And already, I think it's necessary for a student to understand and realize the, not only the community guidelines, but the, the contributing guidelines for the particular project. And you get to know that when you're raising PRs, you're writing, I got to know a lot from Mark's PR, then there's a way he left reviews before even I started the project, how he expects me to create a raiseable request and how it should be done. So that, that was a great thing. Yes. On the mentor side, I would say that the activity of students also was, it's a, one of the most important inputs for us when making decisions. Because, yeah, for applications, we're actually able to do educated choices of what application we would like to accept. Yeah, a lot of these additional information we collected through applications, basically all 12 projects, we had clear expectation about what would be of chances of success and also of additional challenges, which might need to be addressed during community born and later. It was mostly invisible to students, but as mentors and as actually there was a lot of discussions about that. So far, it's actually application phase and, yeah, all this new kind of friendly issues, office hours, etc. They helped us with the innovations a lot. Okay. Anything else to improve? Or should we move on to the next phases? Yeah, I have a point about the application phase on what did go well. I think I agree with all the project ideas. I think we have provided a web page included as ideas listed and we listed the project ideas with the potential mentors, the scope, and even some guidelines. I think that's as that could be a real big reason for the students why they choose Jenkins instead of some other projects. And so I think we should keep going on that and providing the project ideas early for with details early for the students. And the second highlight about the application phase is that we just provided his shared his development environment for me. And he just shared it on GitHub so I can get a quick start and start hacking Jenkins quickly. And that's friendly to the newcomers. Yeah, having quick start guidelines was an expectation for every published project. I wouldn't say that these guidance will be extensive enough for any of the projects and it's something we definitely need to study. But in principle, when we are moving the project idea to the published state, we will require a quick start guide and also become friendly and ambitious to be present. It's documented to some extent. Yeah, we definitely should keep doing that. Okay, so we have eight minutes left. Should we go to community bonding and start discussing that? Because usually people have a lot to say about community bonding. We already covered some items before. And personally, I think that community bonding is actually the most important part of the project. And for the most of the projects of community bonding, you can actually forecast whether the project is going to fail or not. At least you can definitely forecast that it requires involvement of all that means to succeed. And if you see what went well and what you put them through, it would be much appreciated. I think from my side, particularly from my project, whatever the challenges I faced, those could have been the personally I could have been initiated or from in general from our team, I think the challenges of planning could have been solved within the community bonding. We were more focused towards actually doing the project. We were more focused towards performance benchmarking and eliminating the finding results we could actually use. But we should have first before that we should have made sure that we need a concrete plan for the three months. And I remember that one of the mentors, Parichet, who was who did GSOC 2019, he did suggest me to write a design document. And I was and when I started to write a design document, I was actually, I was, I was finding it difficult to write it because for my project, I was not sure in terms of code, in terms of design, what I would design before even I needed to find performance improvements first, and then I would design a way to inculcate those improvements within the plugin. So that was like a step two for my project. But at that time to forecast the whole design was, I could not do that. And I remember that I did not publish a design document within that phase, which I did not like personally, but I was, I was confused how I can do it. I did, we did try categorizing how the benchmarks, the process of searching the areas within the plugin or how the benchmarks would go, but not the part of how we would implement it. So I guess that that was something where we, I could improve or where we in general could improve. Yes. So what would you suggest to change? I am not, yes, Mark. I am actually not sure, I am not sure if this is a problem from, if there's something which can be done in terms of maybe, maybe submitting a design document by the end of, by the end of the community phase, bonding phase could be like, I am not sure, I am actually not sure. This would be me being pushing to provide an improvement. I think it was more of an issue within, within our team or with myself, rather than the Jenkins, like the whole GSOC project. So I, I don't think I can suggest an improvement here. Well, but, but I think, I think it's a valid point that we did not have, we, as a mentor team, we failed to guide and failed to heed Parachay's good advice that a design document was a good thing. And, and that was something the mentor team could have encouraged. It would have shifted the work though, and that, that would have had a cost, right? We would have done some of the benchmarking later in favor of doing the design earlier, a document earlier that was expressing something we didn't yet know. And, but, but that's something I think the mentors might be encouraged to, to be sure that you have a detailed design discussions and the result probably should be a design doc. So maybe we should somehow reward the technology. So design doc might be confusing. We expect to see at least the forecast vision, whatever, well, to good energy, which would be possible at this stage. Because again, you don't need to design doc just for the design doc. The whole objective for that is to actually help the team to do the planning and to ensure that they can iterate when the coding phase starts. So do you find design doc technology consuming confusing from that point of view? Yeah, I'm, I'm fine with whatever phrasing, because the goal was to understand the kinds of things we would be doing. And I think we didn't do as good a job in that, in that community bonding. We had really good, good interactions, and we had really great results. But it wasn't as focused on the plan, I think, as it was on getting started on the results. Rishabh, did it, was that a fair way to say it? If not, feel free to correct me. No, no, I think that's 100% what I, what I want to say. But I would, I would also say that maybe we should ask the other participants that, is this something they, because I'm not sure if this could be a general improvement to, if you're writing, if you're creating a new plugin, I think there, they would not feel the same. What I want to say is that we should ask the other participants as well before considering this an improvement, because maybe this was something we particularly faced. And it was something we could have corrected as a team. But in general, I think requiring a design document or just telling that to the team is, should be enough for them to know that they should plan well before heading into the coding thing. So how would you work it? I'm actually not sure if this, Sumit or KZ, do you, do you feel that this is something which, because I'm not sure if we want this as an improvement in general, because I think if it, if it affects everyone, then it's okay. Okay. I think, I think what was something more specific to your project was, so you had to do some research and then your entire coding was, so if I'm getting it correctly, I'm not sure, so correct me if I'm wrong, that you had to do some research, right? And later on, your entire code was dependent on the results of that research. So, so I think, yeah, so that's a tough problem in itself, right? I mean, so it's just something that's hard to, that's hard to predict for you. So you're really dependent on the research. I think the best thing that maybe you can do is build a flow chart or something, but I'm not sure if that's a, so I think that's a, I think that's a problem with the project that's heavily research focused and then, you know, so I'm not sure if I have a solution for that. Answer your question, yes, that I was not, I was not dependent on any research per se that I had to do. So, so yes, that problem was not something we faced, but I can see that that's a problem that a research-based project can face, yeah. Okay, so fair enough, I think this is a project-based, more of a project-based problem than like something the Jenkins admin or Gadmonds could do. Okay, so probably I will take an action item, just to define it a bit and actually come up with a proposal, how we would change it, because I also do not have a clear answer what to do now. I understand that it's potentially a problem. Well, some projects, yeah, very iterative, well, assuming that we're in general, it's supposed to be like that. At the same time, we still want teams to do planning, and unfortunately not the entire JSOC structure is rather waterfall. Well, even if you read JSOC guidelines, yeah, now a project we rather prefer to do it more continuous, but yeah, for example, even if when you submit evaluation for student, basically there is a question of, has the code been merged? Then you can answer yes, no, but there is even no notion of Jenkins integration there or whatever iterations, so there is no way to specify, let's say 50% merged or whatever, and yeah, maybe it comes from that specific there. In our case, we definitely advise teams to do re-planning at the beginning of each phase to ensure that everything is on track, but yeah, I understand that the workshops experience with the research-focused project is different. Okay, I'll take an action item for myself, and yeah, we are going over time, so again, retrospective is a bit tedious task, and yeah, our main objective is to collect feedback, so I think, honestly, if you submit feedback for other phases, for example, just proposing changes in this Google doc or submitting feedback form would be much appreciated, and yeah, work admins will be working on merging everything into a single document, and then you will keep working on action items, so thanks a lot for all your feedback today, and yeah, if you could provide more feedback, I promise it will be appreciated because it improves the efficiency of these meetings, okay, so then thanks all, I'll stop the recording. Thank you.