 Welcome to the February 4th Galaxy Developer Roundtable. Today's topic is how to do a better job of bringing new Galaxy developers on board. So we're going to spend some time talking about possibly training and then we want to discuss other ways to make this easier as well. So this is a one, two, three, four, five person effort at least to start with and we expect it to grow. And I encourage people to ask questions as we go and at the end as well. So, let's see. So if you're a Galaxy developer, which I am not, I want to make that clear. What challenges are you going to face when you're new? One is the size of the project. Another is where do you find documentation? Where do you get help? What are the standards if you want to contribute? And what is the process for contributing? There may be more than this. There's a footnote at the bottom, which is my attempt to highlight that Galaxy is actually much bigger than just the Galaxy server code. There's a whole ecosystem out there. And we need to do a good job of communicating to people that they can contribute to that as well and make it easy for them to find those resources too. Okay, so what can we do to make this easier? And that is the central question of today's call. Again, we're going to seed that discussion with one particular idea about training, but we really want to hear from the rest of you about other things we can do. Okay, so this is my central proposal. How did we steal the admin training model? So we started admin training topics in Chicago at GCC 2012. And then we launched admin training as a separate event in November of 2016 with five days, 35 people, all of us tethered to Dan's phone at one point. It was quite the experience. We finished the Galaxy admin also five days, but with over 80 participants free online and global. That was coordinated by Helena and I don't think everybody was ever connected to her phone, but we've come a long way in those five years. Okay. So, from 35 to 80, we've done it several times in between. A couple of really cool things happened along the way. One is the first GCC after that first one we had a coherent day long admin training track added. And that was a result, I think, of the Salt Lake event, the November event. More importantly, I think we now have a robust library of Galaxy admin training tutorials. And that gets updated twice a year. So that's in the GTN and it's a lot of material and it's current. Okay, and it's kept current because we do admin training at least twice a year once is a standalone event and then once as part of GCC training. Okay, and that is it has been a huge win for Galaxy administration, in my opinion, because they no longer have to dig through obscure old emails in Galaxy Dev to find particular solutions. They have an excellent starting point for almost any topic they would care to cover when they're doing admin and configuration. And that's a result. Or that's because of this robust library of training materials. Okay. And again, I think that happened because of the training. So the question then becomes, would offering a three to five day developer training be useful for people. Would there be enough interest to justify the amount of effort that it takes to create and run that. And something else to think about is, can we do this without inducing instructor burnout. So the community already does a lot of training already does a lot of curriculum development. So those people would be some of the same people who do developer training. And so how do we make developer training. If we do it. Let's see be as active and vibrant as admin training without burning people out. Because if we do that. It's not worth it. Okay. So, if we do this. Yeah. I think that would be a wonderful side effects. As mentioned, we would have tutorials. And they would become more robust over time as have the admin tutorials. And the amount and quality of galaxy developer training at GCCs would also increase. So we've had some developer training over the years at galaxy meetings. But it could become on par with the admin training track we run every year. Okay. I forgot this. So, assuming people think it is here's a couple things to think about and we can discuss this in two or three slides. But what should that training cover what should the curriculum be. Let's see me Bay and Dan are taking the lead on their curriculum. If you would be interested in helping with this then contact us, contact us. And maybe possibly teaching this at a developer training event sometime in 2021, possibly as soon as GCC 2021, which leads us to the next slide. Get ahold of us will have a follow up meeting after this one to discuss that. And if we don't think this is a good idea then boo on you. No, then we respect that because none of us have free time to waste. Okay. Okay, Saskia Helena, I'm going to throw that at you. Okay, so this year, unfortunately, GCC will also be virtual event. We announced today. Our idea is to have one week of training before the conference. Like Dave said, last week we had the admin training was all virtual so we had pre recorded videos that people could in any time zone could start whenever they want they could follow these videos and get support online. So we would thinking of doing the same sort of format here. So we could just reuse all the admin training that we developed for last week. Similar training coming up in a week and a half for scientists called smorgasbord which uses the same format so we can then also you reuse this. And I was thinking it could be nice to try to have at least some core curriculum of the deaf training GCC as well. This could be something nice to aim for to to develop. So you don't have to fill all the five days with eight hours of content. In fact, it's nice or if you don't for our participants do, but maybe you could have some core topics on galaxy dev maybe tool dev visualizations. Things like that so could be something to aim for. Great success with this format in the admin training. I'd say it went fantastically and students got a lot out of it. I know a lot of instructors are somewhat uncomfortable with the idea of completely asynchronous training but for students. We had really good feedback that it worked better for them worked with their schedule worked with their accommodations that they have in their own setup their environment their own home. And last year at the GCC we had live trainings to different time zones, but the result of that was that we first of all had to identify two sets of trainers and we couldn't always so the offerings weren't the same. And this allows everybody in the world to have access to the same program the same materials. It's a lot less stressful for instructors having to wake up at odd hours. We're not recording the videos, but during the week itself it was less stressful for instructors. So, definitely, and we would have maybe everyone in remote where participants can drop by to ask questions and interact with with the teachers and each other, and we did support on slack mainly for admin training. What is the current enrollment for smart is worth 700 last time I checked. Okay. So this will be a, I'm sorry to say it's a good stress test. Yeah. And GCC probably won't be that big but maybe it will knows. But yeah so this will be a good stress test for that model. It worked last year with machine learning, which is something private put on, and there were 400 people signed up something like that. So we mulled it a little bit after their approach 400 students then and that worked fine. So, and that was all in the same time zone so here of course it's spread out over different time zones. So there'll be fewer active at once. Okay. Thank you. Oops. Okay, and I'm going to stop sharing and we're going to open the floor for whatever people want to talk about. Okay, so how do we onboard developers that's the big question and what can we do to make that better and is training one of those ways we want to do that. Okay, I can pick something more controversial to throw out there, if people want. Okay, that's a decided lack of enthusiasm. I'm just trying to get myself unmuted here. Being the developer that's on like day three here currently being onboarded. One thing that I would find very handy is just sort of a welcome to Galaxy development team stuff, just with links to the documentation. I don't know the stuff if I'm poking around in that the GitHub repositories but you know, oh, here's the guidelines for submitting your poll request, you know, create a branch name, follow this naming convention. I want to contribute how I'm expected to do GitHub poll request just sort of a welcome packet there's a lot of that around, but it sort of scattered all over the place so just a common document that says you know, here's your reading material and your homework for the next week. Come back when you've got your first poll request ready. That's a great idea to consolidate all that in one spot. There's some pretty standard files and like the galaxy main repository that contributing and read means and stuff but since the project spans like, you know, 100 repositories and who knows how many different domains it makes sense to come up with a single place to find all those, all those things. Yeah, just sort of you know a view of the project from 10,000 feet. Okay, so that's an argument for for better more coherent documentation. Like developer training, like, there should be no question about this. The admins love it, you wouldn't have the admin community you do without admin training like of course you need this for developers. Yeah, I guess I'm my, my, not concerned but but what I'm thinking about is how we should how it should be positioned. And that's the hackathon and things like that because the hackathon is kind of like one of them, like my favorite things to get out of the hackathon is that hands on sort of developer training and walk through kind of those those pieces of it. And that's, of course, going to be especially hard to replicate this year. So coming up with something useful here. So I agree with Elena but I'm on the inside so sorry I interrupted someone. I said I agree with Helena as well I mean I think galaxy has become an impenetrable mammoth for newcomers. And so organizing it based on like what Keith said, this is the first step to this is how you run the tests locally this is how you run an individual test locally how to speed up that access experience for the developers how to make it less intimidating. You know I mean there's the back end there's a client components that can each be either parallel tracks or subsequent phones because I'm guessing not everybody's going to be interested in everything development wise so it's more like a menu type structure rather than a, you know, you have to learn everything like get the ball rolling and pick your thing. So would those be more like, kind of like the, we made a couple of them but the videos on YouTube, where like here's how to do that client develop on the client with hot reloading it's a two minute video that shows you just getting started, or more in depth courses. You need both. You need multiple different types of documentation you need the tutorials that walk you through really hand holding step by step. You need the API documentation you also need like the developer focus documentation but I think we're talking about just the tutorial level sort of like really hand holding click this button. Now you can set up VS code with hot reloading blah blah blah run this command precisely something you can copy and paste just to get people up to that speed. I think that local environment setup is would be a huge help in this in this because just going from cloning Alexi to being able to change something and know that the effects are seen is challenging for the first timers. That's a tough one because I'm sure everyone in this call does it differently. Right. That's fine. I mean, once you, you know, learn the tricks of the trade, do whatever you want, but like the very basic open source tools, things that are accessible. That's the ideal scenario. It just needs to be something that works on a vast variety of systems. Who, who on the call has tried the. Well it's been updated just recently but the, you know, developer integration with VS code where you can run and break points and things like that. I mean it's the only way I work. I would do it if I need help. And so that, well the reason I was asking is we have the instructions are pretty good for that and they're concise, and I'm wondering how we can make those more discoverable because they work out of the box and it's really, it's really nice if you know how to get there, I guess. And, and where is there and know to ask for it in the first place, right. I guess that's a documentation. There's a developer documentation like docs.galaxyproject.org documentation. And then the other half of us uses women PDB and silly print statements. So maybe we could do both for for a given topic for example how to debug galaxy how you do it with VS code how you do it with the shell. Sorry to interrupt. It's nice to have definitely different environment setups that work for different people I just wanted to show this graphic that I saw the other day. And we've got a lot of reference documentation, we've got some how to guides but we're missing the tutorials. And I think this was a nice framework for saying okay these are the different types of documentation a project needs and where we have gaps. I'll put the link to this in the chat. Thanks Elena. When as a mother new developer. The most useful thing I found was the, I think it's like galaxy internals GTN course that goes through a lot of stuff. It's pretty long as the server is pretty complicated at this point I guess. I mean, and it's, it's pretty long and it's still kind of skin some parts, but something like that, but even longer I don't know broken up. I know it just takes more time to write more documentation though. The problem with very detailed develop a documentation is that it's fundamentally different from admin documentation admin documentation persists. Because what we do with galaxy persists back end stuff developer stuff is implementation detail, which changes all the time. So, if we have very, which is not a bad thing if we have very detailed developer documentation going into details, it will be a considerable effort to keep it up to date because things might change on a daily basis of how things I mean, so I think maybe we shouldn't like go into that rabbit hole. But instead like it should probably not go deeper than what is in the galaxy internals. Because that's sort of the high level view, and then just, you know, describe one component fully in depth and you know the rest is the same like, yeah, you know, like, I actually think galaxy's gotten a bit easier, and like the concepts are all the same. So, you know, if you describe one API controller you can get started. It's all the same. Good to hear. Ask this question, but have we defined what developer training includes. Like tool development that won't change very often things like contributing to the client that changes more often but it's very important on board developers to have another type of building visualizations that's another type of training that's missing. We can start a list. Maybe we can start a list of these kind of things of what do we want to add to how to how to write a tool how to do visualization how to write a test, how to write different kind of tests. That's a great start, and that'll be a great jumping off point for the schedule for GCC. I'm thinking to include these in the standard GTN repo or so that because there already is slightly outdated but visualization training there. Yeah. We'd love to have them there. We're happy to help with any infrastructure changes you need things like that. So after this general introduction might also be important to identify like a good project for the for the person who just get started because having like a clean project makes it also clearer to what look at right. Yeah, that's where I really like clean goal. I mean, I think people might need assistance to find a proper project. And then we would also find a person maybe even to assist and then that would be probably. That's what usually comes out of the hackathon part of it that I really like is someone either has something they want to do or they're trying to find something to help with and then you can find that project to use as a vehicle for them learning like the paper cuts are probably a good list to pull from for that. Yeah, I like that idea. I mean the paper cuts are like they go from one day to two years effort rate. I think we didn't do a good job separating them. We need to relabel the two year effort one. That's not paper cut. We can de paper cut them. Yeah. The mentoring angle has come up in several calls in the past week for me so we're talking a lot about paper cuts talking a lot about the community in general, and like how to bring on new communities so like climate when climate came on. They had a, you know, specific people that were walking in them. Thank you Europe. But, you know, if we get more communities coming in and we should. Yeah, we should mentor them also new developers would be great to have a mentor. Once we identify what new developers are interested in. And we can point them to the right people. I think that would be great. Yeah. I mean, I think like, you know, if you get somebody from the community that has interest that's a good path but if you hire somebody that does it for a job I think that's another thing like there should probably. I don't know. I don't know what goes into that but like there should be a project and there should probably also be mentors from different teams. Yeah, you know they have more than one person to talk to. Yeah, and I don't think we can hire someone because we are so huge so broad. We can't get the expertise in one person. So, so I don't. Yeah. I can't imagine who that person would be. So, because again, everybody doesn't know everything. So, could this be a way to incorporate working groups in terms of like, if you want to figure out what people are interested in doing you. Find someone from a specific working group to be that, or maybe more than one person to be that mentor help them figure out their first PR or something like that. Yes, that's brilliant. What it requires that the working groups are set up in a way that, you know, I mean, we know about the project so that we are able to help, which is like departure from how we used to do it which is just do it, which is not very helpful. So, I mean, now we have our roadmap right but if somebody comes in in between the roadmaps. I mean, of course they can say well I want to do this and is there anybody here to help me but like. Yeah. Supposedly we have 40% of free time. But maybe there should still be a way to have that planned out. It seems that there are useful links pages videos spread out. So, for a new person would be difficult to know about all these so maybe the first step is would be to collect all of these and put them in a galaxy developer onboarding page so at least so for example john's YouTube videos are great. I had forgotten about them honestly I reviewed them I watched them involved back and I forgot about them. Or there are some documentation and galaxy project that are pretty good on how you set up a visual studio ID E debug test all that stuff. So maybe one would be to collect what we have already and put it in a page that we pass on to the next developer and said okay these are what we have. Next would be we obviously add to those. One other thing I don't know how useful this is but you know I'm up in here for a year now so not exactly that new, like some other people, but a office hour would have been perfect. I could collect my questions and then maybe once a week, we have an office hour, somebody can. I mean, I don't know how many people are there to justify this but for me, six months ago. If there was such a thing as an office hour I could gather all my questions come 15 minutes asked them and then that would have been very helpful to me. I'm going to chime in on the first one we've solved on the admin side admin training being all in the GTN. They just go to the admin page and there's everything they need to go from zero to 100. And maybe paper cuts would be a good idea. I think so you said for overlap with office hours, we could achieve something similar there. Yeah, right now that's once a month, and it's around the world so it's every three or four hours. Yeah, and we'd have to change the marketing slightly, which I would be all for. So, mentoring new students who have questions. Yeah, but Taiwan did not suggest paper cuts as a way to do this, which tells me something. So, yeah, I'm wondering how we can encourage. Sorry about the kids, not buffering. Not buffering stuff for office hours either like how can we encourage a sync first just ask the question as soon as you've got it, and someone answers, you know, I'm going to mute myself sorry. I think there's this also balance like, I don't think it's productive to wait an hour to wait a week or whatever until a certain hour, if you want to work on something. And there's enough people in enough time zones that you can ask a question and get an answer. So I don't know maybe our onboarding should also include just ask a question when you have the question. Sorry to interrupt. I think this is connected to to the mentor concept right so if you if you know that you have a mentor that could answer your questions. And at least you, you have some person to point to your questions instead of launching them in the void and hoping for someone to reply even if that most of the time happened. There's also like a social component to asking for help. I'm a relatively new developer and I am just starting gets the point where I'll ask questions publicly. I mostly will ping Dan and directly or john directly because I don't want to sound. I don't want to sound an experience I don't want to sound like I'm asking a stupid question. And so that's why having that one on one connection is really helpful. And I mean, I feel like if I were in a It's so much easier being in a in person office environment I mean, I just swing by someone and ask him a question. I mean, office hours would be great but you know, just being right next to the people who you can just shoot a question to is it valuable. But yeah, it is there is something to make one thing. There's a way to encourage maybe still going the public route. I mean, you know, you can do hybrid. I understand it's a problem. Like when you knew you want to show that you're competent and you can, you can work it all out that there's also so much time to be saved. Still, I mean, you know, there's also the question like you can get different answers from different people. I mean, it's in public. There's a lot of advantages if this is all in public. Yeah. I think, yeah, I mean, I guess when it's digital, like right now, like just throwing it out to the general chat room is efficient. And that's, I guess what I would do these days. But I don't know, I think as soon as certainly onto something and maybe argues for some kind of mentorship relationship. It's this is it's a tough thing to balance right because I think there's a lot to learn by sort of osmosis in the channels to right so like you ask a question and 20 other people see it and they're like oh I didn't know that now I know that for next time, even if you hadn't intended it so there's a huge community benefit to just asking. I don't really get the, you know, new to a project you might be asking a question to 1000 people on the main, the main channel and it's it's intimidating so there's something to, I don't know, I don't know, I don't have a good answer for this but it's definitely something we need to work on. Yeah, and from the, from the, like, as a person who probably answers more questions than he asks, it's like the 30 minutes that I like have to set aside for a meeting with a sooner each week is nice. I mean, it's like, it's, I mean there's just so many questions being asked publicly, and I, there was a time in my life where I had the like, you know throughput and social energy to do that but I just, you know, the world being what it is today I just don't and so I was like, oh, like, having that like explicit relationship and explicit time to do that, let's be focused on those things and sort of make them a priority to. But yeah, I agree though that there's real downsides and figuring out how to make it public would be great. I mean, the other thing is that you know if it's common questions to be consistent and then afterwards put it in some sort of how do I or like that you know you can have both you can have the private conversation, but keep the knowledge around, which was great about the mailing list right. I mean, you can still find all these old answers and a lot of it still applies. I mean, we lose that a bit with guitar I mean it's not a search room. No it's not. Although searches better now and get her but still not. Yeah, I can't get Google to crawl get her archives. I don't get it other projects can I but they should figure that out. The mailing list was good too because it was. Oh, I shouldn't admit this but it was like they were to do this like it was a tool list it was in my email is like I'll get back to that and it was like persistent in a way that. It's like healthy help is and I don't check that and probably should, but in the chat David said that you know the working groups feel like a little bit more like concentrated areas where you feel more safe asking questions and I have seen in the working groups I'm involved with some more specific technical questions and discussions and I had, like in the, like dev room or galaxy project room so I think those are a step in the right direction here. But yeah. It is where the groups. I was sorry, I was just gonna say I do worry a little bit that those are more isolated that the greater channel could benefit from a lot of the conversation, sort of this osmosis learning. But yeah, it's something to balance. Because a lot of people only sit in dev right. And they wouldn't know to go to the back end working group or this that or the other. You're probably joining a working group could be intimidating new people. It doesn't apply sort of that you take on something right. It's not just. I mean, I couldn't I guess, but I sit in some channels where I don't contribute right because I do want to see the things and I, you know I see the UI discussion even though I tend to try to stay out of it. It's just good to see so I think people should feel free to sit in on the channels at least. Yeah, maybe it's just knowing to sit in those channels. I don't think right now. So Bjorn mentioned the working groups being the primary entry point for new contributors and restructuring where that's the focus. And maybe that would fix that problem where you know okay I got to join, if I'm interested in UI, I'm going to join this working group instead of just sitting in dev or the lobby. We maybe alter the death description to link out to the rules. Each working group could if we do compile a document like a central landing place for new developers. Maybe that is contributing the contributing document in the galaxy we pull maybe it's something else, but maybe each working group can go through and sort of say like, here's a paragraph like what are your links, what are your, what are your key pieces of documentation for people want to contribute. Maybe that could be there. I mean we, I think it should be in contributing that's great. It's, I think we've kind of assembled most of that content already on the hub to, so we could just move it. So this could be just me because I'm the hub guy but in contributing people are going to have a harder time finding it. And it emphasizes that it's all about the main server, which I don't think we want to emphasize. So, I see everything through a hub lens. It makes sense. I never go to a hub. I know which video see there's just work on you Marius. Start putting jokes planted here and there so just to check that everyone's on board for we need training for GCC. Three days or so. I'm not sure to me if that's a lot of feeling of agreement on that topic before you segue it into onboarding. Yeah. So the people who expressed opinions said yes but not a lot of people expressed opinions is that a fair summary. Yeah. I mean, my, my preference would be to scale up the architecture training. I mean I'm, again, thinking of lenses right I do the, you know, I spent five hours doing the architecture training last year. It's nice to sort of, but that was really just what it's three or four hours of content if we could like, say what could we do if we're going to double that. What would that look like. I mean we have a whole day of tool training, at least a whole day of tool training available right. And there's the visualization training that's another two and a half hours API. We've got a lot of the pieces here already if we just want to sort of repackage it and fill out the missing fill out the gaps. And make sure they're updates and things like having training on setting up VS code with hot reloading I think would be very key to all of that coming off successfully. Yeah. I have flashbacks to contributing to galaxy workshop that four people came to about five years ago. I think about that a lot. John I would also second your overview document or view training. I found that useful when I first attended GCC I see comments in the chat about that too. It's definitely something we should base the developer training around that is the first introduction here's how it all is laid out and then individual trainings on specific topics, which work really nicely with asynchronous mode because people can just choose which topics they're interested in and go to those maybe around the content piece, it would be nice what I mean the architecture training is a little bit more reference than hands on. So I think maybe a key piece that's missing here and that I remember really seen is maybe we should just have like a go to example like adding something to galaxy that like doesn't actually get merged but like everyone adds a controller that does. I don't even know what that controller would be right. But like some some example that everyone could work through and get to that last stage where you like see the PR but then just close it. We obviously have that for tools. I'm assuming we have that for visualizations. I mean the API has a bunch of specific hands on examples. So all of those developer pieces have that but the code contribution piece. I don't think has that and that's a good it's kind of hard to come up with a good example that's going to stay relevant and stay updated but Yeah, I know the feeling we have a tutorial for the training materials which is how to contribute and focuses on things like get and making PR is that sort of thing. I'm not sure how useful it is. Do you have a feeling. No, it's useful. John just said something that prompted a thought in my head. So you know, I don't even know what controller what I add so what prompt what would prompt the developers to either attend, or, you know, take this training, or ultimately contribute to Are we targeting new people that are being hired by either galaxy or related projects and this is going to be their job, or is it people that are scratching an itch in which case they would presumably have a controller they want to add. So for admin training we just teach them no matter if they're working for us or not. I think it has the most social benefit that everyone learns how to contribute. We're not getting something out of it right even if we're not getting a trained developer or someone directly contributing to controller. I mean like the, the, my thing is one class of users here or one class of consumers of this content has long term benefits and they want to learn a comprehensive process for how galaxy code is laid out the architecture. And they are willing to invest, you know, potentially days of training to get up to speed. The ones that are trying to scratch their itch are like, I'm just more annoyed than annoyed by having to spend an hour or so setting up a local environment and fixing something quickly. So I think those are actually like two target audiences here. One we just need to enable them to set this up super quick, solve their problem and move on and be happy. I think I think we have material for those for that type of audiences just the two minute video they can watch scratch their itch be done with it move on. As for the controller. I don't think the point is that this is a very specific control that is going to be relevant to this that person. The point is that the controller, possibly is a good, a good vehicle through which you will be able to teach the different parts of changing something in galaxy or adding a feature to galaxy so a controller would be a vehicle for backhand adding exposing some functionality through an API so how you would add an endpoint to an API how you would test an endpoint and API and API how would you test this new functionality you're adding and then how you would expose it on the front end so it's just all these different parts and the control is just one possible relatable vehicle package. So throughout another use case or another potential audience, which is researchers. I curate papers and I come across a lot of papers that extend galaxy for provenance or a different way to do workflows or you name it. And that's actually a fairly common publication type, where they're making some significant change to some underlying plumbing to see how it works and make their argument for their thesis. So that would be another group of people that we would reach is researchers who want to use it as a platform for research, computer research. Do any of the other PIs want to voice support for this, or rejection, that's not important. Sorry, I talked to you Daniel. No I said it's a great idea I yeah we need more documentation we get it all arranged we need easier ways on board. I think what Anna said is correct you need people need the deep dive. We also need the quick start right so yeah. Anton that sounds good to you to see still here. Of course I'm the one who still hasn't give you videos for smoke. Yeah, about that. So in principle yes. In principle yes because we have infinite resources. I'm, I can't find it now there's a, there's a, like a really hot app right now that like you develop a whole, you put a whole developer environment into a container and attach it to your project and get hub and you can launch it. Are those called workspaces. VS code has yeah integration with its VS code workspaces or GitHub workspaces. Yeah, VS code. I think it, it might still be beta invite only. Yeah, it's better and I've tried it though it's really, really slick. That might be the scratch or particular edge version of this right. I mean, one thing that the admin material is doing well is that it's streamlined with the answer but playbook so yeah maybe the code spaces could be like the thing to get started without passing the setup and stuff. Okay, so we're at 10 minutes left. I'm going to try and summarize. I think there's not an overwhelming consensus but a consensus that we should do some sort of depth training. We should go after low hanging fruit like actually centralizing what we have and creating easy ways to find it, not centralizing everything in one place. We have engineers in one place you know have a landing page for new devs which, which makes sense. Mentoring is big. What else. Okay. What I propose doing is that the five of us or the three of us depending upon how interest is that presented today. I hope a summary and distribute that let's see what I want to suggest for all of you is get a hold of us if you're interested in pursuing making developer onboarding better. Okay. In a time commitment sort of way. I'd like to volunteer Sergei for that, just since he started coming up with a good list. I'm all for volunteering Sergei for pretty much everything so Sounds good. That's on tape so Okay. Okay, in our remaining well yeah I mean we can end early to that's always a win. But we got nine minutes left does anybody have anything else they want to talk about say make a point. Okay, I'm going to declare victory then john you can stop recording. And thanks everyone for your input.