 All right, take it away. OK, welcome, everyone. We have a long agenda today, but we wanted to start off with some recent updates to the Galaxy Training Network that you may or may not have been aware of. I'm going to share my, John, can I share my screen? Is that a thing that can happen? It seems to be an option. Yeah, one second, sorry. No dramas. Should have. Now you should be able to. Awesome, OK. So thanks to a lot of work by Delphine and James's lab. We have automated slides to video conversion so we can turn some nice notes into slides, including subtitles, which don't display in this for some reason, but they're there. I promise it's super nice. It's pretty easy to work with. We could tack it on all very existing slide format. It was great. It only works for slides, though. We'd love to figure out a way to do it for the trainings, but we haven't gotten quite that far yet. The other feature is the new fancy little button on Galaxy, which some of you may have noticed, to see the Galaxy Training Materials. And within there, we have linked tools. So you can click on them and activate the tool within Galaxy, which is life changing for a lot of our trainings. We had a lot of issues of, oh, well, which cut tool is it? Or you paste in the tool name and you can't find it via search. So this fixes that for us. So that's been a cool thing. We're also planning a global Galaxy course in February. You want to talk about that? Yeah, so the idea here is to do a global course. So we were going to prerecord all the sessions. It's going to be five days. It's going to be free and open for anyone to join. So prerecord everything. And then on the week of the course, we will be available in Slack to help people out. And all the participants can work through the materials at their own pace, so manage their own time. So it'd be great if people would be willing to join the Slack and just keep an eye on if anybody needs some help during that week. Another thing that could be nice to think about is if we can make at least all the usegalaxy.star servers be able to support most of these tutorials so that people can go to the different servers and not all overloading one of the servers. Yeah. Yeah. We're also planning a global Galaxy admin training, which a number of you are involved in, in the end of January as well. And that's going to require some creation of videos for students and also is going to necessitate a lot of admin helpers. So if any of you are willing to volunteer to answer questions, we would love it. OK, with that, I'm going to stop sharing. Have you picked the topics for that course? Which course? The global Galaxy course. Yeah, so this really started as a small local course, and then the pandemic happened. So it sort of grew on me. But the first day is going to be a standard introduction that we often do, just introduction to Galaxy for newcomers. We're going to do some quality control and mapping. Then we have a day of RNA-seq, single-cell RNA-seq. The fourth day is going to be proteomics. So the Galaxy team has agreed to do that, and Proteor and then Melanie. And then the last day, I wanted to make sort of choose your own adventure day so that everybody in the GTN who wants to show off their tutorial can, so they can make a video. And then we supply the participants with the options you could do today. If people don't have time to do a whole video, because I know it can be time consuming, they're willing to be on Slack on that day. Then we can say, here's another set of tutorials that we know will be supported. And if people want to do any of the other ones, of course, they also can. So yeah, I was hoping to show off. Just care, you used to have a document with the outline. Yes, I have a program. Do you have that? You don't have it, but I didn't see it. So we can paste that in chat. Yeah. And I know Anton has kind of agreed to do the introduction portion. And Simon will do some assembly, he told me today. So I really want to show off like instructors from all over the globe. So if anybody here has a training they would like to give on that last day or one of the ones that are scheduled, feel free to let me know. Perhaps a stupid question, but do we have the date set? Yes, it's February 15 to 19. How soon can we advertise that? I was hoping, but it's by hackathon this week. But I'm going to make a little website with the global outline of the program and set up a Google forum for registration. So then we could, I think, start it. Thank you. And I will, closer to the course, organize some instructor training or helper trainings so that people know how to help out. And I just want to comment then. And choose your own adventure idea. I really liked, which is why we're also doing after the admin training I stole it. Dave, so you put this on the Galaxy event schedule. Can you link that document? OK. Yeah, so once it's ready. So once Askia says go, yes. So I'll make a little web page, I think, maybe in the GTN or maybe somewhere else with the program. And then we can update it. So I also wanted to ask Alex, oh, are you here? I am. Yeah, so you were coming up with this design of web pages for introduction. Are you talking to Saskia and Helena? I have not been. I mostly have been talking to Dan. And this is in relation to a PR that they are aware of. Or rather an issue that they have both commented on, if I remember correctly. So it's the one that's got like a whiteboard picture from Dan and James and Anton. I got put on that one. I have a question you're referring to. Would you mind reminding us? See if I can find it. Give me a moment. There it is. It is called New Welcome. Overhaul users' first experience on the main one. I don't know that I can share screen. No, that's fine. I can find it. What was this in connection with Dan, Tom? That's what I'm trying to figure out. I just want to avoid a situation where we're doing various training related things and not aware of each other. Good to hear. I can share. It's just a landing page when a user first creates an account. So it's like a quick start, get into Galaxy, and then getting to know the Galaxy community tools and workflows is just a quick intro. Fantastic. Here's someone's working on that. And it would be, yes, very helpful from a user training perspective to have more graduated introduction. Then here's the whole thing. Have fun. My question. So when you plug in those videos, where do they go on the GTN page? They go to an S3 bucket. Current. No, no, no. I'm not talking about storage. I'm talking about where they actually get displayed. I mean, go to the introduction. Let me share my screen again. There is one now for the short introduction slides I made. And there's also one in the admin training. It's just if there's a video icon, it'll appear next to the slides icon. All right. So I have a question. So I'm making a bunch of narrated videos as well. I haven't publicized them yet because I want to get some critical mass, maybe about 10. So right now it's about data upload and it will be explaining collections in depth. So I'm making about one a day at the current rate. So I'll have substantial amount of them by next week. What would be a good place to put them? Of course, we have a galaxy video channel. It's called galaxyoutreach at gmail.com. That's the channel. I mean, it's not publicized yet. But that's where I'm going to be dumping them. But I actually hate YouTube interface. I would rather embed them somewhere on the GTN page. Should we create maybe a separate page for videos where we can group them? This is about upload day, uploading data about collections. They'll be very short. They'll around three minutes max. So we can definitely think about doing that. I would have been more against that, except for something Dave said to me the other week, which was if you call them webinars, no one expects them to be updated. Currently, all of the videos on the GTN get automatically updated whenever changes are made, whenever we update the slides. Well, you were talking about videos which are made out of markdown. Yes. But I'm actually making these videos the old way. I understand you're talking about something very different. Yeah. The point is they will get outdated eventually. Well, so the new thing here is that before when I was doing this, I was making them relatively long. Right now, it cannot be more than two, three minutes. So if it's a subject which is more than two, three minutes, I just split it into multiple videos. So actually updating them in a way easier than updating markdown for me. Anton, can you show one? I think no one knows what kind of video you are talking about. You said that kind of while it's in talks, I think. I'm pretty sure I've seen some on the Fimeo channel. Here's an example. Well, you guys continue talking. It will take me a while to find it. Yeah, I think the concern here is just the naming of what we're calling them. Yes, I'll do that. Well, so this is all your things, webinars, or something other than the videos. These are not webinars. So I think that the automated videos, these are like a little snippets about particular interface elements. Then webinars, that's a whole difference because these are long ones and these are more of a We'll call them webinar apps or something. Or webinars. Shorts, maybe. Sure. They used to be called quickies, but in the new world, we have to be careful. So let's name them. Yeah, that's not a great term for them. Yeah. I know. I'm just kidding. I like John's proposal. I second that. What's the John's proposal? Okay, thanks. Anyway, so we can bike shed the name later. We would be happy to have them. We can definitely come up with a video, a place to put all of these videos that are not automatically updated. Maybe just every paper cuts, co-fests go over them and see if we need to deprecate any or re-record just so they don't get too far out of date. Because that's the main concern, I guess. It'll be okay. Oh, I hate YouTube. Quick dives. Also a good suggestion, Dan. Yeah, well, that's what I named the ones on the... Excellent, but then too much credit to Apple. No, not quick time. Quick dive. Quick dives, okay. I don't know. That's what I called the one. Well, I've only made one, right? But I was going to make some of the developer-focused ones. That's what's on the YouTube channel now. All right, here's the one. 39 seconds. Good. Fits in my attention span. Basically, it's basically about... So this is the new front page for them. And it's sort of pangalactic. And then it's just, you know, it's just how you do shit. That's good. And they're all going to be in that sort of time. They'll be short. So actually they're easier to update. The problem here is I'm making this mistake again, where I'm bottlenecking stuff on myself, but I like making videos. So that's... So I guess I'll just do it. And I think our long-term solution to that is that Freiburg and Oleg are working on automatic video recording, but... Well, the problem... So the problem, and this is something I also discussed with some of you, is that the automatically generated videos are fine, but they're a little dull just because of that voice. You know, we can mimic your voice. We can make a nice Anton deep fake. No, I know. That's fine. That's fine. And I said, yes, there is a... I don't know how many of you watched this deep fakes from South Park, people. Yeah, that was what I was thinking. So I don't know. It's just... Yeah, it's... As I was saying, one of the highest, one of the most successful screencasts I've made was poorly edited. So it was, I screwed something up and I was saying, fuck, repeatedly in it. It was very successful. I mean, people just loved it. So that's the thing. Anyway, so my only question here is that if you can tell me or think of some special page within GTN where I can be adding them because I don't want to just use YouTube interface No, no, no, we can definitely link those also from the relevant tutorials which cover the same topics. Should be easy enough. So next week we have the GTN Co-Fest. So we can put it on the schedule for that to work on it then. Can we get a raise of hands though on this no YouTube thing? Like who is likely to just watch it on YouTube if it was on YouTube? Like there's some reaction. I mean, if we add it in the content, people are going to find it there rather than finding it on YouTube. We can still have it on YouTube, right? Yeah, you can embed YouTube in the page. GTN just for findability, you can still. Yeah, I think YouTube's great. It's free, there's no, I don't, I don't know what the black people wouldn't like it. Google? Who does he use a Gmail though? I mean, right, that's if it's sailed. Sorry, don't get me into that anytime soon. Yeah, we can embed YouTube, it'll be great. Everyone will be happy. Essentially, if we continue this discussion, we're going to have to make our own silicone because we wouldn't like Intel or Apple or whatever. I mean, we can use the new built-in video format in Galaxy and just host them on galaxy.main, you know? Okay, more bad suggestions for me. So back to our agenda. One of the next topics on it was the GTN and Galaxy web hook is nice, but it's probably not the final answer for how to integrate training with Galaxy. And there are surely better ways we can do this. One of the main concerns for us is that workflows don't stay updated. And how can we maybe combine training with workflows in something like how the workflows have a report section? Could we also have a training section? Does anyone have any feelings about this idea that they want to throw out? It's a long-term plan, obviously. Feelings are good. Yes, so it's a good idea. Everyone forget their coffee today? So what exactly is the problem with the webhooks? Like how do you want to extend them or what's missing? I'm just curious. Just that the training doesn't stay tied to the workflow. It's the main issue for us. We write a training from a workflow but then no one ever updates the workflow unless we ask them to. And we also lose connection between this tool with these parameters. And here's a textual description of the parameters. If it's some format that's closer, maybe we can generate the list of parameters that you have to send. Maybe. I mean, it sounds like an interoperability problem. Like I guess you would want to find for a tool workflows and trainings and then sort of integrate that. And relatedly, we should have easier ways to update workflows to new versions. I mean, that doesn't seem beyond reach. We can probably do that. At least for repositories that we trust, we can probably do that. The workflows, I mean, especially when they're in the testing, you update them and if they pass, that's fine. It's a nice hack project, I think. OK, no negativity. Extend the tours somehow that we don't have to always. Yeah. What to test them for. But I don't know how the tools work at this point. Yeah. And it's worth an attempt. Yeah, that's worth thinking about. I know we had talked about it in the past too, using tours as a vehicle for walking through this stuff. Just never sort of leveled to the top of the pile, I guess. Paper cuts. That's more than a paper cut, probably. I'm missing hand. Yeah. You talked about the idea a lot a year ago, as it goes with the last Biohackathon. One thing that came up is that the invocation reports right now are like a document. It's like one big thing. But the way that you probably want the training reports is like, here's some markdown for each step. And probably just the relevant steps, right? And then you could, those could be the things you highlight in the tour or the things, I don't know. Yeah, sort of different paradigm. But the idea of attaching the markdown to the workflow is still really solid, I think. And you could reuse the widgets and stuff. And I mean, more markdown in workflows is always good. I think the instructions we can provide today with workflows are very limited. So if we can extend that, that would be great. Well, if anyone is excited about working on that, let us know. We'd be happy to, Liaise. OK, if there's a lot of silence on this, then a long, long, long time ago, John proposed that we have somehow met a tracking issue for the GTN and our priorities and what we need, what's important to us. I went ahead and made a project board. Yeah, long time ago, John said, hey, maybe we should just say, keep track of our paper cuts and can't promise anything, but it could be useful to help us. So does a project board work for everyone? Also, if we maintain this, will anything come of it? I mean, I will, so my, just some background about why I thought it was a good idea is because, like, a lot of developers don't use Galaxy, right? And so if we are fixing issues that we can see in training, then we know those are user-facing things. And so they would seem to be high-priority, roadmap things, sort of like paper cuts, but maybe even they could be a little bit larger. And so, I mean, if you maintained it, the thing I could promise to do is review it before it's like, if I'm still involved in roadmap discussions or whatever, definitely at least, you know, monitor it and sort of see what that is and sort of have a better impression about what, you know, a broader range of like real users who are not just users, but users who are like, you know, on the ground, writing content and building content for users, for other users. So I think it's a great idea. But I mean, thank you, thank you for throwing it together. Sure, we're happy to update this at the CoFest if it's going to be useful for developers, if it gets on Anton's radar and the roadmap. So what are the big, you wanna like walk through what the big issues are right now? Sure, tool panel search. Marius, how was, you said you had a bug? Or you were finding something in Woosh? Yeah, but still trying to get the release on and this has been a bug forever. So yeah, I mean, the weights for, you know, we use this multi-weighting thing. In some situations when you have a good match, the weight just goes to zero and it's multiplied. So you multiply by zero and then that's it. So the best match falls out, which is unfortunate. And didn't have time to look further because that's a huge... Issue. Yeah, a lot of math. Can you replace zero with 0.0001? Sorry. Can you just allow zero in weighting? I still didn't hear it. So that it never gets multiplied by zero. I mean, there is some math problem in there, but it's not like somewhere in the code it says multiply by zero. I didn't understand fully what the problem was, otherwise I would have taken the note. Okay. EU took all of the weights which we had set to 40 and set them to 0.1 and things are for some queries a lot better. Multi-weight queries especially seem to work a lot better than they used to. Others work badly, but I don't have data on how they worked before. More visual cues in the GUI for options that are only available when logged in. This is one that constantly catches some of our users. I had an issue where, yeah, someone else was complaining about not being able to rename a history only to figure out later that they're not logged in when they finally got around to screen sharing. All of these sort of things could be, could make our life a little bit easier when we're training. There was another user who had some strange bug about they hadn't had their account confirmation approved yet and it was approved in the background, but then they tried to run tools in the meantime and the tools wouldn't run even though they were approved and they had to refresh their page. And when they did refresh their page, everything worked. And these sort of things have been causing us troubles lately. So those are our high priority ones and updating, automating the screenshots, but that's an ongoing project I hear. We have lots of low priority ones and some pie in the sky ones like the integration of training or TOS support on all of these galaxies and that sort of thing. We threw this together without a lot of discussion. We'll update it at the next co-fast. Hopefully it gets more useful for everyone from there. That's awesome. I think for the, especially for the paper cut kind of issues, it's a good way of, I mean, even those ones are like, that's a very clear paper cut and good user story. Thanks for putting that together. Yeah, definitely the tool panel search should work. We can ask every GTN co-fest all the instructors because when you watch someone use Galaxy for the first time, you see a lot of these sort of like little UI pain points. So we can collect them then from all the instructors maybe. Yeah, it's super helpful. Okay. Were these also linked to issues in the core Galaxy repository? Like if they, okay, cool. Thank you. I don't think I have the board automation or whatever turned on, but yeah, they're linked. Okay, next up is GTN events. We would like to start thinking about synchronizing events. Dave, are you still here? Yes. I wanted to make like an events page on the GTN itself and have instructors because they're already used to adding stuff to GTN, add their events there. And then maybe the hub could work more as like an aggregator from different places. And I saw this was proposed before, but if you're in I think, so then like the EU could have their own list of events, the GTN could, other places. And if we just decide on maybe like a minimum set of metadata for the hub to maybe, how would this work? So should we use the bio-schemas JSON-LD as an interchange format because we published that already so tests can consume it? Yeah, that's right. Do we just want to delegate the entire problem to tests? They collect events. I've never actually looked at tests events page before. Yeah, so our code needs to be updated because it's not quite a perfect match anymore, but... We trust long-term to... We just got added to the test club. They've told us they're going to support it. Before that test is done, I don't know. We've gotten a lot of mixed messaging on tests. Okay. I think the bio-schema is JSON-LD. I don't know, off the top of my head, I think that's the best way to do this. Okay. So even if tests gets mixed messages, but we need to update our code as well because we haven't kept up with the spec. But yeah, pulling from GTN would be great. We're currently looking at redoing the website. And so with an eye towards aggregation, and when we were talking about aggregation, we would aggregate events from multiple community sites. So for instance, Galaxy Project EU, we could pull in stuff from there. Fantastic. And vice versa, all based on tags. So that's what we're looking at. It's early days for that. So Anton hopes to get that out within the next six months. We'll see. Where's Anton? There he is. Okay. Okay, cool. Good to hear that there's motivation for that. Let's encourage him for us. So while you're talking about events, it might be really cool if while you're just browsing through tutorials, if it said the next time one of those would be presented, right? People don't track that sort of metadata in their event. No. Good. You could ask. If we do it in GTN, then we can say, if you know the identifier of the tutorials, you're covered. You could do something like that. That would be a nice feature we can add. How do you envision aggregational work for like other communities that don't necessarily follow our standards natively? Let's see. If they don't follow our standards natively, I don't know. Like front of Galaxy community or something. Yeah. But if we... So a couple of ways. One is we're hoping to create a generic platform, which will be used by the three big useGalaxy.star communities. Okay. So that's in the works. And then we're also hoping to create a platform that can be used by useGalaxy or by GalaxyProject.fr if they want. Okay. But if they don't want, I would say we push the JSON-LD format and just say here, you know, just generate this behind the scenes and that will make aggregation easy. So that's not it. Sorry. Go ahead. Sorry. Please finish. Yes. I don't think that's a ton of work. You know, it just requires some JavaScript knowledge and yeah, I don't think it's a ton of work, but I don't have that JavaScript knowledge. So maybe it is a ton of work. I don't know. I was going to ask, what did you mean by a platform that could be used by? So hoping to create just a generic software package will post on GitHub. And then if people want a look and feel that is similar to the use, to the galaxyproject.star platforms, then they could easily do that. So if you want to launch a Galaxy India and you don't want to get your own, create your own platforms from scratch, we'll have one which will automatically integrate things like events, like people, like all those stuff that's directory-based. Sort of like how we did on Galaxy Europe except everyone contributed to the same thing. Yes, not that we're stealing ideas from you guys, but yes, we're definitely stealing ideas from you guys. So exactly. So in theory, that will be attractive enough that people will at least consider it. Now there's gonna be people who just hate stuff about certain design choices that are made and they'll just go their own way. But if we can create standards for how to export stuff then anybody can conform to that no matter what their website looks like. That sounds good. Does anyone want to complain about the trainers they work with? Like, you know, I expected more discussion on some of these issues. Any, I hear mains finally getting around to Tios maybe. That's exciting. Yeah, Sergey, you want to talk about that? We are in early stages. We have our own fork, so we are developing. It will be the grand idea, which is again still very much not set in stone. The grand idea is to have one code base. One code base which will be customizable to hopefully 99.999%. Well, and the rest, I guess, will be in the config. I'm sorry, I have double noise coming from someone having lunch here, my apologies. So right now, we are building, well, Helena has built the Django app. We are just making some adjustments so that it's customizable because a lot of EU stuff is hard coded. And also we may or may not have our own requirements for how we accept registrations for training. This is something me, Nate and Jen have been discussing lately and hopefully we will be discussing it all together very soon with Helena. And once we have an idea of what our combined needs are, we'll figure out how to generalize the code base. So it's again, it's one, one app to rule them all. That's the idea. Yeah, cool. So what's the backend on the.org site you wanna look like? The backend as in the dynamic job destination? Is it, I mean, is it gonna be like a separate app or I forget how Tiosk works? All it does is it puts users in groups. That's literally it. So they still use EU's main server. Yeah. Okay, all right, yeah, okay. Well, we will be, it is my understanding that our app, the Django app, which accepts registrations and also which, what am I trying to say? When the user accesses a URL and is logged into Galaxy main, the app will read the user's Galaxy session cookie and add that user to the, actually create the training group on Galaxy, in the Galaxy database as needed. And then we will have a dynamic job destination in place which will route the jobs created submitted by this user to designated resources. How that is going to be implemented. It's something we will be discussing with nature. It's long and short. It just puts users in groups and has registration form. It was intended to be a stupid of a service that could be possibly made and everyone can make their own choices on scheduling. Sorry, I tried to do my best. I have a soundtrack and a background here. My apologies. Okay, continuing to pick on people. Anas, you said something about having a security lecture for an hour. Yeah, sorry, go ahead. When do you think that'll be ready? I'd love to see it before. Yeah, so it's being worked on the first half of December. The document we have is going to be, I guess, reviewed and the engagement with trusted CI is not gonna be done until first week or two of January. But I just commented, the guy that's largely assembled the document is willing to do the talk and put a sort of an outline of the talk by early December. I think I wanna say maybe like December 2nd was even the date he suggested if that's a workable timeline. Yeah, that's work. Will there be a tutorial to go with it or? No, no, so the idea is, I mean, there's the document itself that will be made available as a resource to the community but the document is more like a template that each site has to customize and yet it's a 60 something page document. And so the idea would be to talk about the document in sort of a structured format so that a site when they choose to adopt it has a bit of a background why or how to go through the document because it's not really feasible to do it, I guess, especially not in the context of the training. It's a lot of organizational stuff rather than technical. Yeah, yeah, of course, just. All of the trainings we'll be giving have associated tutorials that people can go back and review with their own convenience and text they can read if they can't understand the speaker and that sort of thing. That was mostly my concern is just that everyone can access this. Well, the document will be available once the document is available. But I don't know, I mean, if you have ideas on how to turn it into a training I can take a stab at putting it together. I just, I mean, it's read a document and make a decision that's based on the organization needs and abilities. So I don't know what. Okay, we can discuss that then whenever it's available so we can have a more informed conversation. How is the end of our agenda? Really expected more discussion like how far off track we got last time with the tools. But I guess tools is a topic that's a lot more applicable for many of you been training. He's right. Go ahead. Sorry, I did have something that I had discussed briefly in the getter but might be worth throwing out there. I don't know if it was related to who's in the meeting but I have a long-term target to rewrite several of our runners and it's gonna be using a template engine and Galaxy only use one that ships with Galaxy already. It looks like you guys have three template engines in Galaxy. You have Mako, Jinja and Cheetah. And I don't know if we wanna pick one. They've been working to converge. Is there a solution that we're aiming towards? In the getter, they said Cheetah might be good but I don't know if that's... I guess it depends what exactly... I would say it's safe to not use Mako though. The Mako's likely mostly going away. Yes. Between Jinja and Cheetah, I can't imagine we're getting away from Cheetah anytime soon with all the tools that use it and Jinja's pretty standard. I don't know, we use it in Ansible and all that. Is Cheetah maintained anymore? I mean, it had a push for Python 3 compatibility but since then, not much. Yeah. And I mean, it depends on the target. So what do you wanna do with it? Well, the idea is that the runners will have a template file that can be customized by an admin. So let's say the Kubernetes runner, the Kubernetes YAML document that's submitted to the Kubernetes API can be templated. And then that opens up the possibility for any job configuration, including sidecars and anything rather than trying to implement incrementally every single bit of the Kubernetes API through runner programs. Yeah, I just wanted to say I started work on that. So that is in progress. Hopefully in the next couple of days, it'll be ready for a PR. I was actually thinking of taking it a step further with the, and standardizing so that I can integrate a Nomad runner and also redo the Docker runner. What template engine are you using for your PR? Ginger, but if we can work on it together, then maybe we can try to coordinate that work and maybe I can push what we have right now or something or we can just have a chat offline. Sure. Yeah, I would say ginger is totally safe. Like that's no issue having that as a dependency. And yeah, even if she does end of life, it's so central to what we need. It's what we would maintain it ourselves at this point. Nobody else has anything else. I have one other idea that I wanna throw out there. I don't know if this isn't a greater, I don't know who's involved in runners. The idea of pulling out galaxies, job preparation and metadata into a sidecar so that it can run independent. So like it's a smaller container in itself that can run before and after the tool container and it pulls data out of galaxies database and pushes it back in while also doing the metadata calculation and that allows removing the need for a shared file system. Pulsar does this, right? If you submit a Kubernetes job with Pulsar, it sends two containers, I mean, it runs two containers. One, it's the bio container that runs the job and then there's the Pulsar container that stages the data in, sends the data back, runs the metadata calculation. I always recommend people sort of not assume the staging data in and staging data out. Is it... Like I'll see as soon as it's got access to those files. So I really think that that should be built on Pulsar but it's a big, heavy thing. I understand like sometimes you've got to rewrite things but I would definitely take a look at the Kubernetes Pulsar work. There's a test case in the code, like test Kubernetes staging. I mean, I would take a look at that. I think that's the right way to do it. And we've already extracted the library that does the metadata calculation and stuff. It's all been sort of siphoned off and packaged up. And so it's got the smallest part of Galaxy that it could have sort of attached to the Pulsar inside the Pulsar container. Yeah, that was like, I mean, so I would take a look at that because it sounds a lot like what you're doing. It doesn't talk to the database though, would be the one difference, I'm trying to sort of... We're also, I mean, that's also on the roadmap. I think the part that is prior to actually, I mean, the part that takes a request and turns it into a job, that part is also something that we could put and process on a message queue, right? So that's also something that we could push out of Galaxy. Yeah, there is an issue. I'm sure I have an issue. I worked on this for a year, just on the other side, the metadata calculation and extracting that out of Galaxy Core. I think the inside piece also could be extracted. It's a huge project to do correctly though. But yeah, I mean, this is part of a project we wanted to be able to take Galaxy workflows and make them into CWL workflows. And so that requires taking everything to do with the calculation of the command line and the parsing of the tool and all of that stuff and making that a node in another workflow engine. And so that really requires doing that. And so there's work in progress on that and there's an issue on that, but I, yeah. It's a... Is there a way to unify this across the Kubernetes runner and the Pulsar runner? So we can use the same two container staging. I mean, I think that you should just use Pulsar. I mean, the way to unify them is to extract all the common functionality, I think, out into the PyCube library and put as much as you can there. And then like, I mean, with Pulsar, you have the option to turn off whatever you don't want, I think, right? Nate's not on to like contradict me, but so it should be a super set of what you can do with the Kubernetes runner. So I don't know, that's my extremely biased opinion, obviously, but yeah. So how does the sum of this, like at the slightly different level, like a lot of effort is being put into iRods and iRods has the object store plugins and then this whole two container job pods that are resolving their own data. How does those two, like today overlap? Is it one or the other? What's the fit between this? So the two container job pod ships the Galaxy object store. And so whatever the underlying object store does to stage the data and can be done on the container. And so I think it would lend really nice. I don't think iRods breaks anything in that model. I think it would work just fine. But it doesn't, so in that world, so the two container or the object store from Galaxy would just, we would just use the iRods configured iRods and iRods would do the data transfer between an object store and the local file system. Is that the architecture there? Yeah, I would imagine that. I mean, I obviously haven't put those pieces to get verified at work, but I think like from a Galaxy architecture standpoint, I think that should work fine. And if it doesn't work fine, it's a bug that we should fix it. So as a follow up, is there a, I mean, can we, is there a reason not to adopt Pulsar than as the standard job runner and push everything into Pulsar? I mean, the only, I mean, at this point, the Kubernetes runner is more, it's got more eyes on it, right? But I mean, Nate's now using the two container drop pods for the ITs on main. So they, I'm sure they'll get a bunch of bugs there, but it seems like the superior way to do Kubernetes because now you've got a way to sort of put some logic on the remote node and it's logic that's well tested and it's with Galaxy and yeah, I mean, the only reason why you wouldn't want to do that is because the Kubernetes runner is better tested or because there's too much overhead. And if the overhead is really a problem, then let's like figure that out and work on simplifying it and not simplifying it, but like get the performance better. But I would, yeah. I was wondering about all runners in general and just have Pulsar as the key. I mean, I would love that, right? Nate and I talked about that sometimes. There's, yeah. In general, Slurm, PBS, et cetera, there's a Pulsar backend for all of those, right? And so, and Pulsar can just be started up right inside of Galaxy with the embedded Pulsar runner. So it's not really, it doesn't even sometimes, I don't know how people envision Pulsar, but part of it is just like a client library or not even a client library, it's a library for staging the data in and out and sort of creating abstractions for saying like, these paths go here and this way and that way and so, yeah, it's, it would be nice if all runners are Pulsar runners. It's just sort of momentum that prevents us from doing it. From a design perspective, if we were starting over, obviously, yes, they would. But, yeah. I mean, so all the command interface runners, they are pretty much the variants from Pulsar, so there should be import stuff from Pulsar library for that. So, I mean, those are almost there, right? We could easily move them out. Yeah. I mean, we should move in that direction, I don't know. It's just a sort of legacy, you know? I mean, the last thing with a lot of the runners is that they are conveniently contributed. So in a sense, I mean, if we create one migration show how to use that with embedded Pulsar, that might be a way forward. If we want this to happen and we don't want to spend the work ourselves. I also think, yes, that, but we also probably should be using embedded Pulsar or something on main and you and stuff, right? Like we should, you know, I mean, I think Nate's not here, so I can just put words in his mouth. You know, he agrees that all runners should be Pulsar runners, right? But we haven't done the work to make it happen, even on main. Great segue, thank you, John, it would be, we need more training on that. How to use this new embedded Pulsar. It would surely be helpful if someone would write some admin training on that. Well, Nate's not here, so it's my nomination. It's Nate, the only one who can do this. I think so, I don't know. You can't say no right now, that's what you're talking about. Yeah, just looking at all of your faces, I would love to see more contributors to the training materials for the admin in the dev section specifically. There's lots of developer education that we could be doing and we could be highlighting there that we'd love to do. We just need developers to start contributing to that. Is it completely out of the question that we would do it in a way that there's nothing to configure? Like we just make the switch and that's it. It's a good idea. I mean, not everything needs to be configurable. So if one of them is better and there's no versions, then switch it. We've done that with a couple of things, so I don't know. I don't know if it's feasible, but if it is, then maybe that's the way to do it because why have something configurable if one of them is better? Yeah, it's a good idea. I think perpetually these admin configuration things don't, they just don't make the roadmap, right? They're not, it would be nice to spend some time working on that. That's a good idea. Well, I think we've only got a couple of minutes left, so we should probably wrap up this conversation. Thank you all for joining. If you are a trainer or involved in training, please let us know. If you're available to help with the admin training or the Global Galaxy course, we would love to have you there. Please, again, let us know. We can do that in the chat here or on Gitter. If any of you care about training, please come to the co-fest on the 19th of November. We'd love to see you and have your input on how we can make some of these improvements to Galaxy that we need. Anything else from your side? We'll get some paper cuts written down for next week. Helena, can you stay on that goal? I just want to talk face to face. Thanks so much for presenting and thanks for so much for coming, everyone. Have a good Thursday. Thank you.