 Hello, everyone. My name is Francesca Bonatt. I'm today's host of the Bite Size Talk, and with me is Maxime, newly member of the Zekera team, and he's here today to talk about, and of course, sub workflows. Very interesting. Hello, everyone. Let me share my screen. I'm trying some demo as well at some point, so I think it's a bad idea, but we need to try that. Okay, full-size screen. Hello, everyone. My name is Maxime Garcia. I'm working at Zekera Labs in the scientific development team. I'm still working in Stockholm, but remotely for Barcelona, but that's Zekera. So I'm going to talk about sub workflow. So basically, what is new with the sub workflow, like in NFCOR, what is our plan like currently and maybe in the short, long, medium term, whatever, and some demo time. So yes, tiny disclaimer to start. I think it's always important. More or less what I'd say are more or less like my own takes of what the community is doing. So other developers might follow my ideas. Other developers might have other ideas, but I think it's good because that's how we are forging the best practice. So it might and it probably will evolve some of the logic, some of the syntax, some of the stuff, but I think it's good that we try stuff and we figure out what is the best way to do stuff. So all of the new stuff. So I think the most important part is that we have no like sub workflow in a specific folder in the NFCOR module repository. We have a look there. If we look at the repo, we have a sub workflow folder, and in sub workflow folder, we have like an NFCOR folder and in that folder, we have all of the current like sub workflow for NFCOR. So this is the most important thing that we did during not the most important thing, but for me, the stuff that had the most impact of my work on the pipeline was actually the naming convention that we had, which is basically like explain all there. We want to have like a naming convention because that way it's much more easier to understand directly what is sub workflow is doing on it. Otherwise, like we have a lot of documentation here on this page, which is, which is DSL to sub workflow that's in the doc contributing DSL to sub workflow. So a lot of the logic is inherited from the module so we still keep the same terminology, which I think is super important. So remember like in NFCOR like a module will be like just the atomic process and then the sub workflow is a chain of module. And then all of the logic is like is pretty similar to all that. All of the underlying logic, then we also have like a lot of documentation for that and what we did new is like some new tools. We don't have everything there yet so I will, I will wait, where was I? Let's finish the presentation before we actually start the new commanding tool so I will show that in the demo time as well. So the plan we do have a plan, of course, so at least I have a plan myself, which is like translating all of the local like sub workflow that we had in Sarek and putting all of that into NSCOR, because that way I think it's very good for the community and I'm hoping also to help other pipeline to do that and to convert more and more like local sub workflow into and of course that way I'm pretty sure we can find like a proper logic to be smarter and to do smart things. And that's my plan. And I think with that we could find like really some new way to do stuff for new pipeline. I think like for me one of the example is that in Sarek we use a free base. It's one of the tools that we use like many tools but at least I know that free base is used by other pipeline and I'm pretty sure that we're doing some stuff in Sarek. We have like a spread and gather like solution that can help like speed things up. So we are trying to import that into the NFCOR module and I'm pretty sure once we have that in the NFCOR module, then other pipeline might be able to import this sub workflow and I'm pretty sure that will be a huge game for the world community. So I'm really hoping like really looking forward to project. So and now let's go into demo time. So I'm going to demo how to install a sub workflow. So I'm going to install like a sub workflow that I just created yesterday. Maybe I just ported a local sub workflow from Sarek into NFCOR module. So I'm just going to do that. So, first I already did that so I installed the dev version of tools. This part is the most important I think that I did that already. So now I'm going to my local repo. Okay, it's my own fork. Okay, I'm just creating a new branch. That's fairly simple. So now we have a new NFCOR tool. There's this like command sub workflows that's infrastructure team so Mattias, Julia and everyone else. I don't know who else is involved in the infrastructure team but they did a pretty good job before that. And I always like what they do like it looks like so fancy like what they're doing. Okay, no command sub workflow. Oh yes maybe just without the S sub workflow. No. Sub workflow. Yes, the S was there. Don't misspell stuff. Okay, so for the pipeline we have info install this remove an update to develop new sub workflow that will be like very similar to the same command that we have in for the modules. So I will not show that, but I will show all of that. So let's try like NFCOR sub workflow info. I want to have like an info about like the new, the new like sub workflow you want to install. So is the sub workflow local install. No, because I want to install it. Please select a sub workflow. So I want to select the VCF annotate VCF. So we have some nice information about all that. It does perform annotation with snippet fight and then bgz plus tabic index resulting VCF. That's perfect we do need like a metamab VCF version of the snippet effect database and optional pass to the root cache folder for snippet effect and then we have like output. So as VCF five plus tabic index HTML report and of course the version. So what were the other commands that we could see install list remove update. Let's check list. I want to list list local. That's some logic and this remote. Okay, that's the same one. Okay, then let's go for install. Wait, before I actually install, let's remove the, the one that we had. Let's remove sub workflow local VCF annotate. So I removed my, sorry, I removed my local version of this sub workflow. I will now install the new version. So that's sub workflow. And I want VCF annotate. I could copy from there but I want to try out what's happening if I don't do anything. This is so fancy. I love that. So I am a big fan of auto completion. Okay, that was super fat. So it's done. Use the following statement to include sub workflow. So I will grab that copy. I'll launch my code. I have a sub workflow. I have that local. Okay, this is my meta sub workflow where I do everything. So I'm just copy pasting this new command here. Okay, let's just align everything well. Okay, that looks good. So annotate. I will use this new sub workflow which is located there. That's nice. I don't have my VCF annotate. It's not perfect enough because it's not there. That's wonderful. That's what we expect. Okay, where are you with the kit? So we deleted some file, we added some new file and we modified some file. Let's add the new file. Okay, so we removed two files. So the meta is different enough so it's like a new, it doesn't register as a renaming. The script itself for the sub workflow is exactly the same, which makes sense because I created it yesterday and I basically copy paste everything. What is happening is a module.json file. Okay, this is new installed by module. Okay, that's interesting. It's just looking. I like to check everything. I think it's important. Okay. So let's add this new file. Let's commit everything. Replace local. Okay, let's push. And let's create the pull request. We want to do that in dev. Replace sub workflow. That's one VCF annotate. It's not perfect. Okay, create the pull request. Okay, that looks good. I'm thinking there is just one lesson that I need to do, but this is like very specific to Sarek. Yes, I need to change the path to the file here. We are doing like pie tests with like tags and we are watching like if some of the files are being changed or not like from one PR to another, and then we are triggering the test just on that. So for that, because of the past is not the same anymore. I just update the pass. He's done. Let's commit that as well. Let's push. And I see I'm hoping that we are done with this request. So, yes, we can see that it was failing before I'm pretty sure because of the test that we're failing now everything is triggered. And as we can see, hi test workflow is being triggered at the moment. I'm guessing once it triggered it will like figure out like which test it has to run or not but that's like something else. So I think that's good for for that. So I like to hear. So I'll paste like the wealth of the history like here in my slides, and I will share my slide at the after this talk. I think now it's time like to thank everyone and to go for the question so these are like the institutes that are participating between the four. I really need to update the slide because I think it's already one year old and I'm pretty sure we have like multiple now. I'm not a contributor but I really want to like thank everyone that is contributing to NFC because it's a community and that's a community effort and like without like everyone else we wouldn't do anything. If you have any question, please ask them because that was mainly like just a demo and that was fairly simple, but you're like people have more questions. Thank you very much. So you're now able to unmute yourself. So if you have questions either put them in the chat or ask them straight away. I think I saw some questions. I will stop sharing. It is not a question it was a comment. Okay, yes. Someone is being very happy that they're already 24 sub workflows. Yes, because we started like the sub workflow at the hackathon like properly. So that's, that was when last month or two months ago. Yes, so 24 like just in a month. That's good. I'm pretty sure we'll have like more and more coming. And I know that Mattias is working on adding the command line help for NFCore tools like soon. I'm guessing we're waiting for release of tools for that. John, do you have a question. Yes, can you hear me. Yes. So thanks. Very interesting talk. I'm, I'm quite new to this but I use next flow. And I am also a little bit used to NFCore but this thing about sub workflows is this specific to NFCore or is it something that can apply to other next flow pipelines. So this is something that can be applied like to any any pipeline like we developed that like, I think first in mind like often of course like the module, but then every module like, like in next flow, everything can be a module like every process can be a module, every chain of process can be a module even like the workflow itself can be a module so you can import whatever you want or whatever you want. So basically what we are creating here with NFCore like this sub workflow stuff. It can be used like in the broader next community without any issue. Okay, okay, thanks. So I also have a question or maybe comment also. Okay, so that was a great presentation. However, I was going to ask a, maybe my first comment is similar to what John just said. So my other question sounded more like sub workflows where NFCore thinks instead of a next flow thing, you know, so I think that's why he was asking that question about whether sub workflows were NFCore or next flow. So my other question is that what's the naming convention for sub workflows in NFCore so is it like the first word is a verb followed by the, the names for the tools that you're chaining together in that sub workflow because I noticed some pattern like that but maybe I'm wrong. So we have this convention is definitely like an NFCore thing only. I'm guessing like other people that develop like stuff might want to follow convention as well so yes I'm happy like to talk more about that. But I think we have this convention I think it's the file, the input file type which is like first, then it should be like a verb and then the list of the tools that are used. For example, like in that case what we were doing with like this sub workflow that I just added it was VCF underscore annotate underscore snippet. Okay. Yeah, thanks. Thank you very much. Are there any more questions. It doesn't seem so. So if you have more questions as usual you can go to Slack, either in the bite size channel or there's actually a channel also for sub workflows. A channel for tools as well obviously. Obviously, or you can directly ask Maxine. Otherwise I would like to thank Maxine for the talk and as usual for funding the Chan Zuckerberg initiative, and you all for listening. Thank you very much. Thank you. Goodbye.