 We will continue now with a talk by Philip Iwals from NFCore and he will introduce how, how do we plan to use right now DSL2, next flow DSL2 in the NFCore organization. So, stage is yours, Phil. Thank you very much. Thank you for the introduction and thanks for the excellent overview, Evan and Pardo. It's really nice to see you guys. I've seen that talk a few times now, but it gets more polished every time. So it's good to see, see a kind of the features becoming more and more stable. It's great. And so this talk was originally scheduled to be harshal. And so he's put the slides together originally, and then kind of both been working on it and he asked if I could, I could give a talk but it's going to be a bit more kind of hopefully a bit more interactive than some of the earlier talks in the week. So please jump in with questions as soon as you have them. Maybe you said they will like interrupt me if there's any good ones. And I'm going to try and pull harshal in for a bit of help every now and then we'll see how we go. And I'll start with a quick kind of introduction to what we've done so far and kind of what the perfect, what the current state of play is for NFCore modules. So let's see if I can share my screen. So, yeah, so like I say, most of these slides were made by harshal originally, and then I've kind of tapped my name on at the end that really well by him. A little bit of history. I think I first heard about DSL to in November 18 2018 at the next low kind of meetup, which was made to great fanfare and a lot of excitement. One of the main arguments against using next row from other other communities was the lack of this kind of feature so it's kind of been highly anticipated for a long time and when Paolo announced it, it was really, really well met. So that was a talk by Paolo and kind of a demo and then there was a blog post in May of last year by Paolo kind of explaining the main features and giving some code snippets and explaining how this was going to work. We of course were discussing this within the NFCore community like from day one about how we were going to work with this, what we should do, what we should aim to do and kind of what kind of features we can build around this new syntax. And the obvious route to go was to try and create a central repository of software tool wrappers which could be shared across different NFCore pipelines or any next row pipelines. So try and maximize code reuse. And so in July of last year we started a new repository called NFCore Modules to try and kind of spearhead this. There was very little code added to that pipeline so that repository for quite a long time. There were several GitHub issues created though to discuss approaches within the community and both there and on Slack there's been a highlight so far of this effort has been a massive amount of discussion. Because once you start to really get into the details of how to implement this thing at scale across large numbers of developers in a very stable reproducible way. There's a lot to consider. And it gets very complicated very quickly. So there's lots of discussion and then we had a hackathon earlier this year in London at the quick organized by Harshal. And there was really one of the first times that being some kind of little efforts at some of the earlier hackathons like it's I Life Lab. But really the one in London was the first time we made significant impact and actually started writing real code. A lot of people started writing different wrappers for new tools so we actually had more than just FASQC in the Modules repository. Started a repository to kind of play around with a DSL2 pipeline which we called DSL2 pipeline test. And kind of tried to work out how this would work in reality. And we also started, well I started building mostly some new continuous integration GitHub actions kind of automation to try and help some of this. And missing from here is that we also started building some of the NFCore tools software so the helper tools. We started adding some new features into that which would also help developers work with this new system. That kind of trundled on for a bit, had a bit of a quiet period and then a couple of weeks ago we sort of thought maybe we should try and get this in shape before this hackathon so we could work on it. So the last two weeks have been pretty frantic activity on the NFCore modules. Harshal's done a lot of work along with other members of the core team especially to start work on converting the current NFCore template which is like our kind of central golden kind of reference point. Converting that from DSL1 to DSL2 and that work is ongoing on the branch on the NFCore tools repository, going really well. Okay, so that's kind of how we got here. Now I'm going to go into a few of the discussions that we've had to talk about the approaches we've decided to take and why that is. So we do have a question from the audience already that is fitting to this slide. So we have had already a lot of development in DSL2 and a question is, is the DSL2 syntax considered stable enough right now to start using it in production? Excellent question. So we've had kind of competing pressures for modules in DSL2 since it was announced with NextFlow. Of course we want to stay relevant. We want to move quickly enough that we are not left behind by other people and that people who want to use DSL2 won't kind of give up on NFCore. And we also of course want to use all of the powerful features that come with NFCore. And as you quite rightly pointed out, DSL2 has not been considered stable yet. And NFCore pipelines are the first thing I say in every introduction talk is that they are portable and reproducible. So stability is critical before we actually push anything into production here. So there's a kind of a powerful balance there. Yeah, we are going to release next week. There you go. We are there. You heard it from the man himself. So exactly. So as of next week, there'll be a stable release of NextFlow. We're going to hold parallel to that. And then after that's there, then we can start using DSL2 for real. So we're nearly there. Any more questions before I move on? So this is partly the work of this week actually. There's been a lot of discussion on Slack for those who've been kind of following it. And it's also covered in one of these issues on NFCore module repository. We've started tying down some of the best practices that we're going to ask people to adhere to when writing process definition process wrappers. So, right, how shall I button here if I get any of this wrong? Module file should only define inputs and outputs. I don't completely remember what that means. Can you help me out? We initially, I think, are going to try and keep the process definitions as simple as possible in terms of parameter, providing parameters. And so ideally we'll try and follow the NextFlow model, which is what I've envisaged. This is still sort of up for discussion, but I think we're going with a simple approach. So NextFlow, by definition, requires inputs and outputs. Sometimes not inputs and sometimes not outputs, but I think you get the drift. And so what we're going to do is we're just going to, for any tool, for example, FastQC, you'd only be providing FastQ files to FastQC. And you'd be taking zip files or HTML files as output. We won't be providing any other parameters within the process definition. And the main reason for that is that if we want these wrappers to be flexible and for other people to be able to reuse them, they should be able to provide their own parameters when they're running these module files. And this, at the moment, seems to be the most flexible way of doing that. And so the other point on that slide, which says that all the other parameters can be provided by a parameter string, which can then be used by the tool. So this is kind of what we're thinking at the moment in order to take this bit forward. Also, so threads and other stuff, which you can use standard NextFlow variables like tasks.CPUs. If the tool supports multi-threading, then you can provide that as a parameter for the tool because it just makes it easier than having to do that. And it's quite standard anyway. But yeah, all other parameters ideally should come into the module via a string of some sort. Thanks. So, yeah, I guess my way of rephrasing that would be we don't have hard code specific options for any tool other than the mandatory arguments and everything else which is optional. So yeah, fast PC options or trimming distances or adapters or anything like that should be put into that single parameter, which will be just a generic string. And then of course the pipeline, which uses the module, then that pipeline can have whatever parameters it wants. So it can still have all the same customization that we currently have with NF call pipelines. It's just that the module wrapper itself should be cut minimal. And yeah, stuff like tasks.CPUs because that's hard coded from from NextFlow that's always a bit. But when it's not that we can check for that in the code. Okay, and then the final one is a simple one that we should call our processes by a name, which should be all uppercase. This is purely aesthetic. It just helps when you're reading a pipeline, it just makes it a bit easier to kind of know what's a process and what's a NextFlow function, et cetera, or a variable. And so that's purely stylistic that just makes life a little bit easier to read. Okay, so extra stuff that should be in that process. Currently, for those of you who have dug around within NF call pipelines, you'll have seen that we have one process which calls every single tool used in the pipeline with a version command and writes all of these to one file. Sorry, multiple files, but those files are then used to kind of grab the versions from the software and put them into a multi QC report for reporting at the end of the pipeline. And now we are kind of instead of having all of those version commands within a single process, we're asking that each process does the version call itself into a file. And we'd also like to use some batch rejectors to clean those those version strings up as much as possible so we have a minimal file with just a version number for that tool. And then we'll collect those outputs and again still report on them in the same way. It's just that that part of the version reporting is delegated within the process actually doing the job. Modules should be minimal. So this is kind of simple we just, you know, don't try to chain too many things together within a single module. It's fine to have lots of modules which change together, but each module should be absolutely minimal where possible. Ideally that means a single tool doing a single thing. The exception is when we need Unix pipes for really heavy lifting. For example, using sam tools here to convert some to cram or to bam, where it was just kind of a bit unrealistic or as a large time penalty for creating intermediate files. So it should be minimal with some exceptions. This helps with a lot of stuff. It helps with simplicity of the code. It helps with maintenance and it also helps with the software package packaging inside the Docker containers. Okay, last one is a bit of a just mentioned because it's a shift from what we're currently doing within NF call. So most NF call pipelines that work with genomics data. There is a params dot single ends. So if you want to use single end ATD minus minus single end and that applies to the entire workflow run. We want to shift this partly so that we don't have to handle programs everywhere. So now we still want to have an explicit variable defining whether the data should be treated as single end or pair them. But that should be passed in in the channel as a value with the input channel alongside each set of data. This has some really big advantages. For example, you will now be able to mix paired end and single end processing within a single workflow run. And we'll also tie into sort of more fine grained inputs in the future. And it is possible to infer whether data is single end or not by looking at the input channel, for example, looking at how many different files are supplied, whether it's one or whether it's two. But almost as possible, I would advise from years of experience that typos are easy to make within glob expressions. And other things quite easy to accidentally run paired end data as single end. We used to do this years and years ago and we opted for that explicit program instead because it just saved us from a lot of pain. So we're going to stick with that. We're going to say no inference of single end must be an explicit variable, but it should be passed in with input channel. Thanks Phil. So we have a question about this light in the audience. So what about multiple processes in one module? For example, should it be one module per SAMtool sub command or all sub command in the same module? Yeah, I'll show you. I'll show you the live demo at the end of this and we kind of look at what modules are there already. It should be one module per sub command. So SAMtools has multiple sub kind of folders, as one for view, as one for index and so on and so on, to keep them minimal. So we can organize the directory structure so that there's just SAMtools and then we have all the subcodes there. That's no problem so that we keep things organized. Does that answer the question? I hope that answers the question Maxim, otherwise just ask again. And we also have a question from Steve. Are there any guidelines about users of custom scripts inside modules? Good question. I don't think we have any guidelines on this yet. There definitely will be plenty of cases where we want custom scripts and I don't see any problem with including them there at this point. Maybe. So I have been thinking about this. So at the moment, because we're still trying to find our feet, I think we will reserve NFCOR modules or it makes sense to reserve NFCOR modules just for existing software that's out there. And so at the moment, during this hackathon, actually, we're trying to reimplement the Chipsy pipeline and SARIC actually in DSL2. And so part of that is also to identify problems in porting to DSL2 and to actually write some of these guidelines as well. But at the moment what I'm doing is in the pipeline repository what I've put is a modules folder and within that I've got another subfolder called local and another one called NFCOR. So all the stuff coming from NFCOR modules will be in the NFCOR folder and anything that is more customized and things that I need to keep locally in order to get the pipeline running I'll put in local. So it would be best to keep them there for now until we find our feet with all of this is what I would probably suggest. Yeah, that was something I meant to cover in the introduction and I completely forgot the policies for that. So the NFCOR modules is for common tools which are possible for other people to use in other pipelines. So what I expect NFCOR pipelines to have a mixture of processes from the centralized repository and also processes which are specific to just that pipeline, which are in this local folder. And of course, pipelines can carry on using scripts in the bin directory and everything as they do currently. And so that's kind of expected to have a mixture of both of these. But yeah, I am sure that in the future we're going to come across really obvious use cases where we want custom scripts within NFCOR modules, but we will have to figure out how to do that when we get there. We haven't encountered that yet. We don't have any guidelines for that yet. We'll be learning experience. Your question also reminded me of another common question we get so I'll just preempt that and answer it myself before anyone beats me to it. There are some workflows that Paolo described. So with the SL2 we can have module wrappers around specific tools as processes and we can also have these sub workflows. And it's quite easy to foresee a future where it might make sense to kind of start a module to provide those in a central repository as well. And we're aware of that and we're ignoring it for now. Again, that's another one of like, we'll tackle that when that becomes an issue where we really want to do it. I very much expect that to be something that happens, but we haven't really thought about that yet. And I would very much recommend that pipelines themselves in the local pipeline or do set up mini workflows very often the moment NFCOR pipelines are littered with minus minus skip this minus minus skip that the workflow scapes will make that much, much more elegant to use. If you want to use just salmon in the RNA seed pipeline, you'll be able to do workflow salmon if you want to use high sat star, etc. So you can imagine having these sub workflows within each pipeline working very elegantly and then having the default workflow which changes those together. And so sub workflows. Yes, definitely in pipelines certainly and probably in the future in modules, but we don't know how yet. Any more questions. Software dependencies. This is something where those of you have followed the discussion and of course about modules might be a bit confused to see what's on the slide because we've changed our minds a few times. Currently NFCOR pipelines we recommend that pipeline has a single environment file with all of the different software requirements and the single Docker wrapper where possible. This is an early decision. And the main reason was that a lot of people were using NFCOR pipelines especially at the start we're using singularity. And basically if the pipeline had kind of 20 different containers you had to have 20 different singularity images and the file size started to get quite large. However, that in some cases cause is issues. And that means it's more difficult to install software if they require different versions of Python things like this. And so really as a developer especially it's much easier to work with a single process a single doc file per process with shared modules which we're talking about here this is just mandatory there's no other way to do it we have to have a single dog a file per process. So that kind of forces our hand. So essentially we thought about having about recommending that we use other publicly available Docker files, for example by containers is a big one that hopefully will be a talk by beyond about by containers later in the week. And we're still thinking about that. And but for now, because we still want supports for condo, we recommend that every process has its own environment file, which will hopefully be very small just be the channel declaration and maybe a single tool in there with a pin version. And also a dog file alongside that which again looks like all the same. And of course dog files we have at the moment we should very very minimal just take the environment and build a condo environment within a Docker image. And then we will build those Docker images ourselves and using within the pipelines. It's still some outstanding questions about how we're actually going to do this, especially with versioning. We would really like some ideas on how to solve solve the versioning issue with Docker Docker files within within the pipeline code. But this is how we're going to start off at least and we will explore other approaches later on but for now this is the easiest and the most stable way to do it. We've used Docker Hub so far for everything within NFCore but we're probably going to use Docker GitHub Docker repository going forward with NFCore modules. It allows us to have multiple different images Docker images within a single repository which is really nice the automation works well. And we're kind of centralizing everything into GitHub which just simplifies much of the management. So for example if you want a new pipeline one of us in the admins has to go and create a Docker Hub repository for you as if we're using GitHub directly for everything it's a bit more streamlined. Okay so environment and Docker file probably went into too much detail there sorry but that's how we're expecting it to work. Yeah explicitly pin the software versions in the condo file. And there's a lot of questions and a lot of discussion about how versioning of NFCore modules will work. I'm going to talk about this more I think on the next slide where we talk about how we're going to pull modules into the workflow but effectively what we want to have is on the NFCore modules repository the shared repository. We'll try and track the latest versions of those tools. So if a release comes out that will be updated on the NFCore modules repository. A new Docker image will be built with a new version and the new pipeline code will be automatically tested against it. And that will be committed as a new a new commit to the NFCore modules repo. If your pipeline needs a new version, you go to NFCore modules and you do it you don't do it within your own pipeline. And everyone benefits and it also keeps this very strong link between modules and your pipeline. I'll talk a bit more in a second about the details of how that will work. Okay. We need to document what we're doing in NFCore modules so people know what's there and how to use it. This is slightly up for grabs but the suggestion so far is that we write a YAML file just a structured text file which sits with each module and kind of describes what the module does. What the inputs and outputs are and who writes it. We're not doing anything with this yet but the intention is in the future that we can use this for automatic documentation. Maybe some kind of validation and testing and things like that you can imagine. So we're kind of pretty much stolen much of the syntax from Conda YAML files here and we just kind of set this out very early on as a template so that we collect this information from Word Go release. GitHub actions. So this is we used to use Travis now we're using GitHub actions for everything for continuous integration testing. Just like all NFCore pipelines, every module should have its own CI with a minimal test data set. This is one of the main reasons actually from the early stage why we wanted to use modules is that we have kind of end to end testing of pipelines but we really wanted module like unit testing of the specific steps within a pipeline. So we actually run each individual step by itself and ideally actually check the outputs. Mostly at the time moment we just run the pipeline make sure it doesn't crash and then have to manually bet that the inputs and outputs look right before releases. But now we want to actually check and validate the file from each plus each module as it runs. Just like other pipelines where we have a test data sets repository, we've decided very recently that we think the test data should be on the modules repository. Ideally, if you just need some first few files or something you should share in between different modules so we don't load it unnecessarily. Okay, so this is roughly what the NFCore modules repository looks like currently the directory structure. We have the kind of basic stuff like the read me and get her workflows, and then we have a directory called software. This used to be called tools, and then how she pointed out that I keep calling everything tools and then of course, and it was very confusing. So he renamed it to software. Within software, then we have the names of the main tools which we're supporting. So here we've got PWA, we've got first you see we've got Sam tools. And then you can see in some cases we have sub directories nested under there, which is what I described earlier. So the different samples commands, for example, and these can be nested arbitrarily. Then the kind of the leaf leaf folders the end of those folders we have what looks like kind of a mini next way workflow almost have a main file which just contains that one process. We have the metafile which describes everything going on there and we have a test action.yml file, which is used for CI. I'm not actually sure if that quite how that's used currently. I think that might be wrong. And there's also missing environment and Docker files here, but you get the general picture of it here. Each of those folders should also have a test directory, which will contain actual an actual next play workflow, as well as all the input and output test data, which will be used to validate that unit. Right, importing this is where I said I was going to come back to a bit more detail. So Phil, now that we go to the previous slide again so we do have a question from Maxime, and it is how can we specify the version of a given program within an NFCore module. I'm about to tell you. So versioning and importing are one and the same thing, basically, or at least we're very closely tied together. This is probably the single issue that's had the most discussion so far and but I'm pretty confident we're coming to a conclusion now. If you look at that issue thread that's very long. It's a lot of discussion there. And we've gone through a lot of different suggestions and we've even tested out several of these. So if you go and look at that old DSL to pipeline test repository, someone dug it out earlier we actually deleted it from NFCore but I forgot that I still had my personal fork floating around. It's outdated. Don't look at it anymore. But in the early days we tried using Git sub modules. This makes quite a lot of sense. You just say, okay, we've got the modules repository. I just want to use that here. And then I'll be able to access any of those module wrappers. The downside are that when you clone the repository you need to clone with sub modules get clone minus sub modules recursive initialization, which is kind of a hassle doesn't happen by default with Git needs extra flags. It's just kind of liable to cause issues. The other problem is that then you can't have a sub module for parts of a repository. So you have to pull the entire NFCore modules repository, which could get very large and could be a bit problematic. The nail in the coffin for this one is that you can only import that repository once. And so that means you have the NFCore modules repository at a single commit across all of those different software wrappers and they're all locked to the same version. And that's important because you might want to have different versions of different tools running and we'll do that by taking different versions of NFCore modules repository. Hopefully that makes sense. Another suggestion was to have a library of modules. It's kind of very attractive given the possibilities here and where we could post modules on an existing library infrastructure. The team that came up with MPM node package manager. And of course, condo, which is a lot of people are very familiar with. We even got as far as making a condo channel called NFCore, which is that currently empty. We're now ready to have these tool wrappers. And we have a branch on tools which has a working implementation of using condo. And so basically using condo works. The downside of using condo is that it's complex adds a lot of complexity into the automation and to the management of the modules. So we're looking into how we could use by a condo utils to kind of manage some of this and really it was going to turn into a kind of a massive job to really make sure all of this works. It's entirely possible we go down this, this road in the future. If things start getting really big and unmanageable by ourselves. And it does have some advantages but for now we've decided to kind of put that on the shelf and come back to it later if we need to. One of the main arguments for using condo is dependency management. But it's an important concept here to remember is you're pulling in flat files from NFCore modules into your pipeline. And you pull those in and chunks so they should never really be cross dependencies certainly as for someone for running your pipeline. The person running your pipeline just grabs out flat file repository and runs it all in one go. So dependency interconnectivity should be fairly complex unless we start getting very complex with sub workplace. Okay, so what we ended up with was basically a homebrew solution using NFCore tools, Python package. And on now merge into the dev branch which will soon be released in version 1.10 is kind of a skeleton early version of a new suite of commands called NFCore modules. And I'm going to just quickly demo that. So hopefully you can all see my content. I'm just in an empty directory called demo. If I make a new, a new pipeline with NFCore create. I can just call it demo, demo pipeline. What this does is it takes the NFCore kind of template and just builds me a skeleton workflow to start from scratch. You can see it's made a folder here called NFCore demo. And if I look in here, I've got my main main.nf and my next row config and everything. This is a DSL one pipeline, but currently got that work going on I described with converting his template to DSL two. So what I can do now is also start to play around with these. But do NFCore minus minus help you'll see I have a new command here modules, which is not being heard before. And if I do NFCore modules help you see it comes with the same sub commands. And these three don't do anything yet. They'll just tell you it's not yet implemented, but these two do work to some degree. So as the two ones do they fetch the available modules, the NFCore modules repository and list them. And this one installs one of those into your pipeline, which basically just means it copies the relevant files. So if I do NFCore modules list. We will hopefully get a nice list. There we go. We have a different available tools. We've got Bowtie, we've got the Sam tools, different wrappers here, we've got FastQC and so on. I've got my pipeline here remember if I do NFCore modules install, I give it my pipeline directory and I say I want to install FastQC. I will check that it's a pipeline folder that I gave it or check that FastQC exists in NFCore modules repository. And if I go into here, I can do get status and you can see I've got a new folder here. It's just not been seen before. If I look in this folder, you can see it's copied all of the files for this wrapper. And so then I could use those within my pipeline. I've just import from module software FastQC. And this probably will need fiddling with a bit. I just realized that Hartrell said he had these tools within an NFCore subfolder. So I need to add that. But you can see the concept here. This Python script is very simple. It's just looking what's available on the GitHub repository and then copying those files and avoiding the test folders and stuff like that. What will come in and is more validation to check that will make sure that when you push changes to your pipeline that you haven't changed these files at all. You should match exactly to NFCore modules. If you want to change something, you should do it on the NFCore modules repository. And also to like update them and make sure that they're locked against specific git hashes of the NFCore modules repository. So eventually coming back to your question maximum about versioning tools. On NFCore modules, you'll have a history for each tool. So we'll start when it's first added that will have its own version with a commit hash of a code and of a Docker image. If it's updated, you'll have new commit hashes for that tool and new Docker images for that version. So we recommend always that you always just update all of your modules and just have them all of the latest commit of NFCore modules, which will then also have the latest versions of those software. But if for whatever reason you want to have a specific earlier version, you just pin that git hash for that wrapper and then you'll have that software version and that version of the tool. So it means that when you make your pipeline, you can be sure and confident that stays stable. It doesn't matter if people start updating NFCore modules. Firstly, you've copied those files out of NFCore modules that are contained within your pipeline and stable. And secondly, they're tied to a specific hash of the NFCore modules repository. Okay, any questions before I move on. Maybe put everyone to sleep or everyone's got a headache. Right, I'll move on. I think we're nearly at the end. Yeah, okay, so this is the end of it. Yeah, that is the end now. Okay. Cool, right. Haasual, did I miss anything? No, I think that's about where we are with DSL2 at the moment. If people do want to get involved, we're having a bit of a sprint at the moment, like I said, on Slack in the Hackathon channel for DSL2. So get involved. And yeah, thank you for listening. So thank you everybody. So now we can have a discussion question. There were a couple of questions that I didn't ask yet because they were not directly related to the slides, but you have now the chance of asking them yourself to any of the speakers today. So raise your hand here in Zoom if you have some questions, please. Do you think you could repeat the questions that I guess, Anna, which are written in the chat? Because there's a couple sat there at the moment. Yes, I can also repeat the questions. So a question to Paolo. Are there optional and or named inputs for modularized processes, like for outputs with emit, let's say I create a process for my tool, which has dozens of parameters, a lot of them optional. It could be nice if I could just call mine. Yeah, it's it's difficult to read because there's a bit of coding between the lines. So if that person could speak up themselves here or raise their hands, I can unmute them. They can probably ask themselves better. Yes, this is an open problem. Some ideas, but it's not going to be available in the coming release next week. But yes, this is something that need to be, need to be something next to have a better handling for optional input output. So not next week, but hopefully soon. Okay, perfect. I hope that this answers your questions. We had also another question before. I was wondering how difficult it would be to have multiple pipe connected script blocks per process task. Yeah, I think I'm going to go on that one. So the idea would here would be imagine that you had several processes that you wanted or several process definitions that you wanted to run at once. So at the moment, each next load task is submitted out to the infrastructure that you need to send it on. So for example, if you're going to send it out, who would each go on to a virtual machine on AWS batch or a node in your cluster. The idea would be that if you could link two of these files together, two of these processes together automatically, you would no longer need to transfer the output files to the input of the next process. There's some ideas on doing this. In some ways it would break cool could potentially break the pipeline across different infrastructure, because if you were doing this if that could only happen on some infrastructure. It would break some pipelines. So also other ways of having be able to imagine being able to stream the data from one process to another. And these are all open ideas as well and things that we will be working on, because as the, as the person asked the question, I said this would make things much faster. You wouldn't have to have intermediate files, you wouldn't have to necessarily write to disk. And you could really speed up a lot of the execution time. So it's a very good point, but something that we'll be working on for the next little bit. Okay, perfect. Thanks, Evan. We have another question from Rad. Any chance of using version tags as well as commit hashes for modules? Yeah, I can take that. It's not the first person to ask, it's come up quite a few times. We've thought about this and it's, it might happen. We have the software version tags in the environment file. And so we do have that available. The downside is, if someone wants to add some more functionality to the wrapper code itself, not leave the tool version unchanged, the change the process wrapper, then that process wrapper needs to have its own version as well. So then you're maintaining the software version, wrapper version and the Docker version. And I kind of felt that this was very complicated. So my gut feeling is that in the end, we're going to have a single identifier, which is going to be the git hash. That contains everything. It contains a wrapper. It contains a software container version contains everything. Whether we add some kind of helper functionality, which finds a software version and converts that into a git hash at a later date, that might be possible. But it's not going to be there straight away. For now, just use the git hash. I'm hoping to be honest, I would like to try and push everyone to use the latest version of all tools for the time. So really, I'm hoping that all this question of versioning is going to be a little bit mute because you'll just run NFCore modules update all and it will just update all your modules to the latest version. And you check it works and then, you know, one happy but of course there will be some circumstances where you're forced to use older versions of software and we need to make that possible. Perfect. Thank you, Phil. There is a similar question from Maxime. Could there be named inputs for processes as there is also now as also now named outputs exist? Maybe that's more a question for Evan or Paolo. Yeah, I think it's for Paolo there. Yeah, I fear that's for me. No, at least I think that we're confused even more. It's already quite start to be complicated. Maybe the syntax, I fear that that will open the possibility to have too many options. I don't know. So short question, I would say no. Long question. Maxime says, okay. Any more questions from our audience today. So Rad is commenting, I would argue against latest in favor of explicit versions and a way to have both version and hash similar to how you have tag and digest for Docker images. Yeah, exactly. I think we're agreeing. A Docker tag resolves against a digest. At the end of the day you always have a specific hash and it will be the same for NFCore modules. At the end of the day you will always have a specific hash and we might add kind of nice front functionality to make that easier to use in the future. But first we're going to start with the hash and go from there. I want to add a bit more about now optional and name it input parameter because this is actually is related to optional input. Now, because if you allow people to have a process in which the input are specified, not with the position but with the name, you can specify one. Imagine that you have a process with many inputs. And you specify just one with the name and then you omit the other. So maybe if we have a good way to specify the default value for and then and therefore the optional value for some inputs we could have also a way to manage name and parameter because they are linked to. So maybe in the future. Would a map go anywhere towards solving that? Yeah, the map is doing that the problem is that what is the value for something that you don't specify. Now we don't have no. We should also way to specify default value for a process input, but then we can be weird because I'm not sure that they have much sense. In the fall from now the full value you can have some specific cases but that is more an optional input my opinion. You should specify a default value for each input. Otherwise, when you specify, when you invoke the process each time you have specified the name and actual value the name that makes very verbose the composition of the pipeline I think. Or could the default value be initialized as false or not or something and unless. Can you repeat sorry. Could the default value for any given input be null or false or something unless it's overwritten being provided to the person. Yeah. A default to zero to say. I don't know. I think that will open. Create more confusion. A couple of people have chimed in on the chat here and I was thinking the same thing, we could potentially try and steal how Python does this. I guess other languages as well. But if no name is given you can just use the order of arguments and if names are specified and then use those. Maybe this is getting a bit in depth for this for this form. Yeah, we can take this on slack. Yeah, that could be an option, even though not sure how much will improve the experience with the language. It seems to implement this then is a big a big work so I prefer to focus on other stuff. Sorry, there's just hop on a question here about from Steve about how DSL to works with params. And whether basically will lose the ability to use params to kind of customize the way that script blocks run and things like that. Yeah, basically, the model in every the parents from the cooling model. So you can use params as any other script. The model received all the parents that are in the invoking script. But you can still override if you need when you invoke when you make the inclusion. There are some example in the documentation. And we will be making use of this quite a bit in NFC modules as we saw in my talk earlier. If you remember said params dot module name arguments. So we're using a regular prams there to to customize the script block already just in a kind of standardized way. And so, yeah, prams will continue to be a core feature of pipelines even with the SLP. All right, there seems not to be any more questions. So if there are no further questions I'd like to thank everybody who contributed to the session. It was very interesting to have this discussion and if people would like to contribute to us porting all our NFC pipelines to the SLP to join us now join us for the rest of the week. And yeah, this way you can make an impact on NFC.