 Yeah, we can start with the new pipeline development. So there were several people this time working on new pipelines. I'll share the screen to show those lights that they've prepared. Do you need to enable everyone to unmute themselves so they can present? Maybe you've done that already. I've taken care of that. Perfect. So Kamel, if you would like to present. All right, I'll make just a brief introduction. So we had four projects for this hackathon. And this one was focusing on developing new pipelines. Those were the people, the group members that were involved in developing new pipelines for NFCOR. And we have, first of all, the QTL map pipeline. Is there somebody here that was involved in the development of this pipeline that would like to give it a quick introduction for the rest? You can just unmute yourselves. All right, there doesn't seem to be anybody right now. But just a brief description. So the QTL map pipeline was proposed by Nolan Keremov from the University of Tattu. And it's supposed to automatize quantitative trade logic mapping of the connection between genotype and molecular phenotypic data. So that's quite an exciting pipeline to have along. So we hope to see more developments this week of this one. We have also the RIFSIC pipeline. So I can mention that briefly. So this is basically a pipeline that I've been developing previously in Snakemake. And for various reasons, wanted to convert it into NFCOR. Many of those reasons are all the bells and whistles that you guys already have implemented and everything that is very nice about NFCOR that you have streamlined everything. And everything is all very sort of working in a very reproducible kind of manner. And so yeah, I don't see much of the space. But basically, this is a project collaboration with the Mats Nielsen group in Stockholm University. And they have developed a new method for highly multiplexed RNA isolation-free RNA-seq. So I'm writing the pipeline, the bianthematic part of this pipeline for them. And it's relatively simple in terms of gene expression QC and that sort of thing, except that it's got lots of steps of splitting and gathering samples and needs some more advanced metadata in order to be able to run because it also wants differential expression and some downstream analyses after the sort of standard stuff that the NFCOR pipelines are already doing. And the weird thing about this, I guess, is that since the group hasn't really published this yet, this is still a private repo. And I'm not really allowed to share much of it. I was given permission for this hackathon to share if anybody wants to look at it and help out. But seeing as we're all very busy, I've been mostly sitting working on it by myself. And just trying to get it ready for NFCOR inclusion once the group actually publishes their paper, which they are in the process of starting to write now. And I started it with DSL1 and has since started working with DSL2, not with any modules and stuff, but just the syntax. So quite keen on seeing what the modules can be doing and enjoy the talks so far today. So, yeah. OK, perfect. Eric, thanks for this short summary of what the pipeline is doing. Is there any interested people also in contributing than just getting in contact with Eric? We have also the same secure pipeline. I think Camel mentioned that he would be willing to present it himself. Yes, yes, indeed. Hello, everyone. My name is Camel, and we have this time, we have a team for a huge, well, quite a larger team from Archigen. Me, Paolo, Daniel, and Tomak, we are working on Simsecure Pipeline, which main aim is to allow a end user who has set of annotated antibodies, for example, in a form of CSV file to look out for antibodies in a reference set, which are, for example, matching annotations, such as VGNs, canonical for or other annotations. Can you go to the next slide? And this reference may be, for example, one of the publicly available databases like OAS database or our own antibody database, which we have. And the main problem which we commonly have to deal with is this particular sequence was already seen somewhere in our data or any other similar antibodies. Have we found them already or not? And these two should be simple and should give this first overview of a particular set of new sequences. And the Simsecure is named from the Maltese world, Hogue, and it simulates the activity of Hogue in finding what it seeks. Yeah, don't ask why this name was chosen. Can you go to the next one? So our progress, well, all four of us yesterday spent a huge amount of time on task definition and architecture design. And I think that we are ready for actual development if you can switch to the next one. In the meantime, Daniel was writing a wrapper for Paracell tool. This tool we would like to use for actual sequence alignments. And if you can go to the next one. Also in the meantime, when we were brainstorming, Tomek also started our repository with NFCore template. I hope that one is in the SL tool. Yes, it is. Oh, great. So we will be developing this pipeline in the second version of an X-File domain-specific language. And right now, Tomek is working on module which will validate our input data. There is also one additional slide, I guess. Yeah, and currently, our pipeline is residing in a private github repository. But of course, if anyone is interested, we will be happy to share it. We don't have clear restrictions, but we also don't have any permissions to publish it yet. So we came with new idea for our common problems. And we just use this hackathon to make something new, which will be helpful for us. OK, thank you. All right, perfect. Thank you, Kamil. Does somebody have some questions towards this pipeline? Please jump in yourselves. All right, so we can now continue with the GWAS pipeline. If there's somebody here that would like to introduce it. I can introduce it. So it's a genome-wide association study pipeline. And I think this is quite new in the NFCore world because all other pipelines are sort of sequencing focused. And this is at least a tradition that's been starting from the genotype micro-race and then doing this wide association study. So we have been in the Slack channel talking about this addition for some time. And I thought it would be nice to initiate it now during the hackathon. So we have been three people discussing and talking about it. Now this morning, we dropped down to two. The progress is that it is in the NFCore repository only with what the NFCore tools create gives. So we will try to add some test data and get all the checks and so on running and then create the basics for a basic GWAS. That's the ambition for the hackathon at least. So any questions on that? OK, sounds great. Hopefully now that the pipeline is already on the NFCore repository, there will also be some further contributors. Otherwise, I think that's OK. Exactly. I think 18 people in the GWAS channel showed interest, so someone should show up. Yes, it definitely raised some interest. So I think that's it for the new pipelines team. So far, we can continue with the existing pipelines. I think the key to sex should be there after. Yes. OK, perfect. Sorry for that. Yes, so my goal for the hackathon or one of the goals is to write the documentation for KSAC, which is feature complete, at least from my side now. So if I have the documentation, I think it's good for a release. And if I go to the next slide, I finished the readme and then before writing the user documentation, or it's already there. I just need to update it and fix some things there. I decided to go to an issue in the tools repository, a website repository, where there was the idea that we could use the parameters and the JSON schema to automate the usage part of the session. So now I switched over to this. So I'm now split in two projects. All right. Sounds great progress so far for this pipeline. It's been one that it has been already up for a while in the NFCore repository, right? Yes, I think at least a year. Yeah, great. So let's see if we can push it for release this week. And if you can also find some further contributors here. Yes, I would be very happy for new hands and eyes on it. Yeah, perfect. So we have yet another pipeline, pop bomb pipeline by Matthias Marquardt. So are you here available, Matthias, and willing to present it? Maybe he's not, but I can say a couple of words about this pipeline. So pop bomb is apparently an acronym for prediction of phenotype based on metagenome. And it's supposed to be a pipeline that includes some metagenomics analysis as well as some machine learning training, which is quite exciting as we don't have yet NFCore pipeline that includes a machine learning process. So we'll see how this evolves. And I think now we are done with a new pipelines hackathon project. So we'll move forward with the existing pipelines hackathon project. Let me share the slides. We have the deep variant pipeline. I think that uses machine learning. All right, that's true. So an earlier comment, we do have a few non-genomics pipelines as well. We have a few proteomics and there's an image analysis pipeline as well. But yeah, they are mostly genomics as well. Okay, I'm just finding the other slides. All right, so this is the next hackathon project team, which consisted in developing existing NFCore pipelines and the group members are collected in this slide. They were working on many different pipelines. So maybe they can also jump in themselves and explain the projects that they were working in. So first of all, we have the NFCore eager, somebody willing to present this one. So the main developments for this pipeline was finishing remaining structural changes. I think there were also some problems on running it on AWS, which hopefully will be solved now and finishing documentation and preparing basically the pipeline for the 2.2 release. There is also the NFCore Sarek pipeline. So Maxime, I'm sorry, but you cannot hide yourself. So we... Okay, I'm here. Yes, so sorry, I was not on Zoom, so I just joined right now. So yes, so we've been working on Sarek fixing issues and updating the pipeline. And we've just started working on DSL2, but I guess you will talk more about that on the next project. Can you change slide? So yes, so we created the Sarek team of core developer on Slack and on GitHub. We fixed control-freak issue. We fixed two issues for... No, we fixed one issue for Mutex2. We added one option for Mutex2 and we switched from BWA to BWA main two. So I'm hoping we should have a lot of improvement and a lot of speed up on the mapping steps. And that's all for now. So hoping we can be as productive tomorrow. Perfect, sounds like great developments for Sarek. Then there were some improvements of NFCore Mac as well. Is there somebody here willing to present? I have a quick question for Maxime before we go ahead. Do the indices have the indices changed for BWMM2? No, they do say that they provide the exact same result, just faster. So you can still use the old indices. That's not changed. No, that's what I did at the moment. I haven't changed anything. Okay, cool. But I'll do, of course, more tests, just to be sure. Okay, perfect. So we will move towards the NFCore Mac pipeline. So yeah, I could say a few words to this. Daniel, Maxime and me, we're working on the Mac pipeline. For those who don't know, it's a pipeline for assembly spinning and annotation of metagenomes. And it was originally developed by Adrian Goulet and Daniel Straub. And I also started contributing a bit in the last weeks. And since we added some features, such as hostread removal and also some parameters to ensure the reproducibility of individual tools, we are working already towards a release, which we thought we could push this week as well. So there are a few remaining bugs and we still had to address. So Maxime, for example, fixed the memory conversion issue with Spardos. Daniel also worked on the compression of assembly files for the results output file, which reduces the site significantly and we changed some error handlings. And I'm still working on a problem with the busco process, which is still work in progress. Yeah, that's it. Okay, perfect. Thanks for the progress report, Sabrina. Let's see, by the end of the week we can release this 1.1.0. Okay, there's also another pipeline and of course, DIA proteomics. Can somebody brief us on this one? Yes, maybe I can say a few words. So I created this pipeline a couple of, almost half a year ago and just as an initial experiment and now I thought I had time to fix a lot of bugs. Including template update, the container, which had to be updated. I didn't get very far yet, had problems with the container, but at least that seems to be fixed for now. And I'm looking forward to get at least the basics running by the end of the week. Okay, perfect, Leon. Thanks for the briefing. And that was it for the existing pipelines. So we can move on now to the next hackathon project. So we will continue with improving NFCOR framework tools, even though I think this hackathon team was not very popular, unfortunately. It was an elite group. Let me share the slides. There they are. All right, talk for too long because we've been talking for a long time already. But yeah, this project is about everything NFCOR, which is not a pipeline, basically. So it's the Python helper tools and the website and kind of a generic automation. So there's three of us, I think, if you go on the next slide. Yeah, there's me, Stephen and Matthias. And we've been tackling a few different things if you can just hop onto the next slide. So we kind of chatted to Stephen yesterday and talked about the kinds of things there are to be done on tools and kind of going through some of the issues. And then my suggestion was that he kind of starts off with some of the kind of more standalone issues which were kind of not necessarily too ingrained and didn't need too much pre-existing knowledge. So he got started with working on NFCOR lists. I don't know if he's part of this group here. And Steve, I don't know if you want to say anything about this. Yeah, so yeah, like Phil said, I was just going through the issues list and this one about the NFCOR list showing archived workflows was one of the top ones that I saw, so it was pretty simple to implement. So yeah, now it does not show archived workflows by default, but you can use this new flag if you want to see that. Thanks. Although simple, it's quite an important issue actually because these functions that power the NFCOR lists of command are actually used by loads of the other parts of the helper tools. So archived pipelines were trying to be synchronized for example, which they shouldn't be and other stuff. So this was a good one to get fixed. Yeah, next is just sort of generic my work where I've been working on NFCOR tools. It's been a long time since you've had a release every time we say or try and make the next release sooner and it's very difficult to do. So if you hit on to the next slides, we've been, I've been working through the kind of different issues which are pending for the next release of version 1.10 of NFCOR tools. You just hit next and again. So basically we have used GitHub issue milestones to try and group things together so stuff doesn't get missed. I bumped a few of them over onto version 1.11 because I want to get this release out. Remaining stuff is, the big stuff is improving how NFCOR sync works. It's a fairly unknown command. Most of you won't ever need to really run it but it's used by the GitHub automation. So when you have a new version of a template you all get automatic pull requests on all the different NFCOR pipelines proposing hopefully just the updates to the boilerplate code. And we run, I completely rewrote this for the last release and it worked fairly well but there was like a few things where just after it had run I thought this stuff could be improved before next time. So I was going through that. And I also looked through a list of pipelines that failed last time and just vetted them and made sure that they were still, they were ready for the next synchronization. A few of you will have got messages from me where I went through trying to tidy things up. So I think we're in a really good place now for the next release. And also the NFCOR, so we have Python unit testing for most of the stuff, the NFCOR package. I'm absolutely none for NFCOR sync. So I'm just writing a few unit tests for sync now. Next. All right, this is Matthias. Do you wanna jump in Matthias? Yes, I can. So as teased already in the case sec I want to use the JSON schema parameters to generate at least parts of the usage documentation. So if you go to your power plan and go to documentation you will usually see the usage and then the outputs which are currently rendered from the markdown file. So everything has to be handwritten. And at least for the usage we could, the plan is now to use at least the parameter definitions as one part of the documentation. And if you need more you can still have the markdown file attached to this page. So I got familiar with how the code looks like to generate the sites and I was like, yeah, sticking out all the parts which should be done. And yeah, I hope I will not follow this XKCD too much for this project, but we'll see. Just for those who are not familiar with this this is going to tie onto a huge new body of work for NFCOR which we haven't mentioned at all yet which is the JSON schema for pipelines. Don't worry too much if you don't know what we're talking about. I'm going to talk about this in quite a bit of depth on my talk on Thursday where I talk about new things in the NFCOR tools package. But basically, yeah, it's a way to describe all of the parameters used in a pipeline in a kind of computationally friendly way if you're like in a structured manner in the JSON schema file. And so we'll be able to have documentation auto generated from this. So this is really starting to make use of this that Matthias is doing here. It's really good. I think that's it for our group. Okay, perfect. So thanks for the developments and we'll be at your talk on Thursday then to learn more about this. And the last group missing is the next flow DSL tool modules which I know was a very active group. So I'm going to share the slides and does somebody want to take the lead who wasn't this team? So the primary focus of the team was working on the DSL to modules or porting pipelines to next flow DSL tool. And members were Harshel, Gregor, Jose and Frederika Hansen. And yeah, does somebody of you want to brief on the first slide? Yeah, so this is actually been quite a bit of fun and from the previous talk, it's been a bit of a learning curve really the past couple of days. It's been intense, but I think we've needed to be put in this situation to actually make any progress with DSL too. And so now we're sort of in the fire pit as it were. And so the idea that we took was that we would attempt to implement a pipeline end to end because it gives us a particular focus and also it'd be able to help us identify limitations and fixes and other things when you can get. I mean, I personally had to do things possibly in a messy way with DSL one, but with DSL two, all of that will be tidied up. Things will be more modular. And so we thought we'd implement the re-implement the chip seat pipeline from DSL one to two. And because I've just released the version of the chip seat pipeline, it makes sense to now just port that pipeline directly to DSL two. And so the aim was then not only to start adding modules to NFCore modules, but also to try and implement them within the pipeline. So Gregor has done quite a bit of work on updating our main read be on NFCore modules because it's become quite out of date and we're sort of changing these requirements as we go along essentially just so we're clear about things. Jose and Gregor also have been adding modules or attempting to add modules to NFCore modules with the things like bed tools and sand tools. And I've made a list as an issue on the chip seat repo with a list of modules we'll need to implement in DSL two to get this working. And so yeah, it's just been a place you go and from some work that I've done last week on the pipeline template, we've managed to strip out a lot of the boilerplate code that was in the template. So I've been getting my groovy on for the past few days trying to create a separate groovy library that we can then use to import specific functions for the NFCore template within the main script as opposed to having big chunks of code in it. And now it just, it looks amazing. It's just incredibly satisfying deleting large chunks of code from the main script because we've been planning to do that for ages. And I think Paolo done some work on that for the RNAC pipeline a while back and Patrick done some more for the methyl seek and we've sort of needed to bring it all together really. But that would be quite nice because then it will not only simplify the boilerplate code for the pipeline template but we can then also use modules which again is a condensed version of processes to even make things even that much more smaller. So yeah, so this slide is basically us talking lots at each other trying to figure out how to go forward. Do you mind the next slide please, Giusella? Thank you. Yeah, so Gregor's been working on this. I guess we've kind of been using this module as a test case to identify things we want to do and don't want to do with our modules. And we're sort of making steady progress on that and it's sort of the typical software. People use as an example really. So it was a good starting point. And Gregor was working on this. Next slide please, Giusella. So Rike is working on this, I believe along with Maxime. Do you want to say something Rike? Yeah, I'm pretty new to modules. I mean, I've heard about them for a long time but basically yesterday was my first exposure to them. So I'm really glad about the talks today and this morning we discussed which tools we need and luckily there's a lot of overlap with the chip seek pipeline on needed tools and currently I'm fighting with figuring out how to create my own module with BWMM2 since we just this morning also added it to the SARIC pipeline to the current version. And Giusella has also done a lot of work actually I think on SARIC. Yeah, I was mainly working on trying because there is now a first version of the template for the SL2 in tools thanks to Harshil and I was contributing a tiny bit in the end. So I was trying to also see how SARIC would look like with based on the DSL2 template. Yeah, you got multi-QC working, which I couldn't sir. Thank you. Yeah, after a bit of fighting as well. Okay, so I think that's it for now. I'm gonna stop the live streaming right now.