 Great. Okay. I'll make a start. Thank you very much to everybody for joining. Either joining us live or catching up on YouTube at a later date. Today's bite-size is just a short one. It's going to be about half slides and half live demo. And I'm going to talk through some of the tools that we use for code linting. I'm specifically code style linting. So I'm not going to talk about the NFCore lint commands in this talk. That's kind of as other bite-size talks specifically about that. This is more about the general use tools that we use for linting and code style. What do I mean by that? So as I'm sure anyone who's done even a little bit of development work will know how you format your code is a contentious issue. I started looking for a good like gift for me to put in, of course, XKCD gave up the goods with some rather excellent insults about code style. And once I started, there's just it's a pretty deep trove. Actually, there's a lot of a lot of good comics out there about about edition crises about code formatting. And any of you who watched Silicon Valley on TV or other TV programs will be familiar with the tabs V spaces argument. Anyway, the point being here is that everyone has quite individual views on how code should be formatted, not the content of the code, but just the white space and how things are formatted and how things are done. This is where code format is come in. And the idea of code format is or lint is that they set the style you're going to use. They decide how things should be formatted. And that means that you don't have to. It's good because they mean that you kind of free up a little bit of space in your brain for it. You're not kind of having to think about whether to use single quotes or double quotes, how much indentation you should be using, things like this, because it just happens. And the other thing is that they're really good for projects where there are lots of contributors like NF core, because then we have lots of different people coming in and working on the same single code base. Everyone has their own idea of how things should be formatted. And if ever you've come in and read someone else's code, you'll find that sometimes it's very difficult to read through that code if it's not in a style that you're used to. Code format is forced everyone to write code in exactly the same way. And that makes it much easier to collaborate together. And also means that when you change things, the differences in the code, the code diffs that we see on GitHub are just the important things, the actual code that you're changing. And we don't get lots of spurious diffs where white space is being changed. So yeah, code format is think about style so that you don't have to. I've put a little asterisk down in the corner here that they're highly addictive. I think code format is a bit like wearing a seat belt in a car or gloves in a lab or using git to manage source control. Once you start using code linters, you'll find that it's very difficult to work without them. I've been using them for a year or two now, and now I find it really irritating whenever I'm in a project that doesn't have them. I notice always little inconsistencies and I find myself wasting time on thinking about this. So once you start using code linters, I suspect that many of you will find it difficult to go back. I hope so anyway. So these are the different languages within NFCore that you'll come across. Apart from the obvious next clue, we've got a bunch of Python code, especially in the NFCore tools package. Also some scripts within pipelines. We've got a lot of documentation written in Markdown. We primarily use YAML for configuration files, also JSON. And then on the website, we've got a few extra languages, websites written with PHP, CSS and NHGML. And these are the main languages there that I'm going to talk about today. And there's not very many tools that we use. We use basically two tools, maybe three. Python, we format using a tool called Black, which is the most popular and commonly used code linter for Black, code formatter for Python, used by a lot of big projects now. And it was Black that got me into this in the first place. And then we use a tool called Prettier to do everything else. And so recently, until we last released the tools, we used to use a tool called Markdown lint for Markdown and YAML lint for YAML. And we had numerous different individual lintes here, but we replaced those in the last major release of tools. And now we just use Prettier for everything. So if you did a pipeline sync recently to a template, you'll see quite a lot of minor changes, things like quotes and things like that in your YAML files. And that was because we switched using Prettier and now using that standard. And then the little mouse down in the bottom corner is a tool called edits config, which is basically a generalized, but it's a standard where it tells your code editors what the code style is for a project. And both of these other tools tie into that config as well. And it's also a standalone tool that's generic for any language. Now that one does stuff like indentation should be at multiples of two. So if you try and indent any code with the multiple of three, then it will complain, stuff like that. Okay, so these are the three projects we're going to talk about and these in their web sites. They all come with command line tools. So they all come with a command line package that you can run that look very similar. With black, you can do black, minus, minus check. Prettier, minus, minus check. And then edits config checker, that tool is specifically for checking. And you just give it a bunch of files. So normally it would be asterisk for all of the files in the projects. Black and prettier especially will just ignore any files that they don't recognize. And they will go through and they'll check up with syntax of all of your different files and they'll throw a warning if anything doesn't conform to the style that they like. What is particularly good about these tools and one of the main reasons we switched to prettier recently is that these commands also will fix it for you. So you don't have to just go and look up every single line like we used to have to for markdown lengths like our line 432 is using the wrong style of bullet point. Now you can just do prettier, minus, minus, right to go and prettier will run through and it will format everything for you and just fix it. That's great, that saves a lot of time. Okay, so before I go on to the live demos I'm just gonna walk through some stuff. There's three main ways that you're likely to come into contact with these tools. For many of you, the first way you'll come into contact with them is through the continuous integration tests. So you'll push a commit or you'll open a pull request and you'll have a failure on the CI. You'll have no red cross and it will say that prettier failed. And it probably says something like, oops, forgot to format your files with prettier. And usually pull requests will not be merged until that turns into a little green tick. So that might be the first time you see this and that might be how you ended up on this talk. One of the first places that we might go to to then to fix this problem is you can run these tools on the command line. Like I just showed they have commands so then you can go back to your code place run prettier minus minus rights which will fix the problem and then push another commit. And then hopefully that red cross will turn to a little tick. But really the best way to use these tools is to have them set up in your text editor in your code editor. And when that's done properly they will run automatically every time you save a file every time you edit anything. And you don't ever have to think about running them because just everything is automatically formatted properly. So that's the best way to work really. And once you've set that up once you can just forget about this and it just works. Right, so let's see if I can screen share a little portion of my screen over here. Hopefully everyone can see that bit of screen and see it throws everything around my window. So wave at me if you can't see what the website I'm looking out for the black documentation. Good. So just to show you the websites for these tools this is the black documentation. You basically don't ever need to look at this. But just so you know this is kind of what it looks like called the Google Python black, you'll find it. This is a website for prettier. And again, it kind of talks about how it works and with different languages that supports and for different code editors that you have integrations with which is kind of basically all of them. So if you're using something other than VS code which is what I'm gonna show you in a minute you can go along here and, you know install a plugin for Atom or whatever else. Yeah, and it's got quite good documentation. And the final one is this editor config which is just like I say it's a way of sort of standardizing config files across editors and projects for things like what type of new lines to use character encoding, indent size use and stuff like this. And certainly prettier will find these editor config files and load these settings from that. So they kind of integrate together. A lot of editors have built-in support for editor config and some you need a plugin for. So again, you can come here and get your Vim plugin for editor config. And basically then that this if this file is in your project it will override your normal local settings. Right, and then I said it ran on continuous integration. So just to give you a look of where this is this was within the pipeline template and there's a file in the GitHub workflows called linting.yaml. So just so you can see this is running editor config here and this is the exact command it runs. So if ever you want to emulate the CI tests that are running on GitHub you can run these commands yourself on your local system and you should hopefully get the same results as long as you've got the same versions of the tools installed. So you can see it's just running prettier check and editor config. And if we look in the main tools package where we've got a load of Python code you'll see it's also running black. Python black, you can see it's running but it's actually running GitHub action for running black. Okay, so just to do a quick kind of live demo now I've got the RNA-seq pipeline open here and I've just made a couple of basic kind of dummy changes I do get diff, I wonder if I can do diff dunk so you can see here that I've just made a few changes I've added something to a change log file here and I've just made some white space changes to this YAML file just for the purposes of this demo. Now right away you can see that series of changes here, the GitHub changes you can see it looks a bit weird so it's probably not surprising that this is gonna fail but this is a valid YAML file. This is the NFCore config file so it's customizing a bit about how the NFCore tools work the linting tests and stuff but all of this is valid YAML I haven't actually changed any of the real kind of meaning of this file and it should have run fine but you can see my indentation is a bit wonky here and I'm kind of using some single quotes here and whatnot. So if I, I've turned everything off at the moment so if I hit save then I could do prettier check and this will check all the YAML markdown files and sure enough it says, there's something wrong here the NFCore.YAML file has got a warning and also a change log where I added a little note about what I've done here. So I know something's wrong and if I had pushed my code anyway I would get a warning on GitHub, some GitHub actions. So I'm just gonna commit this now. So now we'll be able to see what changes after this. Okay, so I'm in BS code so I can run prettier, right? That goes through all the different files it recognizes and you can see if I do get status that has actually modified these two files. So you'll see I did get add first I committed those changes first and I think that's good practice before you run code linens is to commit your actual changes what you're thinking about first and then if you're gonna make big linting changes you can do that as a separate commit and it's very easy to see the diff of what's changed in case you're nervous about it. Later on you get used to running it and you won't need to do a separate commit but just at the beginning. And then if I look at the diff you'll see that both these files have been changed and you can see it's basically kind of messed around with some of the formatting and whitespace and sure enough in this file now all the indentation is fixed or using two space indentation everywhere. My single quotes appear converted to double quotes and all the extra line rates were removed and if I'd had any like trailing spaces at the end of lines like that then they would have been removed. Over in the change log you can see my markdown now correctly has a blank line after this heading and my bold text has been changed from double underscores and italics with asterisks to the standard that we use which is underscores for italics and double asterisks for bold. Minor changes didn't make any difference to the actual contents of the file but now we have a nice consistent usage of markdown and YAML. Okay, great. If I kind of... Hang on, if I just did it, where was it? And then here we go. So if I just put this back to how it was and I will show you the other way of doing it. So, say this... I said, so that's on the command line but I said the best way to do it is with the browser with the code editor. Now to do this I'm going to use a couple of plugins with VS Code and like I say, you can do this with basically any editor. So for VS Code, if you go to the marketplace or you just go to the extensions tab here you can search for prettier and there's a few ones but it's the obvious big one that's got 20 million installs. And basically you just hit install there and things should work. The same goes for editor config that there's a plugin for VS Code. You may not need this. And then black is kind of a bit more complicated because it's actually built into VS Code. So you need the Python add-on and then if you go into settings and type black you'll see that the Python formatting provider you want to set that to black so that it runs black whenever you edit Python code. If you're never going to edit Python code then it's all packaged, don't worry about that. Okay, so I've got them installed and set up. Now, if you're doing this on some other projects that's outside of NFCore, it's important to note we have a few files in the roots of the repo here which are helping us out. For the edit config tool this guy we have an edit config flow that sets everything up. Says what the indent line should be and things. And for prettier we have a prettier ignore file which tells it, it's like a get ignore file tells it to ignore certain stuff. And a prettier RC file which has some settings. Here we just specify that we want the width to be 120. I can't remember what the default is but it's quite narrow. So we don't change very much. So we have some config files in there. And then once that's installed I can go to the command palette here. So that was command shift B for me because I'm on the Mac. And you can see there's an option for saying format document. So I can run that and boom, done. So that's a bit better but command line it's still not great because I still need to actually remember to run that command all the time and we're trying to get away from having to remember anything. So if I go back into settings here if I look for format on save there's the magic tick button here editor format on save. And this tells VS codes to do what it says to format your files whenever you hit save. So now if I go to the change log markdown file and make some change if I hit save you'll see that prettier ran and it fixed this extra line break and it fixed the markdown here. I can put for the bunch of extra training spaces in here which are there and if I hit save they disappear because prettier runs every single time I hit save. And this then is the state that I recommend you run in where every time you hit save everything just is formatted automatically and you don't need to worry about it and all the tests should pass. Okay, final, I'm running a bit late so final little bit. Sometimes prettier breaks or doesn't do what you want it to do or there's some specific reason that you want to ignore stuff. I said that there's this pretty ignore file. One of the things that we ignore is actually like in the new release it's gonna come out for the template any minute is this email template file because this is actually broke because of prettier. So we've got groovy code mixed into HTML and so that confuses it. So there are sometimes legitimate reasons to ignore the code lenses. If you want to ignore an entire file you can stick the file name into the prettier ignore file and if you look at the prettier documentation you will see that there's the section about ignoring code. It talks about that file that I just mentioned and it also says that you can use the keywords within the file. Basically you make comments and you say prettier ignore and that will just ignore a chunk of that file basically. So there's a way to do this. This just ignores that one line and then prettier will continue to pass. But of course this is kind of exceptional use case only. Most of the time you don't need to do this and you should just let prettier do this thing. Right, final couple of slides just to wrap up then. Screen, come on. All right, very good. Yes, the final mention. I mentioned the YAML and Markdown who's the two of the biggest offenders here and also like Jason and a few others. But of course wouldn't it be amazing if we could do this with next flow code? And we had a standard about indentation after inputs and output blocks and script blocks and everything. Those of us who have worked for a while and especially who are used to code and using them and like them would absolutely love this. But it doesn't exist yet. However, prettier can handle plugins such as you have a website does the PHP code is like a sort of semi-unofficial plugin. And there is some interest within NFCore and next flow communities to build something like this. So the other day we were talking about this and Edmund actually kicked off a new Slack channel called Prettier Plugin Next Flow. So there's nothing really to see there yet but if you're interested and especially if you have any experience in doing it so would like to help out or just get involved go and check out that Slack channel and join in because please, please I wanted. I imagine you'll make a lot of friends if you can make this work and make all NFCore is all about standardization and best practices and this would really be icing on the cake for that. Right, with that I'm happy to take any questions be it about Lindsay or anything else who can hit me. There's no one moderating today so I would just keep an eye on the Slack chat there's a couple of things in there and or just unmute yourself, I believe. So it's a question about what do I use for next flow at the moment? Yeah, editor config, that's one of the main reasons that editor config is in there that's a general use tool is very, very simple. Like I say, it just checks like the indentation basically kind of two space for space and that's about it. So it's still possible to kind of do pretty variable formatting, but at least it's there's something. Like I say, hopefully we'll have a prettier plug in one day. And this asks, have I ever come across that black introduces bugs in my Python code? No, I quite like that actually when I hit save, if I don't see things move around that usually means that my Python code is invalid and there's a syntax error somewhere because black can't run on it. So it's actually the other way around. I think black probably saves me from bugs by alerting me early to the fact that there might be something fishy going on. I've never seen it introduce a bug. No, sometimes you might not agree with the choices it makes, especially black. It defaults to very narrow column width. So it breaks loads of stuff over lots and lots of lines quite quickly unless you change that default. But I've never seen it actually introduce errors or bugs. Moritz says that he likes the shorter 80 or 88 character line length. This is personal preference. I think that we've got it set to 120 at the moment partly because that's what I have everything else set up to when I came into this. There's quite, with the configs, so you'll see there's kind of a couple of too weirdish bits. I don't know if who was paying attention might have really spotted it. I'll go back to it again in this window. So if we go to the pipeline templates and go to the editor config file, you'll see here that we've got some stuff to be set up to be indent size four. So that's all files. So that would be like net flow and config files and everything indent size four. And then some stuff we've got indent size two. And then some stuff we've got like modules with kind of unsetting a bunch of stuff so that it's not clobbered, but yeah. We're doing some stuff here. And some of these choices you might be like, well, why are we using two for some files and four for some files? That doesn't seem very consistent. And it's the same for line length. There's a pretty simple answer with that. It's because we tried to minimize how many changes would be introduced into the code base when we started using these files. So Jason was already set as four, I think. So we tried to stick with four so that it didn't break everything. So we tried to actually minimize the number of changes which were introduced by these tools when we started using them. And stick, if we already had some kind of standard we tried to stick with that. Yeah, line length you can be set in editor config. I think that's what might be where we're setting it, I'm not sure. Okay, that's nice actually. If we can set the pristier line length in editor config then maybe we can just completely get rid of a prettier config file. That'd be nice. Any more questions? She checks that. Great, well thank you for sticking with me on what could be a bit of a dry topic but hopefully a useful set of tools for everybody. And as always, if you have any questions or any questions or any questions please jump into it and of course Slack and we'll be more than happy to help you out.