 So good morning, everybody. Welcome to part two of the EasyBuild tutorial. Let me make this a bit bigger to start. So this is part two. Part one was two weeks ago and was recorded so that recording is available through the EasyBuild YouTube channel, which I think I shared in the chat yesterday. So if people want to get back to that, it's there. Today's session is also being recorded. The next session, which was originally planned for tomorrow, has been moved to next week to give partially to give me a bit more time to prepare. And if there's any suggestions there for specific topics, they are still welcome. So in here, there's an overview of things I plan to cover. So these four I will probably cover together with the GitHub integration that EasyBuild has. So to show how you can contribute back to EasyBuild quite easily. But if there's additional suggestions for topics, do let me know and I'll make sure to prepare some content for that. Today we're going to do part two, which is going to be about troubleshooting, creating easy config files and implementing easy blocks. So there's quite a bit of content here to cover. I hope you can get through all of it. So this is the rough time table I have in mind. We'll start with troubleshooting and hands on exercise there. This may feel a bit backwards because we're going to look at easy config files before we've actually covered easy config files in detail. But even if you're writing new easy config files, you'll need to do some troubleshooting. So I think it's easier to start with looking at troubleshooting and giving you a broken easy config file that you can fix. So gradually looking into errors and log files and figuring things out, because though that will come in handy when creating new easy config files as well. So I'll spend some time here to explain the details about easy config files, how you can create them, what should be in there and how. And then we'll have a hands on example and also an exercise so you can create one yourself from scratch. And then hopefully we'll still have time to look at implementing easy blocks as well. And also here. Two exercises well one but in two different ways, and hopefully we'll have time to cover some of that as well. So just a bit practical information again, just like last time that most of the content will will be through the easy build tutorial website so this one. There's a last section here, which has the overview so the introduction was what we did last time using easy build is what we will do today. If you click the overview, you will see the four parts, you click using easy build you will see the three parts will do today or you will see those same three things here troubleshooting creating easy config files and easy blocks. So that's what I'll be focusing on. And usually I'll do a split screen so half of the screen, the easy tutorial website half of the screen, a terminal where I will play around. So for the hands on demos and the exercises will use put the again at CSE in Finland. So hopefully everybody has an account there. I already saw some questions this morning in the chat. So the exercises and the demos mentioned slash easy build. But you should translate that to this on put so the slash easy build is under our project directory for the last preparation. So here you will also find an easy build module now so once you've done this module use, you can load easy but as a module, you don't have to install it again, or use your own local copy. You can, as long as you have the latest easy build version it doesn't really matter which one you use. But this is an easy way to use the latest easy builds that installed already. So make sure you're you have properly configured easy build so tell it to use your home directory or the easy build sub directory in your home directory, except for the build part that should be a local directory. This last one is important I noticed if you do software installations on. I think it's a luster file system in your home directory things go wrong so the compiler doesn't like doing live compilations on the luster file system it seems. So that's, I don't think that's an easy build problem it's it's the file system or compiler or combination but if you use a local directory for the build directory that should work fine. And I'll get back to that as well when I do the hands on demos. So let's start with the troubleshooting part. I'll jump to that here. And I'll walk through it a bit first before I do some hands on examples. So troubleshooting. Even when using easy build things will still go wrong, you'll get compiler errors. The installation may fail because of external reasons being out of this space or out of memory or what. And there's a couple of things you can do to figure out what went wrong and how to fix it. So when something goes wrong easy build will throw an error message at you for a whole bunch of possible reasons of course. The error messages are not always very clean. I will admit so this is something we will we will work on in the coming weeks and months. We have some ideas on how to improve this significantly typically when something goes wrong you'll get an error message like this or something failed. It will tell you, at least when it was running a command, it will tell you in which directory that was run so you can jump in there to see what was wrong. Sometimes if you're lucky you'll see the actual error in here already. But depends on how much output was generated by the command that failed, because you only see the first 300 characters of the output message here. At the start of that output you will see it straight away if it's somewhere in the middle. You won't see it here and you'll have to jump into the log file. So when easy build doesn't installation it keeps track of a temporary log file. And whenever you get an error it will print the location of that log file and you can grab that. Open the log file and check what's wrong. So in this case, it's pretty clear what was wrong. You were using a compiler option that is not supported by g++ or by the particular g++ that was being used. And if you look into it. You see here it's using user bin g++ which is probably not what you want to do. Since this will be using the system compiler and not the one in the tool chain that easy build is using. So that's basically the problem here. It should be just g++ to make sure it picks up the first one in the part. Then easy build log files you will often need to dive into those, especially when the error message or when the actual problem is not mentioned in the error message. So the location of the log file will be printed in the, in the outputs also. When you start an easy build session, and maybe I can already show that. I'll need to make sure my key is in place. So I'll do the setup. I'll do the module use of the project space I will load easy build because I don't have it installed. And I will do the configuration so built about in the local directory and prefix. Basically everything else in my home directory. When I do an installation of anything, I will force a rebuild here. You will see that very early on easy build already prints the location of the log file. So that's one of the first things it does it will start a temporary log file to track what's going on. If the installation succeeds it will throw away this log file. So this is really only useful when something goes wrong or if you're quick enough to look at it before it disappears. If the installation completes, it will copy the log file also into the installation directory itself. So afterwards, if you notice something wrong in the installation, even though it seems it seemed to have completed successfully, you can still check what easy build was doing, which commands it was running, how it set the environment, what the output was of the commands and so on. So the log file is always there either in temporary location or in the installation directory. And then let's take a look at one of these log files. If you use the right parts, so the right project. 866, then you can copy paste this part and not look for it should be there. So this is a log file for the HDF five installation. This is pretty verbose so easy build gives you a lot of information just in case you want to get back to it so it's a bit daunting to navigate through it. So all the log messages that easy build emits will have a format like this, where you have a timestamp. The name of the Python module that is part of easy build that was printing the log message and also which nine number. And what type of message it is, whether it's an inform, informative message or a warning or a debug message, I think this one already only has info messages, and then the actual contents of the log message. There's a couple of tricks. I'll get back to the last locating in a bit. There's a couple of tricks to navigate the log files. So when easy build doesn't installation it does it in different steps. There's a configure step build step install step, a module step all of those. So if you look for build underscore step for example. You'll jump straight away to the build step and a little bit lower depending on whether it's a debug debug log file or not. You will see the commands being run in the build step and the output that generated. So the steps are definitely useful way to jump around in the log file to a particular part of the installation. Whenever it's running commands, it also always does it in the same way. So you'll have an info message. So if you do info space running command colon, you can jump between the different commands that were being run. And there's a bit of noise here as well because easy build run some commands just for logging purposes. So figuring out which TCC version or LDD version uses that to pick up some information and also unpacking sources. Yeah, so there's a bit of noise here but most of the commands are the ones that you maybe care about. Is there is there any tool that structures the log file with the markdown or markup so that it can be further processing to organize you into a page, a page which you can click through only the command outputs or only the commands that were run and stuff like that. Right now no, we don't have a tool like that but that's an interesting idea. So really, most people who are used to using easy build are used to navigating log files and initially it's a bit daunting. But that could be an interesting tool to have indeed to make it a little bit more easier. While the installation is running you actually also gets partial log files, especially if you have trace mode enabled. And here I'll have to be quick but I can cancel the thing halfway. So during the build step here it was running the make command as a part of the build step. And it was logging the output of the make commands to a separate temporary log file only for this make command. So if you have a long running command you can tell this or you can check what's in there while it is still running. So there's a bit of log files flying around all over the place but hopefully that won't be too much of a problem. When something goes wrong, these are a couple of good patterns to look for, error in various ways. So this is typically what you see when you're running a CMake build, you get error one, which is the first error that occurred. And depending on which build tool is being used, the errors may be reported differently. And of course, seg faults or files not being found are common patterns to look for. So especially when doing a parallel build like this, make minus J40, the output will be a bit, yeah, all over the place you will get output from different processes in the same place. So the errors will typically not be lost but somewhere higher up that makes it a bit difficult to pinpoint the exact issue. So it can be useful to do a rebuild to the parallel one. If something goes wrong, so you, you get the, probably you get the error message last and just by scrolling up you can figure out the cause of the error. And of course, yeah, navigating with less or them or don't have to explain you that that can be useful. Yeah, looking for steps. It's handy to jump to different parts of the installation looking for commands being run can be useful and easily build also when it sits environment variables. C flex set to, for example, it will tell you how it's defining environment variables and what the previous value was if it was there. So this could be useful as well. And then next to the log files you have the build directory to so since I canceled the visa builds. I should have the visa build directory still there. It will be software software software name software version and then tool chain here it was an assistant tool chain so there's no separate actually is separate build directory based on the tool chain and then this is the unpacked source. In here, and in this directory is very easy but we'll start running the configure to build the install commands and you can check in here. What happens for conflict of log or seem make output files. Whatever else. And usually you get the place where that was happening straight in the error message. And if you I think if you do a debug log. Yeah, if you do a debug log. We show you that. That can be useful sometimes cancel it so I have a log file here and then running commands. It also tells you in which directory commands were being run so that that can of course matter sometimes you can. You can guess it quite easily but sometimes maybe it's running in the wrong directory and then you want to know so the debugging makes the log files quite a bit more verbose so you'll get a lot of information that you probably don't care about. But it does happen that information is missing from an info log that you may be looking for. Okay, that was a bit quick I think but the best way of getting a feeling of where to look for and how this all works is with an exercise. So the exercise here has an existing easy conflict file that you can copy paste and put in a file to use the name of the file doesn't really matter. You can name it example or whatever. Copy paste this put it in here, and then you can give it to easy build try installing it and see what goes wrong so it should go wrong. Pretty early on and you can try and fix the problems you run into. Maybe take a couple of minutes to try and fix those this first issue, and to make sure we don't wait too long if you've figured out the problem and you know, and you have fixed it, maybe raise your hand here and zoom. So we don't wait too long to continue I won't wait won't wait for everybody to complete it but let's say half of the people have it. We can show and show it and continue. So let's try and finish this and maybe 1015 minutes before we jump to the next one. And of course meanwhile, feel free to ask questions to make sure you have your environment properly prepared test it again here. Don't forget to add the project directory in front. So the module use you want is this one. Just as a hit maybe for the exercise we don't want to install a whole bunch of things so everything we want to use in terms of dependencies is actually already there. So don't try and look for the gcc 8.5. You won't even find an easy company file for it it's a version that doesn't exist. So, if you raise your hand I'll assume you've. Yes. So, finish the exercise or if you have a question go ahead. It's a bit obvious because the path for the, for the download this is not there. So I assume that it is something like, like in the Docker file you need to get to the file in a local place and point to it or to put the URL right. Yeah. I was showing a different error here so because of my testing it's already finding the source in my case. Yeah, I think in my case it was showing a different error message that. Yeah, let me clean this up. Sorry about that. So what you should be seeing is still finding it. This is a matter I get in the wrong error. So it will tell you is it's not able to find the source file right so it's able to find it for me. I can just lost this away actually don't really care. I was removing the wrong source file of course. Yeah, so it will try to download it. This may take a while because it considers a couple of locations and tries a couple of download locations and that will fail. This means easy build doesn't have the source file downloaded yet. So it doesn't have it available on the file system in the location that considers. So that means in this source path, which is in my home directory. There's nothing here I just removed this directory so it's not there. And it's also not able to download the sources. And if you check the easy config file. It just specifies the source file, but even though there's a comment here that tells you where to download it easy build is not able to use this of course. So you have two options. I think I have this in the slides as well. So the source file is not available yet. It's either you download it yourself with girl or W get and then you put it in the right place. Or you somehow convince easy build to download it itself when it can't find it yet. So just a question about the source code. So, so it seems like there is a defined source code directory for for easy build. Is that the standard procedure you would use for let's say, installing a package which has some kind of commercial license so you cannot download the source but you have to get the source to some other place and then you put it in this directory. Exactly. Yeah, if easy build cannot download the installation files itself you just do it manually and you park it either directly in here or sim link it or doesn't really matter. So if easy build can find it in here, they will not try to download. And usually we have these checksums in place as well to make sure you're actually using the right file. So you don't have a corrupt download or a file that's different from what easy build or what this easy company file was tested with so easily will check this checksum as well. So to show you if I go in here I have sources. Easy build sources. You'll see it already has created the file itself so it will if it's built downloads the file it will put it in the first location in the source path so you can have multiple locations there. But it will only right try to write to the first location the other ones are just read only so you can actually have sources spread over multiple locations. And then easy build will only use the first one as a right location. What I can do here. I can just submit or GZ is that the one it's looking for looks like that's not the one that has a dash source. Okay, they just create a file for it there. It will find it and then say okay I found this file but the checksum is different so this doesn't make sense to me. So either this is a corrupt file or you try to trick it into using something it's not happy with. It's not there it will not be able to check it. But usually all the easy country files we have an easy but you have these checksum someplace. Okay, to avoid losing too much time couple of people have already fixed it. There's two options let me show you both. So the easy country files actually has a comment that says where to download it. I can just download this manually. That should work. It's not the small one but should be quick enough. And to make sure you have the right file. You can verify the checksum which 256 some so this looks exactly what's here. That's good. Just leave it in place. Next to the easy country file is able to actually also be happy. So the location where the easy country file itself is is also considered when looking for source files. So if you put this in the same directory it will work. So what you want to do indeed as Peter suggested is you want to move this into the source path where easily built will look for files first before considering to download it so if you now try this, it will be happy. It will find the file. At least they will be happy in the sense that it won't complain anymore about not finding the file it has also verified the checksum so that's okay. This is a manual step. We can avoid that having to doing the download manually, at least if it's a publicly downloadable file. We can tell easy build where to download it using the source URLs. Easy config parameter and they should not include the name of the file itself. The name of the file and easy build will then for each of the sources here it will try to download them at this location, they will add the name of the file automatically here. If we do this now. This is the wrong one will remove this one. To show you the download happens. Now do the same example again. Easy build is able to do the download it will verify the checksum and only then will it continue the rest of the installation. So as long as it's a publicly available downloads, you might as well let easy build to the downloads itself. And either you have a checksum already maybe you get the checksum from the website of the software. The checksum that's in here is a shout to 50 shout to 56 checksum, but it also supports in the five some and easy build will recognize automatically what type of checksum it is, or you can also tell it if it's a particular different type of checksum. It knows about a whole range of different types. So that gives us the source files at least we can try to start the installation now now something else goes wrong so that was the one which I showed originally, which is the tool chain. Can I ask about one line in the eb file? There is the replacement of a version and past the brackets there is an s which will appear since it is past the brackets not inside the brackets the s would appear to be part of the text. However, in the actual file name the s is not present. Where does this go? The s is part of the Python templating mechanism. The s stands for substitution. So if you do this in a Python shell, let's say source equals this. What's actually happening is easy build will do this in the background. And even though the s is here, after the replacement, the s is not there. So it replaces this part with the version. All right, thanks. Yeah, this can look a bit confusing if you're not used to this. And there's there's different ways of templating of course and in Python as well. Depending on which Python version you use. So what easy build does it just basically executes this easy config file as Python code. So whatever you can do in Python should work here as well. So if you if you're a bigger fan of different string templating mechanisms they should work as well. Easy build just make sure that that's all properly replaced. So the error itself is pretty clear. It's trying to load a module for GCC 8.5 since that's the tool chain that's mentioned in the easy config file, which is not there. This GCC version doesn't exist. So you, there's no point in trying to install it. And the easiest way forward is just using a different GCC version. You're able to find if you did this multiple use. And here I guess there's actually also two ways to fix it. I'll show you both. And I'll maybe also explain why one is better than the other in this particular case. All right, I'll continue. So we don't have GCC 8.5. We do have a GCC 10.2. In the prepared software stack. So let's try using that. There's two ways that maybe the most obvious way is changing the version here in the easy config file itself so editing the file and changing the version. Before I do that. That's the recommended way I would use here before I do that there's another way, which I believe you want to show you. So there's a try to chain version option in the command line. If you do this. Oh, it's actually not happy at all. Okay. I was expecting this to work, but it's not because then it really wants to have the GCC tool chain available already. Okay. So, say you had a different version in here that is available that is aware of. Even though you don't have it installed. If you do a dot three, I think they should. This should work. Oh, and already found it even better. One as well. That's what you get when testing. Now if you try this, if you start from an existing version, meaning a GCC version where easy build has an easy coffee file for it doesn't have to be installed. And if you try to change version, it will replace the tool change version automatically and try to continue. So here you see it was indeed using GCC 10.2. If I do a trace, if I enable trace mode, you will see it even better. When it's preparing the environment it's loading this as the tool chain model. So this is this is one potential way forward using try tool chain. What will try to change does is it takes the easy coffee file it creates a temporary copy. It changes the version there and then tries to continue. If the installation fails. That's still annoying because your original easy coffee file hasn't changed and there's probably additional fixes that need to happen here. So for that reason, in this case, it's actually better to change it in here and then try to enable trace mode again so we have a better view of what's happening. So now it's happy with the tool chain it was able to load the module. As a part of the repair step where it sets up the build environment. And then it tried to run the build command. And here it's not very happy. And in this case, the actual error message. I think it's not shown here. So you only see the first part of the output and it's cut off here so there may be stuff after this. You can see an actual error message here so now you have to dive into the log file to see the actual error. So let's say in the more general scenario where you have an easy build or an easy config and you want to change the tool chain or actually it doesn't support the tool chain you want to use. So how would you normally proceed what would be the recommended way so it seems from what you are demonstrating now that the way you're supposed to do this is that you copy copy the easy config file into a new one for the correct tool chain and and change just to change the tool chain rather than using this try to change command. And it depends a bit what what your goal is. So if you just want to take an existing easy config file that you know should work. So not a broken one like this. And you want to install it with a newer tool chain then try to change version is okay. Because you'll, it will you will get a temporary copy while doing the installation for one wants the installation completes, you'll actually get the resulting easy config file in here. Easy config archive, where all the, and actually the fixed sub read is already in here, because I was testing this yesterday. Whenever installation completes the easy config file that was used will be copied in here and not only in here also in the installation directory. So, and at that point you're not really worried. About the temporary copy of the easy config file because you know you'll get the final copy anyway. So for this HDFI for example the easy config file that was used is also copied into the sub into a sub directory. So in that sense it's okay to use try to change version. Because you probably don't want to mess with the temporary easy config file anyway so it's useful in that here it's less useful, because the installation fails and you probably need more fixes in the easy config file so you're going to have to fiddle with the easy config file manually anyway. So there's little, little use. There's other other try options as well, which I think we'll be using in the next part. So they are useful. The reason they are named try is because you're indeed trying something that maybe people haven't tried before. And actually in this case will will, I had to fix the example when jumping to GCC 10 because it was failing because of the new GCC version so every now and then you run into surprises of course. Yeah, depends a bit what you're after. Do you are you're trying to fix something that's broken or are you hopeful and you just expect it to work when you jump to a new tool chain. That's okay. And another aspect when you make a new tool chain is that you usually also update dependencies as well. So this one doesn't have any any additional dependencies. Yeah, but if it depends on three other software packages you may have to update update those versions as well so it's a bit of an iterative thing we have options for that as well. There's try update depths and I think somebody was playing with that yesterday. And yeah, also there it's called tried because we do like a best effort approach but there's no guarantee that things will actually work. It depends on whether you have easy conflict files for the dependencies already with that tool chain or not. You may have to do manual job anyway. Okay, let's continue here and see what goes wrong. If we look into the log file. So we get the location of the log file here so we can open it up. This may be interesting to show the last locating as well. Easy build has a last log option. We just print the location of the last log file. That's handy because you can do this and define an alias for it. So if you miss your favorite editor you can just run last log and give that to them as a location and then you never have to copy paste the location of the log file. It will be different every time it will be in a different temporary directory. So last lock can be very handy. Typically when something goes wrong you start from the end of the log file so you jump to the end and you start working your way up, maybe looking for patterns like error. Easy build also emits log messages with error but here we see the actual problem where GCC is not happy with the dash fast option and it's giving you a helpful hint to use dash op fast because it knows that exists. So we'll have to fix this by editing the easy conflict file so the fast comes from here. So build ops is the additional options we give to the build command so in this case make. And this is where it has the dash fast hard coded the obvious way to fix this is as GCC suggested is to use of fast instead, and that will work. At least that will work in the sense that it will jump to the next problem. This is the recommended way of fixing this one. Because easy build sets up an a built environment for us, and it likes to control. So it will pass the build step and go past the install steps with actually continuing and easy build sets up a build environment, which I can show you in the log as well. So C flags for example. It sets up this environment variable dollar C flags is set to this where easy build tells the compiler to generate code for the machine it's been. The compiler is running on so MRG native and it will change these compiler options based on which compiler is being used whether it's Intel or GCC or C lang or something else. It's smart to to adjust these. So it will be good to use to use this prepared environment variable than hard coding it to that show fast. And to do this, you have to be a little bit careful. So we want to use. We don't want to override dollar C flags like we're doing here we want to add to it because this F common thing still has to be there. That's the problem I mentioned with GCC 10. It's the lazy way of fixing the problem with GCC 10 but it's good enough for this exercise. So instead of hard coding the minus or fast we want to use the existing C flags environment variable like easy build defines it and just add dash F common to it. And here we have to be a little bit careful because we have to make sure that we use double quotes. In the command here to make sure that the shell will replace those C flags with the actual value. So it looks very similar, very similar so inner single quotes and outer double quotes from a Python point of view. This is pretty much equivalent but when running the command it's not. And I'll show you what happens. If you look into the log file, you will see it has a hard coded for flags. Here you get GCC gets very angry at you because you gave it flags it doesn't know what to do with that and of course the dollar C that was in front here was replaced by the shell as an empty variable and then you get flags there. First of all it gets very confusing because it's not really clear where the flags come from. But you're not getting the behavior you expect so rather than the shell replacing the dollar C flags it replaces the dollar C and you're in trouble. This is not what you want. If you do the double quotes it works. So the dollar C flags that we get from the easy build environment, we will be building with minus or two minus mRG native and some other compiler options that easy build specifies and also those you can control in various ways. So that's well covered in the easy build documentation. So this brings us to the last problem. So fixing this two options an easy one and the proper way I would say that brings us to the last part which is the failing sanity check. So whenever easy build doesn't installation it runs a quick sanity check at the end where it can do two things it can check for it's better to do this in trace mode again, because you'll get a bit more detail. It will always check for certain files and directories to be there and depending on what's in the easy conflict file and what's in the easy block, it will check for different things. So here it knows that for separate specifically it's expecting these two binary files to be there. And this non empty directory to be there in the installation at it seems it's finding those so that looks okay. But then it also tries to run a command so one of these binaries with an option and that one failed. So something is wrong here. And the output already gives a strong hint to what's wrong. And then when checking the easy conflict file again, you will see this is specified here. If you check the output. You will see the full output of the attempt to run dash dash version. This is a partial output you have to scroll up a bit to see the full output. Here, this is the full output. So this tool also gives you all the supported options and the one you're actually looking for is minus capital V I think that's also a useful way to jump around and log files all these. Statement starts with a double equals space and then a date. So if you want to jump to the start oven. The start of the output of a command which can also have multiple lines like this. You can search for double equals at the start of a line. Anyway, it's somewhere in here but the way to fix this is to use the proper argument which is lowercase v here I don't think that's 100% correct but it will at least make the sentence check pass. So what the sanity check amount actually does doesn't really matter too much, it has to do something useful in terms of running a command that makes sense dash dash help dash dash version. So something quick. So here with fixing this we can redo the installation that will work. But since it only failed in the sanity check we don't have to start from scratch. So if this was a build that took half an hour to complete. And it only fails in the sanity check because you made a typo in a file or in a command it's very annoying to have to start from scratch again. In this case you don't. If it's only a sanity check issue you can tell easy builds to only do the module file. So in that case easy build will skip the configure build install steps it will assume those have already happened and have happened correctly. But it will do the sanity check and it will do the module generation part, and only if the sanity check passes with it actually generate the module file. Let me do this again with trace to see what's happening. So here you see it's skipping a bunch of steps, it's not even unpacking the source files, because it assumes that that won't be needed to generate the module file it's skipping all the actual building installation steps. So it's only only proceeding with the sanity check part checking for files running the command checking the exit code this looks okay now so it's happy and it generates the module file. And now, as long as we make sure that we pick up the modules that are installed in our home directory. Subread available, we can load subread, and then run the feature count commands ourselves. That completes that installation. One, one, maybe small reminder here if people are seeing issues with modules not appearing. So if you want to make sure that your lmod cache is outdated so lmod generates a cache file by itself. So this one was generated. Just now I think. Yeah, just now. So if any additional module files appear in my home directory, I'll not may not see them. So that's actually a bit of a misconfiguration maybe on putty. So that is just to remove the cash file. So lmod will be generated you can there's an lmod configuration option that can prevent this from happening. So that's a small detail if you see modules not appearing a module avail. It's probably the lmod cache. That's getting getting in the way. Okay, I'll continue with the next part unless there's any questions I still see some raised hands but is that questions or is that finishing. I have one question can it sure this is much is speaking. I've managed to broke something because after I changed the minus oh fast. I'm getting error that I will copy into the chat. In one second. Unknown error from the linker. Oh, okay. Yeah, I saw that when testing as well. Are you using the temporary directory as a built part this not really. That's the reason that's important. Yes. So apparently, the LD gold linker or the compiler or the combination of both doesn't like it when something is being built on the shared file system on putty. Okay, I see. I was, I was very confused by this as well but if you use a local directory you will not see it. Okay, thank you. It's a problem that happens more often with parallel five systems. Yeah, it's it's usually not a good idea to do build directories on the, on the federal fastest, whether it's last or gps. Is it related. Weird issues. There are similar problems from time to time. Is it related to Python IO or to easy views. It's due to the fact that the last is, is doing quite bad with a very big amount of small files and in fact, for example at the front that we are doing all nearly all comp, and compilations on the MFS so we are doing it in the RAM using RAM disk. And it's, and by the way it speeds the, the thing quite, quite, quite dramatically because you have usually you have bigger, bigger amount of huge amount of small files and if you if you put it into the RAM disk is very, they could be accessed very fast. And, and last or gps or other, these parallel five system are, are very bad with, with locking files and creating small files and so on. So so so so it is, it is very bad idea to use the, these shared systems to compile anything. I understand the solution but I was rather asking about who, who really whom really hurts this problem. Definitely the file system just the file system and this that you, these five systems are used to you, they should use the big files and they, and for example, creating the file opening the file it is quite, and how to say. Yeah. It is. It's, I think it's an issue of IO intensive and also timing. So, I think the compiler or the link or things it has created a file, and then when it checks again the file is not there yet because the parallel fastest and doesn't react quick enough. Especially that, that to create file you have to, you have to go through the all the way down from the, from the server and meta server and, and up down to the OST so so it is quite how to say. Yeah, it costs, it costs time and it costs resources on the on these shirts by systems. Yes, I understand but I, I'm asking because I'm pretty good. I think if I only run may make me new j 40, I will not receive such problems. So I was wondering actually you do I was just doing it. I was just doing it manually and you do, you see the exact same problem. Okay. I, I rather make minus j in my home directory so just by unpacking the source and running this. And you see the exact same problem. If you do it on the local fastest and you will not see. You know, it is a build or fight and or I get such. I get such error messages frequently when building on, for instance, media has also I got them occasionally on GPS also. It's just a matter of five systems are not made for builds and I think it's actually the timing problem that cannot mentioned. In some cases I managed to fix it by putting a very, very short and extremely short sleep a fraction of a second behind the command where it went wrong. And before before the command where it went wrong. I see, thank you. But yeah, it's a confusing problem I was scratching my head as well when I was casting and not being careful with using local directory looks very confusing. It's very hard to tell it stuff. Okay. Let me look at the timetable again, I think we're already a little bit behind. It's not a big problem. I will walk you through the easy config files parts, and then you can take a break before we do the hands on example, and maybe some exercises as well. And for this I'll use the tutorial website here so I'll jump through this. So creating easy config files, whenever you do an installation with easy build, there has to be an easy config file for it. So either one you created manually, or one that exists in easy build already, or one that is generated automatically, for example, with the try options. And at first. One question that pops up often is, do I just need an easy conflict file or do I also need an easy rock. So to remind you about the difference, an easy conflict file is a dot eb file, which is basically a key value definitions name version tool chain dependency so what, what should be able to use. And the easy block is the actual implementation of the installation mechanism. How should that installation be performed. There's two types of easy blocks will get back to that in the easy blocks part. Here are some rules of thumb to figure out whether you need a softer specific easy block or whether you can get away with a generic easy block so if there's very important values for easy conflict parameters. That maybe even depend on the tool chain or depend on which dependencies or which version of dependencies that are being used. You'll have to align multiple things in an easy conflict file, which can actually be derived automatically so if that's the case it's probably better to look into an easy block instant. If you have an interactive commands, you can't really handle that from an easy conflict file at that point you probably need an easy block unless there's a way to make that make the interactive command non interactive. There's lots of custom configure options that are specific to dependencies, or if you have to create configuration files that are not easy to copy or modify based on files that are there already. Or if you feel you're hacking around in the easy conflict file by putting hash characters and build ups for example to escape part of the build command that easy build will use then you probably need to look into an easy block. Things like open foam tensorflow and all these more complex applications typically have their own easy block because they're just way too complex to handle in an easy conflict file. And you get a better feeling of where the line is it's a pretty thin line and it depends a bit on on your personal preference as well. But hopefully by the end of this session you'll have a better feeling once we've written some easy config files and look that looked closer at easy blocks as well. When writing easy conflict files, usually there they have a dot EB extension but that doesn't really matter easy build doesn't care about the name of the file or the extension, as long as you specify directly on the EB command line. When easy build goes looking for files, and we discussed this in the in the previous session. And then the naming does matter. So it expects your name like software name software version to chain version suffix. And so when the robot mechanism tries to find easy conflict files it does this only based on the name, not on the contents of files. Easy conflict files themselves are in Python syntax so they're basically executed as Python as well and easy build just takes the new variables that are defined as the result of that. So you have to figure out how to do the installation. The order of easy conflict parameters mostly doesn't matter. So you can put name at the bottom and put you in the top. That works fine so it basically just key value so order doesn't matter with the exception that if some things are defined. Usually it was of other easy conflict parameters then order does matter of course something has to be defined first before it can be used to define something else but that's expected. And usually we try to keep the same order of parameters in easy conflict files just to make it easier for humans. And they're more or less in the order that the installation goes. So configure options go before build option sanity check goes last things like that. So there's five easy conflict parameters that have to be defined by every easy conflict file software name software version or not a surprise. We always require the homepage and description for the software. This is used to generate the module file. And these are in the easy conflict file because they could differ between version of software so that's, they're typically not handled in an easy block. The tool chain always has to be specified as well so what compiler tool change should easily use to do the installation. These five are hard required name and version are pretty obvious these are just string values specifying the name and the version homepage and description are also just string values and the description is often a multi line string. So you have these triple quotes that are the way in Python to specify multi line strings. The tool chain is very often a proper tool chain like this like we saw in the example, the troubleshooting parts. And there are exceptions where we use this special system tool chain which means I'm just going to use whatever is available on the system in terms of compiler. There's two use cases where this makes sense. There's a binary installation so when there's no compilation being done at all so the system the compiler use doesn't matter. Or when we're installing our own compiler, we need the compiler to build that compiler of course then for example and installing GCC. We also use the system tool chain as a way of bootstrapping the GCC installation. We try to limit the use of the tool chain of the system tool chain to make sure we're in control over which compiler and which version is being used to do installations. So those five are the only ones that are actually hard required that may seem a bit weird because there's no sources here there's no dependencies here, why are those not required. Because there are cases where you can actually do installations without having sources, for example when just bundling things together. When you want five different modules that are loaded by another module, you don't actually need any sources to build that bundle module. So that's why sources are optional. That's also explained here so usually you do have sources and maybe patch files. But for most installations but they're not a hard requirement. When you do have sources you hopefully have source URLs as well where you can download the necessary files so easy build can download them automatically when they're not available yet. And usually you have checksums as well. In this case we have two checksums one for the source file one for the patch file. And whenever checksums are in place easy build validates them before continuing with the installation. List values, so also sources even though there's only a single source here is always a list same for patch files is always a list so that's what the square brackets mean. That's Python syntax for lists of strings for lists of anything. Are checksums required or just optional. By default, they are not hard required. You can configure easy build to require checksums if you want to enforce that in your own. Easy build installation, but upstream so for all easy config files we have in the central easy build repository. Since a couple of years we now hard required checksums for everything. So they're required when contributing something back. They're not hard required and easy build itself but you can configure easy builds to ensure there are checksums for everything before it install something. Thanks. And I suppose the order of the checksums does not have to match in any particular order the order of sources or patches. They are just it's a it's a set operation, not a list operation to match them. No, it's not it's a list so it has to match. So this checksum is for the source. This checksum is for the patch. We do have support for sets of checksums so you can actually give that sounds a bit weird but you can you can give multiple valid checksums for for a single source file. So if you make this a tuple so with round brackets and you give it to checksums that's two alternative checksums for the first source file. But in this case, this is a checksum for the source this is a checksum for the patch. So easy build will not say okay if the patch file matches this I'm happy no it's going to complain that the patch file doesn't match this check checksum so it does a one to one verification. We will use a list here and Python lists have order so. Another common parameter to define is the easy block. So which easy block should easy build use to do the installation. This is optional because by default easy build will go looking for an easy block that matches the software name. So if you do name equals TensorFlow, easy build will look for an easy block specifically for TensorFlow. And if that's not there, it will complain, unless you specify with the easy block parameter that it should use configure make or Python package as an easy block. You can see that in the example as well so this is optional because easy build has an automatic way of looking for an easy block based on the software name. There's a distinction between generic easy blocks and software specific easy blocks, which are already mentioned some examples of generic easy blocks or these see make make Python package you will use some of them in the examples. The software specific easy block always starts the name of a software specific is but always starts with EB underscore. So that's how you can easily tell whether it's a software specific one or a generic one. The generic ones, we can we can freely pick those names so these have not is don't have the EB underscore prefix. I will go back to that as well when looking at implementing easy blocks. In the example we are using make CP easy block, which is not in the list of the common ones. Is it related to any figures in the right from some. It's not listed here. I would say it's actually pretty common. It's derived from configure make, where the configure step is just skipped. The small step is not make install but this is copying files. So it's basically for software that only has a make file without a proper make install target. I didn't list it here but it is listed. If you look into the documentation which is linked. You'll see make CP here. It gives you an idea of what's what's going on. The configure step no make step and it will make CP will require that you specify the files to copy list. So which files should be copied into the installation director. There's a set of easy conflict parameters which you can always define so name tool chain dependencies these are all supported for everything. And you can check which ones are supported using EB dash a support for a fail easy conflict parameters parameters so this has a long list of mandatory ones. It actually lists these two as mandatory but they're not in practice. And this yeah this gets very long of course but you'll see things here like conflict ops and pre built up so the ones that we briefly mentioned and a whole bunch of other ones as well. You can get extensive, but all the ones listed in the output of EB dash a are supported for every easy block. Some easy blocks have additional easy conflict parameters which are only supported for this particular easy block, and you can figure out which ones by using the dash e option or short for dash dash easy block, and give it the name of the easy block. So if you look at the output of this, you'll see below the mandatory section you'll see an easy block specific section. So these only make sense when using the configure make easy block or an easy block that is derived from configure make. These don't make sense for example when using the Python package easy block and easy but will tell you as well if you try to define build command and you're using an easy block that doesn't support this, you will get an error. And it will tell you that doesn't make sense dependencies is very common as well. This discriminates between two types of dependencies runtime dependencies and build dependencies. So runtime dependencies are needed for actually using the software. They're also the modules for these dependencies are loaded in the build environment also when doing the installation. The module file that is generated will include load statements for these runtime dependencies so whenever you load the software, the dependencies are loaded automatically as well. For build dependencies these are only loaded in the build environment and the module file will not load seeming because that doesn't make a lot of sense that you don't need seeming to run software usually. These are, this is a list of dependencies each dependency is a couple to name version. Sometimes dependencies have an additional element in the top which is the version suffix. So if you have a non empty version suffix for the dependency. You may have to specify it as well to make sure that easy build finds the module or finds the easy country file if it's not installed yet. You see also again these template values. So we saw a version before. In this case, it's by ver. So this is a template that's automatically defined based on the Python version, if Python is included as a dependency so by ver will be There's also a pie short for which is only three dot eight, which you need and send it to Jack for example you often need only the short Python version. And there's a list of not linked here, but there's a list of defined templates in the documentation, and also on the command line. Easy config templates. You can ask easy build which templates it knows about like for example version or version minor. Or for dependencies. There's by short for which is a major minor version of the Python or the full Python version and so on. So all of these are templates you can use in easy config files. For dependencies that the version suffix is basically a label that you can add to the module name to discriminate between different configurations of the software or for example a version that depends on Python two and the version that depends on Python three. So this is a mechanism we use very often. And that's also why here you have an additional element for sci-fi bundle, which specifies in this case, we're looking for a sci-fi bundle that matches this Python version. A common way of doing that is for example for stuff that still requires Python to which does exist unfortunately. So when you have a CP to K installation for MPI only so P opt or the hybrid version the PSMP that's how you can easily discriminate between these different configurations. Then option or parameters to customize the installation procedure, which we briefly saw as well in the separate example. The, the configure build test and install steps have two parameters available, a pre in this case config opts. So this is just a string value that is glued before the configure commands and a config opts, which is glued after the configure commands that easy build will run as usual. Same for build test and install. So a concrete example here. If you use config opts equals dash dash enable HDF five support that will make easy build at this part after the configure command it will run by default, which is typically in case of configure make dot dot flash configure dash dash prefix, the location directory and then it usually stops unless you give it config opts then it adds those as well into the command. The pre options are useful if you have to run stuff before the command actually gets run. And here you have to be a little bit careful typically you use the double ampersand to create multiple commands after each other and make sure that this command run successfully before you continue with the other command. So with pre built up in this case, we're making sure that this part gets glued before the make commands, which will be run by configure make by default. To make easy block is smart or actually easy build is smart enough to determine how many cores are available so it will it will always do a make minus J number of available course. So that's dynamic. Unless you tell it not to you can restrict it to just one core or two cores if there's a good way to do that. Or a good reason to do that. So install opts prefix equals install there so this is another template, which will be replaced by the actual location of the install directory. And this make sure we run make install like this where the dots are replaced by the path to the install directory. So, you can see through these options we get a lot of flexibility to steer the installation done by a generic easy block like configure make. So having to implement an easy block ourselves. And then the Senate to check we've seen this so the Senate to check it, I think is a very important part of easy build where it tries to make sure that the installation worked that the stuff that should appear is actually there. That simple commands like dash dash help or dash dash version can run. And it's a very good way of, of at least recognizing a totally broken installation so even though make exits with code zero it doesn't mean the installation actually fully worked the way you want it to so we're strict is maybe the wrong word but we're making sure that we have sanity checks in place for all the easy country files we have in the central repositories we feel this is a very important part. So it will do a sanity check by default by checking for a non empty bin directory and a non empty lip or lip 64 directory. So if you don't tell it to check for something specific. As long as a bin and a live directory are there, it will be happy but it's probably not a good idea to rely on that. And then the model class is usually there, we have this categories of software. At least for the central easy config files we have these model classes defined as well. So that's the most important parts of an easy country file of course depending on the software you're installing. You'll get additional parameters that may be mandatory or maybe very, very much required to get a proper installation but that depends a lot on which software you're installing. And then in terms of generating easy conflict files from existing ones. So the trial software version I already mentioned that can be very useful try to chain can be very useful as well if you have an existing easy country file, and you only want to change the tool chain you can avoid doing it manually using try tool chain. Again, the try aspect here is important and this may fail if you're certainly when you're switching copilers you often run into surprises. Even when doing trivial version bumps, maybe they changed from configure make to see make in the new version and then of course you have to change the easy block and change the configure options and things like this so it's it's a good idea to try this but the chances of success are not always that big. And there's additional try options as well. This try to chain also collaborates with the dash dash robot for examples if you do try to chain and dash dash robot they will try and generate easy conflict files also for the dependencies that are still missing. There's a lot of things you can, you can do here or at least try here to help you and avoid that you have to copy and edit easy conflict files manually. So doing easy conflict files there's a small option as well that can be useful. If you want to take an existing easy conflict file and you don't really know where it is. You could use eb dash dash search first, and then get the full location copy paste that location and just give it to the CP command, or you can do eb copy EC so to let easy build copy and existing easy conflict file from no matter where to a specific file or specific directory. So that can be handy as well. And here I have an example prepared. Which I can build up gradually, but maybe we should take a short break first before we go into that. Does that make sense. Yes, break sounds very good. Let's let's take a 15 minute break so start again 10 to 11. Thank you. Continue. So what I want to do here is a hands on example of creating easy conflict file from scratch for a toy software package which I came up myself with some help from others. This is a CMake and I absolutely despise CMake so I got somebody else to write the CMake lists for me. It's a pretty basic thing it's a single source file with a header file has a CMake list so we're probably going to use CMake to get this installed and it has a friendly read me file that gives us basic installation instructions. So that looks pretty good. So I'll get started with this. Make sure our environment is set up properly which I did here. Use the central software stack load easy build configure easy build. Make sure we use the temporary directory. In this case it probably doesn't matter too much, but that's a good practice to have things properly set up. Okay, so to start with will need to make sure the mandatory easy conflict parameters are in place name version home base description and tool chain. In here, I have the tool chain missing deliberately just to show you what happens when you do that. So this is a decent starting point for an easy conflict file. We try to install this easy build well yell at us, even first yells about the easy block. Okay, that's a bit surprising. I was expecting it to yell first about the tool chain, but it will yell that yell about that next I guess. So here it says there's no software specific easy block for EB minus. So it's looking for an easy block that's specific to the EB tutorial software package. That's what it does by default if you don't tell it to use a particular easy block it will assume there's a software specific easy block that can't find that it will just give up. So in this case, jumping ahead a little bit here I'll get back to the tool chain parts. So hitting this issue. We see a CMAKE lists in the sources in the unpacked sources and the installation procedure tells us we will have to run CMAKE so this is probably going to be a job for the CMAKE make easy block which we briefly mentioned before. So I'll include that here. Usually the easy block goes on top. It doesn't really matter too much in terms of order. So we have to make sure we get this right or easy build will again yell at us. This is a generic easy block because it doesn't start with EB underscore. So you can find this in the list of generic easy blocks in the documentation, or certainly and lots of example, or other easy country files you will find that this is used quite often. So after it's happy with the easy block now it knows it will use CMAKE make. The next thing it yells about is the tool chain. So we didn't specify the tool chain. This is a mandatory easy config parameter. So we'll include that as well. We'll again use the same TCC version for this example as well. So it will be happy in terms of mandatory easy config parameters and it will actually try to do the installation. It looks like it's managed to start the configure step, but then that went horribly wrong. And from this we can't even tell what went wrong. So we can check the log file, jump to the end. And here we say CMAKE command not found. So because we're using the CMAKE make easy block, easy build knows it should run CMAKE and the configure step. And it tried doing that it even nicely generated the full CMAKE command line, which is quite long. So it specified the installation directory to be the release build it specified the names of the compilers, the compiler flag so all of that was automatically done by the CMAKE make easy block. And it tried running that command and it failed horribly because CMAKE is not available. Indeed, if we check on the system, there is no CMAKE. So that makes sense. That's this part here, CMAKE not found. We do have CMAKE available though as a module. If you set up the central software stack you should see this one used. This one available rather. So this particular version of CMAKE we can use as a build dependency in our easy copy file here. We can add build dependent dependencies, which is a list. And each, let me do it like this is a little bit cure, each lead each entry in the list is one built dependency, and just giving the name and the version in this case is fine, because there's no version suffix for CMAKE. So it will happen now and to return back to the previous session last week. We can always check what easy build would do without it actually doing it using the extended dry run option or dash x. So if you use dash x here will get a lot of outputs. So we better pipe this to less. Easy build will tell us what it will do. After a couple of seconds. So we can make sure we got this right. We make easy block we already knew that we go to the configure step. We can check, we can check what it will do. So this is a bit more. It's actually setting up environment variables as well. It wasn't showing that in the trace output. It will try running the CMAKE commands. And to check that it's actually picking up on the build dependency we can scroll back to the repair step and here it says I will load CMAKE as a module in the build environment. So basically did this what you load CMAKE. And because the module already exists, it knows what kind of modules will load it at the time of the installation. And it also tells us the full build environment. In terms of environment variables. If we drop the dash x it will actually go ahead and do that. It should be able to find the CMAKE command now and do the configuration. It's still failing in the configure step, but probably with a different error. Jump back into the log files, go to the end. So now this is the error we get this looks like an actual proper CMAKE error. So it found CMAKE, it tried to run it and now it's CMAKE itself that's complaining. And it's saying the source directory here does not appear to have a CMAKE list.txt, okay. So we can, this is the location where CMAKE is being run. Or actually it's the source directory that's passed to CMAKE as an argument. If we check in here, we see this directory so whenever easy build uses the CMAKE make easy block, it will set up a separate build directory because you're not supposed to build CMAKE software straight in the source directory. It has to be in a clean empty directory on the side. So easy build does that for us automatically. But there's nothing here. This is empty, even the top directories empty so there's no files here at all. Well, why not? Well, we didn't specify any sources. So sources is not required. Easy config parameters, it's optional. But of course if you have software that actually if you have a proper software package you want to install not just a bundle of things that are existing modules you need to specify source files. So we'll go ahead and do that to fix this error you need to, of course, give it source files like this. Or at least we could use this. This would work. But this hardcodes the version number in the name of the source file, which we is a bit silly since we already know the version here and we want to reuse that. We can do that with this template. So again the same version template here we saw before. That's a little bit nicer because if we then want to bump the version. We just have to change this version number either directly in the easy config file or with tri software version. So what tri software version will only do is change this line. It will not touch lines like this, which have the version hardcoded, which is why we like to use this templating mechanism. And in this case because this is a pretty standard name for the source file. We actually have a constant that's exactly this. So name dash version dot r dot g set so we could use this as well. So let me use that one and this is a constant template value. So we saw the value the templates before. Source star. She said. You'll see it mentioned here source star GZ is exactly equivalent with name dash version. She said, they should work. Of course here the same issue. We need sources. They will not be available yet. And we would like easy build to download them. So we'll give easy build the location where to download this file. The full URL was mentioned. On top. Here. This is the full location. So we just need this part. Without the file name. We put this in source URLs and this is a string value so we give it quotes. You try this again now is able to actually download the source files to the unpacking and then when we run C make it will now be able to actually do the configuration, whether and complain that it can't find stuff. It fails again in the configuration step. I want to blame C make here but it's not really see makes fault, but it's failing in a different way now it was actually able to do the configuration but there was something it didn't like. It says here eb tutorial message is not set. So this is a required. Configure option for this software package. So this is a very handy read me file that doesn't tell you everything. That should be no surprise. So there's a required configure option here. So we need to give it an extra option to the C make command and for this we can use the config ops is a conflict parameter. So we can tweak the C make commands as we need to. So we're giving minus D this configure option and we give it this value. Hello from the easy build tutorial so that should make the configuration step happy. Do this again now it should at least pass the configure step. There it goes. It does the actual build it's a single source file so that's quick it does the actual installation which is make install in this case, and then it proceeds and it fails in the sanity check. And here it says sanity check failed. No non empty directory found at lip or lip 64. So the standard sanity check is expecting to find both a bin directory and a lip lip 64 directory. If we check in our target directory so easy build software name of software version. We check what's in there. There's only a bin directory with a command no live directory is no surprise for a small software package like this. So we'll have to tweak the sanity check. So let's do that. There's two parts to a sanity check. So the part that's failing is the files and directories it's looking for so that's done through sanity check parts, which is a Python dictionary. So that's the curly braces, which expects a dictionary with two entries files and directory so files and theirs. And in this case will only have a binary file bin eb tutorial we expect that to be there. And there's no point and also checking for a non empty bin directory because that's already implied here, since we're looking for a specific file. There's no other directory so we can really list anything useful in here. That should make it happy. And again since it was failing in the sanity check we can use module only. We don't actually have to redo the configuration and the installation. We just want to redo the sanity check and the module generation. So that worked fine. It was happy with the file it's found. Now we could actually do a little bit better in the sanity check. We know the file is there but does it actually work. This is a binary so we can try running this sanity check commands is another part of the sanity check where we can just run the tutorial. We don't have to give it the location where the command is because the module file that easily generates will update the whole part so the command will automatically be added in our bad variable. So automatically if it sees a bin set directory it says okay this probably needs to go in part so we'll go ahead and add that by default. If you're not happy with that you can change it but usually you want this. I can redo the installation and enable trace modes to show you that it's running the eb tutorial command in the sanity check. So it's finding the file it says okay and then it's actually running the binary. It says okay so this looks all good. And let's take a quick look at the module file generated as well. So there's a bunch of things here. We always set prefix path for all installations we do so see make us aware of them. And we update the door path variable to add the bin sub directory so the commands that are provided by this installation are found as well. So everything specific to easy build eb root eb version are always defined ignore the eb devil thing will get rid of that soon. The model file will also generate the tool chain because that's assumed to be a runtime dependency. And it includes some description of the software based on the homepage and the description that's available in the easy config file. It gives us a full working easy config file for this example software package, which is also shown here basically exactly what I came up with a small difference. The checksum is not there. I can add it manually, or there's a command line option that does that adds the checksum for me. I can inject have to be a little bit careful. I can actually show you what goes wrong. If you do this. Easy build will yell at you and say example dot eb I don't know what type of checksum that is so the inject checksums takes an optional argument. If I do it the other way around. There's no optional argument to inject checksums and then it will assume. We want a shot 256 checksum which is the default one the default type we use in easy build if I do inject checksums it will first check if there's any checksums already. If there are it will refuse to replace them unless you force it to. If there's no checksums yet it will determine the checksum of the file it found and go ahead and add that into the easy country file in here. So you can also do this manually of course especially if the software mentions checksums on their website you want to use that one and make sure that the downloaded one matches that. Check checksums is quite useful as well. So that's more or less concludes this this part I do have a couple of exercises but I don't know if we should spend time on them. Right now, or if we continue with the easy blocks part and try to stick to the time. I don't know Nicolas what makes most sense. Yes, probably, if you're short in time probably it's better to go with easy blocks in the exercise something we can do anyway. Because they're very well done in the tutorial so it's. Yeah, so there's a small extra like let's hear what the other people say. Does anyone object to moving on to easy blocks. Let me ask that. So the exercises what do they do actually. So the exercises. There's four different ones for small ones so the first one is taking the example which I just created here and. So if you run it manually let me show you that maybe as well. So what we compiled. It's a small binary that just spits out the message you give it. And the first exercise is to change this message. So redo the installation and change the message and make sure it mentions your username so you can use the dollar user environment variable to make it produce the username. But you have to be careful with quoting to make sure we don't get a lot of user here but your actual username. So that's the first exercise to change the existing easy config file to make it do that. The second one is to install a newer version of the software so there's in here. We installed 101 there's also a 110. So you can try doing that. So that should be very easy as well. Yeah, and also it tells you here not to make any manual changes so that's a tip. So don't edit the easy config file but do it in a different way to install a new version. We mentioned that a couple of times. And then the third one is a bigger one. So this is for another software package, which is actually quite in bindings to this TV tutorial. And so that's another easy company file you have to create from scratch. And it gives you some tips on how to do that. So I think this is a good exercise to try yourself and basically follow the steps I did. Start with mandatory easy config files and gradually work your way towards working easy config file and try to fix the problems you hit along the way. And the full solution is here at the bottom that you click this you'll get the full solution. Yeah, I was just wondering in general how let's say how complicated they were because sometimes it's nice to be able to ask questions. I mean we can do that later so it's okay. You can ask questions in the rocket chat and I'll hang out there and anything that doesn't work or any questions you have feel free to ask them there. Certainly the rest of the day today and even the rest of the week I'll hang out there so feel free to pick me. I'm happy to answer them. So they are introductory level they should be relatively easy and give you lots of hints and if you really want to you can peek at the solution. So it should certainly be doable to do this yourself. Okay, then the easy blocks part. This is another big section. I will only cover the basics here because you can take this very very far. But I also have to exercise is here at the end, which again include the solutions. And maybe if I have enough time I will actually do one of them as a demo and then you can do the other one yourself. So easy blocks are Python modules Python scripts in some sense that talk to the easy build framework to get an installation done, and they use the information from an easy conflict file to figure out what exactly needs to happen. So which versions which to which versions of dependencies, which tool chain to use where to download sources. So that kind of information is taken from an easy conflict file. Again there's a difference between generic and software specific easy blocks I already explained that that's also important in terms of how things should be named. So, there's two naming aspects to easy blocks. There's the name of the Python class, and the name and the location of the Python module file itself. So it's important to get both of these rights in order to let easy build pick up on them, at least for software specific easy blocks that's very important to get the naming right. So the first aspect here is a Python class name for software specific easy blocks this always starts with EV underscore. That's mainly done to discriminate between generic and software specific easy blocks, and also because we want to be able to support any software name out there so we have this escape mechanism or this encoding mechanism. Because very early on we saw software names that have like open pipe character speed shop or whatever funky things people come up with hash hash characters or they seem to make a sport out of it to come up with something funny or cool looking. And then you need a good way to handle that and a tool like easy. So we came up with this encoding mechanism. You take the name of the software and you mangle it a bit so you can use it as a class name in Python. And what you do is you replace spaces by underscore you replace dashes and other characters by a translation of the symbol into something that starts and ends with an underscore so dash becomes underscore minus underscore underscore becomes underscore underscore underscore if you want to say it like this. There are rules to follow here, but you can basically ask easy builds what the class name should be. So with the symbol Python script basically you can import the function that does the encoding, and you can just give it the name of the software and you'll get back the answer, what the name of the Python class should be. That CDF dash Fortran becomes that CDF underscore minus underscore Fortran. That's the class name. The class name for generic easy blocks don't you don't need to do the encoding thing here because there's no direct relation to a software name you can pick anything. So configure make bundle see see me quite the package. It doesn't really matter. You're in full control here. So if you do implement your own generic easy block. You're fully free to pick the name and it doesn't start with an eb underscore either. The name of the module files so the Python file itself also has to be what is expected to be. And here you basically take again the software name you do lowercase, and you replace a dash with an underscore and any other special characters you just drop. So gcc becomes gcc.py net CDF dash Fortran becomes net CDF underscore Fortran and anything else like braces or spaces you just kick them out. Like this, well, I guess the space also becomes an underscore. Same thing here you can ask easy build based on the software name. What should the name of the module file, the Python module file be, or based on the class name of the easy block what should the module file be so for both of them you can use the same function and knows how to tell these apart. If you look into the central easy blocks repository. You'll see that we organized things a bit based on the first letter of the software. And we have a separate directory for generic easy blocks. So see make make configure make. Python package, all of these live in here. And then in these letter directories you have things like the software specific easy block for TensorFlow, and if you look into that one. Everything lower score, and the class name, which is a bit lower. This is a quite complex easy block. The Python class name is EB underscore TensorFlow with the right capitalization. If you're coming up with your own easy blocks. They don't have to be a part of the easy build installation itself. So you don't actually have to put them in this organized directory here, they can live anywhere. You can include easy blocks configure option to add one or more easy blocks into your Python installation. This is both for new easy blocks and also easy blocks which you have changed, and you would like to overrule what's in easy build itself. If you're, if you made a copy of the scenic make easy block you made some changes to it anyone use that one. As long as you give the parts to include easy blocks, that one will be used and not the one that's included with easy build itself. So you get full control over what easy blocks are used. And I will show that in the, in the exercise as well. And looking into the easy block itself. Overall structure is bunch of import statements and then a class definition of the easy block. And in the class definition have multiple methods that are implemented for the different steps that you want to customize in this easy block. In this case it only shows configure step but if you're deriving from the base easy block class, then you always need to define configure, build and install, because those are not defined in framework at all. You will get a not implemented error if you try omitting one of these. The minimal easy block when starting from scratch is configure, build and install, you have to implement those methods. The different steps that easy build do all have an corresponding underscore step method. And all of the methods that you can customize in an easy block you can check in the API documentation. Here, this is the definition of the base easy block class. And you'll see methods here like build step it says here abstract method so that means the easy block class doesn't implement this it just gives it as a handle. And if you actually want something to happen you'll need to implement the build step method yourself. And there are a bunch of other ones as well. You can redefine or enhance if you want to, depending on your use case of course most of these you probably don't want to change. Some of these you have to like configure step. And there are cases like prepare step for example, where it sets up the build environment. And one could be interesting if you want to get additional environment variables defined, or you want to change something what easy build does by default. You can override this method as well and make some changes. Okay, that's the most important part of that. Then you can also derive from existing easy blocks typically from a generic easy block. You have a software package that uses configure make but has some specific stuff that needs to happen for this particular easy block you can start from configure make and just customize the things you want to customize. In this case here we're extending the configure step and we're first copying a file from an example file I guess to a file that's actually going to be used for the configuration. And then we move on to the actual running of the configure command so we just call back to the parent module. Yeah, so deriving from an existing easy block you start from configure make for example, and you derive the class from this rather than directly from the base easy block class. Then the easy config parameters which are defined in an easy config file like this, you can access them in the easy block through self dot CFG so CFG is short for easy config in this case. And all all the easy config parameters are supported, also the ones that are not defined in the easy config file, you can access this way. Some of them we have a more directory like you do if you do self that name that's equivalent self that config. That will become more clear in the example as well so also things like config ops pre built ops, all of those are accessible through this self config class variable. You can just check the values of these you can also update them. If you want to add additional configure options before you actually do the configuration. You can change values like config ops, but you have to be a little bit careful. If you want to just replace the value of an easy config parameter in the easy block, you can I just reassigned a value a different value. That's not always smart because then you're totally ignoring what's in the easy config file. So usually you either want to you want to honor what's in the easy config file, since that's where the installation is being controlled. But there could be cases where you know it is safe to just ignore what's there. That's possible. Usually you want to update an existing value. And for this you actually have to use the dot update methods. Otherwise you will run into surprises. So in this case where we want to update a list value so an easy config parameter that's a list. We want to add the example string as an extra element to that list. We do it like this with self config update it'll work. If you try this so grabbing the value, the existing value and use the dot append method which is a standard Python function or method I guess for adding stuff to an existing list. This will not work. This existing some list value will not be updated because of some some implementation details that the value you get here is not a reference to the actual value that's being kept but it's a copy and you will run into surprises. So whenever you want to update an easy config parameter use the self config updates mechanism instead and then it will work nicely. So just to standard easy config parameters you can define custom ones as well that only which are only relevant for this particular easy block. And this is done through this static methods named extra options. So here we're defining two additional easy config parameters one that's required one that's optional. You can also tell easy build that this is a mandatory one and this is a custom or option optional one. So that means this one has to be defined by the easy config file, and if not easy build will complain. And this one you can define but it's not strictly required. So and that's done there through this static method which is a little bit different is the standard Python constructor. It's useful to use if you want to initialize some class variables so things that you know that you will define later on in the configure step for example or in the build step and then also reused and other steps. So that's useful to do that in the constructor. There's an important side note here which I'll get back to a little bit later. You have to be a little bit careful when you do things like this when you're sharing variables between different steps. So I'll get back to that. And then common file operations reading files writing files copying files creating directories changing files by applying regular expression substitutions. Removing files all of that is functionality provided by the easy build framework. So you can use the existing functions rather than figuring out how to do this in Python. All of these functions are available in the file tools package that's part of the easy build framework. And they're all listed here with some hopefully informative help and documentation for each of the options. These functions provide. And similarly for running shell commands like make or see make. There's functions for this available as well as a run command function to run shell commands, which are non interactive. And there's a separate function to run interactive commands like sometimes you have installation scripts, which ask you a question and then you have to give an answer. Interactively so we have a way of automating that and you can give it a Python dictionary with patterns of questions to look for and then the matching answer that should be given. If or when that question is being asked. So hopefully you don't need this because getting this right is a is a bit of a hassle. But once it's there it worked really nicely. And there's some options then for running shell commands. What easy build will do by default is it will run the shell command and check the exit code if it's zero it's happy and it will continue. It will log the output in the log file. And then continue with the rest of the installation. But sometimes you, you want to check the output before continuing on so checking for a pattern or making sure error messages for example some commands always exit with zero and maybe you may need to be a little bit careful with checking that it didn't fall over halfway. So if you need to do that you can do that by grabbing the output from the commands, grabbing the exit code from the command and then checking that yourself. If you're not happy with what easy build does by default. You can miss also tell it to run the command and it's in a specific directory rather than just in a current directory. And so on and then additionally for interactive commands, there's options available there as well to ignore questions or ignore output that sits there for a while that may look like a question to easy build but it's actually not. There's a lot of details there as well. If you're manipulating environment variables in the built environment, you can just use the standard OS and Iran, Python mechanism or OS dot set and that will work, but then easily will will not lock those or or will not report to doing extended dry run, for example. So it's better to use the set for function that easy build provides in the tools of environment module, because then easily will keep track of which environment variables are being set. It will report on them and extend the dry run and so on. So using the easy build mechanism for manipulating the environment is recommended. And then logging in errors. In your easy block you can also emit your own log messages if you want to inform the user about what's going on or how far you have progressed or whether a particular decision was made to go left or right in the easy block. You can also emit a log message as well. For this you can use the self log instance that's already available to each easy block so you don't have to set up a logger yourself you can just use what is there. With self log info you will emit an info log message with self log debug, you will emit a debug log message which is only included in the log file if he has been configured to run in debug mode. There are things as well which pop up a little bit better in the log files. If that's worth doing. And if something goes horribly wrong wrong and you want to give up the installation halfway you can raise an easy build error. And then this will result in easily reporting the error to on the command line and giving up the installation. As we've seen before. A quick example of that is that you do the necessary imports to be able to raise an error and to run a command. We run the command here and we take the output. So the output and the exit codes. We make sure that the word success and all capitals appears in the output. And then we log in a message and log message and we say okay this looks great the command seems to have executed correctly. If not we raise an error and we say success was not there so something has to be very wrong. I will just give up here. And then the sanity check. This is often included in the easy block as well. If you have specific easy blocks, you typically have a custom sanity check that says these files these directories have to be there, and these commands have to be run. So this looks very close to what you can do in an easy conflict file as well. But if it's always the same for a particular software package you might as well do it in the easy block. So you don't have to do this in the easy conflict file. This is how you can do that. What I'm mentioning here is the sanity check that's specified in the easy block is a default or fallback. You can still overrule this in the easy conflict file so if you have sanity check parts in the easy conflict file and the easy block specifies them as well. The ones in the easy conflict file will actually be used and not the ones in the easy block, and you can tell easy builds to not override them but use both at the same time as well. So sometimes you need to, if you configure the software differently, maybe you need to run a different commands and you can specify that in the easy conflict file by means of exception, rather than using what the easy block does by default. And then version checks you may have to use in an easy block as well. If you need to do something different, based on the software version, you can do that with these version checks typically that's done through the loose version. Class I guess that's provided by Python itself. So that's just to make sure your, if you're comparing versions that's done correctly and not by alphabetical sorting. So here, the name of the pin file has changed for old and new versions of the software. So we added this condition in here. And then the last thing is the additional remark. In terms of using class variables, you have to be a little bit careful. Yeah, so if you're doing a dry run DB dash X or extended dry run. You have to take into account in the easy block that if you specify to run a command that in dry run mode to command will not actually be run but easily will try to continue and report what the easy block is doing. So that means, even though you expect output to be here, it may not be there when in driver and mode and in that case you probably don't want to raise an error but you want to say okay we're in driver and mode so there's no output but that's actually fine. I'll just continue and pretend everything is okay. So this is an updated version of the example above where we check for success and the output. If it's not there. So this else and we're in driver and mode then we say okay. The pattern is not there but we're in driver and mode so I'm just going to ignore that and continue. And if we're not in driver and mode only then we erase the error. So that's an easy block that's aware of the extended dry run mechanism. So what similarly, if you're doing a module only installation so eb dash dash module only you're skipping a lot of the steps configure build install or just skipped and the code is not executed at all. And it may be important that your easy block is aware of that as well. So this has an example where we have a class variable self dot command which is initially set to none. And eventually this will become a command that we will run in the sentence check, which we also have to specify in the configure commands. So here we have a separate method, which is fully custom to this easy block it's not a standard one where we say the command that we should execute is the name of the software. Plus a dash plus the name of the compiler. So this will be different based on the tool chain that we use. We call this set command method here before we define the additional configure option. And then we reuse that self command in the sanity check to make sure that this command is actually working after the installation. The reason we have to be careful here is that this configure step will not be run when we do a module only. Which means this set command will not be run. So this will still be none when we run the sanity check. So that's why we have the separate part here if the command is still none, then we make sure that the command is set before we continue with the actual sanity check. So sometimes you have to be a little bit more careful in your easy block. If you wanted to be compatible with module one. Is there a question. Okay, I heard somebody enable his mic. That's a lot of information. I think the examples are, are hopefully helpful. But to make this a little bit more concrete, let's maybe work our way through the first exercise. Together, and then I can leave the second exercise for yourself which is building on the first one. And you actually get the full answer here in the solution box. Well, let me try to build this up. Gradually, to give you a feeling of how that's done, how to implement easy block from scratch. So before we begin, can I ask a general question? So I haven't programmed this easy blocks before just worked with easy config. I imagine that in practice when you look around in these easy blocks, there is a certain programming style or best practices and so on. How you do things. And you've shown a few tips and tricks here now, you know, smart things to do in easy block and so on. So is there any cookbook available with examples or are there specific easy blocks which you would recommend that you would check. The ones you have written, for example, or yeah, from where should one copy paste the code to make the easy blocks because I think that's what happens in practice a lot, right? You can't hear you. And if we don't hear you anymore, you may have problems with your internet connection. No, maybe the Bluetooth is better. It's very silent. I think it's my headphones acting up again. Let me see if I can fix that. Is it better now? Yep. Okay. My headphones die every now and then I have no idea why. So the question was best practices. Okay. I think it's a difficult question to answer in general that the things I mentioned here are general best practices. And if you're looking for examples of how to do a particular thing. I would say the source, the search option in GitHub is your best way of finding something or asking around in the easy books like for example, the issue is the answer you may be looking for depends a lot on what exactly you want to do. So if you want to run a shell command and do something in addition like checking the output or interactive commands or there's different easy blocks that do interactive commands, for example, but also in different ways depending on how long ago they were implemented. Like in Worf, the Worf easy block for example I know has an interactive configure command, but that was one of the very first easy blocks that we wrote. So that's probably other ones that do things in a smarter way or in a better way. So, yeah, I think it's difficult to answer and if you can't easily find what you're after, asking around in the easy books like it's probably the best way there's probably people there that are certainly the easy built maintainers that are well aware of what's out there and if you ask a specific question it will ring a bell for them and say okay. For example, look at so brief because that looks quite similar to what you want to do. Okay, so there are no specific coding styles and guidelines which are enforced in order to get it accepted upstream so is the format quite free so to speak. The coding style itself is there's actually style checks in the and the CI that we have in the easy blocks repository. So, there are certain things that the test that will make make the test fail or make the ball yell at you. And other than standard that eight compliance so that's a standard fighting coding style. Not really we, when we do the review of pull requests we will probably make specific remarks like okay if you do it like this it may work but then, for example it's not not going to be compatible module only and this is how we could fix that. So it's more of a back and forth I think during the review. There are somebody who's more, more familiar than how easy blocks work or how it may affect other things that tries to help you out there. I'm just asking because you know in in some projects that they might be very strict requirements and they might be a guy that you're supposed to write the code in this way and do these kind of things in order to, you know, not really we don't have very strict things like this. So basically when, when the, the CI tests are happy, and the installation works, unless we have a particular reason like incompatibility with model only, it will probably be okay. So, so it's not like we're going to, we're going to force you to write declarative code or anything like this. What we will do is if we see you doing something that we know there's a there's a function for in the easy build framework will probably tell you yeah it's probably better to use the easy build framework function because it has side effects like logging or extended dry run reporting or things like this. So, but other than that I don't think we have anything very, very strict. Okay, if there's no more questions let me try and finish this example. I'll make sure I have a clean environment here so I'll just log in again and do the basic setup again. So, no easy builds under the basic configuration. So the, the purpose of the exercise is writing a custom easy block for eb tutorial so that the one that we wrote an easy conflict file first. We're going to try and implement an easy block for that and minimize the easy conflict file as much as we can. So I think I named it example here. Yeah, so this one we're going to try and clean up. There's a couple of things where we want to get rid of here, mainly the configure option. So we don't want to do it like this. Because this is CMake specific and we don't want to have CMake specific stuff in here in case even tutorial. For some reason things it's not a good idea to stick with CMake and move to something else we want to make this a bit more generic. And the sanity check of course we can do this fully in the easy block and then the easy conflict file doesn't have to worry about it anymore. And we don't want to use the generic CMake make easy block for whatever reason, we're not happy with it. So the purpose of the exercise is to write to implement a new easy block that starts from the base easy block class so basically starting from scratch. The second exercise is then to do the do this again but starting from the CMake make easy block which actually makes more sense, rather than redoing everything ourselves but for the purpose of the exercise. It's okay to start from scratch. So, let's try doing that. So the naming of the easy block it's EB, the software name is EB-tutorial so the name of the easy block will have to be EB underscore tutorial dot pi. So again, you can ask this function here get module part. Actually, let me do that. I'll show you how to do it with Python 3. Do the import and then get module part. EB tutorial. So this is the name of the Python file and of course you have to add both pi to that. And the name of the class similar with a different function. So in code class name will tell us that we need to do. I have typos at least. So start with EB underscore and capitals and then a software name where we replace the dash with underscore minus underscore. So, EB tutorial dot pi, the name of the class we're going to directly derive from easy block. And let's do pass here to make it an empty class. And of course we need the import statement as well. So the easy block class is defined and the easy block module that sits in the easy book framework. So this is a very stupid easy block that won't do much. But we can already try using it. Before we do that we have to change the easy config file that we copy it so I can show the difference with the other one. I'll use the second one. So we want to get rid of this easy block statement. So easy build will try to find the software specific easy block for EB tutorial. We just write like this it will still fail. Because easy build doesn't know the location of this custom easy block, but we can tell it to pick up on that using include easy blocks. And we just give it the name of the Python file. So you can do some wildcards as well so you can actually do dash sorry star dot pi here as well. If you only have easy block Python files in the current directory that will work too. So that will pick up the output that will pick up our custom easy block and try to use it. In this case, it already sees the module so I have to force it to do a rebuild. Now try to do the installation with this custom easy block that will fail pretty hard because we haven't implemented the configure step. So it says I found the non implemented error as soon as I was hitting configure. So everything else up until the configure step is done by the easy build framework. So downloading setting up the build environment unpacking the source files all of that is done automatically so we don't have to do that ourselves we leverage that functionality from the framework from the standard easy block, but the standard easy block does not implement any configure command. It doesn't make any decisions towards what what the standards installation procedure is in my view it doesn't really exist. So the first step will be to define the configure step. So we define a method like this. So we'll have to the exercise a run this CMake command. So we'll do run command. Basically CMake, which means we do need an additional import tools that run import run command. I'll try this but of course this will need some options. So first of all, here it says you need to specify the installation directory. So we'll need to give it CMake install prefix as an option. CMake install prefix equals something. So we'll write in templating to inject the actual location of the installation directory here, and we'll also need to provide our required eb tutorial message configuration option, which I can already do it here. eb tutorial message without this underscore, and we'll give a value here. This is getting a bit long. So in terms of code style will break this up. And we'll compose this command based on the list of values. So here we're already formatting things a bit to make it look nice for humans and to make sure we pass the standards. So this is Python code style. And if we haven't made any typos this looks okay. So this will be the different parts of the CMake command that we will execute we will join that list with the space as a separator so we get one long string. For the template value we of course have to replace with the location of the installation directory there's a variable we can use for that that's defined for every easy block self installed there so this is set up by the easily framework and will contain the value of the install there. Actually very early on, certainly by the configure step that will be already in place. The eb tutorial message here the value here we need to somehow grab from the easy config file. So for this, we will need a way to communicate the string we want to use to the easy block. So that sounds like a custom easy config parameter that will define. If you want to access easy config parameters will use self config and let's call this message. It's not an existing easy config parameter will have to define this one ourselves. And for this we use the extra options. Method that's a static method that's a bit of a special case. And what the extra message extra options does it returns. What is it a dictionary to think about this myself. It returns a dictionary of additional configure options. So in this case it's message and the value we have to use it as a list with the default value. There's no default value help message here, which I won't care about I will make this a mandatory. Easy config parameter. Since this has to be there for the installation to work. Which means we need an extra import for this mandatory constant like this. So this looks okay. We have a custom easy config parameter which we use in the CMA commands to pass it to the configure option here and we also specify the location where we want to have the software installed so let's take this for a spin already. Let's just use the existing easy config file which won't make sense. We have a typo line 18 missing a brace here to close the run command. Try this again. So here it says mandatory parameter is not provided in our easy config file message. So it's picking up on the custom parameter that's mandatory. And indeed, our example easy config file doesn't define it. It still does this, which doesn't make sense anymore. First of all, because we're totally ignoring config ops in our current easy block is probably also something we want to fix. But now we have our custom message easy config parameters so we can use this instead. Let me comment this out. Yeah, it's not really needed to comment it out it won't have any effect, but it's irrelevant at this point. So now we've added the message parameter, which is mandatory, and now it should be happy. We do that again with dash dash trace. We're being run in the configure step. And indeed, we're running CMake, we're running, we're giving it the CMake install prefix option framework has injected this value as the location where the software should be installed. And we're picking up the tutorial message and the value here and here we already see this doesn't look very good because our value has spaces. We're not properly passing this down. So that's, that's not going to end well. And probably if we look into the log file, CMake will already tell us it's not very happy with that. It's missing the source directory. Yeah, okay. We're trying to run this command and it's assuming that the last argument it sees tutorial is the name of the directory where the seeming command should be run. So that's why it says source directory yada yada slash tutorial does not exist. Okay, so this is a bit of a weird way of how the spaces is manifesting. We can fix that in our easy block by making sure we wrap this in single quotes. So the message can contain spaces so we have to be a bit careful. We try this again. The CMake will probably succeed. So it was running the CMake command. This has accepted with zero. And then it tries to proceed to the build command where we again hit the non implemented error, which makes sense we haven't implemented the build step yet. So the very basic way of implementing the build step is run command make. And we might as well do the install step as well, while we're at it. Make install. Now here of course I'm ignoring a whole bunch of things. We're not running make and parallel. Like the CMake make easy block would do. We're ignoring things like built ops and pre built ops, which is not a very good practice as well. So in terms of Peter's question like are there best practices, making your easy block aware of pre built ops and built ops is one of the things you should probably pay attention to. So that would definitely pop up in a review of pull request. Try this again. Is there a question or Okay, this seems to work. So it's doing the make, doing the make install. That's working fine. It's doing the sanity check using the information we have in our eb config file. So this stuff. Looks that looks okay. And now the last bit I guess for the easy block is to push this into the easy block itself so we can clean it up in the easy config file. Let me open both. So this part we want to move into here in the sanity check step, sanity check step. And then here we want to define. Let me just grab it from here. The names of the variables don't really matter. So we can define both the parts and the commands here but we have to pass this down to the actual sanity check. So here and build step and install step. All we really needed to do was run a command and then that step is complete. In the sanity check we want to specify what to check for. And then the actual check we can just divert to feasible framework so we can call the parent method essentially. So here we need to give it this super call. And say we want to call the sanity check of the parents. And then we have to pass down. So custom parts is the list of parts we want to check for and custom commands. So this custom parts and custom commands you will see these in the API documentation of the easy block. If you check sanity check step, you will see that it takes this and this as options when calling it. So we're just passing down these two to the parent. So the standard implementation of the sanity check. So as you can tell we don't actually have to customize the sanity check step ourselves as long as the easy company file gives us the necessary information the framework and do that for us. But in this case we want to take control and do it in the easy block, which means we can remove this from the easy company file here. Let me kick out this one as well. And have a minimal easy company file and divert as much as we can to the easy block. So now we've cleaned up the sanity check in the easy company file we've added to the easy block. And if I haven't made any typos that should work. So we can do another rebuild using the easy block and cleaned up easy company file and we can see that it's still doing the same check but it's now coming from the easy company file. And to show you if we want to check for something else. If we want if we're not happy with what's being done by the easy block. We can do it in here. So we can only customize. The echo hello, which doesn't make sense. Very stupid sanity check to run. But if you want to customize it, we can do it in the easy config file and then what's specified in the easy book will be ignored it will only run the echo hello, but it will still check for the parts that are specified by the easy block. I think that's basically the exercise unless I. skip part of it so I'm not picking up on things like config ops or pre install ops. That's shown here in the solution. So if you want to be a little bit more careful you want to make sure that the configure step picks up on pre config ops picks up on config ops and same for build and install. It takes into account these rather than just blatantly ignoring what the easy company file gives it. If we take this a step further we can make sure we built in parallel, but then we're gradually going towards what what CMake make does by default so it's better to leverage that generic easy block. And that's exactly what the second exercise does. So that says implement an easy block for the tutorial but start from CMake make and only implement what you really have to. That really means we won't have to do a build or install step ourselves, since what CMake make provides should work just fine. That's what I had in mind for today. We skipped over the exercises a bit but other than that we've covered everything I had in mind. Any additional questions. Yeah, I mean you're asking for specific file there to be available. That's it that's that's a good thing to check for. So it's also in general I understand but just I mean this is artificial example so I mean if you make an easy block you would expect that you have multiple easy conflicts and then you sort of you have different binary if you install. So why do you check for a specific binary. That's my question. Compared to having this. Is that the question why not check for just the bin directory and why check for specific commands. Yeah, but why do you check for specific commands. But no it's actually I think it's a good idea to do that, because you're looking for specific commands that you want to use the software with. So, just checking for just checking for a non empty bin directory, who knows what's in there maybe there's a read me file there. I would expect that the name of the command is different for different easy conflicts for different software packages to install. Yeah, so yeah that that's just I guess that the example that's a bit stupid here but. Okay, then the name of the command here happens to match with the name of the software. Of course you could have multiple binaries or you often check for for library files here as well. Like example.so or whatever. Yeah, and there's there's a bit of a balance between a very specific sanity check, and one that's good enough so if you have an installation that gives you 100 commands. I don't think you want to run. Check for all those hundred commands and also try running them with dash dash help pick a couple ones, the ones that are mentioned in the documentation or whatever. I think I understand the questions maybe more okay. In that easy block I would expect things to be checked for which are common for all possible easy conflicts that use this easy block. And in this case it makes sense to check for eb tutorial because we're, we're implementing an easy block specifically for installing eb tutorial. Okay, it's not a generic. It's not a generic. Okay, okay, then. For see for see make make of course there's there's no custom sanity check for see make me because you wouldn't know what you wouldn't know what to check for for Python package on the other hand which is a generic easy block for installing Python packages. So one will always check for lip slash Python slash side packages, because that always has to be there when selling a Python package, or it will always do an import check. So Python imports example, because it knows if you're installing a Python check it and Python package. Usually, the import checks should should work then there's exceptions, but as a standard way of checking something that makes sense, but for see make make or configure make. Check for sure. Okay, any additional questions. So again for next week. This is roughly what I have in mind. The custom tool chains maybe a bit difficult. And we'll definitely get back to that when when doing the cray part to the last one. And I do plan to show some stuff here about the GitHub integration as well. But if there are specific things people want to hear more about. Definitely mention it. I'll prepare that by next week. So feel free to reach out through the rocket chat or just fire me an email if there's something specific you want to hear about. And if not, I think we can wrap it up here.