 Okay, so now we'll talk about our module. This is a spin-off project from Biltest. So maybe some of you may be familiar with in previous talk on like the Biltest module features. What we've done is taken some of those Biltest module features and you're exposed into our module. And I'll start off with one of the user surveys. I think I was conducted last year in EasyBuild. So, the survey indicated like, how many modules were installed at the facilities and what I found was like approximately about like 60% had like 500 plus modules. There were a few sites that had like up to 1,000 or even more than more than that. So it's got, it's a lot of modules actually. So the idea is like, well, if you have so many modules we need to have a mechanism to automate the module, like just automate module testing, right? It helps, obviously gain conference in your system like your software stack through just simple sanity check of the modules. And I have also seen instances where, we've written like an invalid module and then we may have got like few tickets from them because some people like to use specific modules and the goal is to help even reduce the number of tickets that come up. So what is our module? It's basically a Python API for module system to automate like the module load test. And as I mentioned, this project grew out of built tests. So previously all the features were in built tests. They were deprecated in version 08 and now it's just a standalone Python API. And the goal is maybe other communities, other projects may find this useful if you're using modules in some way. El module uses spider tool to like actually query the software stack. And if you want to install El module, you can just use pip to install it. And the documentation, you can find it over here on the right. So just an overview of El module, it currently supports three main classes, module spider and module load test. The module class is just like, it implements some of the features of the module command. The spider is an interface to the Elmod spider tool to basically query the software stack. And then there's the module load test class that automates the module load test of a software stack. And that's used mainly just for just testing the software. So let's start off in module class. You know, in order to get started, you can just import the module and you know, from Elmod module and you can invoke the module class and create an instance. And it expects a name of a module either as a string. So in this example, we say, I want to pass in GCC core and you can use the get command module to get an actual return string code. You know, of what the module command would look like. So it just shows you some module purge and then loads the module. You can also pass in a list of modules so you can expect either a string or a list. So we can say, I want to pass in GCC core and Zlib. And if you say get command, it will get you the, both modules are loaded. You know, you can also test the modules. So if you load some module in some instance, you can use the test modules method and it will actually test the actual module load and give you a return code. You can also find if a module is available. So you can say, I want to create an instance of module and say, I want to use the isAvail method and say is module GCC available and we'll give you a return code. And it could be, you know, basically it's using the module isAvail command. So by default, we purge modules, but you can tweak that behavior, right? So all you need to do, like, you know, let's create an instance of a module and say I want to load open MPI, but I want to say purge, I don't want to purge it. So I say purge is equal to false. And then it would just only load open MPI without module purge. You can also say, I want to force purge. So if you say force equals true, then it will append the force option to the module purge and then load open MPI, okay? And if you, for instance, say I want to force purge, but not do purge, like if you say purge equals false, then we ignore module purge because force is just an option to module purge. So we support module collections, kind of, you know, the Lmod module collections. So like, you know, saving modules, showing the modules, like module save, module describe, you know, like to get the collection name using module restore and actually testing them. So if you create some instance of a module class, you can say save, just call the save method. If you don't specify any argument, it will save the modules that you have into the default collection. So it's equivalent to module save command. If you want to specify a collection name, you can specify that as an argument to the save. So let's say, I want to save GCC Core and Zlib into the GCC under Zlib collection name. So it will just store that as a collection. We can also show the collection name. So, you know, if you're familiar with like module describe, we have a describe method. So if you don't specify any argument, it will show you the default collection. You know, that's equivalent to running module describe. You can also specify a collection name. So in the describe method, like I want to say, show me GCC Zlib collection name. So it will show you that. You can actually get the appropriate module restore command using the get collection. It will return a string. And that also operates on the collection name. So if you don't specify anything, it will show you the default. If you specify an argument, it will show you the collection name. And finally, you can actually test the collection. So using the test collection, underscore collection method, and it will give you a return code. And what it does is just running the module restore command and seeing the exit code. Next, we'll talk about spider. So our module uses the spider tool to query the software stack. You know, to get started, you just import the spider class. And what it supports is getting unique software, getting the module trees. It can get the module names. It can retrieve parent modules and also versions of particular software. Okay. So to get started, like you just invoked a spider class. You can specify, you don't have to specify any arguments. And what it will do is it will just search all the modules in the current module path. And you can use get names. That's a method in the spider class. And that will show you all the unique software names or yeah, basically unique module records that spider finds. And if you want to, you can also filter by a specific module tree. So in this example in the bottom, you can specify a root of a module tree into spider and then use the get names and it will give you just modules in that particular tree. So you can also do some interesting things. Like, you know, you can use spider module to like class to do like filtering. And this is just a simple example. Like we create an instance of the spider class. A is an instance of spider and we're gonna loop over them using the get modules method and filter by all modules that are of CUDA and GCC. And then we can invoke the module class and actually test them. And you know, if you run the script, basically what we've done is the script is just testing all the CUDA modules and GCC modules on your software stack. So I think it's worth mentioning the content of spider. What our module does is it runs the spider command which is it has multiple options. One of the options in the dash O for output format is spider-json. It gives you the spider record in the json format and you can specify a list of directories for the module path to search. This is an example from Lmod eight spider record. And there was a change between Lmod six and seven in the sum of the records in the spider. So here's a record of diffutils. So the top level key is diffutils. Then there is a key that follows it with the full path to the module file. And then it shows you all the metadata of the module file including the full canonical name. That's what we use to find the actual module name used to load it. The version property here in the red is the actual module version. And we also can query the parent modules that's used to load modules before you load this module. And it's just a list of modules to load. So now I'll talk about module load test. That's the class that's used to kind of implement module testing using spider. And this is compatible with Lmod. And the goal is to help automate module testing. You currently, we can test all modules by module path. That's the default behavior. You can test by specific module tree. You can tweak the module purge behavior and filter modules by name, also full canonical name. And you can also like test by login shell if you want. So you just import the module load test class and you can just create an instance of it. If you don't specify any argument, it will just test all the modules specified by module path. So this is just an example and it will just start testing everything. In this example, we have two trees that we're testing and the output is an entry of the module name, the full path to the module file. And if it pass or fail and the output can be quite long depending on how many you have. So that leads to the other part, like you may wanna test modules in different ways. So you could test them through a module purge or you can test them without module purge. That really depends on your site and how you want it, but you can use the purge option. So in this example, on the left, we specify in the module load test the path to module tree. So you can actually do that. You can also specify the purge option that tells, okay, I wanna say purge is true and it will run module purge before each test. So run module purge and then load the appropriate module. If you say false, then it would not load module purge as you can see. And you can also enable like debug if you want to see the actual command that's being run. So you can say debug equals true or false. We can also test modules in a login shell sometimes. So by default, we test in a sub shell, but running a login shell can help solve some weird issues that you may have in your current shell. So in order to just do that, you can just say login equals true. That's just an option to module load test and it will just run everything in a new login shell. I guess in this example, just a bash shell. You can also filter modules. So using the name property, and that expects records from spider. So when we say, and it's a list of spider records. So if you say filter by CUDA, it will go into the spider and find all the CUDA modules and then test every single one. In this example, we just test only six CUDA modules for this specific module tree that we have. So now, what if you have environment module? How can you use this? Well, you can actually use L module for this. So first thing was that currently L module or environment modules didn't have support for parsing modules until like version four, four to five. And this issue I raised up with a Javier about having some kind of ways so I can like an equivalent to spider. So, you know, in order to parse it. So they added this dash, dash, Jason option. I'm not sure if this format's gonna be like if it's gonna change over time. So I haven't implemented anything like this yet but I have a script that I just implemented to kind of parse environment modules, output, like, you know, we're using module avail and just test them. Just forgot a little bit of code but the idea is we take the output of module avail and use dash T to get like the a list out place that's abbreviated and then we just parse each entry and we use the module class to for loop over them and test them. We get the return code and then right now, you know, you can just do like a pass, like we just have a counter to see all the modules that pass and fail. And, you know, if you wanna use the same source file you can just download it and it should work pretty well. So what we have done is we have actually taken that script and I have this tested on Corey. I have this tested on other systems also using environment modules three and also four. So what we have is that we have this script called module test.py and we test it for every single module tree. So we have, I think like five different tests and we're expecting a pass or a pass of 100%. That's in our expression. And we found that one of the tests failed. It had a 95% pass rate. This, one of the trees that was the opt-cray PE module files and in that output we saw one of the modules perf tools light failed. It had a return code one and it failed because it required another module to be loaded. So this test actually kind of shows how you can actually do like your entire software stack just testing them through modules and you can add as many trees as you want. Yeah. I just wanted to bring up some known issues. So if you have environment modules three which we do have here in Corey if you load an unknown module it will return an exit code zero which is not the right exit code. As you can see on the bottom right if you were to do this so what we're doing is we're just loading module load X and it's not a valid module but it gives you an exit code zero. But if you were to do the same test on like environment module like four like I guess the latest version you get an exit one and in Alma is the same thing. So just one of these weird issues that's present if you have an older version. So finally, I wanna wrap up the modules it's to the primary interface for the users and the software stack and it is important for the facilities to actually test their software stack just basically a simple sanity check on module load is sufficient. Currently we have one release of it this was released in March and yeah I think it would be interesting to know like if others have interest in this and you can probably improve the API a bit. If you need help you can please file an issue or you can just join the Slack channel. There are also some examples in the documentation to help you get started if you wanna write your own scripts and yeah you can just yeah that's it. Thank you. Okay thank you very much Shrizaad. We already have a question by Maxime. Yes, hi. So we install basically 99.99% of our modules with Easebill which already tests loading the module when it gets installed. Do you see use cases in that situation for L-Module? If it's not installed through Easebill if it's some user software stack they install themselves sure I'm not sure how it's being well I mean I'm sure it may be okay if you already have it tested but I know that for instance there are some software packages that may have to be installed that are out of scope of like sometimes even Easebill or spec like you have to have custom modules for this thing like that still requires testing. Right, yeah those are pretty rare but yeah okay. I guess there could be a use case when people start cleaning up installations and are not careful enough. So since modules generated by Easebill load other modules if you happen to remove one of those you may not be aware that you've broken something. So that's where L-Module could help with flagging those modules as broken. Shrizaad maybe you can kill the screen share so we have something to look at yeah. Any other questions Maxime? He's already muted again, I guess not. No okay, I did have a question as well Shrizaad so you mentioned relying on spider provided by L-Mod and when you're running the module load tests does L-Module take into account that there may be other modules that have to be loaded first before you can load the module? So if you're in the module hierarchy where you first have to load a compiler and an MPI to reach an application is that something L-Module does automatically when it sees that L-Mod reports so you have to load these first before loading something else? Yeah, I believe where was it? If I could share it's using the parent module. So I have to test that part out. I believe I had this feature tested but I don't think I have it, what was it? In this spider record. So I don't have it with me right now but one of the features is there's a spider record called parent AA and that is responsible for, these are parent modules that load module path to other trees where these modules are and these modules need to be loaded first before you load like in this example of diffutils. I don't believe I have that support or I have to test it on a L-Mod system. I can test it on another system. I just don't have it on Cori but I believe I had this feature in in build test when I was trying to migrate most of this. However, I think you can, if you don't have L-Mod system you can still use the module class and specify your custom like list of modules so you can go into like this example on the bottom left. You can say a list of modules to load or you can have a script that says like, I wanna load some modules before I load something else. Yeah. Okay. It's not clear exactly, automatically do it yet. Yeah, that may be important because if you do an L-Mod spider on a module name L-Mod may give you multiple options that says, okay, if you wanna load this module you either have to load A and B first or DNC first. So L-Mod will have to iterate over different options and only one of those may be broken depending on what happens in those modules whether the actual module is actually there. So maybe what could happen for example is that in the L-Mod cache the module was recorded. So L-Mod thinks the module file is there but when we actually try to load it that's when it reads the actual module file and that may still fail even though L-Mod is still reporting. So if your cache is out of date or something like this so it may also be useful to actually test whether what L-Mod is telling the user is still working. Yeah. Yeah. And the spider actually, where was it? It was in, I'm sure as if you have an L-Mod system you would be updating your spider cache on like some cron basis or something like that. And if you have, if you invoke spider or if you invoke the module load test class it will just use the spider command. And during the interval when you submit it and spider record is not up to date it won't pick up the new modules. It's just trivial, but I think, I mean, you could still work around it. I think that's, I don't know if that's in scope for L-Module to do that, but I didn't consider that yet. You can also tell L-Mod to ignore the cache. If you wanna bypass that and wanna actually look at the system but then it will be slower to walk through all the modules and load them, that will be a bit slower but you can just, it's an environment variable. You can tell L-Mod to ignore the cache and just go to the actual files on the system. So depending on what you wanna test do you wanna test the cache or the relation between the cache and the actual files or do you just wanna check what's on the file system? There's again different options there depending on what you actually want to test, of course. You may sometimes also want to check that the cache isn't corrupt. That can happen. Isn't corrupt or isn't outdated. So usually people have a cron. So we, for example, run a cron that updates the L-Mod cache every hour, but if that cron dies or the node where that current is running is dead or cannot reach the software or whatever happens, then the cache doesn't get outdated. So having a check that verifies whether the cache and the module files are not out of sync could also be very, very interesting, I think. It could be an easy check that people could run in their monitoring. And if they see the, if L-Mod says, okay, the cache and the files are out of date or out of sync, then it could flag a red flag in the monitoring. So that could be a good use case for L-Mod as well. For L-Modul as well. Okay, I don't see any other, questions popping up and we're getting close to the next talk. So we'll wrap up here. Thank you very much, Shazab. If people want to ask Shazab questions on either build test or L-Modul, I suggest you can move to the breakout room. I think Oka will have to recreate the room. So feel free to jump in there. I'm not sure Shazab if somebody will show up or not, but you can check what we have to get ready for. Next presentation. Thanks a lot.