 Okay. So the executive summary, if you look here, you always want to monitor jobs to make sure that they're actually using the resources like you think. So both not too large and not too small. As you're trying to make things run longer and longer with more and more you want to monitor very carefully to make sure that it scales well. Because it can often happen that you use a lot of processors and it runs just as fast or even slower as when you had fewer. And that's just a huge waste. And then there's several different commands here to use. Maybe we can... Yeah, maybe if you share my screen, I can show the actually the like we can use this as an example of this script that I previously run. So because like if we think about what we just did in the example, we monitored the output using tail, like we had the job running, we can let's submit it again, the job that runs there. So let's use SBatch and use to sleep for. So what is monitoring? The first step of monitoring that your job should provide some output that you can use to see what it does. Like that's the first step. Like you should have something in your code that provides some output, some print statement, something that you can then use for your advantage. Of course, like your job shouldn't be outputting everything constantly because that creates noise and it reduces the readability. But let's say your job is running something and it will produce like every thousand integrations, it will produce some output. You know then, okay, it's currently running here. If your job is seen like a complete black box that like you just fire it away and you know nothing what happens and then it just crashes here out of luck basically. And it's very hard to debug what's happening. So you should have some sort of like debugging, not completely debugging, like bringing everything what the job knows, but like some simple output that you can follow if it performs correctly. But if that's not enough, you can always rely on the slurm. Or if you don't have that available, or if you haven't used that, you can always rely on slurm to provide you some of this output. So the commands that Richard highlighted here, so we have the sf commands. So let's see some of the jobs that I run. Let's look at one of the previous commands. So let's see for example, this first one, this run host name. So if we run the sf command on that, we will get some output. In this case, we will get like cq efficiency zero because it basically didn't do it didn't do anything. So not a very good job is it. But but in a in an actual job, you might get something between zero and 100%. You cannot go over 100% because 100% like 100% represents that all of the resources were used correctly. Like if you ask for 10 CPUs, then 100% means that all the 10 CPUs work constantly. If you ask for five CPUs, it means that the five CPUs work constantly. And the memory efficiency, well, that can go over 100% if you had asked more stuff than you actually needed. But then your other help is the SRAM Q. Okay, yeah, if you had does 100% CPU efficiency mean the job is doing what it should? Like not necessarily doing something, but still be inefficient. Yeah, of course, like it just means it's just a mechanical or like simple calculation is the CPU occupied. It doesn't mean that it's doing the correct thing. It can like be as inefficient as you want. But it can like, it just utilize the CPU. So of course, you will have to like add into your code some timing or something like that, that lets you know how long it takes in different steps. And it's good idea to check a profile code or something like that to check where the bottlenecks. So like when you're doing real scaling testing, you'd not measure the CPU efficiency, but you'd also measure the results efficiency somehow. Yes, yes. So we will talk about this SF a bit more when tomorrow we will look on multi CPU jobs. And then we'll check how efficient they are with SF. And then there's also another command. I think it's mainly auto specific that you can check the GPU efficiency, but I'm pretty certain that other other sites have something similar as well. But basically, the idea is that make your code so that you can read the outputs. Like that's the simplest way of like that's the best way of monitoring your job. Make it so that you understand what the job is doing. Like my way of making it print something every now and then. Yeah. Okay. So let's see. So we talked about monitoring after submitted with SQ. We talked about the while it's running. Oh, did we talk about while it's running? So we can tail the files to see as things are coming to debugging checkpoints. Yeah, I think we covered. Yeah. If you want to, there's more, more information in the documentation that you can browse to more hints. But those are the main points. Like look what the job is doing. Use the SQ or the SQ to check what it's doing. Look at the job output. After the job is finished, look at the Slurm history and the SF. That's about yeah, too fancy. And after it's finished, we did. Yeah. So I guess we basically done here. So that's the main ideas. Let's see if there's any questions. Nope. So should we go to the last section then? Yeah.