 case. Great. Okay, so now we have seen Numba. And I think that Numba is like kind of great. And it does most of the time perform super well, but that's automated if you will optimization. Now we can also see another method which is Scython, which is a way to do that, but then the optimization is not so automatic, which can be a good and a bad thing. It's a bad thing in the sense that then it asks you to do more stuff manually yourself. So you have to spend a bit more time on that. But then it's because it's manual sometimes, that means that it's easier for you to actually specify what you want. That means that basically if you play with Numba and Numba doesn't want to optimize your thing because it doesn't understand what you want to do, then you are sort of stuck. Whereas with Scython, you can always delve deeper and specify more stuff manually to actually get the optimization that you want. So it's not an interesting to know about both. All right. So then Scython, let's look at what it does, how it works and how it can helps us. All right. So what it does is it tries to compile your Python code to C or at least part of it. All right. So the goal is sort of the same as Numba, but it does it like semi manually, semi automatically. So in Jupyter, we can load this extension and then we use this here magic command to transform stuff to Scython. Outside of Jupyter, you do it in command line flavor. So we call Scython or we write a little file to set how it should be compiled to C and how we should interact between C and Python elements. All right. I give you some example, but here we'll use the Jupyter version. So I load this extension. Then we will work our way through this little example that they give in their documentation, which I find fairly nice then to have a small progression. So they have on one side the function that we just worked on, okay, which is how to integrate a function. Now we have just the function in two, in two sub function, right? One main that does the integration and one we just compute this x squared minus x. So this is here our Python and now what we can do is that we take the same thing. I just had percent percent Scython there. Okay. So I execute this one as well. And now I can compare the two, right? So it takes a bit of time to compute the native Python we know should take 200 and yeah, 200 and something. And then simple Scython. Let's see. Okay. So simple Scython. Here you see it's better, but we are nowhere near the sort of optimization that we have seen up to now. Like remember with the NumPy, we had a factor, for me it was a factor seven, for you maybe a factor 20 something. So okay, something is better, but you know, it's only one, it's only 30%, right? So okay, it's nice, but it's a bit lackluster. So we can fortunately try and see what happens with this dash, dash annotate. And then when we see that we see, so it's still compiled, but we also get this little report telling us what happens, what Scython does and what worked and what not. So the yellow line ends at Python interaction and the further Python like there is, the more complex the C code becomes and so we get less performance. And you see that here almost everything is yellow. If you want, if you are curious, you can do something like this, click on the plus and then you see here for this line, oops, sorry, this is the equivalent C++ code. So you see that here, that's why it's a bit complex is that just this line gets replaced by all of this. And so what it does there is that it defines a function, but now Python functions are not typed, that means that they could return anything. And they take here a one argument x, but which could be anything. But C function have much more constraint, right? And they must have a return type and they must have like, they must know exactly what is the type of what they are given. And so to circumvent that, Scython creates all of this super, super, super ugly code, okay, which kind of circumvent the sort of flexibility that we that Python have and that C doesn't want to have. So that's why it's super bad, all right. And as you can see, everything is yellow. So all of our code is super ugly like this, maybe not that one. Okay, yeah, here, it's much simpler, right, but that's the simplest one that there is. Yeah, right. So now what we can do is we can start giving some hints to Scython to help it compile the code better. For instance, we can say, okay, here, you put something super ugly because you don't know that for instance, X is supposed to be a number. And we swear we won't give you anything else than a number. And same thing for A, B, and N. So we can start doing these sort of things, all right. So we add some manual annotation of the type. So we say here, f takes a double that is X. And then here, this function takes a double and an integer double is the equivalent of a float 64. It's a long float if you want. Then we say that I is an integer. Here with a C def, we tell it that you are like, we tell it, all right, you are inside a function and you can define it using the C type and not the Python type. So if we come back there, you see there, it has defined a Python type integer, all right. But if we have a C type integer, then we have less Python, more C, and so we go faster. Same thing here for S and DX, all right. And then here, we just call this new function that we created. Now, if we do this, you can see that now suddenly we have slightly less yellow in there, okay, than before, but still quite a bit, okay. There is less yellow there. And here, of course, now there is not too much yellow here. And now this part there, you can see it's fully white, okay. There are still some stuff happening in the four because they are, they are the viable name I've been replaced. But otherwise it's, this is very simple C code, all right. It's just using weird viable name because I is an integer there and int and n here has been also recast to a C integer. And that's it. So already there, we can gain something from that. And then of course, we can go even further and even further and even further until we have almost only C. For example, we can react and say here, there is a lot of Python interaction there for this function, but we only need it inside this function here, okay, which is also something that is compiled by Thyssen. So this could be pure C in the end, right. There is no need to be interaction between this code and that code because this is a simple float that we give a simple double and we catch a simple double. So what we can do is that we can have more typing where we say explicitly that this function is not a Python function, it should be fully compiled. And so we say here C def and not def to say should be a C function. It will be like it will return a double and take a double as input. So that's kind of the structure of this is that you do C def return type, name of the function and then for each argument you give the types. And then here we have not changed additional stuff. And now when we compile, now we see much, much, much less yellow. There is yellow here because it's a function whose role is to interact between Python and C. Here there is a little bit of ugliness because we use B, A and N which were defined there, all right. And here we use again A. So there is a little bit of interaction but you can see that it's much lighter yellow and there again because we return back outside of C toward Python. But then for this function there you can see that now it's fully white because it's fully compiled. All right. So that's kind of how we go at it and it can become a bit more complex, a bit more ugly. But you see that you have then fairly high control on what you go and where you go and how you can specify to Python what should be, what should be compiled fully and what not and so on and so forth. There is also a good support for NumPy array. Also you need to, the syntax let's say is not the absolute easiest. So there is a little bit of legwork involved. All right. So then we can sort of try it out. We have our native Python. We remember that it takes about 200 milliseconds to run. Then the size is about one quarter slightly, one third better, right. Then we have the some typing there where we just give some hints there. And then we have the more typing, the one that we just saw. And that's what we get. All right. So then we see, okay, at the beginning we don't have much but then we see that as we have more and more typing we have more full C functions. And now we get an almost 100 times speed up. So we are much, much, much closer to what we would expect. All right. Are we good so far? Everything's still making sense or are there any questions? Okay. So there's a question by Tess. How would one go about debugging the size and nice code? I would personally be a bit careful. I would not, I would not size nice something before. I'm sure that it already gives the right and expected results. So I would write my Python code, make sure that it works exactly as expected and then size nice it. And as I size nice the stuff, I would check that I still get the proper expected output as I go. Does that make sense? Great. All right. So then let's now see a little bit where this thing can go for a more complex example. We'll see about the size nice version of our pairwise distance. Now brace yourself because there is some ugliness there when we work with multi-dimensional arrays. But once you kind of get there, it works. So Syson annotate of that. Okay. So we import numpy as NP and then we see import numpy as NP. All right. So what this says is that like you import both the Python version and the C version of numpy. As I said, they have ported numpy inside their Syson libraries. And we see import Syson. Then the numpy array will have a type float 64. And so we kind of define a type just for that. We see type def there. These whole stuff there, these are stuff that you kind of find in their documentation that they say that if you plan to play with numpy arrays of type float 64, then you should use that. So you don't invent it. Then I add there two directives that they say make stuff better. In practice, I have not necessarily found that it improved performance immensely, but they say it helps. So I put it there. The first stuff is that you turn bound checking for the entire function. That means that what it does is that when it tries to access elements in an array, if you don't put that, it will check that these elements exist in the array so that you don't go overboard and you don't try to ask for stuff that don't exist. And then I just say, don't check. I'm certain about what I'm doing. Of course, only do that if you're certain about what you're doing. And same thing also the negative index wrapping. So in Python, if you write list and then between square bracket minus two, then you get the second to last element, minus three, third to last and so on and so forth. And for this, I just disable that. So again, that's Scythe has less stuff to try and account for. So that's less if and else everywhere each time I try to access stuff. Then here, this is something a bit ugly. So let's always say that this is a 2D array there. So an array and then one dimension and then the second dimension. And then this is again some little boilerplate code, just a way to say how this should be iterated and what is the stride of that thing. That's basically some sort of default that we never change. So to say. And then inside that are stuff that we have already seen. I define my element with cdef and using simple ctypes. And then here you see the same thing as before there when I create my output array, which is then again a 2D array defined as an numpy.empty and I give it the float 64 type. Then this part doesn't change at all. So the hard part let's say is really this thing there when you try and create these elements here. And now when I wrap that, okay, I get this. And now this is what you see. You see that almost all of it is white. It's really there the creation of this array because of course it's an numpy array. So there needs to be interaction between C and Python. But the rest of it here, this whole operation there will be full C. All right. So it's actually quite nice work. And of course, if we do try it out, we see that we should get a slightly nice performance. So the numpy takes a bit of time to run. See, all right. And so then you see now we have about a times 200 gain. If I'm not mistaken, no times 50. Yeah, times 50 gain when it comes between like the numpy and the Python. Of course, this comes at the cost of having to play around with these little ugly bits of code. But at least you get some performance. All right. Now, Python is sort of a process to learn. But ultimately, they do have a fairly nice tutorial, which I've actually shown you little bits of earlier on. And one also nice thing to add is that Python is a great way to interface some Python code and some C code. That's a good thing to know because there is already a lot of super nice and super well written and efficient scientific code written in C. And so you know that you can leverage all that by writing Python extension that just lets you write interfaces between these C libraries that are super nice and your Python code. All right. So again, also if you have some C code or C++ code, which you'd like to use in your Python, that's a way to do that. All right. And also if you do yourself a little bit of C or C++ and you would like to do a sort of extension for Python, you can also use Python to create your interfaces. All right. And so last but not least, they have made it, and here again, I give a little link, but they've made it fairly easy to do some profiling. So debugging, I don't know, but the profiling of Python code to then understand which are the part, which are the Pythonized version of functions that are fast and less fast and so on so forth. This is quite doable as well. And it works with C profile as well, if I remember properly as well. So with at least the same interface. All right. All right. Are there any questions? All right. So apparently there is a link that was dead. Ah, too bad. Is it? Yeah, that one. Okay. So then while you have your exercise that is to come, I will ensure that I find a replacement for that link. But thank you, Tess, for letting us know and we make a note. Are there any other questions? Okay. If not, then while you think of eventual question, I'm just going to run now. We are going to just compare our different implementations of the pairwise distance functions. All right. So we start with a native Python, which should be super slow. Okay. 11 seconds to run. And then the other one should be much faster. And if I remember properly, here we should expect the best performance from Numba. Yep, indeed. There we go. So NumPy is about a factor 10 at least on my computer. Normally, I think it should, on your computer, I think it should, you should have even a better speedup. And then Scython about 27 millisecond and Numba, 20 millisecond. All right. And I think on your computer, you will see even better speedup. But for me, I think that Zoom is doing so many things that, yeah, things are slowed down. Yeah, it should be said that this is a, oh, yes, there's a question by York. Yeah. For me, for some reasons, Scython is very, very slow on my machine. So Scython is like 500 milliseconds compared to one second from NumPy. So it was the same in the previous examples. So interesting. So maybe your, your Python version has little troubles there. Linux and Windows. I don't know if the others with Windows have the same thing. Maybe. Maybe other people can share the sort of speedup or times that they experiment. All right. So while this runs, yeah. So one thing also that you can check is that indeed here, you get most of the stuff turned white and that you don't have yellow remaining too much. Yeah. That was the same. That was the same. That was the same as in yours, but the results, I can paste it. Okay. So then it might be that indeed just the, to see libraries for some reason run slower on your, on your computer. Let's see. We have a few other people sharing. So yours and then, yeah. So you see here, for instance, some other people get very similar performance for Scython and Numba. Yeah. That's closer to what I experimented as well when I'm outside of Zoom. I tend to see very similar performance. Victor, maybe could you tell us which OS you are using just to know if it's maybe a Windows problem or any other person for that matter on the very old Mac. So on the very old Mac, it works, works the same. All right. So anyhow, that's to give you also a little idea here. As I said, this example will favor Numba a lot. And to me, that's also a generic observation that I can give you. I've always seen that when your problem is numerical and simple Numba is, I would say, really your go-to. Okay. And when you go for weird stuff and so on, then if Numba doesn't give you super good performance, then try Scython, but also know that it will be a bit more involved. Though, I mean, you can get very, very, very good performance. And once you know the few tricks that there is to know about Scython, it's actually not too, too, too complex to actually do and get good performance out of it. Right. Okay. So that's most of the stuff that I wanted to talk to you about, about like the trick to get better performance and so on and so forth. And now, of course, I've shown you a bunch of stuff. So the most important is that you go and practice a lot. And so that's why we have now at our second