 Do you hear me well and see my screen? That's perfect. So hi everyone, I'm Michael and my talk is about how to extend Python using Rust. First I'm gonna go through reasoning for such a thing. Then why Rust is such a great choice for this task. Then we'll go through some libraries that provide bindings from Rust to Python. Then we'll switch to demo and build a dynamic library that we then can load into Python. Then we'll build and run a simple web service example that uses both Rust and Python code and we'll build and run it using Docker. And finally we will compile and build a PIP package that contains both Python and Rust code. So what would you even want to extend Python? First of all, most obvious probably reason would be because Python is great but it's easy to use but that comes at a cost. The dynamic nature of Python means it runs on the very high level and it's very slow, it's somethings. So you possibly might speed it up with an extension. Then reusability, why would you want to, you don't have to necessarily port the whole code to Python if your main code base in Python. You can just use some code written in another language from Python. For example, large parts of NumPy or SciPy still written in Fortran. And if you, when you install these packages you will see how Fortran is a compile. The next reason would be a collaboration. For example, when in your company, one team might work in Rust and another in Python and it would be nice to find a way for them to use code each other, each other's code. Another reason might be migration. Suppose for some tragic reason you want to rewrite your whole project from Python into say Rust, then you don't have to do that all at once. You might do it module by module and then import what's already ported into what you still have in Python. So how you might approach this first and the classic way would be to write an extension in C or C++. It's, Python provides a great beingings for that. The thing is that writing something in C or C++ it's cumbersome, let's say that, and not exactly safe in terms of memory management. So another way would be to apply some Python improvers, I would say, like a site on that recompiles your Python code in C somehow or the newer library code number that compiles your Python that if your only goal is speed it's a nice way of doing things but I would be wary of doing that on large projects because it's a use control of how your code is run because these libraries perform some magic on your code. Why Rust? Rust, well, I have to say I'm not an expert in Rust. I've been using it as my hobby language for some time and I think it's absolutely great. It's a systems language compiled and fairly low level but it uses a very innovative system of borrowing. Basically it controls the memory memory of your code and it controls the memory management at compile time. So if you don't properly manage the memory it just will refuse to compile your code. And Rust is maintained by Mozilla and it is voted as one of the most loved languages by developers and recently it gets a lot of attention from companies to notably Microsoft and what is important for, in our case, it is that Rust has a very minimal run time so no garbage collector, no anything like that that will run in parallel to your code. If you vote your compiled code extension code into Python, you will not have anything else rather than your code running inside the Python interpreters. This, which will be not true for a garbage collector, garbage collected languages. But I wouldn't, as I said, I'm not a Rust expert. I wouldn't write the whole project in Rust because it's, Python has its advantages as a, in terms of development, development speed, most of all. So it would be nice to maybe write some critical code in Rust and just vote into Python and some common code like a web service or CLI you can write in Python. It's not maybe performance critical or safety critical. So how would we do that? There's a few, there's a couple of Rust libraries that allow you to build a Python extension. First of all, it's a Rust C Python. We're gonna use it in the examples during this talk until then, PyO3, which is a fork of the previous one. The reason I used Rust C Python because that's because PyO3 works only on nightly Rust and Rust C Python works on stable Rust as well. I actually, I have seen that PyO3 recently migrated to stable Rust. I mean, it can be run on stable Rust as well, but it just didn't have time to check. The both libraries are very similar because one is the fork of another, obviously. They also give you ability to run Python scripts from Rust, but it's really out of scope for this talk. We're gonna use, I'm not gonna go through setting up Rust and syntax and so on. There's plenty of information online. I will recommend reading Rust Book and Cargo Book. The Rust is pretty nicely documented. I'll just say that we're gonna use Cargo. It is a package manager and a general CLI for Rust. Okay, so let's switch to demo now. Now, hope you see my dev environment well. So let's look at our Rust project. I created a very small, very simple example Rust project and it basically consists of, if we disregard the files I get ignored and stuff, it consists only of two files. The first is lib.rc, which contains our Rust code. And another is Cargo Tomo, which is a metadata file, sorry, metadata file. So lib.rc contains the Rust code. So here's all our whole Rust code. Just one, again, I'm not gonna go into details on Rust syntax, it's, well, if you worked with the C-style languages before, it's pretty obvious what's that, and it's also somehow resembles Python in some ways. For example, the typing syntax. So the first one says that we are gonna use C-Python create. Create is a package, is a Rust term for package. We're gonna import these things. Then we run the pyimodule initializer macros that creates Python extension wrapper for us. We create doc attributes with some helper string here for Python to see and we export get result function that Python will be able to use. Then we have our actual function. It will return, it accepts a string and then returns the same string back to the color prepended by Rust says, that's all it does. So it returns actually a pyresult, which is a wrapper for Rust types that kind of converts them into Python types so that is transparent for Python. So that's all here and then cargo Tomo set up some metadata like name of the library, my credentials, name of the create, and this is important. It says that the resulting library will be a dynamic library that it's not statically linked by, but dynamically loaded into runtime. So it will be a .dll file on Windows or dylib here on Mac or .so file on Linux. For some reason Python only understands .so file so we're gonna rename the file. So let's build the Rust project. It's a CD to the Rust library and everything that we need to do is to run cargo build and we indicate that we will build for the release or no debug information and basically it's done. So here in the target directory, you can see it built our binary. It's a dynamic library that you can use already. It is a Python extension already. So as I said, we can't use it right now so we're gonna copy it on the top level and rename it as mylib.so. So now it is very simple. We run the Python 3 interpreter and we can just import mylib here and it imports. You can see that help mylib will show our help stink and that's the function get result is available here. So then we can run mylib get result. Hey, now it's a typo. So yeah, so we just run the Rust code from Python. Now we're gonna remove that file. It's not needed anymore. So this is nice, but how we, let's look at a more real world example. So here in the web folder, I created a small web project. I use Flask and I import the same library here and when you call the main endpoint, it will just return the result of mylib get result hello. So how can we run it from here? So let's use a Docker. We're gonna use a multi-stage build technique. So because when we gonna run Python environment to run our web service, we don't want to have Rust and all the Rust machinery in our Docker container there. So we're gonna use the first stage from we pull the Rust Docker image and we give the stage a name Rust build. So then we can just copy our two files and then we set up the target. The thing is that previously I ran the example on my MacBook. So I used the Mac OS compatible binary, but the Rust, the Python 3.7 slim image is a Linux environment that is Debian, I think. So we're gonna build for Linux now. So it's very easy for with cargo, you just set the target and you will have to write binary as an output. So then we go to this next final stage of our build. We copy our Python file, then we copy the requirements file and then from the previous stage, we copy the library build. Is the same way as I did locally, it will be done in a Docker environment, but this time it's called .so because it's built on Linux, but in the end it's the same mylib.so. So then we install the libraries and run the service with the GUnicorn to have it production in a production manner. So we're gonna just build it like this. It gives us a lot of caches and it's fine because we don't have much time. It copies the library and so now we can run this image and we're gonna expose the port 8000 for tests. So let's see how it goes and it works. So the last says part comes from the last library and hello comes from the Python part of the code. Let's queue it now, right? So you can build it like that, but the thing is that imagine you work in the company and you work primarily in Python and there is this project that contains Rust code as well and you have to maintain all this Rust building part. So probably you don't wanna do this every time you want to do pip install mylib and have it installed on your machine. So you don't have to even think about cargo and stuff from the Rust world. This can be achieved with Python packages. So here in the package directory I have the same Rust code is exactly the same cargo Tomo as against the same. The only thing that I added a few files here for a pick package machinery and the main file is setup.py which will build the Python package for us. We're gonna use the library called setup tools Rust that provides a binding class and Rust extension class. So apart from packaging the Python code it will build the Rust extension for us and we'll use the binding Rusty Python which we use but it can be set as PyO3 as well and we want to use PyO3. So it will build it and includes the Rust binary for us in the package. So for this we gonna let's create a virtual environment for us. Okay, so we can activate the environment and now we can just go to the package directory and run pip install dot and we will see, we will see. So it's building the package called mylib Rust and it's built the wheel for us. The wheel is a format WHL extension. It's basically an archive with the binary and Python code together for Python. 3.7 because it's my current version of Python and for macOS because I'm running it on my MacBook. So this, what it just did, it built the package from sources. That means you're gonna have to have Rust on your machine but at least you can import mylib something then run and yeah, I know what it is. I just need to go up and then it will import it from the environment. Yeah, but the point is that it builds the thing from sources which provides a lot of complications as well. What can do now, we can pre-build the wheels and upload them to some, to PyPy repository and then when you run it on your machine, PyPy repository and then when you run PyPy install mylib Rust, it will download, it will automatically select the wheel supported by your environment and your Python version and they'll download it for you. So you don't have to have Rust installed on your machine. It's the same way as it works for C extensions. So I don't have time to really do that now but I will just show. So I can build things for macOS using the setup.py, the dist wheel, it will build this dist, it will build the wheel for me like this. It's a wheel for my macOS but suppose I want to do that for my little web service. So now I can use a different way, I can just build it, upload somewhere and I can remove the first Rust build stage and install it from requirements TXT where I have just this. So in order to achieve that, I will need to pre-build the wheels for Linux and upload it to somewhere. Somewhere would be the test.py.py website in my case. It's a playground for Python package developers that you can use, they clean it up once in a while but you can upload packages there and test it before polluting the actual global repository or in your company you might have your own mirror or your own private PIP repository where you can store your packages. So I created my Lib Rust here and uploaded the wheels for that library. It's macOS here and Linux here. So I don't really have time for that. So to upload it, you can use the tool called twine and you can build for Linux on macOS. You can use Docker image called many Linux. The tricky part with Linux is that there is lots of repositories and they differ from one another and the many Linux projects claims that it won't maybe cover all the Linux repositories but it covers enough to call themselves many Linux. To build for Linux, I'm gonna pull that many Linux image and run my build inside the Docker container and I'm gonna run this, I'm not gonna run it but I would normally run this build wheels script which what it does, it installs Rust and cargo and then it goes through all the Python versions present and build wheels with the same command we would use locally, it just builds the wheels for this. Hi, Mikhail, apologies for interrupting you in between your talk, it was going nicely but we have gone much ahead of the time. There are two, three questions that people have raised so maybe we can focus on them and then you can take the questions in the breakout channel. Yeah, I'm gonna post a link to my GitHub repo with all the code there and we, and also everything will be in the Discord breakout channel. Right. So sorry for the apologies for that. It was really going great but we are short of time. We have three questions here. One is for Matthew, he's asking that the flow with which Fortran gets, Python gets extended with Fortran using F2Py is slightly different from what you showed for Rust. So is it because of setup tools Rust library? Well, probably yes because we use setup tools Rust for Rust and there is something specific for Fortran built by NumPy developers. I can post a link in the Discord channel. I have a link for the tool they use for Fortran. Okay. Right. And Radu is actually asking of quoting some examples where you would actually write a specific Python module in form of Rust. So kind of rewriting the specific module, the Python module in Rust. What can be the use case for it? The use case, for example, you have a, well, you can just want to speed up your Python code. Then you can rewrite it in Rust in the more low level language or sometimes if the code is critical and you won't have more control over it, the statically typed language would be better than Python. It doesn't mean you have to rewrite everything in the systems language. Right. Thanks a lot for answering these questions. I think we are running really short of time. So I have to onboard the next speakers. Yep. So thanks a lot, Mikhail, for such an interesting talk. People can further ask you questions in the breakout channel. I'll post the link to same in the Microsoft track. Yeah. Thanks a lot. Thank you for attending. Interesting. Yeah.