 Thank you. Thank you everyone for coming here. So yeah, we're gonna talk about trust so kind of weird to talk about trust in a Python conference, but we'll see that it can make sense if we want And so we'll cover the very basic of first as well The interest of building native modules to integrate trust within Python and not just writing bare rest code and And as well the performance improvement that you can have when you're working with rest within Python And you can have this in a quite simple way So first things first I'm Arthur. I'm from Paris. You probably noticed with my kind of poor accent So you can find me on Twitter and Github at R049 So I started working in Python For something like eight years ago Yeah, okay So I started working in Python Approximately like eight years ago, and I started working in embedded systems and building some AI tools for robotic applications So this is what really got me started in the Python ecosystem And then I started as well doing some back-end development and discovered the first API and that's where I think kind of got Offtrack and I started working in open source and trying to build my own project because I got so much like power from Existing tools that I really wanted to Bring something back to the community So I build the OD Montic, which is a simple ODM for MongoDB Based on pydentic. So we'll come back to pydentic later because they use rest as of like one month ago in production And then also I'm a typo contributor to CPython since probably two months ago And Right now so as of last year I started building code speed So it's a tool for continuous performance monitoring and it makes a lot of sense when you're building some native rest module So we'll come back to that later So all I got started with rest so first like probably a bit more than One year ago. I started with the rest by example The wrestling the exercise and the rest book, but it was just like purely theoretical and not really I mean applied and Then last Christmas there was the advent of code So as probably most of programmers when I want to learn a new language I got started with the advent of code. So if you're not familiar with it It's a set of problems you get around Christmas one problem every day algorithmic problems And you just need to solve them every day to get your stars and I tried it Few times ago like with a go long and type script and I never got it finished But with rest I finally have my Christmas tree. So I was quite happy with it It took me a lot of time, but finally I completed one advent of code Then I worked on some small proof of concept to just get a bit the feel of the language and Later, I probably four months ago. I shipped my first cut to my first first code to in production So I was fairly happy with it and it works well for no and it's in code speed So we're talking about trust Probably you heard about it. It's the probably the most loved language I think in the Stack Overflow survey it had been the most beloved language for like seven years You probably talk to people and they tell you they program in rest before even telling you their names or something but it is there is there are some reasons and so Some pros for the rest language. So first is the performance. So since it's a compiled language You will get the most out of you CPU. So it have some advantages But we'll talk about the disadvantages of having a compiled language just later And then you have the compiler. So as I just mentioned, so I put it in the pros because As a difference with like for example the C++ or other compile language, the rest compiler is extremely friendly So it can be a bit annoying sometimes But most of the time if your code compile it will work as you expected to work So this is a this can be a really good point. Then you have memory safety So if you work with C or C++ when you have to do some memory allocation you need to make sure that you are Freeing the memory you allocate because otherwise you have a memory leak and if your program execute for a long time The memory will just the memory use will just increase increase increase and then your system will crash or just your program if you're lucky So this is this has some advantages But also some downside because with this you get also the borrower checker I won't go in the details about the borrower checker, but probably in the later talks in this room people will explore this more in depth and Then you have the package manager. So this is probably the reason why we are not having a talk building a native C++ add-ons for Python Because the main difference with other compile language Is that in rest you have cargo which is an amazing package manager and you have a lot of library on crates.io So in rest package as we call it in Python is called crates So you have a lot of crates for doing many different things and it's quite close to npm or pipi for if you if you know Those kind of package managers and it's way easier than in C++ where for example for installing a library You need first to clone a github repository or copy .h files And it's it can be extremely complicated to install an external library And so now the cons So first there is a kind of steep learning curve So that's why I think it's quite good to start by building rest modules in Python So it takes quite a lot of time to really have a decent level so you can write what some codes that do exactly what you want Then again, it's a compile language So you have the compilation time and as well some additional CICD steps So you need to build your modules to make sure they are available when your code will run So instead of just having a build step with a Docker image for example, you will need to as well build your rest libraries to use them in Python and then the compilation time which can be a bit annoying but it's not much of an issue and lastly The verbo city of rest. So this is why I think it makes a lot of sense to combine Python And rest because for example if we take a simple function To increment a list of integers like this in Python, it's extremely straightforward and you Everything may make some sense just in the business logic There would say so everything is there for a reason in your algorithm. But if you compare it with rust It's kind of more verbals. So basically it's the same idea, but you have a lot of specific details for example, here's a moot and as well here they're referencing the value and It makes things way more verbals Because it's a compile language so I think it may it brings a reason for using rust with some kind of sparseness and not put right everything in rest and so that's why you still using Python for high-level business logic makes sense while Using rest for some small parts Is as well complimentary? So you noticed as well in rest you have some curly braces So we are not very used to it in Python, but it's a trade-off to make so they're no more intense And as well as there is a little crab here, so it's it's called ferris. It's a rest mascot So why would you we use rest within Python? So as I mentioned we have kind of the boss of both world first We have the expressiveness of Python and its conciseness and as well we can use rest for extremely performance needy part of code and as well it makes a lot of sense to do an incremental transition to rest because probably not a lot of people on a team or on a project no Working with rest already, so it just brings a capability to build some sub part of the module in rest So for this example will try to build a Simple graph parser so the idea is to be able to pass a graph So we'll work as well both in Python and then in rest and see the differences between the two approaches So the idea is really to pass a graph and it's a huge graph like a graph with more than 50k nodes and The idea is to do it in a reasonable amount of time. So we will see the performance detail later, but the idea is really to work on performance and so the format to the example format we can take here is a dot graph format and So probably you saw it I think it can it can be used in mere made and it's used mainly to display graphs But here we'll just build a simple parser to take this text input and create the graph structure in our code and so why would we do this in rest because mainly because It can take really a huge amount of time to pass huge graph in Python And so we'll concentrate on the subset of the dot specification, which is this one. So basically undirected graphs with the names and edge names So to build a parser The first first step will be tokenizing so it's kind of the same as spelling for word We'll just extract world and make sure there are words that exist in our language and then we'll Pass this which means that we'll apply grammar rules to determine What's our graph structure is like So the tokenizer So first we need to talk about the tokens so the tokens are basically as I mentioned world So for example here we have the graph world as well We have some identifiers which are the name graph name and as well the name of the edges of the node sorry As well we have some edges. So undirected edges. So Talking about an undirected graph. It's like for example, if we take back the example We could consider that for example, it's a graph representing people who shook hands between them and for example, it means that a shook his hand with B and as well be shaken and with C and be so check event with D and it's Reflexive so it means if be shaken with D, this shakes his hand with B as well So yeah, then we have some semicolons in those in those tokens and as well some brackets So it's a very simple path so we'll build because just an extremely toy example basically So if you want to do this in Python first to declare our token we can use an enum So this enum for example will declare like our static tokens so the ones that are fixed words and can't really vary and then we have Token that is an identifier that have an input actually in it and so for this I use the data class So we kind of removed the hassle of building a fully fledged Constructor and having this extra verbosity and then I finally declare a type that is a union of the two So this is our token either a basic token. So remember of the enum or It's an identifier So if you want to build this in rust it will be an enum as well, but as you can see We can declare everything in the same enum and we can have a member of an enum that contains variable actually and so this is a kind of specificity of the Rust language and It is really where it shines in my opinion because you can really construct complex compound types and And mix them together in a simple way So if you want to talk about compound types first you have the power arrays and vector So in rust the main difference with Python is that If you have an array it's fixed size by default because the memory is allocated at the compilation time so you can't really Expand it as you would with a list in Python and then you have a vector type if you want some dynamic sized array As well Rust have an powerful enum system as I mentioned just before and so for example in Python you have the known types that allow you to do some Optional typing But here in rust the option is an enum and so it's either some of something or none So while in Python for example, it would be a value or none here You have to put some and it's a member of an enum As well in rust I won't talk in-depth about error handling But the error is under with an enum type as well So it's either okay of a value or error of an error so it can seem pretty complicated in the beginning but actually the rest language have a Quite and descent syntax to really make some error bubbling and to make it way simple to to handle errors and As well you have struct so structs is kind of like a class in Python, but it basically a Mixing of multiple types and so for example here We have a strike that use an enum and we can wrap this isn't in an enum of option at the end So this is here kind of simple example of how powerful the enum and strict types can be with rust So if we come back to our tokens, so we have an enum. No, it should be a bit more simple and So if we want to Convert a word into a token In Python we can do it with a nif, elif and else statement. So here for example we pass the first or static tokens and Then if it's nothing we can consider that it will be a dynamic token of an exactly an identifier But since Python 3.10 we have the pattern matching So it makes it a bit more readable. I mean in this case It's really exactly like if an elif statement But you can use the match the pattern matching to extract some some more details And it's quite close to the red syntax as we'll see just later And so out of curiosity who used the pattern matching in Python here Okay, okay great So for those who didn't use it yet Take care because it's unfortunately not backward compatible. So if you're working with open source of where it might You need to wait some time if you want to support Python 3.9 and and lesser for example So we'll still have to wait some time for writing that kind of code in open source of there And so in rust we have exactly this pattern matching system And it's as well it can be combined with the strict and enum types And so it's it brings something that is extremely close to what we saw in Python And as well in the case where we don't match our identifier or our token with a basic token will return an identifier Okay, so now we built our tokens We are we managed to extract them for from our graph and so the next step is really to Bring in the grammar. So it might be a bit more complicated, but I will try to make it simple and to skip some steps because It's really not the most interesting part So first if we take back our previous graph We can just think about it as a sequence of tokens so as as described here and we can try to make it more abstract by just Considering that we'll start with a graph token than a identifier that will be the graph name then a last bracket Then the definition of our edges and then a right bracket Inside the node definition, it's actually pretty simple also it will just be a sequence of identifiers defining the node name Followed by some edges and then some identifiers and then the sequence will be ended by a semi column and This can be repeated to create multiple chain of connected nodes So this can be represented as a state machine So we'll see why making a step machine here makes more sense And so the idea is that the state machine will transition from one state to another by getting one token And so we have two loops in this state machine Indicating that for example, we have we can have an unlimited number of tokens Of of node names on a single chain and we can have multiple chains So implementing the state machine in Python can be done as well with an enum. So for example here I just described the states And then I create my state machine. So it's a I created a class I put it in the initial state And then when it receives the token we use the pattern matching again to transition to a new state So for example here in the start is a start state. We just expect the graph token So when we get this graph token, we switch to the next state Then we have another step where we get the graph name. So this is this one and Lastly, we so then we have all our other definition of state transition But I won't list them here and then in the end we can handle error case And that's why building a state machine for this is extremely interesting Because we don't have to manually handle each error case and we can just say if we don't match and don't find any Then don't find any state transition. It means that The received tokens are not matching our grammar. And so we can just show an error like this So in rest, it's pretty similar. So we start with an enum. So here there is no variable member Then we create our strict. So it's kind of a class, but not exactly. So in rest, you don't have the class and Inheritance things but you have some you provide some implementation It worked like this probably in the next talk There will be more details about it and straight or you can just check it out But the other trade system is a kind of different paradigm paradigm than the class system But it's pretty similar. So for example here, we have our constructor So we create all state machine initialize it in the start state and then again We do some pattern matching. So it's really close to all rest code Here same we pass the graph name. So first step was to get the start and the graph token Then we get all names or graph name and then we have other state transition and we'll finish by handling Exceptions in just one statement. So again the benefits of having built a state machine And so Are we done? Not really. So we've built our rest code, but it's not yet connected to Python. So that's what we'll see just now So first we need to bind our rest function to the Python code So this can be done with pyofree. So pyofree is a library providing binding I mean the most popular library providing binding between Python and rest So it lets you expose a rest function to the Python interpreter and as well It gives you some control the other way around, but we won't go in those details here So this is the first step. We need to bind our rest code to Python and then we need as well to build our rest code because rest is a compile language and The difference with Python is that we need to build it before executing it. So for this The industry standard, I mean the you can have the standard is to use maturing So it's an extremely powerful library allowing you to build rest modules for Python You got you have some cross compilation option. I mean to really have the possibility to support most of the platforms but in this example We'll use something newer and More useful for proof of concept, which is called rest import and rest import actually is pretty simple the idea is just On top of your rest file You mentioned that with a comment that you have rest import and that you're using pyofree Then you import pyofree just here Then you declare your function in rest and you put a py function decorator kind of so it's not exactly a decorator but this is the same idea as in Python and Then you can just Import I mean set up the import hook of rest import and then you can import your rest file directly And it will compile on the fly and then you can execute it straight without any configuration And it's extremely easy. So for just prototyping it can make a lot of sense and You can really test quickly as well. You don't have the need for any extra build steps So for example, if you have a complex CI system, you don't need to add an extra step to Just build your rest module and then put it in your image or in your on your machines As well, you can still configure everything Just below the rest import pyofree you can specify some additional packages rest packages that you want to use for example And then you can really easily migrate to match around when your thing get out of the box state I would say and Yeah, the immigration is really straightforward So with our code if you want to do it first we can declare our graph class so with a name a list of nodes and then the agent adjacency map So a hash map is quite close to a dictionary in Python, but this is the name in rest So this here is a pie class Then we declare our function which will just pass the file We provide it You can see that it's returning a pie result and this is quite a fool for hero handling So for example if as we saw before you can return either ok of your value or air of your error And so if you return an error here, it will be converted to an error in Python and it you will be able to catch it for example And so yeah, then we have our function that is combining our parser and our tokenizer Then the last step we worked with really hard type in rest probably fought with the borrower checker for maybe hours probably we feel like you're less than hours and If we just keep it like this we won't have any typings In our Python code, so it's still quite helpful to just write some type steps So Piy Piy I file To remind to Python what is the structure of our module? So we just create this parser the Piy I file and then when we really import our rest module It will behave With the typing and it will be very easy to access As this is not automated yet. It has been done for Node.js modules Because then you can generate some typings in TypeScript, but in Python it's not automated yet probably it will be done soon So then we built our module so what kind of test should we write There are kind of two different approaches But the easiest one if you're just learning rest and getting started is to write test in Python Because it will be some kind of integration testing where we will really test your rest library and As well, you will be as close as possible to your rest code If you're working with for example teams that know rest knows rest and don't know rest It will be quite easy for them to understand the expected behavior of the rest code without having to Understand the code actually because they will be able to see the tests So here we can just write a simple test file with for example PyTest We just have our import hook at the top. So when we import our pass file function, it will get compiled on the fly Then we define our test and here for example we import a graph that is undear dot dot and It has a length of four four nodes and we just check that the nodes are actually the one we set in the graph and so This should compile quite easily And it will pass but what if we should we what if we could measure performance in the same same time So measuring performance is quite hard because it really depends on your hardware It depends on what software is running in parallel on your machine. So it's extremely complicated But if you're working with rest and you're working to improve your performance It makes quite a lot of sense to measure it because if you're not measuring it then what really bothering improving performance so the idea is really to Put this as a benchmark and to kind of shift left performance testing and doing it during kind of unit testing instead of putting it at the end of the chain so with pay test code speed for example, you can just turn your previous test at a fixture and So you will keep your unit test and it will become a benchmark on top of that So if you weren't by step by test without Any benchmarking stuff it will work fine. We'll just check your assertion But if you run it with the benchmark flags then it will turn this in a benchmark and it will measure only the pass file function and Get you some exact timings and it's also compatible with Pytest benchmark and also Pytest speed from Samuel Coulvin And this new syntax as well is supported thanks to Patrick Arminio We did a PR probably last week. So thanks to him for for this contribution And so if we have a look to the performance We can see that there is quite a lot of difference. So here we have the Times that it's taken to pass a graph of different size. So we start with 200 nodes and we get to 50k nodes so The so it really so the number of the time should be the less so less the lower the better We can see that we have kind of time 20 improvement just by having a Our compile rest module and we didn't do much. We just literally Translated our risk code or Python code to rest and just wrapped a subset of our module And so this has been measured using cut speed so the idea is really to do the performance measurement continuously and Before deployment, so I think this is extremely important because if you just measure it when it's deployed then it will harm your users and Probably you don't want that and you want to catch performance regression early on So the approach could be dad is Based on instrumentation, which means it's more reliable than time based Measurement and especially more stable, which means you won't have some crazy variants Even if you're running in extremely noisy cloud environment or github action, for example As well, it's in completely integrated like like codecov to your pull request and see I Workflows and lastly, it's free from open source. So if you want to try it out and give us some feedback It would be awesome So let's talk about resting Python project and people we did some already did some Crazy integrations that are working and used by many people. So first there is by dantic So it's a data data validation library. So if you didn't hear about it I think there is a talk in this conference, but you should definitely check it out It's something that really contributed to The API development ecosystem it really empowered first API and Completely transformed the way you develop in Python. I mean I couldn't build a project without by dantic and since one month they released by dantic v2 and it use by dantic core, which is a rest module which is using rest module for Python and it really Empowers the data validation and make it as quick as possible As well another example is robin. I think some skies as well here It's a web framework with a restaurant time and it's built Almost like a fast API, but since it has a restaurant time It's as well quite faster than regular just async Python and as well there is another project called TS down sample, which is a time series don't sampling algorithm for visualization and Those three projects are using code speed to track their performance improvement and It helps them already to improve and check out their performance So if you didn't know about this project I really highly recommend you to check it out as well It's interesting to see how they did the integration and what configurations they use with matura So if we dive into the by dantic example The initial speed up was about 16 time so which is quite a lot Just by bringing risk code. So it's an average and Depending on the path and the operation you're doing the speed up is different And they are still working on performance improvement more and for example with PGO. So PGO is like profile guided optimization. It's like telling the rest compiler Which execution pass to optimize? So it's kind of advanced, but it's just a possibility to go even Further in performance optimization and I think they are really on the right path As well, it's really kind of cascaded to other Other libraries using by dantic. So for example home types, which is That data class library For biological imaging was using by dantic and their migration to by dantic give you to just brought 75 percent performance improvement. So this was really like This by dantic new release really has a huge impact on the whole ecosystem and all the machine using by dantic at scale And it's really I think Good for the environment So Yeah, so here are my socials I'll put the code and everything working a bit later this week As well just to come and chat with us. We're here with Adrienne my co-founder for code speed and would be really happy to chat with you About performance for example, and as well you can check out code speed so it's free for open source project and Thank you Thank you very much out of all this really interesting talk We have time for a bit of Q&A and we have a microphone in the middle of that So if somebody wants to ask a question, please get to the microphone and you can ask the question Before we get to that. So give you some time if somebody wants to ask a question to get up there Let's just remind the other audience the by dantic talk that you recommended Visiting is in the other room over there in the south hole a This afternoon directly after lunch at 14 o'clock. So if you're interested in pedantic, you can see that there Now let's have another close look in the round if there is a question that somebody with it Yes, please go to the microphone at the back and then you'll have the ability to ask it. Thank you very much Hello. Hello Thank you for the great talk. I have a question What about if you need to distribute your Your code for all possible Operational system like windows linux how it works. Thank you So it can be done with matron. So with rest import. It's a bit more complicated. I mean, it's probably doable because It's compiled it compiles when you import the module directly But then you would need to have all the rest tool chain on every machine that will use a project It's kind of complicated But the easiest way is to use matron and do some cross compilation or just do the compilation on various Architectures for Windows Mac linux and yeah, this can be a kind of downside of building a rest module instead of just a pure Python module But the performance benefit is here upside. So yeah Thank you very much. We can have another question if you have one. Yeah, so as you're probably aware, there's a lot of Things you can mix in Python that promise performance improvements due to switching in native code just in time compilation. All kinds of things Yeah, right. So in your opinion Your alternative that you proposed rustin python. What is like the Main reason that you would choose this over any of the alternatives already in place or coming up Yeah, I think you for the question. So Yeah, totally and actually I think we there is a kind of Release I'm not really aware of the name but I think with starting with 3.11 It's really some performance release with the CPython core So it will be extremely interesting to see how it evolves and the performance improvement It can bring such as for example with the node ecosystem and via via it for example, which is really IV optimized the code So I think there are you know a lot of alternative But in my opinion building rust code is the most First enjoyable one, but it's really a personal opinion And and as well it brings you memory safety, which is I mean to me It's really invaluable because you don't really have to think about freeing your memory So if you are located a lot of for example huge arrays or I don't know you don't have really to think about it And as well you have the rest ecosystem, which is probably the kind of killer feature Which really brings you a lot of crates and it's extremely interesting because for example if you're working with C++ or Things like this. It will be much harder to install libraries and to get some upgrades, but that's also a bit of personal opinion Thank you Okay, thank you very much. We also have remote questions. So we have a question. How does rust compare to Scython? Scython being able to that you can write your C code And have that compiled fastly. Oh fastly, the difference in performance or in well, it's the real question I can't come to so basically just a comparison. Should I like learn rust or should I go the C path that way? Yeah, the Scython way so probably it's the performance is likewise because Rust actually uses LLVM and the other wood so which is a kind of intermediate language before going to assembly and for example We seek all do you can go through the same path so Actually, even if you code in Scython, the performance will be like very likely to be the same But I think really the upside is that Rustcom is with a strong compiler That we is really friendly. Well, if you're working with C for example It won't be as friendly and you might have some headaches by fixing compilation issues Okay, thank you. Let's have the next question, please. Yes. You mentioned fast API and Robin. Yes. Have you compared those two? No, but I think But I think there are some comparison on the Robin website and I think it at least five time faster So I don't really know the internals of Robin But I think it's on scar is here. So few if you can find him. It would be probably a good question to ask him. Thank you We have a few more minutes time. So if you would like to ask your next question, please Yeah, hello. So what happens if you import the Python module it or the Rust module? Is it recompiled every time you import it in the script and what happens if you get a compilation error? Yeah, so yeah, if you get a compilation error, unfortunately, you will get a runtime error in Python So that's the downside of it. Well with matrimon, it's some external build which means that you will get just a build error And then there is an interesting cache system set up in Rust import that will avoid to recompile the module every time And as well you can also specify some additional flags So for example, if you want some high optimization and a release build you can specify it in the Rust import tool and as well Rust import allow you to do some Offline compilation kind of so when you import your library, you won't have to compile it on the fly Okay, thank you Thanks, let's have the next question, please Yeah, I would like to ask about the performance. Is it always that's the rush is like 2015 times faster than Python are there examples where it doesn't really make such a different like I don't know Web scrapping or connecting database Yeah, so we are for example for connecting with database probably the bottleneck is not really around Python So it's probably the database that might be the bottleneck But as well, yeah, if for example if you're working with huge IOs and you're bringing them from Python to Rust There may be some overhead because the memory needs to be transferred from one to another So I don't know if you want to send one gigabyte or something like this to Rust You might have to think to other alternative and just think about the path of the data to try to optimize it So yeah, there are some cases where it's not really interesting To do this and one more question. Are there any ideas that you can use to program in Rust for example, Python does it has support to An ID even yeah, yeah, so I use VS code. There is a Quite nice extension. I think it's available in all languages called rest analyzer and You can turn on so I can choose an example here But you can have some inline types which is extremely full for example if you're changing a lot of first statement It will give you the type of each statement and it's quite helpful Yeah, if you have some for example functional programming and your chaining map reduce and stuff like this So yeah with VS code, it's quite handy and there is quite a lot of documentation with this Thank you. Thank you very much for that question. We have another of the remote question You mentioned Robin and the question is how mature is the Robin ecosystem compared to fast API or flask? Yeah, so I think Robin is maybe newer. So probably a bit less. I'm not an expert in the Robin ecosystem, but I think it's it's still pretty stable and you can build some Small micro service with it and just experiment. This is the same idea as with the building rest module. You can just create a subset of your whole project and Try the the performance of the of the API for example So you would trust it for production use? Yeah So we have time for like one or maybe two short questions. So please ask your question Is it possible to use code speed for pure Rust applications as well as like Python extensions? Yes Yes, totally. So but you need to build your benchmark with Criterion which is a standard for rest But here it's in since you're working with Python It's way easier to just build everything in Python. I mean the testing and everything Okay, and one more final question, please Does memory safety only affect the rust side or even interacts with the memory manager with Python in some way? I'm not sure I think for the rest part where you allocate memory it will be safe because it's handled by a rust and the compiler So every memory that is allocated will be free But the interaction with for example, if you have some memory leaks within Python directly I don't think it can really help for other part of See Python for example. Okay. Thank you. Okay. So that's everything we have time for now Thank you very much for presenting this for us so that more people get interested in that