 So welcome to my talk I'm Marcello, and I'll be talking about performance tips related to fast API and Before we start Who I know here just like who talked to me the last days just for me to you. Yeah. Yeah. Thank you That's it. Okay, so it's just for me to focus on like look to specific people. It helps me It's not like I don't know how to do it. Thank you. I see Now it's much better Okay, so anyway Yeah, this is me Okay, so this is me again. I am I've been helping on fast API for three years by now I've started just helping on GitHub issues and stuff wrote some requests, but I mainly help on Managing kind of the project with Sebastian kind of thing help him a lot But that's I think what I do there and I've also a maintainer of you've corn and starlet you've corn is Well, if you were here in the room before You know, but anyway, so you've corn is a web server implementation that runs applications like fast API Like fast API and also starlet starlet is a web framework that provides it's one of the dependencies of fast API So it provides the web capabilities for fast API I'm also recently joined by dentiq Who here knows about by dentiq. Okay. Thank you. I mean, yeah, okay Good and yeah, so we just released the version v2 some days ago People were really excited about it. We are really excited about it underneath it uses rust to do all the validation and creating of the Schema so Try it. It's it's I don't know if my boss is watching me, but hi boss Anyway, and we just created a company. So And we are kind of looking for People to understand kind of what's wanted So we have this blog post by dentiq.dev slash roadmap Where we explain things that we want to build and we would want your feedback Well, since it's a data validation Coursing and parsing tool I put there specifically like data people data engineers and data science which really appreciate your help and We also are interested on talking to you so reach me after the talk if you're also interested Besides that Yeah, so we'll be talking about performance tips So here is gonna be something like small things that you can do to improve the performance The way this talks gonna be is like I'm gonna have this base application which might not represent what you'd have on production But it's the one that I That I thought at the moment that I saw on the data was creating this talk So that's the one that I'm gonna use which is this one So as I said might not represent a user's case like real case, but so you see that I have from Online three and four I generate Schemas and I use some data which I'm not going to show here But I've created that with something that I'm going to show on the next slide and online from 9 to 11 I just have an endpoint which I am purposely Creating some by dented models online 11 and then returning that for for the fast API like for the For the client Yep, so this is the base application that we're gonna use so I'm gonna show the improvements and I'm gonna compare always to this one Yep, so let's go I'm gonna be using the version 0.98 because The last the last version of fast API already is using by dented v2 So at the end I'm gonna show the like how it improves the performance using v2 Yeah, but if you install like like fast API without the the version impending today, you would already have the V2 by dented installed Okay, so I generated the the data from those two tools json generator and json to pipe to by dented really nice tools and Yeah, so the first thing that I want to say is about The event loop that we're gonna use when we are using Well when they're building an application and usually people tend to like they believe that just installing you've corn you'll have already the benefits of the most performatic setup which is not true and So you've corn it supports way it supports by default Async I also you need to set up UV loop if you want to use it which is another event loop implementation and I you can see also the how it performs compared to the other implementations, that's nice and What else? Yeah, so if you just install it then you're gonna have an increase of 10% of Performance, I yeah, this is important. I run the benchmarks on my local machine and They are going to be available Like I'm gonna make these lights and the with the test I used I Don't know on the discord server But this is just I want you to see the numbers just as a comparative measure not exactly to look at the The number itself, so you just consider that there is an improvement here, please Yeah, so if you just install it you've corn is already going to use it So if it's in the virtual environment, you already have this Disbenefit Another thing that can be improved Is using HTTP tools so by default? You've corn uses H11 H11, I don't know Which is a sans IO? HTTP parser which was also here in the previous talk. I don't show that And it's the default, I don't know what's the default. I don't why is the reason this is the default I think it was because it was the first one implemented and then HTTP tools is an HTTP parser that works with callbacks and It is faster than H11 because H11 is pure Python and Yes, we run an HP tall tools It's building in C and then there is you have the binding what says there is that it's abiding that no JS also uses So this there is abiding for no JS as well for transcript Yeah, so that's the same thing you kind of have 10% of improvement same idea you should look at this as there is an improvement and You also benefit from it if you just install it So if it's available in the virtual environment you've your corn is going to use it and you already have the benefit Yep So this is is Alex here No, no, no the any IO are you here? Okay, not you anyway, so Yeah, so this is another application, this is not part or do I have I have Yeah, so this is an application that at startup is going to increase the number of tokens And those tokens are the number of threads that you can use So if you notice online Like basic like the threads that they do is a curiculum use So if you notice online 17 we have a death instead of an async that so which means that fast API Starlet is going to run this in a Executor and You can define the number of threads that you have available for that so for example if By default you have a number of total tokens being 40 So which means that if you like right now you send a hundred requests for this application it's gonna run 40 then when one thread is available then it's gonna start running the others because You have a limit and this is something that a lot of people don't know and we're trying to improve how to How to customize this but right now it is like this so you get the default the current default thread limiter And then you change the number of tokens That is so by mistake of my mistake, I think the The low-cost Script that I used I didn't do a good job on the on the Like how I set up because otherwise you'd have far more improvement because it depends on how many requests You are like feeling the the the number of threads So I think I didn't get that right, but anyway, it's If the number of requests is more than the number of threads that you have at some time like then you're gonna need a Bigger thread pool and you're gonna see more improvement in this But that is an improvement again, you should look at this as that is an improvement Another thing is on this example And this is pretty much the same example as the the first one that we started I'm just adding the async keyword on line 10 So which means that I'm not using the executor to To execute I'm just using async tasks and Here just because of that I have I don't have the overhead of those threads And then I have the 50% improvement again. It's just that is an improvement focus on that and Yep So my example was kind of as I said, not kind of real kind of But actually it was took for more than one Code source, so I've seen that in some places which people usually create the model before you send him back to To first API and what happens is that first API make sure that? the response model is the It's actually validated because you can actually modify the attributes of the Of the models that you have on the end point So the idea here is that you always like you're gonna be sure first API is gonna make sure that you have the data All right before sending back to the client Well, there are some people that are a bit unsatisfied with this behavior because They say that if I already have the output model, why should I why first API should revalidate again? And that's something that is going to be a fresh kind of it's gonna be optional at some point But it is not right now And on this case well since it's heavily Like the application that we have is mainly Like we're validating the models like Very CPU bound I'd say Then when we remove one of the layers of the validation, then we're gonna have a 25% improvement And then which means that only first API is doing the validation internally This is Okay, first API went on the other line. Okay. Anyway, this is also Well, this is just changes one line right on the line 8 which should be like 7 I'm adding the default response class Which we use or JSON as To serialize the response Yeah, so this is not something that we're gonna have a huge improvement because right now What's happening is that online 12 we're gonna we're gonna validate the models online and then online 10 internally We're gonna revalidate those models. So we're creating them again So that's the huge stuff that's happening here And then you will you're gonna use the RJs and just to serialize and send to the to the client And then we just well with this application that I have I have a 5% of improvement But again, you should look at this as there is an improvement Yeah Without validation Since everything that I did like making sure that my data is the one like it's the one that I want to be I Don't have that then I would have 150% of improvement So if you don't want to make sure if you don't want to leverage the capabilities that would likely make you use fast API then you have a lot of improvement but I don't recommend But anyway at the end I'm gonna show you something interesting Okay, so you can also do it without logging This is also something that improves the performance. So but I guess you also don't want that. I don't know so you can not have any access logs and You are going to have 15% of improvement on the application that you just saw with the basic setup this setup and then Yeah, and then Without any of the improvements that I told you and just using pedantic V2, which means using the latest version of fast API You're gonna have so you saw that before you have this without any validation, right? And then right now you're gonna have an improvement of far bigger than Then you saw but again Is the Application is heavily doing validation twice and stuff. But anyway, there is a huge improvement using only by installing the last the latest fast API and Yeah, there are some bonuses thing Yeah, so for example, there is there was this command Usually when you create middlewares with fast API or or Starlet, there is a structure called base HTTP middleware. That's where you when you use like App dot and then you put middleware and then HTTP You are under on underneath of that you have this base HCP middleware class which is which kind of Makes it easier for you to create middlewares, but it comes with a cost and Historically has been had a lot of bugs So if you use fast API for some time, you probably face some of the bugs the most common one lately was the that if you have a background task and if you have a middle a base HTTP middleware then you could only have One request in a single house it So if you had two requests on the single connection the background task will block the next one So I had one request which will create a background task And then if I like sleeping 10 second seconds, then the next request is gonna wait those 10 seconds And then it's gonna it's gonna run but that's a soft in the latest Starlet And it's important so fast API doesn't have the latest starlet so that bug is already it's currently happening So that's a good reason for not to use right now, but it's gonna be solved on the on the on the next first API release Yeah, so you can use this thing called ASCII Pure ASCII middleware. I didn't put the pure So this is the way you do in fast API. It's kind of Straightforward so you have online 9 you call the next which would be the endpoint With the request and then online 10 you are adding this new header X potato and then I didn't have a value banana But if you want to do it So as I said, this is pretty simple straightforward, but it comes with a cost and you could remove that cost with Using a base HTTP. Sorry a pure ASCII middleware So just to explain here Like to be big I guess but the important thing is that online 10 and so this is following the ASCII interface And then we also have a good tutorial on how to implement this kind of Middleware, but yeah online 10 and 11 I am Just making sure that I have an HTTP request There are other types of scopes that I can use to create middlewares, but I'm interested in HTTP on this example Also to make it compatible to what I saw where we saw on the last slide and then on the sand Croutine we are just making sure that we are adding this new Header we are using mutable headers, which is a data structure from starlet makes it easy I let it helps a bit in creating those kind of middlewares So we are leveraging that and then online 16. We are just adding the that header Yeah, so those two this one and this one they are analogous this one performs much better because that is it doesn't make it easy for you to implement it, but It's faster So yeah, you can read about it on starlet.io slash middleware have a section called Asgip your middleware it was written partially by me and the other part by Fluorimont, I Forgot his name if he's watching sorry But he's really nice and he writes stuff very very well So I recommend like if you just like if you are using fast API I recommend this anyway because you learn about What as good stuff and And you can also use some middlewares for compressing the responses one of those is already in fast API and solid which is the Gzip zip zip middleware But this is like a bonus because On the application that we had it doesn't make it perform better So it depends on the on the On the response on the body of the response that we'd have So in the case of the application that I had it didn't went better You also have another one the broadly broadly something that middleware But it's an external package that can also be used Analogous analogously as this one And what else just that thank you We do have time for a few questions if anyone would like to come up to the mic or ask a question in the discord Hey, hello You had one side where you just declared a function as async and you said there was a 15% improvement Yeah, that's with the same exact code in that function purely it being async Yeah, so that 15% is just from the threading overhead of Python or yeah Cool. I mean I don't have more than that It's really truly just that I Thank you Sorry, thank you for our talk. I have a maybe a stupid question that how did you measure the What's the formula to have the percentage improvement? So I don't get what can be higher than a hundred percent, but yeah, I Use a little cross script I'm gonna share it after as I said I Intentionally said many times that to not focus on the percentage because I Just wanted to see an improvement and Well, not like that. Like there is an improvement. I'm gonna share it after it's kind of better What's the formula basically if not right in the right what do you put on? Nater that's This one Can you see it? I mean it's literally this so Did you get it? I Will follow up this is this is not something that I come up with like my I saw it in a blog post and I use it Okay, thank you very much. You are Hi, thank you for a talk. Sometimes it's not very clear What to try to optimize so profiling can help and do you have any recommendations for how to profile? When developing and face the API Application or should we just use our regular profilers for that? Thank you I think regular like I usually use the the cute one the new one memory ray memory ray something like that. Yeah Yeah, yeah that one I Don't do anything different than that There is the flag thing Forgot the flag to make sure that you are not blocking the event loop Forgot the name of the flag, but that's the only thing that I do Thanks for the job. Yeah, it is insane improvement 265% using the latest version. Is that related to identity going rust or Yeah, yeah, yeah, but you need like I specifically mentioned that That the application that I used was mainly doing validation and it was doing it twice so it kind of if If that was the main goal of the application that you obviously have a huge performance on on the change of the pedantic version. Yes Thank you Hey, thank you for a talk When you were using the Jason serializer, is there any drawback there's just literally free free launch like I Didn't get the question. So do I need to do anything else anywhere in my application to to to to use Jason serializers just plug it in and You mean they are Jason thing. Yeah Yeah, but also you probably don't want to use it anymore Because by then take is faster anyway right now like almost with by then take v2 It's gonna be faster to use it to serialize them to use or Jason. I don't I'm not even sure if it's I Need to check that but I'm not even sure if it Well, it allows you because you can change the response class so you can use our Jason if you want but I think internally is using the Identical Jason serializer so and It's performs better than our Jason. So that's a tip that was from before the the latest version of STPI Thanks. Okay. Thank you You