 Thank you so much And thanks for coming along. It's very nice to see all of you as he said My name is Robin Ericsson and I like a lot of people around this conference. It seems I'm quite interested in async IO and That's what I'm going to be talking about today. You've maybe been to some of the other async IO talks this morning In my mind, they were a little bit kind of theoretical, but I'm here to try to represent the more practical side of things So yeah, you know, just trying to use this thing and make it work So the first thing that I want to do today is to you know Get a feeling of how many people in the audience have have toyed around with async IO in Python shows of hands We definitely are in a Python conference. That's for sure The more interesting question in my mind, however, is the following how many of you have created actual mission critical systems in Python using async IO and cannot, you know, deploy them to production It's still quite a lot but yeah Still very less than the number of people that have toyed around with async IO and this seems to be the case everywhere Where I ask this question Most Pythonistas have played around with the thing, but they it hasn't really made its way into their production systems for one reason or another And this is despite the numerous blog posts and articles and all of this stuff that details how massively async IO libraries outperform their kind of synchronous counterparts, but also like their asynchronous alternatives So why aren't everyone using async IO in production when it's so obviously all rainbows and unicorns? And there are definitely a number of reasons for this I'm just quickly gonna go through like some of my personal favorites First of all asynchronous programming is it's very different. It requires a very different mental model of how to do programming Takes some time to adjust to Secondly, the async IO feature in Python is it's relatively new to the language And by that I mean async IO I don't mean like tornado and all of that kind of stuff because that dates back to 2002 like to But on programmers a little bit kind of reluctant to new shiny things when it comes to their production systems It's kind of interesting because at the same time we are very much drawn to new shiny things. So it's a very thin line So third to convert on already existing Python application to async IO is non-trivial it takes efforts Fourth due to the lack of native support in the language in the past to do asynchronous programming the community has come up with solutions to this and They've come up with the libraries like event led tornado twisted all of the stuff And many of these libraries are widely used and they just work and people cannot just you know stick with them. So Last probably most importantly asynchronous programming is not always the solution to your problems Now it's so happens that I do have some experience with migrating actual in production Python codebases to use async IO And even though async IO is by no means perfect my path like my experience thus far has been quite positive And that's kind of why I'm here today a little bit to spread the word So my talk today is split roughly into three parts the first part in the first part I'm going to be discussing discussing what led me to commit a lot of time and effort, you know into migrating Software that was already working in production to use async IO Then I'm going to talk about some of the roadblocks that I hate during this time And you know some of the caveats that you might want to be aware of if you want to do the same thing last I want to talk about some of the benefits of this work and You know try to back this up with some empirical data I should probably also mention what this talk is not about Just so we're on the same page. It's actually not a lot of things But mainly it's not an introduction into async IO and how it works So yeah, you're gonna have to look that up somewhere else But even though you have absolutely no knowledge of it at all Don't worry. I'm hoping this talk might at least kind of trigger you to explore the subject further um interesting So why would you want to bother with async IO? Sorry, I'm just not gonna touch this thing So your software already works, right? To understand where I'm coming from personally We're gonna need to explore my background a little bit just a teeny tiny little bit so in my past I worked for a company that Was obsessed with using Node.js for everything Admittedly I was kind of one of the people in charge so I cannot really free myself from that But despite Node.js is and JavaScript's Mootable pain points These technologies still taught me how useful asynchronous programming can be and how efficient it can be Especially when your applications are just mostly IO bound By the end of my tenure at this company I had gotten used to like writing everything even what would be very simple sequential scripts in Python in an asynchronous fashion So when I joined my current employer a company called Smarkets I had to I had kind of to get a new like used to a new reality Smarkets is a company that relies heavily on a microservices microservices based architecture and most of these services are Coincidentally written in Python so initially when I started looking into how these microservices were structured and And how they were coated all I could see was just plain old blocking Python codes To be honest, I wasn't too impressed Especially coming from my awesome asynchronous background So at this point in time I started thinking like how are they scaling these things? Because some of these microservices were taking on some serious serious traffic So after a bit of research, I noticed that most of these services were scaled out in a fairly classical pythonic way its service was running in multiple instances application instances and its instance was a goonie corn server that had you know multiple workers So scaling out was a matter of adding new application instances or adding new workers or both So because our microservices are very IO bound It was kind of apparent to me that all of these workers on the end were just going to be stuck waiting on IO the whole time I was just thinking like what a waste And after thinking about this for a while, I noticed that a big part of the services depended on a packets That was somewhere deep down doing this thing here Probably some of you know this thing David actually touched upon this in the keynote this morning but Obviously my Node.js acclimated mind did not realize immediately what this did but after some research I figured out that this actually patches the standard library So you're previously kind of blocking cold will magically be doing asynchronous things behind the scenes cool So before I think IO came along this was apparently one of the kind of standard ways of achieving concurrency in Python In my mind, this is problematic in Multiple ways mostly because this is kind of hidden away from the developer It's quite magical and it's hard to reason about as well Also like many developers on my team and in the company We're not making use of event less multiple concurrency primitives because they didn't even know this was happening somewhere in their kind of software So the moral of the story is that async IO itself and the kind of async slash await keywords make this way more explicit and Someone told me that explicit is better than implicit I should probably know that I have absolutely nothing against event let in general Quite the opposite I find it fascinating that they were able to create something that works so well that I didn't notice it for you Months that it was kind of happy like doing asynchronous stuff behind the scenes But now that we do have async IO. I think we should kind of start using that So yeah to summarize a little bit. There are multiple ways in which we can achieve concurrency in Python, but in my mind Async IO is the obvious way to go because it's explicit and it's a part of the language It's also supposedly quite performant, but we will get back to that later on So assuming that async IO is indeed the future and that we want to kind of You know Make our new projects use it or migrate our existing projects to use it What issues are we going to run into and how simple is it? I've already mentioned. It's actually not super simple But it's definitely getting better. It's getting way simpler than it was Not that long ago when I was taking a look at async IO in Python for the first time I noticed that people were relying quite heavily on Monkey patching again their dependencies just to make them work with their async IO codes For anyone that's serious about their code base and like serious about the reliability of their system That's probably not a great idea But since then for the kind of past, you know a couple of months maybe up to a year It seems like the async IO ecosystem in Python has grown and matured rapidly We now have loads of mature web frameworks Loads of mature kind of database drivers and just a lot of libraries for a lot of stuff and yeah It's it's it's getting quite good So I kind of feel now like it's getting to the point where library authors should no longer be ignoring async IO And instead kind of defaulting to it Now because I'm a very practical person as I spoke about earlier I think the best way to describe how you should go about migrating a Python project to async IO is simply to describe the process that I went through recently when I Moved one of our kind of in production microservices to use async IO Having done this before I knew that the first thing I'd need to do was to kind of map out the dependencies of the service and research if there would exist async IO compatible versions of the dependencies and Also, since this was a microservice that exposed like a restful API Obviously, I had to figure out which kind of web framework to use always a classical problem And when researching these kind of async IO web frameworks that we have now I was pleasantly surprised to see the kind of wide variety of web frameworks that we have access to This slide just shows a few example of those And all of these are kind of specifically targeted towards async IO Some of them aim to be flask like some of them aim to be fast some of them aim to have good web socket support and some of them Aim to be compatible not only with flasks public API, but also its private API In such a way that yours like now you're basically supposed to be able to do a find and replace or flask to quartz And you just sprinkle a few kind of async await keywords around and it just should work. I Haven't tried it out, but I don't know a colleague of mine Phil actually wrote this framework. So Yeah, it probably works. He's a good guy This can be obviously this can be very useful if you are migrating a large kind of flask application to async IO And you just want to get up to speed quickly Now I didn't care that much about API compatibility with flask So I decided to go with another web framework, which is called a IO HTP Simply because it's known to be mature Sputtle test it and I don't want to take my chances with new shiny stuff like quartz But even though the a IO HTP API is a little bit different from flask the flask API. We all know and love migrating to it was super simple and In fact, I kind of felt a little bit kind of relieved to be rid of the global request object that I've always found a little bit odd That just mean Anyway, the same can be said about the migration from Psycho PG2 Which is the old database driver that we used for postgres to async PG which is the new de facto awesome Postgres database driver for async IO Except for the desperate lack of named parameters if the developers are in the room I was a little bit worried that migrating sentry the error tracking software that we use would be problematic and After searching the internet for a little bit I found some packets on GitHub that was supposed to be you know an integration between sentry and IO HTP couldn't really make it work So I figured I just I was just gonna be simpler for me to do this myself So I had already worked with middle-wares in a IO HTP and I knew I could easily implement something like this as middle We're and literally five minutes later. This is what I have Very typical middle where just runs the request handler race and if the request handle raises an uncaught Exception my sentry middleware will will capture the exception fire it off to sentry and then just raise again Super simple. So obviously this also makes use of the fact that The Python sentry client library also has support for a IO HTP as a transport. So so far so good Unfortunately, my migration was not completely without issues One of the features of my microservice prior to my migration was that its API was built and defined in swagger And it had some kind of fancy swagger UI thing on top of it. So you could kind of interact with the API. It's really cool Anyway, this support came through a library, which is called connection probably some of you know it and after researching connection for a little bit I I I It looked like it would support using a IO HTP as a back end But then I kind of tried it out and it didn't really work and the documentation was kind of non-existent So what I probably should have done at this point in time was just to fix the connection library and the documentation But so my to-do list no worries Yeah, instead I just dropped the swagger thing So up until this point, I've been talking about async IO like it's the perfect piece of software and it's not Yeah, that's just it's absolutely not There are quite a few things you might want to be aware of before starting to use async IO and I'm just gonna mention some of the issues that I've personally experienced Yuri spoke about more issues earlier and Yeah, and I think someone is talking even more about issues tomorrow in another talk So plenty of that first while async IO is conceptually simple in Python There is a steep learning curve to understand the whole beast We're talking about event loops event loop policies avatables coroutines generators futures tasks Executors transports protocols, etc. Etc. Etc. Etc So if you're migrating a large important code base to you know use async IO You'd ideally want to understand most of these and kind of how they enter play So yeah, this takes time another issue that many developers that try out async IO mentioned is As soon as you start developing Like migrating your Python code base to use async IO your code base is gonna be littered with async and the wait everywhere To me like yeah, I can't experience this but to me. This is like this is not a huge deal This is just the cost of explicitness and I'm gonna welcome that An issue that definitely did hit me personally was the fact that sometimes the async IO stack would just You know make my debugging live a little bit too difficult By kind of swallowing my exceptions and I had like no idea what was going on but this is definitely getting better every day and This is something that Yuri the one of the Python core committers around async IO spoke about earlier as well So I'm very happy to hear that Now accidentally running synchronous code that waits for IO inside of your asynchronous functions yet another source of issues Here what I can tell you is like it's very simple to monitor the event loop It's basically just about figuring out which tasks on the event loop take longer than x milliseconds and If some tasks are doing that this might be a sign that it's actually doing some IO or some kind of blocking IO Inside of the async functions. So yeah, as I mentioned, this is just some of the examples So to summarize if you plan on migrating large Project to async IO or you know create a project from scratch the most important thing I can tell you is just you know do the research first map out your dependencies see if there exist async IO compatible versions of them and You know if they exist actually make sure that they work and you know check them out like that's very useful What's out for something gotchas and yeah profit So you've migrated all of your projects to async IO. Was it worth it? Are you any better off? Let's see how it went for me I'm gonna start out by taking a look at that kind of before after Evaluation of a microser the microservice that I spoke about earlier I've already mentioned the initial version of it had you know flask to declare an API that's Psycho peachy to to communicate with the Postgres database a lot of other stuff that matters less Also utilized event led to make everything asynchronous behind the scenes. So in theory, it should have been quite efficient We've also gone through how the after version of this microservice looked like a IO STP as a web framework async peachy as a postgres database driver and To run the event loop itself. We used the python implementation of the UV loop, which is apparently very fast And this is kind of our experimental setup quite standard We used the work project or WRK to perform STP benchmarking of both versions of the service We ran each configuration of the benchmark for 30 seconds 10 times using a variable number of connections And noted the median 25th percentile and 75th percentile in terms of throughput request per second For all of the 10 runs, we also had some delay between the runs just to kind of allow the server and client to settle and On this slide, you can see this throughput comparison Of the microservice running in quite a basic configuration with only one guñicon worker and just on my machine So just in development We then use this work thing the benchmarking tool to test the throughput of both versions On a very simple just ping route that doesn't really do anything interesting. It kind of just returns the string Pong And we use this quite a lot in our microservices just to do health checking like are you alive? And as you can see the results here are quite conclusive like the async IO version of the microservice That is the yellow line is able to handle on our like on average fourteen hundred records per second While the event led version is able to handle somewhere around six hundred and fifty So in this case the performance increase is somewhere around two-fold Here we can see another benchmark that we did which is on a route that actually does things It does, you know some interesting database accesses And then kind of returns the results of the the database access and some transformations and in this Experiment it seems like the before the event led version is able to handle somewhere around two hundred to three hundred records per second While the async IO version the yellow line again Seems to be able to handle somewhere around eight hundred to one thousand now the most interesting one is probably this one This is an actual Bandsmark on the microservice both of them in production both before and after the migration and In this particular experiment the number of guinicon workers were just set to three And we only run the bands marks on one application instance Usually we have more application instances in production But testing all of them in conjunction is kind of besides the point here because they are supposed to be able to scale Linearly if the load balancer that you're using is okay So as you can see Here the before version the event led version of the microservice Performs, okay, like it's we're handling one like one thousand and three hundred records per second quite consistently While the async IO version Confusingly the green line In this graph is able to handle around seven thousand records per second And this is more than a five-fold increase in throughput So this kind of means that we can just get rid of like 80% of the like application instances that we have in production Which is quite nice because we save a lot of money So just to conclude the whole thing is async IO worth the efforts We finally have explicit native support in python to do asynchronous programming That is kind of slowly but surely becoming the de facto standard in the community And kind of replacing other concurrency libraries and as we just saw the performance the Performance of icing IO is quite good. It's very positive results and these mirror what I have seen on the web So all in all yes, I think it's definitely worth it. Go check it out if you have not Thank you Hey I Have yes, I was doing some like I was using some stuff in our code That was apparently doing some very interesting requests like using the request libraries somewhere it kind of deep down I Didn't really understand why it was doing it But yes, I experienced this problem and it's completely terrible Very hard to figure out in this case Well, like this is basically an issue that like it didn't take a long time for me to figure out why this was because I Saw that some requests were just taking too long and we were basically Everything was being held up because of this one request that was doing like this thing So it was pretty simple for me to do it I basically just found the code and I just literally threw it out. It wasn't necessary So I replaced it. So that was my solution like another solution would have been to actually start using the AI OSTP client instead of the request library Then everything probably like It would have been able to make this work Yeah So I might need like just yes or no piece of advice So what what do you think about like run async I own then run like thousand Processes and waiting in the queue in the main thread to just pick them up Wait, so running thousand processes. It's with like an east async event loop or yeah Like everything's there like we are waiting for the tasks Yeah Like they're in the meantime You're just waiting in the loop in on the queue and they're just the process are writing inside Does that make sense? I I'm not sure I understand but Using processes for the Using actual like operating system processes. Oh, yeah, I see sending a request to a Website like running something I think there's probably a lot of overhead in that to be honest Like a lot of overhead compared to what async IO gives us which is very lightweight Kind of you know just like some kind of task queue. It's running the queue If you you have to maintain all of these processes and the operating system has to switch between them correctly So I think there would probably be a lot of overhead in there Yeah, we can discuss the point I Have complexity like transports No, I don't I don't but like I feel like I need to I'd be like like I have never done anything with transports or protocols But given that I'm writing an actual, you know in production very important microservice I feel like I need to be on top of what this stuff is doing No, I'm not saying that absolutely not I'm just I'm just saying I should probably know what these kind of basic building blocks that are behind async IO I should know what they do But no, I haven't used them and Yuri actually talked like talked about this earlier Like I'm not supposed to be able now. I'm not supposed to need to know what these things are So maybe I should just take his word Maybe I Have not no, but I still feel like I need to know it Well, yeah, yeah, so basically the API is no longer defined in swagger and we don't have this swagger UI for this particular microservice I have not So that exists. Yes. Oh, that's very good to know. That's very good to know. Thank you learning a lot today So How should I put it like The code that I'm working with The company that I'm working for it doesn't use ORMs a lot and definitely the projects that I have been using they don't so If you are yes, you probably that's like this is one of the research things that you would need to do if you Want to migrate a project that you're using on an ORM You're gonna need to figure out if there is some support for ORMs somewhere else I I don't know the answer like I I we don't really use ORMs We just use, you know, like SQL queries directly. So, yeah research it some of the shortcomings of as in go should be problems with cancellation timeouts and Back pressure. Yeah, you run into any of these problems. Yeah, like I probably run into some of these But like these are not things that I'm dealing with a lot these services that like for example, this service not complex It's not it doesn't have to deal with a lot of timeouts and stuff like that so No, I haven't really run into a lot of these. No Norris, let's give our speaker one round of applause