 Thank you guys Thank you for coming after this tough party and stuff morning. I hope I will not disappoint you I will try to not make you fall asleep Therefore first couple of words. How will we do this today? So I have a 40-minute slot basically and I do not want you to wait till the end of the talk to ask questions Please rise your hands at any point of time. We'll try to make this interactive So, of course, I have the microphone. So I'm the boss now, but you'll get one too. Just raise your hands We have a nice discussion here So I talked to you today about the asynchronous web developments and tornado in particularly How can we make it better with tools that come with Python 3? First of all, of course, you have the question who is this guy and what does he know to present such stuff? So I'm just as you just the Python developer and I work with Python for like five years more or less I love it. I love it so much that I started the pie Munich group that is making workshops in Munich And we are trying to get more people into Python. I love it so much that of course They work with a company that is very Python friendly. I'll say a couple of words about that too My company Scooby is a German e-book subscription service. It offers lots of e-books on the monthly subscription base like a flat rate we have the native apps for Android and iOS and People like that apps because they put five star ratings to them even though it costs money, which is pretty cool, of course Being user-friendly is nice, but it's more important to be developer friendly So the stuff that I'm going to present you today is not some Abstract blog post that I read a week ago. This is stuff that runs in production for us for years so real thing and basically It comes out of the problems that we had so I'm presenting you the solutions that are already deployed there and As I said the company runs Python as a main language. We also like very much to Participate in Python events share our knowledges We are sponsoring your Python and also other Python events in Germany, especially and Let's move to the challenges that we have from one day to another That's lead us to these solutions So first we have a very distributed system meaning that we have not just one server We have many services that need to communicate over the network or there are some APIs Mostly HTTP APIs. So we learned by time that doing that asynchronously obviously makes it way more efficient therefore, we could not use just nice and fancy Django because Internally, that would be not very performance. We are trying to build tools that speak to each other as synchronously and Efficient we also on our way to migrating to Python 3 like probably most of you guys Of course, it's not that easy, but some models run Python 3 some models still run Python 2 most of them actually of course So it's also a bit of a challenge and we see what tools are already usable with Python 3 What tools are not? We love to share and this presentation will be basically a fully practice based This is the overview first. I'll speak to you guys again. Why do we need this in chrono stuff at all? And when do we need it? second How generator delegation syntax introduced in Python 3 makes it easier and nicer Then of course, I think I owe this is Tool that we heard already on this conference wonderful talks by the way I'm going to just quickly cover it again in case someone is still missing this information Then tornado which is a real framework that can't be usable together with a sinker. You're already at this point of time Major thing ready for production and I'll show you guys a little demo of not just tornado of also other frameworks How do they perform a synchronously? Together What issues do they have comparing to each other? Well, I want this to be on the recording. So it doesn't matter much, but okay good. So Why do we need the synchronous? communication at all Let's have a little recap of the web server execution models like how how the how can it work? The first originally idea was to have one thread per connection, right? This is very straightforward. We have clients coming in we create a threat for it. We handle requests on this thread It's very easy very straight. It's very easy to share the memory We have any show obviously Okay, guys who have seen the threaded web server dying Anyone okay? Yeah, so this is not just me which is good A threaded web server runs in obvious issues when operating system goes out of the memory first can do evil things to you and The worst is that when our system is overloaded not only the new clients will Experience bad service, but also existing ones. So the whole thing crashes total disaster. So what would be the logical step? To fix this issue Well, it would obviously be to have a pool of on the threaded server that would limit the amount of threads or processes Doesn't really matter in this case. So by limiting the the threat pool. We will not Push our system into the state where it crashes, right? So we have a fixed number of threads when our system works fine when it's still performant and nobody Experiences any problems with it, but for some new clients that will join when our pool is already packed They will just have to wait and this is bad because Obviously, even if the thread pool is packed that does not mean that our system is already out of resources It just mean that some numbers that we picked is reached But we might and we usually do have more resources to handle more clients So the next step was to have the asynchronous web server That has just one thread But it kind of turns a picture around turns it a bit upside down because this one thread has IOL loop and This IOL loop controls what client is getting service at which point of time and since Usually our web server are IO bound. So we are not performing any heavy computation in our web applications Most likely so our CPU is pretty okay. Our memory is also pretty fine. We are waiting for something We're waiting for the database. We're waiting for cash We're waiting for whatever thing and so the IOL loop can actually decide at which point of time Which client can get service and other client can just wait for Data to be available on a particular socket and IOL loop can jungle this connection jungle the client and just Yeah process every client at the right point of time So it does the event driven switching in one thread But you say of course async is not something new like This is not the first time of course this idea comes and rises up again So you're right because tornado is like six years old and there is also twisted True python tool for the network. I still think it's one of the best And there are also stuff in other languages, obviously Why we still use blocking servers and what's the problem? Because sequential code is obviously way easier to read. It's easier to test. It's easier to maintain an extent We have seen how jango was successful how ruby on rails was successful just because of the simplicity and nice batteries, of course, so these things do matter and Then obviously we think why can't we just have a Asynchronous codes to be structured and written in the way that it can be just readable as synchronous codes Look similar without this callback. Hell and stuff And python 3 let let's us do that This is something that i'm going to present you today. This is just a little teaser. Obviously, you're most likely already seen it but still this yield from is a generator sub delegation syntax introduced in python 3 and It lets us write a synchronous code in a synchronous fashion Again, I'll have this more detailed a bit later. So now yield from it aims to replace callbacks, but first we need to Come to decision. What's so bad about the callbacks? Like it's something pretty common. It's uh in twisted since ages in javascript, of course It's difficult to make it look nice Does any one of you guys work with javascript? Yeah, obviously, obviously, okay, so Did you guys see this? I also do javascript and they see this is the real code besides it's some testing thing, of course, but still I mean even for testing I usually try to avoid this and by the way, this is a nice resource callback.hell.com So if there is such domain registered, I assume that we are clear that we might want to avoid callbacks So sequential code obviously looks nicer and By introducing yield from in python 3 We have this attempt to write a synchronous codes on everyday basis without callback.hell Now it's time for a good example So we have a very simple Function, let's say it's a get handler of whatever web framework We have a huge database query that is blocking obviously The result of that query is saved to the result variable and then we write the stuff back to the user This is blocking, right? How would we do it with callbacks? So with callbacks, we can define a function Just nested or anywhere that will be a callback that will be passed as an argument to the huge database query And the huge database query will call it when the result is ready. Also pretty clear, right? This is not as bad as an example with javascript, of course, because at least it's not defined inline But still we can make it even better with a sub generator based handler So we put yield from just before the huge database query Who of you guys use they think I already saw that Okay, so that I know I stop a bit on this then So what exactly does yield from do here? So huge database query in this example should not return the result. So it should not block It should return a future. This is a magic object. I'll speak about it in a moment But this magic object is like a promise that the real result will come at the later point And we yield This future back to our coroutine to our ioloop That will at this point switch to any other task that it has and whenever the result of the future is ready Whenever future can be evaluated to the result It will push it back to this point as if you can imagine as yield from would never be there magic It will be safe to result and return back to the client So, uh, what is exactly this magic behind it and what I think you're does for this So as it was already told a lot on this conference This is mostly the event loop and a set of tools to use this event loop efficiently Event loop can register a callback for any particular Any particular event let's say most typical case, of course We want to get to fetch some network resource So we need to make the tcp connection And then we just wait so we tell ioloop. Hey ioloop when the data will be ready on this socket Please call this function and it does that's in a glance. What what do we need to know about it? In this example, I have a minimal setup. Uh, how can we call it? So we have a callback function. Hello, euro python. It doesn't fetch anything. It will just be called as soon as possible First we need to get the events loop instance Then we schedule the callback by call soon call soon is call at any point of time when you can And then we start the ioloop. That's a minimal example. So no fetching on the socket nothing Now to get more familiar with this thing, uh, let's talk a bit about the future object What exactly does it do so? Those of you at least who worked with javascript probably know the deferred or promise objects Most likely so this is a very similar thing. So this is some object that Packs a reference to the result. We have this object just to track the state of the Operations that we want to perform. Let's say we want to fetch the database query We call that query and the future object is just Returned to keep a reference to the result that is not ready yet But we'll be ready at some point of time or it will fail So we have the future to keep the reference to something that will be available at later points of time So simplest way to run a future is just to use this Run until complete on the ioloop and print the future result But this is usually of course not the case that we want. We want it nice asynchronous so Then we need to Have a better syntax. We can also yield it Like here so we define a coroutine It's also important to mark that Whatever web framework you are using it has to know that you are using coroutines. Otherwise, it's just a generator So we are fetching for example google com The results will be a future then we yield this future ioloop does is a fetching And saves the result back to the result variable then we print it And of course to make this whole thing run we need to Run the ioloop So what about tornado here? Tornado is a web framework that is already there for six years. It has no ioloop. It has own futures But now it has to be obviously compatible with async.io because this is a common Frameworks that all of the python higher-level libraries should use So tornado is already production ready framework and Its compatibility with async.io is already there Basically, this is a stack how in the best case in the future We will see it in the python world on the application framework level We can have tornado twist it or whatever as a framework Then as an ioloop Async.io should be used and finally async.io is at the end Just a common interface for the operating system specific selectors like kq epol or windows select whatever This is how should it be Right now even if you use python 2 you can use Tornado with an own event loop with own futures with own everything But since we want to develop for the future we want to be future compatible We can use async.io as well So let's look at the event loop. How does it work with tornado and compatibility with async.io? This is first the way that you would get the event loop in tornado just select ioloop current It's a single ton and you start it This is the way that you can use tornado with async.io event loop Basically, again, this is the same two lines of code with just different syntax pretty easy replace one with another You're running it on async.io Futures again tornado has own futures async.io has own futures, but they are super similar We also have concurrent futures back from python 2 also pretty similar thing But for this compatibility We need to use the tornado magic method that will convert one type of future into another type of future Which is not a big deal because again, it's just one line of code Now I think it's a good time that we have a full Get handler written in tornado Let's look at this So in tornado, uh, we need to inherit from the request handler We need to mark the Method as a coroutine so that tornado knows that the thing that we will be yielding is the future and this future should be evaluated And then the result should be passed back We need to instantiate the asynchronous HTTP client that will not just block instead It will return futures construct them and return it for us Then we use the fetch methods to fetch some really slow api, whatever We are yielding the future that it will return back to the ioloop ioloop will call us back at the right point of time when the result is available and we have the response here Alternatively, of course, there may be no result. There may be time out. There may be 404, whatever Then it will be just normal python exception handler handling. So in this case tornado will just write 404 HTTP error That's it. You catch it with normal tricep Then we write response back to the user as pretty common for python web frameworks And then we need to call self.finish because this is a coroutine. So we want to keep control when request this cut This is a full example. Let's say that we want to make the HTTP proxy We we want to pass a url as a parameter and we want tornado with async.io to fetch it for us and just Stream it back to the client So these are all of the required imports. There are not that much actually This you have already seen the way that we construct request handler Then we just get the url argument from get parameters. We again create the async HTTP client We fetch this url yield the future to the iol loop get the response back write response finish response Yep, please Very well. So I repeat the question The question is How exactly does it work over here that HTTP clients would normally block when you call the fetch Wait for the resource to be fetched And yeah, what what is this yield from doing in this place? Uh, well, how does the synchronous magic work? Very good so This is a kind of a spatial HTTP clients Async HTTP client obviously so we have a similar Library called just HTTP client in tornado and it will do exactly this thing if you run HTTP client dot fetch It would block it would wait just like request just like url. Leap It would block and wait and fetch the response Async HTTP client is not blocking. It's not waiting. It quickly constructs a future object So not the real response object a future object and returns that and since we do not need to wait for long time to Just build a future. It's basically just creating an object of a class This happens super fast And then we are yielding this future So we're so to say sending it out of the function back to the i loop that is controlling this whole process Meanwhile, yes, this get request will just pause So it's like frozen in this state until we have the response of the fetch thing But we have other clients meanwhile, so i loop will in the meanwhile It will serve other connections for us Therefore this get handler will be paused but others get service and in practice it happens so fast that you never see it and whenever i loop will notice that okay now the The url is fetched now we have data on our socket. It will look up. Okay, who have been waiting for it Oh, it was this proxy get thingy And it will call it back And by calling it back it will have the result that was fetched And it will push it just over here So you can imagine that after the i loop was done with fetching stuff Imagine that yield from was gone. It was never there. There was just the fetch operation. This is basically in a glance how that will work Yeah, please Since it's stopping anyway on the yield from why do you need a future object? Why do I need one a future object? future Okay, so why do we need a future object here? So, uh, first of all yield from well, there was always yield without a from Yield from is required if we want to get futures From another coroutine in this case. We don't have to use yield from we can also use just yield But if you're building a big system, you might have a stack of the coroutines one calling each one calling another Then you need from but this is I guess not the question the question is why do we need a future at all at this point? So we need a future because if we return After the fetch call if we will return the result we obviously need to wait for results Here so basically this call will block So since fetching takes long time We construct the future and constructing the future does not take long time because it's just creating of a future object without waiting This is basically how we use futures. It's kind of abstraction That lets us not wait for the response But give something back so that color has a reference to the response that will come at the later point of time and If you might find confusing this yield from syntax you can also imagine a raw future so if we Take back take away yield from just get the future and doesn't don't call it response call it just future Take the yield from away Then at the later point of time to get the response I broke it so to to get the response of this thing at the later point of time we can just call the future dot result directly and Then it will give you the result if it's ready. So future is pretty simple. It has A status it has a result. It has an error. I guess so you can access the results directly But it's just nicer if you don't do it And you let the io loop do it because then you can use the magic yield from syntax that Will extract the results out of the future for you. Uh, no, there will be no result. I guess You can also check if result is available. So if you want to write a really bad code You can write a loop In which you would ask is the result there? No, is the result there? No And then you will reinvent the io loop because basically that that's what it's responsible for We had this example as far as I know. So let's go further demo I have the ipython notebook for you to Show this thing running Let's agree on the simple case that we are going to test So we have the client That will be our ipython notebook We have a web server This will be the whatever synchronous framework we are testing and then we have three apis These three apis will be requested in parallel and so asynchronously And we want to check how well web server is handling these three concurrent requests to the third party apis To simulate this whole thing without external dependencies. I have everything running on my laptop I have tornado server or other framework server too, and I have a dummy local server That just waits for fixed time out of 10 milliseconds on every request and returns a dummy hello world greeting And you can see it over here. So this is what my dummy local server does Now we can test how fast can ipython notebook fetch This dummy server responds directly so that we have something to compare to so it's 16 milliseconds From this 16 milliseconds 10 milliseconds is our fixed delay that we have on purpose and six milliseconds for fetching overhead So first example we have just the blocking server Just a blocking tornado server that is using HTTP client not a sink HTTP client Therefore it does not return futures. It blocks and it waits for results In this list comprehension. I have three URLs. So three dummy URLs That I want to fetch i'm calling fetch on each of them, and then I'm just looping over responses and writing it back to the user This is how it will work. So we requested three times same URL We have the response. Let's see how fast does it go? Okay, so this is 54 milliseconds altogether Obviously, we just multiply this 16 by 3 because it's a blocking server. So it did it sequentially Now let's see how the same thing but just using the a sink HTTP client work Basically, what do we need to do to convert blocking codes to non blocking code? We need to change the HTTP client Into a sink HTTP client. So add this thing Then we need to mark it as a coroutine in order to let tornado or other framework in I think yo, it would be also a coroutine decorator We let it know that this thing that is Beneath will yield us back the future so that tornado knows to process it accordingly So we are calling this thing. We have the same response And this should be three times faster It's not because of the overhead of course this whole thing iol loop and stuff has on overhead, but it's 22 milliseconds So comparing to 54 it's Two and a half times less more less and comparing to the requesting dummy url directly 16 versus 22 we have again about five milliseconds of the overhead of tornado Now the fun part comes Not js So this is also a synchronous framework and probably everyone has already seen on the internet some article like hey my not js on the ec2 micro instance is handling i don't know 10 100 000 of connections simultaneously. Yes, this is true This does work especially in the case of web sockets Is a perfect case for a synchronous framework to be used because for web sockets We have many connections, but all of them are mostly idling So we are wasting our cpu if we have resources dedicated to each of these connections. We want To use our resources efficiently therefore at one point of time we can have just one client Getting real work done on the back end others can just wait because Web sockets do not need everyone to do stuff at the same time So this is how it works I might of course have it Not the not in the perfect way as some javascript guru would advise me But I did it in purpose because I tried to avoid external dependencies Not js has a lot of external libraries that would make it kind of look nicer, but this is just pure not thingy So what we do here I created the option Object that keeps reference to the local to the local host dummy url. I have The array that is waiting for responses that will get the responses then I need to loop I'm looping over api count, which is three just to have three requests And they do http.get so http is a library that is kind of the Async http client in tornado And then I need to of course define a callback what to do after we fetch stuff and I'm defining it in line in the best tradition of the javascript Need to set encoding. I need to create the body chunks because this function will be called not when the result is ready, but when the chunk arrives So on the data I'm creating another inline function that will push this chunk to the array of chunks then On the end event that will send us after all chunks are there I need to glue it back together by using puffer concat Then I push responses to the body object And finally when they see that okay, so responses are three just as api counts. That was three. I can say okay everything is ready now then I Create the response text. I set headers and that's it. I end the request At the end we have the same thing and let's see how fast this works We can remind what was in tornado meanwhile. So it was 22 milliseconds for tornado and it's 18 for not just Bravo not just well done. We have three milliseconds faster But maybe four hours more of our developer work Let's have another example scala Wonderful performant language I guys just don't have time to go through all of this mess Believe me. You don't want to read it I of course realize that the people who are experts in scala do this pretty fast In fact, I was not the guy who wrote this because on every conference when I'm showing some tornado examples I ask someone who knows some other asynchronous framework to give me the example So that I can Add it into my comparison But the funny thing is that this thing is damn performant Let's see how in practice does it work 22 of tornado 18 19 slightly more than not. Okay, but of course, this is not a very fair comparison because I have a fixed number of Back-end apis which is three. So I fetch three apis What if we have 10? What if we have 20? We don't need to speculate. I wrote a little benchmark that will answer us This question. So here I have tornado. I have not gs and I have finagle And I'm going to increase the number of concurrent requests from 1 to 20 Then I'm going to plot a graph that will show us How do these different frameworks behave on the increase in loads? So on the increasing of the concurrent requests And now it's ready So here you see Tornado was losing at the beginning, but it is quite stable at least We see the total opposite question with not gs because somehow with five requests it just blocks and jumps up And in case of scala and finagle, it's perfect. Okay, I have nothing to add here. Bravo to scala tornado also did a good job and I have a feeling that I might just have not a perfect configuration for node because we obviously see this pattern of jumping on every five Parallel requests added So I think that's yeah, we can Stop at this point Let me then jump back to my slides This is the end I will be open for my questions in a second Just before I finish this I want to remind you once again That if you want to do this Cool stuff have time for experiments and so on you should maybe go to scooby.de or scooby.es and check for the jobs Then I will be happy to Pass my presentation over to somebody else continue with this research Have some different topic. So guys Check it again scooby de slash jobs and thank you very much for your attention Now questions I was interested in Since async.io and tornado have very similar jobs, what is the relevance of doing tornado on top of async.io instead of just using async.io Okay, good question. So why do we need tornado on top of async.io at all? So async.io does have a higher level libraries too. It's not only the iolup and futures It also has tools developed specifically for async.io already that is I think a http client. I think At least some database drivers that are ready but Tornado is a web framer So you can imagine tornado as a subset of jango. It gives you templates It gives you base class for request handlers gives you basic security stuff like csref protection and so on Cookie handling and stuff and async.io. I can go back to that slides Where I had a stack here. Thank you So tornado is just a higher level. It's application level framework that lets you build the web applications faster Can't be synchronous can be asynchronous Asynchronous is just one of the options that you would use tornado Most likely you will do it because you can but you don't have to and async.io is a common middleware so to say between Higher level frameworks like tornado or twisted and lower level operating system selectors Like kiki or ipol or select on windows So this is kind of a lower level set of tools and yes, you can use it To directly without tornado, but then You will need to build the rest of the web stack yourself and this use something that you would normally want to avoid Yeah, exactly you can also call the low level stuff direct Okay next question I have a question Yeah, I'm not really familiar With this framework. So my question is Is there a built-in orm? If answer is no, is there is orm of choice like preferred one or the one that you use and then Third and the actual question is that how does async stuff works with orms? Okay, very good So first, do we have the orm and tornado? No Follow-up question. What would be the orm of the preference? Well, guys, I was working with Django for quite some time and I loved the simplicity But I've also seen attempts to put SQL alchemy on top of the Django that worked pretty fine So I think that SQL alchemy is kind of the standard orm for python in general And this is also our choice to be to use that in tornado too There is nothing special into connecting tornado to SQL alchemy It's just two totally different things that are easy to use together But you don't have to use them together In fact in some websites, you don't need database at all So actually not having overhead of orm is even better sometimes But if you like to have a database, you like to build the traditional Websites the three tire setup. I would recommend you to use alchemy. It's pretty flexible and nice But generally you're not limited by that. You can use any orm that you like There was I think the second another part. How does it work with a sync thing? Yes, so I also answer that You are very right in asking this question because having a synchronous Execution on the web server level does not guarantee us success. Obviously if something that we are Waiting for like a database or a cache does not support the synchronous request We can fix this problem by having a thread pool And use a thread pool executor, which is the synchronous in python But this is not cool. Obviously because we end up with a thread pool and that is what we wanted to avoid at the first place but now with Asyncio we are having more and more tools that get support for instance for the database You most likely like postgres because everyone likes postgres. We have psychopg2 drivers that support the synchronous mode for edis We also have the Asynchronous driver for mongo. We have by mongo. I think so most of the tools are already covered But not all of them. So you need to to really look but now since we have this common interface I think are your thingy they will grow. That's for sure All right. We are unfortunate out of time, but let's give a hand to Anton for his presentation Thank you for listening. Thank you for coming write the synchronous code