 Hi everyone, welcome back. It's been quite some time since the last stream and it's because I've been on vacation which has been really nice. It was an odd vacation but that's a story for a different time, Q&A or something. So this is going to be another crust of rust, so it's going to be one of the shorter streams and in particular what I wanted to tackle this time around was async await. Async await is something that has, I've been asked about it a lot. I have a stream on async await. I'll link it up here in the final video but that goes a lot more into the technical detail of how does the machinery work and how do the traits work and the bits underneath, but it doesn't really talk about just like how should you think about async await? How should you use async await? What does it look like for you as an end user or as a developer rather? And so I wanted to take a crust of rust, sort of a step back in complexity and just look at the mechanism, but from like a usability standpoint and just talk about more sort of the intuition and the mental model you should have, more so than the nitty gritty details of how it actually functions. For that you can see the other video. In case you were unaware, I'm John, I have a Twitter account if you want to know about upcoming videos, you can go follow me there. I'm also writing a book and async is actually one of the reasons why I'm writing this book, which is the original rust book, the rust program language, which is fantastic, doesn't cover async. Like there is no chapter on async and I think they are going to add one, but one of the challenges has been the async ecosystem has been so much in flux, it's been hard to actually write one. I do cover async in my book. There I cover it more in terms of like the lower level details because this book really is for those who want to take it, take sort of a step beyond the basics and understand how stuff actually works in rust at sort of a deeper level. But go check that out if you're interested in rust, which you probably are given that you're watching this. Also read the rust book, if you haven't read the rust book, it's fantastic. Now there have been some efforts at sort of explaining async programming in Rust. There is the Rust async book, which still has a lot of to-dos. It was written a little while ago, it explains things in at least to me a little bit of an odd order. I think there are some efforts underway to sort of rejig it to make it easier to follow and sort of go through a more maybe pragmatic approach of how should you think about async await before diving into the details of how it works. It might be that by the time you watch this video like much later, this book is the place to go, but I thought for now a video might be a good way to sort of fill out some of the things that it doesn't really talk about. I also highly recommend you take a look at MiniRedis. MiniRedis is a little project put out by the Tokyo project, which is a re-implementation of a Redis client and server. It's incomplete. It's sort of a toy example, but it tries really hard to show you how to structure an asynchronous application, client side and server side. It's really well documented, sort of written to be a piece of code that you can read to understand what's going on and to understand some of the design principles and thinking that goes into it. I highly recommend you give that a read if you get a chance. Now, I will also say that here, let me pull this up for a sec. I will say also that while there are a lot of different executors out there for asynchronous code, I will generally be focusing on Tokyo just because it's the most widely used one. It's the most actively maintained one. Now, most of what I'll be talking about will be independent of the executor. I'm not going to go into Tokyo specifics here because it's not really relevant to how you think about async code, but I just want to sort of lead with that as sort of a flag that I'll generally be having that in mind. Let's see. Oh no, it's got the wrong video title. That's all right. We'll survive. Actually, I can fix that right now. Let me fix that right now. Give me one second. That's annoying, but I can fix it. Update titles, update now. This should not say that. Let's see. In theory, now the title should be something semi-reasonable. Okay, so let's start out with something that is code. I like code. We like code. We're going to call this patience. Patience. And we're making a, let's make it a bin. Patience, right? Because you need to await it, get it. It's pretty clever, pretty clever. So let's go in here and look at your main function. All right. So we'll leave the main function as it is. Let's say that I write async event foo. All right. So the sort of simplest async function you can write. All right. Let's allow dead code. That's fine. So we have an async event foo. What does that mean? Right? What is this async keyword? What does it do? Well, first and foremost, the async decorator on functions just really means this. Output equals async this. These are the same. Now, I guess use standard, future, future. So if I have an async fn foo one, let's make this foo two, these two are equivalent. They do the same thing. In fact, this basically gets turned into this. So the async keyword on a function is not special. It's just a sort of transformation directive to the compiler. There are some differences, but we'll get to those in a second. Now, notice that this output variable for the future is the same as the return type of the function. So if this was a u size and it returned, say, zero, then this would be output u size and async zero. And what this syntax means, so we're returning some type, we're not going to name it, but some type that implements the future trait. And the future trait means it sort of signifies a value that's not ready yet, but it will eventually be a u size. So in this case, for the async block, we just return zero immediately. But this thing that we give back is sort of, from the JavaScript terminology, a promise that I will eventually give you a u size. I'm not going to tell you when. It's just at some point in the future, this will resolve into a u size. And in fact, if I hear, say, let x is foo one, what happens now? I don't actually care about this. Let's go with unused variables too. So if I now create a variable x that is the result of calling foo one or foo two, they're equivalent, right? Then x now is not a u size, right? This would not compile. And in fact, the compiler is pretty helpful here, right? It tells me consider awaiting on the future. And what it's trying to tell you is that foo one is not a u size. What foo one returns is a thing that will turn into a u size eventually. And it's telling me I need to await. And what await means is don't run the following lists of instructions until this actually resolved into its output type. That's all await really means. What might surprise you, though, is if I hear, let's say, at a print line foo, right, which would be equivalent to doing this print line foo. These are the same. Then this will never actually print foo. And I can show you that patience cargo run. So it prints a little world, but it does not print foo. And this might seem counterintuitive, right? Like we called foo one, the first thing in foo one is print foo. So why doesn't it print foo, right? Like this is a future, sure, but this code is right here. There's no await in here. And the reason for this is because a future does nothing until it is awaited. And in particular here, the future just describes a series of steps that will be executed at some point in the future. And we'll get back to exactly how those get executed. So here, the first time this future is awaited, then it will print line foo, and then it will immediately return zero and resolve. Now, there might be other cases here. So for example, imagine that in here we called foo one dot await, right? And then we print line foo one foo two, right? So here you can imagine that if we here do foo two, we await the future that's returned, it's going to start running because we're awaiting it. Then it's going to print one, then it's going to call foo one, which is going to produce some other future. And then it's going to wait foo one. And it could be in this case foo one will return immediately because it resolves to zero. It doesn't do any network operations or anything that might have to wait. But imagine that it did. Imagine that this was like read to string of some file, right? Then that reading of that file might actually take a while. It might not be ready to yield the bytes or whatever that it read or the string that it read immediately. As what will happen is that the program will sort of wait here because we told it to, right? We told it to await until the result of this future is ready. And so it will print foo one. And then it won't print anything for a while because it won't print until this has resolved. If on the other hand we did this, right, then it would print foo two straight away, because this would just return a future, but we're not telling it to await anything. So no actual work happens. And therefore the way that you can think about async blocks is really that they execute in sort of chunks. So let's say let's just sort of copy paste this a bunch and say let's do something like this. And this is going to be file one, file two, file three, file four, right? You can sort of think of this as there's sort of a first time. It executes until here, wait here. And then second time. And when I say time here, what I mean is when this has completed. So it'll start executing here. It'll run all the way until here. And then it won't do anything. It'll sort of yield back. You can think of it as sort of doing a standard thread yield now, sort of in a loop. So it'll do something like, in fact, let's try to write out an example of what this does. So Fute is going to be this while not Fute is ready. Yield now. And then Fute.try complete. You can sort of read it like this. That it's going to check whether that future is done. And if it's done, then I guess here like result is Fute.take result or something like it. So while it's not ready, it's just going to sort of spin and yield. In practice is not what happens, but you can have that as sort of the mental model that while it's not ready, it lets other things run. And then every now and again, it's going to check whether the future has made progress towards completing. And making progress for a future sort of means getting to the next await point. So when a future does get to run, it'll continue executing from the last point it yielded until the next point where it might fail to make progress. So let's say that let me return this to what it was, which was this. So let's say that this future has sort of gotten to run a few times. And right now it's sort of waiting on this. That's how far it got last time. And now let's say that the read of file three finishes. So now that the read of file three finishes, this await resolves. So you can think of like we exit the while loop that's yielding to other people. And then we get to execute. And let's say here that we had like a expensive function, right? In fact, let X pass X to expensive function. So because there's no awaiting here, once this await resolves, we get to run and we get to run sort of imagining that there's no async. We just execute instructions which include potentially calling all these expensive functions, we might call, we might do all sorts of things and we're not going to get interrupted. Other things aren't going to run in our place. Although, at this point we're in the standard threading model, right? Of our thread might get interrupted and some other thread might get to run. But apart from that, there's no, there's no magic related to async at this point. We're just going to keep executing as if we were a normal function all the way until we create another future and then await that future. At that point, we're going to sort of yield again and wait on that to complete. So once this awaited, we got to execute this chunk of code which contained no awaits and we execute all the way until the next await point. All right. Does this basic sort of async lets you chunk computation? Does the rough idea or this rough structure here make sense before we continue? There are definitely differences between JavaScript promises and this. I'm not saying they're the same. I'm more saying that in terms of the naming, it's a useful comparison that a future can also be thought of as a promise, not in terms of the actual JavaScript type promise, but in terms of the word promise. All right. So you can think of an async function or any async block in fact, anything that is a future as executing in chunks. It runs until it has to wait, basically until it cannot make progress anymore. And then it yields, which is effectively what this await turns into. So I gave you one desugaring of a wait. Another desugaring of a wait is something like let's say we write let X is read to string file. You can sort of think of that as being rejiggered into creating the future while it's not ready. We're going to do a, or maybe even while trying to find a nice rustic way to sort of say that this works. So we're going to loop and if we're going to say if let some result is few dot try check completed, then break with result. Otherwise, we're going to few dot try make progress. And then yield where yield you can sort of think of as a, as a thread yield now, but, but it has this additional property that you actually end up returning all the way up to where this future was first awaited. Like think of it as the sort of a stack of things that are awaiting each other, right? You have, right? So here main waits for foo to foo to waits for read to string. Let's say read to string waits on foo one. So you can sort of think of this like a call chain of things that are awaiting each other. And whenever you yield, you actually return all the way to the top of that call stack and sort of return to there. But the next time something calls like try check completed or try to make progress, it continues from the yield point of sort of the bottom most thing that previously called yield. So this is sort of the way to think about a weight that, that it, it is a loop that's yielding whenever it can't make progress. Is that roughly makes sense? So you might wonder, okay, why do we do this? Like, why is this interesting? So the reason this is interesting is because imagine that you have many of these, right? You have a bunch of futures. So you might, for example, have one future for reading from the terminal, right? Like you're waiting on the user to write something into the terminal. And you're also waiting on, let's say, a new connection coming in over the network. So you have two futures. And you don't really control when either of them will resolve, right? You don't control when the user types something into the terminal. And you don't control when some external program is going to connect. So you sort of want to wait on both of them. And you don't really care which one happens first. Well, if you were in a threading system, what you would have to do is you would sort of do, let's say, how would we do this? So we're going to have a read from terminal. And we would like spawn a thread, which would like lock IO standard in lock. And then we would like four line in x dot lines. And then we would do something on user input. And then we would separately have a thread that's like read from network, which would do thread spawn. And it would say mute x is standard net TCP listener bind. And it would do, you know, bind on some port. And then it will while let stream is x dot accept. This has to unwrap. And then do something on stream. And now this isn't too bad. Why does this require an argument? Yeah, yeah, I don't actually care about the compiler errors here. So this isn't too bad, right? We have one thread for handling terminal reads and one for accepting connections. But it gets worse if we have to have one thread for every operation that we have to do, if not even thinking about performance, but it just gets annoying to wrap your head around that now, okay, let's say that for each stream, we need to call like handle connection, right? So we have want to handle connection of the stream. But now we have a single thread that's managing all of our connections. And imagine like a bunch of users are connecting to download some large files or something, you'd really like to be able to use many threads, or at least if like, if one connection is full, you want to be able to write on the other. And that gets really weird if there's one thread, because at some point, like handle connection has to call like stream dot write. And imagine one of the clients is really slow. So you're not going to be sort of stuck writing to the slowest client, even though all these fast clients you're able to write to. So we maybe then need to like, spawn a thread for each stream, right, they're going to handle the connection for that stream. And now we get like a thread handle back, we need to do something with that so that we remember to wait on the thread if we're trying to shut down the server. Now we have all of these threads running. And like, it's possible to write programs this way, and many big programs are written this way. But but it feels weird somehow, right? What I'm going to propose to you with the async model is you can actually make this simpler. The async model lets you model this in a way that matches your thinking better. So let's say that there's a read from network, not, not these, not these variables I declared up here, but some function, right? So I'm going to call that network. And then there's going to be a read from terminal, right? And let's say these are both futures, then now what I can do is I can say, I can do select, and we'll talk about what select means. And then I'm going to say stream is, and there are lots of different syntaxes for here, I'm just going to sort of make one up network.await and line is terminal.await. So this, the reason why do something with line here, the select macro, which exists in a bunch of different libraries, like there's one in Tokyo, there's one in the futures library. What it does is it waits on multiple futures, and it tells you whichever one finished first. So under the hood, what really happens here is that it try it, I'm not going to define what it is yet, but the code, the select macro, if you will, is going to try sort of remember how we wrote the de sugaring down here of try check completed and try to make progress, right? It's going to try to make progress on network. And if it does make progress on network, then it's going to give you a stream and call the code that's in here. If it doesn't make progress on network, then it's going to try to make progress on the terminal. And if it makes progress on the terminal, like if the terminal now has new lines, then it's going to run this code with a line that it got. If neither of them make progress, then it yields. And then at some point in the future, it's going to retry, right, and try to check progress again. And in practice, it's actually a lot smarter than this. What really happens under the hood here is that when you yield, you don't just say yield, you say yield until this thing happens. So for a network socket, for example, what you yield is yield until something happens on this network socket on this file descriptor. On the terminal, you say yield until something happens on this input channel, or in this case, standard in, right? And behind the scenes, whatever is running the future and sort of has this loop of try check completed and try make progress and such, what it's actually going to do is use some operating system primitive under the hood to be smarter about when it retries. So it's not actually doing like a spinny loop or something. It just goes, I'm going to try this, I'm going to try this. If neither of them succeed, I'm going to sort of go to sleep and then retry once the operating system has notified me that it's worth retrying. And it can be even smarter than this, right? It can even realize that, oh, nothing has changed about this network socket. So I don't even have to try to make progress on this. But the terminal state did change. So I'm going to retry this operation. Okay. So that's one of the reasons why futures is handy is because you have this, you have this nice way of trying multiple things at once because a future is not code that executes all at once. It is this chunked operation that also sort of describes how to try again later, which makes it really nice for doing any kind of programming where you have primarily IO, whether that's networking or disk or even timers, like anywhere where it's not just like spinning on computation. Their async doesn't really add that much. But if you ever have reason for some part of your code to wait for something else to happen, then what async allows you to do is sort of give up the thread's time. Like if you have a thing that needs to read from, read from disk, while it's waiting on the disk, async allows something else to run instead, right? And you can see that here. While the network does not have anything for us to do, we can read from the terminal instead. If neither of them have anything to do, then we can sort of go to sleep. We don't have to do anything. And that's what async sort of allows you to express very nicely. The other way to think about async await is sort of as a big state machine, right? Of you're in the current state of the future, and there are multiple paths forward. One path forward is that something becomes available on the network socket. Another path forward is something becomes available on the terminal socket. And then you sort of follow the appropriate edge in the state machine, and then you run the code in the next one. Now in the case here, right where we don't really have a branching, so select, you can think of the select operation as allowing you to branch the execution. Here, there's no real branching, right? Here, there's just, we're just saying if we can't do anything here, we just await. And if we just yield, right? And then we're going to continue running from here whenever it's possible for us to make progress. So it's really the state machine in this case is we're either in the state of being here before let's make before reading file one, or we're in the state of before reading file two, or when the state of reading before file three, and the edges between these are file one has been read, file two has been read, file three has been read. And these compose really nicely, right? So here, let me see if I can explain. Let's add another thing here. Let me tidy this up a little because there's a lot of, there's a lot of annoying errors in here. And then I'm going to do async then read to string, like so, just to make it stop yelling at me, expensive function takes nothing. Great. It's yelling at me less now. So let me get rid of these as well. And say, these are fine. I can leave those in. So let's say that we also here said foo, which is foo two.await. So now, when foo two, when we call foo two, right, we get something that doesn't execute straight away. It doesn't execute until we awaited. And now imagine that the network has nothing to do, the terminal has nothing to do. So foo two gets awaited. Okay, so it gets awaited, which means that it's going to run until it can't run anymore. So it runs foo one, then it runs read to string one of file one, which is a disk operations that has to wait. So it yields. So at this point, there's another opportunity for the select macro to run either of these other two branches. If there's still nothing on the network and still nothing on the terminal, then it might be that the read of file one is completed, in which case it's going to continue from the previous yield point of foo two, which is this await. So it runs this and then it reads file two and then it yields again. And when it yields, control flows back up the stack, in this case, to the select, and the select gets to retry these again. So this await sort of being a return is what enables this whole mechanism to work, is that every await is an opportunity for whoever is above you to choose to do something else. And it could be that they don't decide to do anything else and they let you run instead. It could be that they let some other things happen and then continue to let you run. So here, it could be that, well, this is maybe a bad example, but let's do foo is foo two. So we're going to store the variable out here. foo.await. Let me make this a loop. So imagine now that foo gets to run for a bit, it reads file one, it reads file two, it's reading file three, but it yields. And at that point, the network has a new stream. So we execute this. So whatever happens in here happens, gets run, and we don't even check on foo again. foo doesn't get to do anything more because it's not being pulled. It doesn't get to run in the background. It's not its own thread. It's cooperatively scheduled. It relies on the sort of parent of it, which is the select, to actually continue awaiting it, right? So foo doesn't get to run on its own. It's whoever has the foo is responsible for actually awaiting it again. So in this case, if a new stream comes back and we eventually finish running this block of code, at that point, we exit the select, we go around the loop again, and then the select happens again. Are there any new streams now? Let's say no, there are no lines from the terminal. So now we continue awaiting foo. So now maybe file three finished reading and foo awaits again, but this time I'm reading file four. So it gets to make progress, right? And then maybe there's a line from the terminal, we go around the loop again, and so on and so on. So the way to think about this is really this idea of cooperative scheduling where if I don't run, I'm going to let whoever is above me decide who runs next. And it might not be me, right? And so this brings us into the topic of cancellation. If you don't want a future to keep doing what it's doing, you can just drop it, right? The trick is if you don't await it, it doesn't get to do anything. Awaiting it is what drives its progress. Now, you don't really have a way in sort of straight line code like this to cancel, right? Because once you call dot await, you don't have any control of execution anymore. What you've told Rust is I don't want to keep running until you have the promised value from this future. And when that happens, run the next line. So you don't really have a way to cancel here, but if you wanted cancellation of this operation, what you would do is something like select, let's say done is this or cancel is going to be, let's say, what's a good example of this? Let's say there's a cancellation here. There's a cancellation channel, which is like a Tokyo sync MPSC receiver. So cancel is going to be canceled.await. So here, we're just going to sort of continue or fall through to print line below. If we get a, if, so you can, another name that was proposed for select by without boats, I think, is race. And that is a good way to think about this. We're sort of racing these two against each other and whichever completes first, whichever await can produce its result first, gets to run. And the other one does not. It just sort of gets cut short at this point, unless you sort of, unless you're looping and trying more. So in this case, if we get await, then we're going to return or something. In fact, you could sort of imagine that this is like a result instead, right? But in this case, let's just say that in that case, we just return zero. And we don't execute any of the remainder. So this would be the way that you do cancellation is you describe the circumstances under which you should cancel the operation. Does that make sense? That you get to basically async isn't magical. It just describes the mechanisms for changing or for cooperatively scheduling a bunch of computation by describing when under what circumstances code can make progress and under what circumstances code can yield. So this might then make you wonder, well, is it just turtles all the way down? Like at some point, you need to get to a point where like, like here, right, where I say, I said, read from network and that I'm going to await that. Well, ultimately, that does like a system call. It does a read system call from the operating system saying, give me these bytes. And the operating system goes, I don't have the bytes. What do you do? Well, this is where the notion of an executor comes in. And we're not going to talk in huge depth about executors because there's a lot of mechanisms going on there. The basic premise of an executor is that you're only allowed to await in async functions and async blocks. So this wouldn't actually compile. It would only compile if this was an async fn, but then you see the compiler complaints main function is not allowed to be async because ultimately at some point in your program, you're going to have this giant future that holds the entire execution of your program, but it's a future. So nothing runs yet, right? So you have this sort of top level future that describes the entire control flow of your application, something has to like run that in a loop, right? Something has to have the loop that we expanded earlier that's like, try to see if it completes. And if it doesn't complete, then what? It can't yield because there's nothing above main, right? Like something if it can't make progress, what does it do because it can't yield? You can imagine it just spins in a loop, but in practice, that's not really something we want to do. And so an executor, a sort of primitive executor is one that just pulls futures in a loop and does nothing else. It just keeps retrying aggressively. Instead in practice, what happens is that the, the executor crate. So Tokyo would be an example of an executor crate provides both the lowest level resources like network sockets and timers, and also the executor loop at the top. And the two are sort of wired up together behind the scenes. So imagine that you're doing like a read from the network and you call dot await. What's going to actually happen is Tokyo's TCP stream is going to do a read from the network, realize that I can't read yet. And then it's going to sort of store its file descriptors, store the socket ID, if you will, into the executor's sort of internal state and say, the next time you go to sleep, watch for this file descriptor changing its state, right? Like tell the operating system that it should wake you up if anything changes about this file descriptor. And then at some point when, when we've yielded all the way back up to main backup to the main executor, the main executor, instead of just like spinning in a loop, what it's going to do is it's going to take all of those resources that, that it knows it needs to wait for. And it's going to send them to be the operating system and say, wake me up, like put me to sleep, but wake me up if any of these change, if the state of any of these change, wake me up because I need, then I have more work to do. And this is sort of the EPOL loop for those of you familiar with Linux system calls, but doesn't have to be EPOL, right? Like on Windows, it's using, the name escapes me. I forget. On macOS, it's using KQs. But, but basically, there's sort of a different implementation of the outer executor loop on different operating systems to best make use of the, the underlying mechanisms. So in practice, what you do, so with Tokyo, for example, you, and you do something like this, and that allows you to write async fn main. But what that really is is just a procedural macro that rewires your code a little bit, turns it into fn main, and then does the following. It does like Tokyo, let runtime is Tokyo runtime runtime new, and then runtime dot block on, and then async of, and then the remainder of the function. That's the transformation that it makes. So your main is not async, right? Even though you wrote async fn main, the procedural macro turns it back into a non async fn main, because like when linux executes your binary, it needs to just have a function to call a regular function. It doesn't know anything about rust async. And then what the thing it doesn't mean is create a runtime, which is this executor, this, this sort of big loop that tries to pull the future that's passed a block on. And then if it can't make progress on that, go to the operating systems to say, wake me up, if any of these things change, and then loops that until the future resolves until the future has no more tasks to run. So the example here would be, this is an infinite loop, right? Nothing, this, this async will never finish. It never resolves into its final value. But you can imagine that the moment we get something from terminal, for example, we're going to break, which means that the moment someone writes something on the terminal, this loop exits, we're going to drop the network future and do no more work on it. We're going to drop the foo future and do no more work on it. And at that point, we're at the end of the async block. So this async block resolves into just unit, the empty tuple, that point block on finishes, because it, the future has resolved, the future was given has resolved. And at that point, we're at this, this end of execution, and then main exits. So that's sort of the, the higher level flow of how these futures get executed in the first place. All right. Again, let's, let's pause for a second and see whether all of these little bits and pieces that I've explained roughly make sense together. So a lot more to talk about here instead of how do you use this in practice? What are some of the pitfalls? But let's see that, that everyone's following along. And please do ask questions. Like this is pretty hairy stuff. It's like weird and convoluted and there are lots of moving pieces. So if you have a question, chances are other people have questions. So please ask them. I'm pretty new to this. I only know about how the JavaScript event loop works. Where would event loop fit into this? Like a comparison, you can sort of think of the executor loop, the outer executor loop as being the event loop, right? Its job is to keep running the, the sort of available futures until there are no more futures to run. And at that point it's done and the program is finished. Now the difference between the event loop and JavaScript in here is that you can choose your own event loop. You don't have to use Tokyo. And in fact, Tokyo has multiple run times or multiple variations of run times that you can choose to use. And arguably, you can also write your own event loop. This, this loop select right here is an event loop. It's sort of your own event loop within the context of the larger event loop. You talked about futures yielding. How does this yielding look like in code? In general, you don't have to worry about the yielding yourself. Whenever you do dot await implicitly, think of dot await as being sugar for doing this yielding. You can't actually actually use the yielding machinery itself directly in your own code. If you try to implement a future yourself like manually by implementing the trait, you don't have access to the yield keyword. Instead, you have to basically manually implement the state machine, which is what the futures ecosystem and Rust used to be before I think await landed. And trust me, this is a lot nicer. In general, though, these days, it's very rare that you need to write your own future implementation. Best way to pass data to Tokyo spawn. We're going to talk about Tokyo spawn. Why is Tokyo main in macro and not a simple function? I'm not sure I follow. The idea here is that this is very easy to write. It's like a very simple setup instruction. You could imagine Tokyo having a Tokyo colon colon main that you passed in an async block too, but it reads a little more weirdly. I think there might actually be a Tokyo main like this too. There's no reason it couldn't be one or the other. So does Tokyo re-implement KQ or LibUV? It does not re-implement KQ. Tokyo uses Myo, and Myo is a crate that abstracts over KQ and E-Poll and whatever the Windows thing is, which still escapes me, but it basically gives you access to sort of something like LibUV, basically an operating system event loop, or not even event loop, but just event register. You can say, I want to go to sleep until any of these events occur, and then you just go to sleep, and then the operating system through whatever mechanism Myo chose will wake you up when one of those events happened, and you can make progress. Now, you might wonder, well, what if, so down here, we waited on cancel, which is a channel receiver. So if you await on a channel receive, there's no file descriptor you can give to the operating system. The operating system doesn't know anything about something like a receiver. In those cases, there's a little bit more going on under the hood here. In practice, you should just trust that the executor knows how to wake its own futures up. So in this case, if you're using the Tokyo runtime and you're using a Tokyo channel, it knows how to work that out. And in fact, the pieces that are being used here are fairly runtime independent. So for example, you can use, so there's a crate called futures, which has also a lot of just utility mechanisms for futures. And I think one of the things they have is a channel as well. And you can use the futures channel with Tokyo, because they use the same underlying mechanisms that the Rust language provides. So the executor does have to include mechanisms for dealing with non operating system based events. What happens if network.await runs, but terminal await and foo await don't? Then on the next loop of a select, does network.await get run again? Yes. So the way a select works is it selects among all the given options. It doesn't remember anything about it having been run in the past. In practice, the way this actually works, if you want to sort of redo things, is you would do this. And the borrower checker will complain about this. If I had actually imported the select macro, it would have told me that I have to do this. Because otherwise, the first time through the loop, ownership of network is sort of transferred to await. And so the next time around the loop, the borrower checker would be like, well, network has been moved and you can't use it again. So in practice, you would do something like this, which allows it to be reused across multiple iterations of the loop. But yes, it will consider all of the cases. It doesn't have memory about past attempts. A lot of what I'm saying has a little bit of hand waving. We're not going into the real mechanism details. And you should consult the documentation if you rely on any particular corner case behavior here. But the general idea is that select selects over all branches every time. Okay, so here's another good question, which I was going to get to, but a question came up, which is what happens if an abandoned select arm has side effects? So here, let's say, okay, let me see if I can come up with a good example here. Let's say the one of the operations that we want to do is a file copy. So we're going to have F1, which is going to be like a Tokyo FS file open foo. And F2, which is going to be file create bar. And then we're going to do a copy is Tokyo IO copy mute F1 to mute F2. And then one of the streams here is going to be a copy.await. Let's just remove the loop for a second. So first of all, you'll notice that I'm using Tokyo FS file instead of standard FS file. That's because if you do operations on a standard FS file, there's no await, there's no async functions on it, because the standard library doesn't define async functions in general, because they rely on this integration with the executor that you need in order to get the cooperative scheduling and the sort of smart wake up that we talked about. So you do actually need to use the asynchronous version of IO resources. And then here what I'm going to do is create a future that's going to, it basically takes a thing that implements read and a thing that implements write or more precisely a thing that implements async read and a thing that implements async write. And it's going to read from one and write to the other. Great. So that's one of my select arms. Now, as I mentioned, the way that select works is that it tries all the branches until one succeeds. So imagine that it tries this, nothing happened, it tries this, nothing happened, it tries this, nothing happened. It tries this and it writes like a megabyte and then it has to wait on the disk, like the disk is saturated and so it can't make progress, but it hasn't finished copying the file yet. And then we go back to the stream and let's say that the stream completes. So now we exit the select and then down here, we're now in a state where some bytes have been copied from food to bar, but not all. You can imagine that we copy.await here to sort of complete the copy, but it's very easy to forget to do that. And there isn't really a good solution to this problem. This is something that you need to identify has happened is that whenever you use something like select, you're now in a world where you might have partially completed a future, right? It's hit some yield point in the middle of its execution, but that means that the sort of trailing end of the future hasn't gotten to run yet. And at that point, if you drop the future, it doesn't get to finish. It doesn't get to do the rest of its work. It just gets terminated at that point. It gets canceled. And so now you need to reason about the fact that your, your program might be an intermediate state. This is a particular, I don't want to call it a problem, but this is a, this only really affects selects. Because if you think about something like here, that can't really happen, right? This either gets completed. Well, then it sort of has to get completed. There's no way for you to cancel the operation midway through. In fact, the only way that this operation or indeed any operation that you call.await on gets, does not get completed, gets interrupted is if this is a select somewhere up the chain. So in general, when you write selects, you have to be careful about what happens if one branch runs and then another branch completes. So the first branch didn't run to completion, but the second branch did. Where does that leave you? So this is a, an error case that you need to be concerned about when you're using select. As you presented the executors making the assumption that the futures are not greedy, how do you avoid a future not taking all the compute time and grind the all async to a halt? This is also a great observation and I'm, I'm very glad you made it because it means that people are understanding sort of the mental model here, which is, this is cooperatively scheduled, which means other things get to run as long as the thing that's running occasionally yields. If you have a future that just is a busy spinning loop or uses say standard IO file and then calls like read on a giant thing or on a network socket that just never gives any bites, that thread is blocked. If it used the, the async tcp stream, then the read would yield if it couldn't complete. But if you use a standard IO tcp stream, which doesn't know anything about async and you do a read, its implementation is to block the thread, do nothing more. So certainly not yield just block until the read completes, which might be never, there might never be a bite coming from that particular tcp channel ever again. And, and then you're sort of in the doghouse. Like this is a really bad situation because now none of your other futures get to make any progress. They're never pulled again because the other one doesn't get to run because nothing, the thing that is running, the thing that's holding up the executor doesn't yield. And so it is very important that, that you use this, that you lean into this cooperatively scheduled world. And this is also why you have to be very careful about using blocking operations that is operations that aren't aware of async yielding or very compute intensive operations when you're an async context. There are some mechanisms that exist for, for improving the situation. So for example, in Tokyo, in Tokyo, this is function called spawn blocking. This was the one called block in place. And these are, if you're in an asynchronous context, and you need to run something that you know might block for a long time, whether that's because of computer, because it's doing a syscall or something, you can use these methods to sort of tell the executor, Hey, I'm about to block, make sure that other tasks get to run. The way they do that, we won't get entirely into today. It's a little bit too technical. But I would, I would recommend that you read the documentation for these methods to understand the differences between them and what the trade offs are. But there are ways to sort of signal that, that you're about to block and that the the executor needs to do sort of take the appropriate steps to make that be less of a problem. I'm intentionally not mentioning future poll. Does that mean that select with a huge number of cases will potentially be slowed down by trying all options out every time? You might think so. And the answer is it depends on the implementation of select. In general, if you have a select with like a million cases, that seems like a problem. But given that select actually forces you to write them out, that seems unlikely in the first place. Now, it is possible for you to have such a large select. For example, you could imagine you have a code generation pipeline or something that generates it. And in that case, I think most select implementations are optimized for few cases rather than many cases. But it is possible for select to be smart, smart in the sense of only pull the ones that might be able to make progress. I'm not going to go into exactly how they do that. But basically, there's a way for select to integrate with the Rust machinery for dealing with futures and wakeups to sort of when a future becomes runnable through whatever mechanism like a file descriptor was ready or a send happened on an in-memory channel. Now, there's a way for the select macro to sort of inject itself into that notification that this future is ready and sort of get a signal to update its own state when that happens. So you can imagine that the select keeps almost like a bit mask across all of its branches. And when that notification comes in, it flips the bit for the appropriate branch. And then the next time the select comes in or the next time the select is awaited, it'll only check the ones where the bit is set. In practice, I think in general, selects don't do this because it's a bunch of machinery and most selects have few branches. You could imagine having a, writing a select macro that only did this trick if you got many branches. I don't think any of them do that. Now, this is a good time. Actually, I'm going to mention join in a second, but let me see if there are more questions about selects. Is the select macro fair as in can it happen that only one branch will run forever? So it depends on the implementation. For example, if you look at the futures crate, so the futures crate has a select and a select biased. And the select one is if multiple futures are ready, one will be pseudo randomly selected a runtime. So that one sort of tries to be fair in practice. It might not be entirely fair. Select biased is a variant of select that always runs if multiple already it runs the first one. So that would not be fair. So it depends on which select you use. When you use fuse with select, I'm not really going to talk about fuse. So fuse, the point of fuse is, let me talk a little bit about fuse because you might actually run into something that requires it. Fuse is a way to say that it's safe to pull a future that's safe to await a future, even if that future has already completed in the past. Right. So let's take the case where I have a loop over this. Right. So the way we had it. And we go through the select and let's say both network and terminal are both ready. So the select sort of checks on both of them. And both of them say I'm ready, here's the value. But then or yeah, let's say they both say I'm ready, here's the value. What does the select do? Well, realistically, it's not going to ask both of them. It's just going to ask one and then the next one. Right. So it checks with network network says I'm done, here's the value. And it goes great. Thank you. I'm going to run this branch. We go through the loop again. And now the select still includes this branch, even though this future has completed. But the select doesn't remember that it's completed and that it doesn't need to check this branch again, because the select just knows about being called once the loop is not a part of the select. And so this future needs to be safe to pull again, even though it has already yielded its value. And that's what fuse describes. Let's see. How would you continue a half completed then abandoned arm? You can await the same future after the select. It depends on how you select on it. But if you select like this, then this is awaiting a mutable borrow of this future. So after the loop, you can still await that that value. This is one of the reasons why in general you will want to select on mutable references to futures rather than the future itself. Because if you did this, that would only work if this wasn't a loop, then the network would be moved into the select into this await. And you wouldn't be able to await it later. It would just be dropped at the end of the select. So whereas if you just await a mutable reference to it, then that mutable reference sort of ends when the loop ends. And so you would get to await it again later. How bad of a performance hit is it to use async when you don't call any async stuff down the line? There isn't really any overhead to like async doesn't add any overhead for executing code, right? Like if you just have a, let's say you do like a matrix multiplication in the middle, like you just have, you have a, an fn matrix multiply, which, which currently implements a matrix multiply, you just add the async keyword. This doesn't make it any more expensive, right? It just the same code ends up running. It's just that it gets wrapped in the future type and you have to await it to get the result, but the await doesn't do anything. It doesn't, it doesn't change the generated code in any meaningful way. So there's no overhead to marking something as async. I think maybe the, the analogous question you're after is what's the overhead of doing an asynchronous IO read versus asynchronous IO read. And there is a little bit of overhead there because now you need to, you need to tie into the executor machinery. There's a little bit more work there. But in general, the additional system calls that happen get amortized across all your futures or across all of your resources. So it doesn't really add much. And usually the benefit you get from the benefit you get from not needing to spawn like thousands of threads or hundreds of thousands of threads more realistically, it usually ends up actually being faster to just have the fewer threads that are run by the executor and then have them cooperatively scheduled. And there are a couple of reasons for this. One common one is that you don't need to cross the operating system boundary as often. If you have lots of threads, like actual real operating system threads, then if one has to block on a read, then the read, like you do a system call and the operating system has to context switch to a different thread, which is not free and then runs that thread instead. With async, if a read fails, the operating system returns to the same thread and says, I can't make any progress, it yields, the executor continues running on the same thread. So there's no context switch and then just pulls a different future. And so in general, that ends up being a little bit more efficient. In practice, it's hard to say which one is like objectively faster, because it really depends on your use case. But I would say on balance, if you're doing things that are IO bound, like web servers, for example, async is probably going to lead to more efficient use of your resources. And also I think easier to read code, maybe not necessarily easier to reason about, but easier to read. Are these proto threads? Yeah, they're user space threads is one way to think about it. Let's see. What would select return? So select is a future. This returns, well, sort of, so select generates a future for you and then calls a wait on that future. In general, you can construct a select manually too. It's usually a lot more hassle, but you can construct the select future for a block yourself. It's not quite true. Select expands into basically a loop with a, with a match or a bunch of ifs. It doesn't, it doesn't really generate a future, but you could make it a future by wrapping it in like an async block and then assigning that to a variable. And then it awaits it. Like select doesn't actually expand to a type. It expands to like rust code that does the appropriate stuff to make this work. But you could make it a future if you wanted. It wouldn't really have a meaningful return value apart from a future though. Great. Now there are a couple different paths we can go from here. I want to start with talking about join. So we talked about how select is sort of branching the control flow of saying do this or do this, whichever happens first. The other operation you can do as a join, which is saying wait until all of these futures complete. An example of this is let's say that you wanted to read 10 files, right? So you're going to do like files is actually let me go down here. Let files is zero to 10, map I, no, Tokyo FS file open format. All right. So here I have an iterator of futures, right? And I want to wait for all of them to complete before I continue with my program. Imagine that I'm like concatenating them or I'm computing the hash across all of the bytes or something. So I really need to wait for all of the bytes to complete. So in that case, there is the join operation. And there are a couple of different joins. So in the futures crate, there is a join macro, which looks a lot like the select macro, except that you don't really specify the branches, you just give you just list the things you want to join on. So here, let's say I collected this into a vector. So I could say join. And usually just you assign this to like, so this is going to be a little bit janky. Let me let me make this three instead. File one, file two, file three is join of files zero, files one, files two. So this is saying, actually, this would be something like read two string. So this is going to run all those three reads in parallel. And notice the reason I need to do this, right? Let's say that I wrote let file one is is file zero dot await file to file one to the wait. And file three is two dot await. So compare this to this. So in this first instance, what's happening is I'm first reading file one from start to finish or file zero from start to finish, then I'm reading file one from start to finish, then I'm reading file two from start to finish, and then I get complete with my program. That works. It'll give me the right result. But the downside of this approach is that it's sequential. This means that rather than give the operating system all the read operations and have it like read the disk in the most efficient way or access the file system in the most efficient way or let's say these were network sockets and I wanted to read all the bytes from multiple network streams. It might be that one stream has no more bytes yet, but another stream is ready. I wanted to keep reading those like use the cycles to read those bytes while it's waiting for the first one. In this first case, it won't get to do that because I've said don't execute the next line until this file has been read. So it doesn't even get to this line. It doesn't get to start the next operation until the previous one is completed. What JOIN lets you do is say run all of these operations concurrently and then once they're all completed, give me all of the outputs from the three futures. So this one is more efficient might not be the right word, but it allows the operations to continue concurrently, which is a big benefit generally because it lets you overlap compute and IO. So that's sort of the JOIN macro and it can be a little bit annoying to use it this way because you need to explicitly enumerate all the things you're joining. Like imagine if this was not three but a hundred, you clearly don't want to like list out a hundred things here and then assign it to a tuple with a hundred elements. But this JOIN is really convenient if you just have a few things. Generally, all of these provide you with multiple ways to do things. So you see the Tokyo one is similar, but generally there's also, let me see, I forget what it's called here. It's not, I'm surprised. Well, I think there's one here. Yes, you see those like there are multiple functions like JOIN which joins two things, join three, join four, join five, etc. But then there's also try join all. So try join all takes an iterator over things that are futures. They ignore try future for a second or actually a try future is a future whose output is a result. So this is going to try to join all the things in the given iterator. And the reason why the JOIN space is a little weird is because in general you probably care about the, you care about the fact that the output result order, you can map back to the input order. So I want to know that these are the bytes for file zero. Right. So if I do the, what was the try join all, right, of files, file bytes, I want to make sure that file bytes zero is equal to files zero, right. And try join all will do that it will, it will make sure that the result sort of output is in the same order as the input, even if they completed out of order, right. It might be that the read here completed before the read here. But then try join all was sort of reorder them at the end so that the output matches what you expect that reordering is not free. There are ways to opt out of it. So there's a type called futures unordered, for example, which adds its name implied implies gives you the results out of order. If you don't care about the order, this is generally more efficient. So here you say you create a new one, you can push futures onto it. And then after you've pushed the future onto it, you can then we'll implement stream, which we're not going to talk too much about, but it implements iterator, where the outputs of the iterator are the results of the futures completing. So the idea here is that you stuff all your futures in there, and then you loop over, you basically await the futures in order multiple times. And each time you await it, you get one more thing, what one more output from one of the futures you stuck in, but you don't know which one. So this might be helpful if, for example, the result contains all the information you care about, and you don't necessarily care about the input because the output describes which input it was from. So that can be more efficient. So multiple, multiple weight is cascading. Yeah. So another way to write this would be file zero dot then files one dot then files two, then sort of pseudo syntax. Whereas this one has do all of them at once. And join us like promise all you can think about it that way. Yeah, there's probably also join all, which does the same thing, or you can manually construct one of these, those futures unordered and those futures ordered. And these, so I mentioned how select isn't necessarily smart about making sure to only check the futures that might be ready. But join is because it knows that there might be, there might be a lot of futures you're joining over, like imagine you're downloading like all the dependency as of a cargo project. Well, there might be thousands of them and you want to download them all in parallel or at least some subset of them in parallel. And then there are many branches and you want to make sure that you don't like have to check all of the futures every time you only want to check the ones that have actually made progress. And so in general, the joint operations like futures unordered will implement this little like hook into the runtime system to make sure that it only checks the ones that have had an operating system event, sort of readiness event, tap into them. Yeah, and join all and try join all, use futures un, futures ordered under the hood, not futures unordered. Great. So that's join. And join is great. Select is great. But all they do is allow things to run concurrently. That does not mean that they get to run in parallel. And this is a big important distinction that is often missed when people deal with futures. Let's see where to start with this. Okay, the Tokyo runtime actually starts up multiple threads. And each thread is able to sort of await a future. But remember that awaiting a future, that's not really what I want to say. I'm going to say it differently. Let's imagine for a second that the runtime only had one thread. So when you call block on, you give it a future. It doesn't know that that future contains a bunch of other futures inside the the the one like this function, this is like this is a method on a type in Tokyo, right? It just gets one future. And it can await that future. It doesn't know about this await. It doesn't get to look at the code inside. It doesn't know that there's a join or futures order or anything. It just gets one future. So the only thing it can do is run that future, like try to see whether that future can complete. And if it can't, then go to the operating system, go to sleep and wait for an event to happen. And then try making progress on that future in return. When it does, that future internally checks its inner future and so on down the stack. Or if it's a select, it checks all of its contained futures, but ultimately at sort of the top, there's only one entry point into execution. And that's kind of unfortunate, right? Because it means that only because there's only one future, there's no advantage to having multiple threads because there's nothing for those other threads to do. There are no other futures. There is only one. And therefore, if you had 10 threads, because say you had 10 CPU cores, there's still only one future. So only one thread can await that one future at any one time, because awaiting a future requires a mutable reference to it, an exclusive reference. And so multiple threads just can't do it. And even if they could, it wouldn't make sense. Like what would they do? There's only one piece of code. You don't want to execute the same code multiple times. And this means that even if you do something like a join, and so you're doing all of these operations concurrently, they're happening concurrently on one thread. Which means that that's probably not actually what you wanted, right? Imagine that you're writing a web server, and you have like a loop over accepting TCP connections. And for each TCP connection, you get a future for handling its connection, you stick it into like a futures unordered, and then use the wait on the futures unordered. To do like a, in fact, let's let's try to write this out. So here's what I'm going to do. Let's make up a TCP server. So we're going to have accept is going to be a Tokyo net TCP listener bind to 00008080. No, that's that I haven't actually added Tokyo. So dependency, I don't get completion or anything like it. That's fine. Stop yelling at me. Great. And then I'm going to do something like loop. I'm going to select on, I'm going to have a let mute connections is going to be a futures stream future maybe futures unordered new. And I'm going to select over this and say, either I get a new stream, in which case I'm going to call I'm going to have like an async fn handle connection, which takes TCP stream, and then does who knows what with it, it does things, right? Like there's a let's just say there's a to do in there or something, you know. So in this case, I'm going to say connections.push handle connection of stream. And down here, I'm going to say nothing is going to be connections. await. This won't compile for a number of reasons. I, and it's not really important. I just want to demonstrate that the higher level problem. The reason we need to have this branch down here is because something needs to be awaiting all of the futures. Remember, a future doesn't run unless it's awaited. So if we just have this, there's nothing is awaiting the futures unordered, which means that nothing is awaiting the futures are inside of there, which means nothing is awaiting any of the client connections, which means none of the client connections are being served. So we do need to await on connections. But we also want to sort of see if there are new connections coming in. So we need to wait on this accepting as well, which is why we need select. And you can think of this futures in order. This is basically a join, right? I want all these to execute concurrently. This of course won't compile because I have a mutable reference here and a mutable reference that here and they're being used concurrently, not okay, but I'm going to ignore that for a second. There's a different more fundamental problem I want to get at, which is this is still just one top level future, which means there's still, even if the runtime had as many threads as you have cores, only one of them gets to run at a time. It will get to multiplex across all the different connections, but imagine that there are a hundred thousand connections. That thread is going to be completely busy dealing with all those connections. It's not wasting any time, like it's not a sleep or anything. It's doing work. It's just there's more work than it's able to handle on its own, but all the other threads can't help out because there is only one future. So the way that you can help this problem and introduce parallelism, not just concurrency, is instead of having this, this connections, we're going to get rid of that. And in fact, we can get rid of this too, which means that we can get rid of this too, which means this can become while that. So let's say I have this, there is a function that's provided by basically every executor called spawn. In this case, let's say Tokyo spawn. And what spawn does is it's sort of a hook into the executor, whatever that executor might be, that you give it a future and it gives that whole future. It moves it to the executor. So it sort of is as if you gave it to the runtime. So now the runtime has two futures. It has this future, which you pass to block on, and it has this future, which means there are two separate futures, which means that two threads can run them at the same time. So with this spawn, if one thread is busy doing this work, so accepting connections, another thread can handle the future for a particular connection. Notice that this is not a thread spawn, right? The threads are spawned by the runtime and there's a fixed number of them fixed. But we're giving additional futures, sort of sticking them on the job queue for that pool of threads. This is why spawn generally requires that the future you pass and ascend is because otherwise it couldn't be sent to another thread to work on. There's like spawn local, but we won't really talk about that right here. In general, spawn also requires that the future you pass in a static, because it could be that inside, let's say that inside of here we also do a Tokyo spawn, right? Inside of this with some other thing that does, you know, whatever. It needs to be static because it doesn't know the lifetime of the runtime. And in fact, the handle connection async function might complete, right? This outer async thing might complete, but this spawned async future still needs to be running and therefore it needs to be tick static. It can't be associated with the lifetime of handle connection. Imagine that this had like a, I don't know, X is a vector. And this tried to use add X. Then if handle connection returned, but this future still tries to run, X would be dropped, but this has a reference to X, so that's not legal. So that's why spawn requires static. So this is the way that you introduce parallelism into asynchronous programs, is you need to communicate the futures that can run in parallel to the executor. Now, it could be that the executor doesn't have that many threads, right? Like with the Tokyo runtime, you can set like worker threads to like one, in which case there's only one thread. So it doesn't really matter whether these are can run in parallel because there's only one thread, so there's no parallelism. But in general, you want to use this pattern so that these futures can run not just concurrently on one thread, but in parallel on multiple threads. All right. Is that the reason why we need spawning makes sense? Okay. Yeah. So this is why it's important to remember to spawn and often why when people who aren't very familiar with async await, start writing async await, they find that their program performance drops a lot and it's because they're not spawning anything. So their entire application is running on one thread and when your entire application is running on one thread, of course it's slower than if you had multiple threads because nothing gets to run in parallel. What's the best way to pass data to Tokyo spawn? What are the best practices to handle errors in async when we call spawn? So spawn is a little bit weird because just like thread spawn, you don't really get to communicate with the other thing sort of implicitly. It's just running somewhere else and you have no control over it. So you need to apply the same kind of techniques that you would use in a multi-threaded program, which is if you want to say share data between this and that or let's say that I have two things I want to spawn and I want them to share access to some vector, I would have to do like arc, new, mutex, new and then this can lock and then do whatever and this can lock and do whatever. In practice, I would have to do like x1 is arc clone x, x2 is arc clone x, this gets x1 and it's an async move and it's an async move so they both get their own arc and I need semicolons so they each get their own arc and they both have a mutex that guards the underlying value and everything is happy. Or you can like have them communicate over a channel, you can have them communicate over like if it's read only over static memory, like you have all the same techniques available to you as you do in a multi-threaded program and you really should think about it in the same kind of way. Now there is one exception to this which is if you have a, if you spawn something and you want to communicate the result of that back to the thing that spawned it, at least in Tokyo spawn what you get back is a sort of join handle similar to if you do a thread spawn where let's say that this ends with zero right then if you do you can sort of assert equal that join handle dot await is going to be zero if you don't await the join handle and just drop it it's the same as if you drop a thread handle it just does nothing like it doesn't terminate the thread or anything it just you don't get to learn its result value so this is one way to communicate the the outcome of a spawned operation back to the caller. Now if one thing to keep in mind is that if you spawn just like if you spawn a thread if you spawn a future and let's say that like I don't know it calls definitely errors right and x now let's say this like return to result right and it's the error case of the result what do you do with the error you don't have anywhere to communicate it necessarily because you don't have a way to communicate with your caller I mean you could return it but there's no guarantee that they're awaiting a join handle and you can't really like you don't have anywhere to do it but the question is the same as if you spawn a thread or indeed if an error happens in main like what do you do with that error you need to have either you could just print it a standard out you could log it to a file you could use some kind of logging framework like tracing to sort of emit an error an error event that gets handled somewhere else in general that's the kind of approach I favor where if you have an error that you can't propagate any further you use an event distribution tool like tracing to decouple the production of events and the subscription to events so that'll be the way to go is there any benefit on calling tokyo spawn and immediately awaiting on it so there can be it's a little it's it's fairly uncommon to do the advantage of doing that is that you get to let other things on the current thread keep running while something uh while that operation is running elsewhere if that makes sense so um imagine that you have an operation that has to do like deserialization so it's somewhat cpu heavy it's still i o so you probably want to do it in asyn context and not blocking context but you could do like uh deserialize over here um and then join handle dot await and now that await is going to immediately yield because this spawn hasn't returned because it's spinning due to deserialization so this future is going to yield and imagine that it's in a select or a join or something other futures on other tasks on the current thread like on the thread that's running handle connection they get to run and then this deserialization operation gets to run on a different thread and gets to do the cpu intensive operation so it gets to happen concurrently and in parallel with these other tasks continuing to make progress so that might be one case where it could make sense to spawn and then immediately await it's rare that you actually want to do this um how is the tokyo spawn connected to the runtime instance created above there are sort of magical thread locals that um that are used basically so runtime new just creates a normal value there's nothing special about it block on will set some special thread locals inside of the executor so that when you call tokyo spawn it checks those thread locals to find the current runtime uh and then spawn on there and similarly when the runtime eventually runs that pass in future it sets the same thread local so when that future calls tokyo spawn it can find the executor and so on it's not a singleton right so so you could have multiple runtimes and if you call tokyo spawn in the context of one runtime it will spawn on that runtime not on the runtime and this can be valuable um so for example there are there are some services where you might care about you might have prioritized traffic for example like imagine you have control plane traffic and data plane traffic and you want to make sure the control plane messages are always handled so one thing you could do is for example have but there's relatively less control plane traffic but you do need it to be handled you can imagine dedicating say two cores to control plane traffic and have everything else for data traffic what you do is you create two runtimes one with a a thread count of two one with a thread count of however many cores you have left you spawn all of your control plane operations on the the runtime with the two threads you spawn all the data plane operations on the other runtime and then you both runtimes get to continue running right like they're both active at the same time um but you know that there are two cores are reserved for control plane traffic um if you had a singleton runtime you wouldn't have this operation you can imagine that the the executor itself supported like priorities and stuff that gets somewhat complicated because it also needs to integrate with the operating systems runtime controls and priority controls it's nice to be able to do this explicitly um so there are advantages to that although it's a little harder to discover it's true um what should i do if there's an expensive function like hashing a password that i don't want to block async execution of a thread uh that's when you use something like spawn blocking or block in place um uh what happens if you tokyo spawn before creating any runtime it it panics it says there is no runtime so rust rust futures do not depend on thread locals um this is an important distinction there's nothing in the the async support in the rust language or the standard library that requires thread locals um tokyo uses thread locals in order to make the and like some other executors do this as well in order to make the interface slightly nicer like otherwise uh imagine that you didn't have thread locals spawn couldn't be a free function so you would have to do something like runtime dot spawn but that means you now need to thread the handle to the runtime throughout your entire application in order for anywhere to be able to spawn deep down in the stack you could make it explicit like this um but and in fact tokyo lets you do this like um there's a runtime dot handle i think and then you can pass the handle around you can do handle dot spawn so you can do it explicitly without the thread locals um but in general the observation is that this is so common that it it's done with thread locals because that way the interface is a lot more ergonomic um the downside of course is that this means that tokyo doesn't really work well on like an embedded context where you might not have thread locals but there's nothing in the rust async sort of primitives or language support that requires it um let's see um when you create a runtime it doesn't really allocate a lot of memory there's like some control um control data structures but they're not generally very large that there's not a lot of overhead to the runtime itself it doesn't need to keep sort of actually i'm about to talk about so i'll i'll save that for a little bit of a second um okay so one thing we haven't really talked about much is um what a future actually is um we we talked a little bit about it right like before we added all the spawn stuff um in fact let me go now let me write a new one um so down here uh so i'm gonna have an async fn foo because i like foos and here's what i'm going to do i'm going to have an x which is going to be a uh byte array uh and then i'm going to do tokyo fs uh let's say that there was a read into um and i want to read file dot dat uh into x i'm gonna just ignore error handling for now uh so n is the number of bytes red uh and then i want to print line uh all of the red bytes so this is a fine usas i don't actually give the type signatures um so let's think about what actually happens here uh we talked about how an async function or any async block is really just a chunked computation right so there's one chunk that starts here and ends like this right like this is one chunk uh and then between every chunk is an await right so this is sort of the the operation that happens in between the chunks um and then chunk two is going to be uh let n let's get rid of the annotation um n is equal to few dot let's say output right because at this point it should have resolved because the await sort of finished um and it does this you can sort of think of it as this is how it's uh oops this is how it gets divided up right this is when we talked about a state machine right this is state one this is the this is state two and this is the edge between state one and state two is future completing or food completing here um and once a little bit weird here is let's try to think about where x is right so x is a 1024 byte long byte array where is that stored you might normally say like it's on the stack right it's a it's a local variable in a function so it's on the stack and that's why this works out but if when you look at it here like this is really a yield which is really a return right so let's say yield which is really return but when you return that x goes away right like the stack frame goes away but the future has a reference to x because it needs to it needs to have a reference to x so they can write into it so it's not okay for x to go away so where is x it's not on the stack because the stack goes away when this yield happens in practice what actually happens is that when you write async fn or when you write an async block the compiler actually generates a sort of that that state machine we've talked about chunk one chunk two it really generates something sort of like an enum where each chunk contains the um the state for that part of the state machine and the state here me really means any local variable that needs to be kept across a weight points right so if i hear said like let z is veck but i never actually gave out a reference to veck like z is never used in any later chunks it doesn't need to be saved anywhere it can just be on the stack when fu is invoked or when fu is continued it can be put on the stack and then it can just be dropped at the end it doesn't need to be saved anywhere but x does so really x ends up being here as a u8 2024 in practice it's a little more convoluted to make sure the references stay the same and whatnot but you can sort of think of it this way the same thing with the future itself it needs to be stored somewhere so this also stores a fut which is like a tokyo fs read into future which actually has a sort of lifetime of tick x right because it holds a reference into x so this is sort of self-referential which is another reason why you can't actually write this yourself in an in any meaningful way and then in chunk two chunk two doesn't actually have any state right because it defines n but n is a local variable it's not kept across a weight points so the n can just be dropped so there's no state in chunk two it does continue to use the state from chunk one though so you could arguably say that like this sort of gets transitioned into here but there's not actually two copies of it this is you can think of this more sort of a union maybe where some of the state actually stays in the same place so that references continue to be valid it's all kind of convoluted but realistically what happens is that the what this actually returns when we say it returns an impulse future right that impulse future really is a type impulse trait means a type of that has a name but i'm not going to tell you what the name is and in reality what this type is is this statement just generated state machine type that like every time we try to await this future every time we continue it what we're really doing is continuing within the state machine type we're invoking a method on the state machine type that gets an exclusive reference a mutable reference to that type so that it can access the future so that it can continue to wait that access an internal local variables and the like and this this conversion from acfn really ends up like rewriting a bunch of these so that instead of being feud this is like self dot feud and self dot x they get rewritten to be re-reference be references into that state machine that's a struct that we end up passing around so in main or in this case let me just pretend that it's an async main for a second when i say let x is foo the value of x is of type state machine right so when i do x dot await what i'm really doing is sort of i'm sort of calling the await method on this particular state machine that's sort of what i end up doing in practice that's really the desugaring you can think of it as i'm really continuously awaiting this state machine the reason this this distinction is important is because this state machine contains a decent amount of stuff right it contains all of the values that are kept across await points and in this case that's say a 1024 bytes which is decently large right so if i for example do bar and pass it x i actually have to move a fair number of bytes this is like a pretty decent mem copy imagine that i was like reading a bunch more bytes right then now suddenly calling bar we're bar let's say that bar just takes an simple future and i pass it x which is a future then what i'm actually passing it is this entire state machine which includes a lot of bytes that have to be mem copied and so futures can actually end up getting really large and the other reason for this is let's imagine remember here it needs to store the future that it's currently waiting on so let's say that down here this called my other future or some library execute.await whatever this future is we don't know how large it is it's controlled by some other library that future needs to be stored inside of our state machine if we have a select we need to store all of the futures if we have a join we need to store all of the futures so futures end up containing all of the futures that they in turn await which means the futures can get really really really large one way you can see this is um imagine that you're doing profiling on your application what you what you'll see in some asynchronous code bases and you'll see that mem copy shows up a lot and it's usually because you end up with really large futures that you end up passing around your program even just like returning a future right like this needs to return a future so this is sort of a mem copy into this variable and I can keep happening up the stack as you return futures or pass them around stick them into vectors whatever and so this is something to to watch out for of course the way that you can solve this problem is by boxing your futures so if you place them on the heap this problem sort of goes away you could either have the this this allocation be on the heap or here we could say this is going to be box new of foo in practice we need to use box pin for reasons we're not going to get into but at this point x is still now a future but when I pass x to bar I'm just pointing passing a pointer to a heap allocation that bar can then treat as a future rather than the entirety of the state machine that I constructed this is another reason to use something like Tokyo spawn is because spawn is going to stick that future into the executor and then just keep a pointer to that future this is why it's actually useful to have the the spawn handle is because you can store that and then like here if I instead of doing this did Tokyo spawn of this dot await this now doesn't have to store this future whatever size it might be it just stores the pointer to it and they can await it and so we end up doing fewer of these mem copies we don't end up with the the futures just growing by growing the onion well that make sense yeah it ends up being a union with the size of the largest chunk state is the right way to think about it why can futures assigned on the stack be moved don't they need to be pinned we're not going to talk too much about pin but basically there's nothing that prevents you from moving a future in fact there's nothing in the rust language that prevents you from moving any value but once you have started awaiting a future you can't move it again unless it's on pin but there's nothing inherent about a future that means you can't move it can you have an async function that eternally creates a veck of futures from itself recursively I know recursion requires indirections in other words is putting in a veck count as indirection putting it in a veck is not enough but if you stick it in something like a join handle I think you can or yeah if you stick it in a join it depends I don't think join is enough actually because a heap allocated like a vector is really a heap allocated array and an array needs to know the size of its elements and the size of its and it can compute and it can compute the size of its elements because the size of the pointer is known so I think that's fine although you would have to use a join not a veck in order to actually select or await the set of futures you should be able to do that okay now that we've talked about this there is a little bit more I want to talk about sort of here at the tail end you may have heard about async trait so one thing that the people are sort of missing in one thing that is missing in the implementation of async await as it stands today is the ability to write the following I'm going to go with service because it's a trait that I know decently well I want to be able to write async fn call request I'm going to just no I'm going to make it simple I'm going to have a struct request and a struct response I want to be able to write this right I want to be able to define a trait that has an asynchronous function in it currently this doesn't work functions and traits cannot be declared async async trait functions are currently not supported consider using the async trait crate and the reason why this is not supported is because remember this is really sugar for impulse future output equals this and here we get a different error impulse trade is not allowed outside of function and method return types the reason for this is let's say that I write fn fn foo and I take an impulse service and I say let x is x dot call request let's make that few how large is few here or let's say I take a din service oh of course this needs to let's say this is mute self um this is mute then how large is few tier what depends right let's say that I have struct x and I impulse service for x and I implement async call async fn call mute self request I have to return response and inside here I say here I just return a response immediately so the size of this is very small and for why what I do here is I do let z is zero thousand twenty four then I do Tokyo time delay or it's called sleep now I forget sleep for 100 dot await and then I drop z so this is used across an await point and then I return response and the size of few depends on what stack variables are there in the async block that was used for the future and so few doesn't have a size it doesn't have a known size and so the compiler doesn't know what code to generate here because it doesn't know the size of few you can imagine that if this was like impulse service then maybe it could figure it out but this is now a type that you can't name so let's imagine that I wanted to return a foo call and I want to write struct foo call and here I'm going to do foo call of fut so it's going to hold the future right what's the name of this future it's sort of like service colon like type of service colon colon call maybe that's easier if we say like s service and this is an s and this is s call but this doesn't really work either you can do this with like existential types but it gets weird so really there isn't that there isn't really a good way to deal with async call like this in what we've described so far because the the type of the thing that it produces isn't known anywhere it's not written anywhere so there are there are two ways around this one is the async trait crate so it lets you annotate like this and what that does is it basically rewrites all of your async offense you need to also place it on any implementers of the trait and it rewrites all your async offense into pin box din future output equals response and it does the same down here rewrites that into box so it rewrites the signature and they'll also rewrites your body into async move of this and that works if i remove async trait now well i'm not importing pin but now it won't complain because this is a type with a well known size it's a heap allocated dynamically dispatched future which rust already knows how to reason about so the size in i remove the block that i should call this but if i if i do if i have a service and i call dot call the thing i get back i know the size of it i know how to sort of await it i have all of that information readily available to me because it's just a din future the problem of course is that now your heap allocating all of your futures which means you get dynamic dispatch you don't get sort of monomorphizations and some of the nice rust optimizations also you're doing a lot you're invoking a lot more memory allocator pressure you also have indirection for all of your futures so imagine that this was not service but this was async read and this is a read now every time you do a read you do also do a heap allocation and an extra pointer interaction so that might actually get fairly expensive this is why async trait works really well for sort of higher level things which traits often are it doesn't work so well if you have to use it as the bottom of your stack or rather it might not work so well always measure first and see whether it's a problem the other approach you can take here is to declare a read if we go back to service here is declare a an associated type call future which has to implement future output equals response and then instead of having this be async fn call you say that fn call has to return self call future the reason this works um is that here I can now say type call future is equal to well this is also inconvenient to use I'm gonna get rid of this because it's annoying um type call future is equal to this this is going to be self call future so I can use I can do the same thing here right boxed in future output equals response so I can make this be the same thing as what it was for for async trait but I can also not do that if I want to choose to do so like this could just be response it could just be an async response the problem I now have is how do I name this type so the reason this works is because now the associated type like Russ knows how to communicate associate types to callers so they can now know how large the the actual type is you have a way to name the return type so that you can use it in other structures and stuff the problem of course is when you implement this trade it's not entirely clear how you name any type that isn't pin box didn't there is a feature called type alias infiltrate which would let you write this um which sort of feels like we're almost at uh async if any in traits there are some other corner cases to uh to fix out um but but this is why the async trait crate is often needed and why it's hard to get async traits um without doing something like boxing all right does that does that make sense why why we have async trait what it's used for um yeah so so one of the observations right is that the compiler could in theory generate all this itself right if you write this certainly you could imagine that the rust compiler could like behind the scenes rewrite it into this right and also automatically rewrite any call to it into the appropriate thing here there's a little there's a lot of magic going on there like suddenly what i write is this but it gets turned into like an async move and an associated type that gets auto named for me uh the the name of that type is not always entirely obvious um it's a little bit maybe i now need to also say like where self is sized because dynamic dispatch wouldn't work with associated types um it's not impossible and this is why i think we will get async functions and traits eventually in rust but there's a lot of design decisions to be made for how this should actually be exposed how it should work behind the scenes should you be able to customize what this um additional function is called um there's just like a lot of design decisions that have to be made um and there's a fair amount of just complexity in the types that needs to be worked out to um if you're interested in this i highly recommend like go look at all the discussion that's happened about this uh it is something that like is clearly wanted is just getting it right is is uh is hard um great so let's see are there more things i wanted to talk about so we talked about join we talked about futures in order we talked about spawn talked about number of worker threads we talked about async trait we talked about spawn blocking um we talked about send and static futures pin and box futures um we talked about how blocking code is problematic because it doesn't let other tasks run um yeah there's one more thing i want to touch on and this is something that is i don't want to say controversial because it it isn't really but let's say that i have an async well let's leave that i have an async fn foo uh actually i don't even need these uh i can write this here so let's say that i want two futures to share state so i'm gonna say x is arc new arc new mutex new of zero uh and then i'm gonna do tokyo spawn actually i'm gonna do let x1 is arc clone x spawn uh async move bear with me for a second while i type out some code this is going to be x2 all right so i really just wanted to okay we're going to do mod tokyo we're going to do fn spawn uh we're going to say it takes simple future and does nothing with it you stop yelling at me now async pub fn yeah pub async fn great stop yelling at me um so here i have uh two async threads and both of them are accessing the same arc uh and they're using a mutex and in this case they're using the mutex from the standard library so there's also tokyo sync mutex i'm gonna move that down here just so that i can make it not yell at me anymore uh oops pub mod pub mod sync pubs rock tmutex i'm gonna tmutex okay so there's an argument ongoing about should you use the standard library mutex or should you use the tokyo mutex or like an asynchronously enabled mutex um and the answer and the reason why there's a discussion about it is let's imagine that i have this do uh let x is x1.lock and then i did uh tokyo fs read to string file .await and then i did x plus equals one this is clearly a very stupid um loop but let's say that's what i wrote so now i'm in a position where i take this lock and then i go await now at this point um let's say in fact let me make this a select that's what i'm gonna do that'll illustrate the problem better uh so i'm gonna move this up here okay let's imagine that i have a runtime with only one thread now imagine that this gets to run first so it takes the lock and then it goes to read a string and then it calls .await and now it goes to run this uh like this yields because it can't read anymore from the file so now it goes to run this other future instead it's been spawned right so the the thread sort of drops this one which is yielded or it doesn't drop it but it it just sticks it on i'm gonna run you later and then it tries to run this future and then it tries to grab the lock but the lock is held by this future and so therefore this blocks and because it's a standard library mutex it just blocks the thread it doesn't know anything about asynchronous it just blocks the current thread but that means that the executor's one thread is blocked which means that it doesn't get to continue reading from the string because that would require continuing executing that future which means that this future never ends up dropping its lock guard which means that the lock is never released which means that this lock never completes and so we have a deadlock this is why it's problematic to have standard library mutexes right it's because you can end up in these deadlock situations now this would still be weird if you have an asynchronous mutex right if we use the the like a tokyo mutex or some other asynchronously enabled mutex what would happen is this would lock this would try to read to string it would yield this would try to lock but because it's an sort of async aware lock when it fails to take the lock it would yield rather than just block the thread it would yield which lets this future run again which lets this complete eventually which lets it drop the mutex guard which lets this continue and eventually succeed so that's why you need async enabled mutexes now that said one downside of async awaited mutexes is that they are a decent amount slower and that's because they need a lot more machinery in order to be able to do this sort of yielding on demand and know when to wake each thing up it's fairly complicated what they have to do internally and so this is advice that in general you actually want to use a standard library mutex as long as your critical section is short and does not contain any awaited points any yield points so in this case our critical section that is the section under which we hold the lock contains an await which is just ripe for deadlock potential like we just saw and so here it's not okay to use a standard library a standard library mutex but in the case we had before of it just does this here the critical section is very short and it doesn't have any await points so there isn't really a risk of deadlock here the reason I say the critical section has to be short is if the critical section was like do a giant matrix multiplication then sure there's no wait point but you're still holding up that thread and not letting other futures work so it's sort of similar to any other operation that is like a long-term operation but it's a little worse because you're also holding the mutex which might block other futures on other threads in the same way right like they may be unable to make progress imagine that um okay you imagine you have a runtime and executor with two threads and there's a future on like this future runs on one thread and this future runs on the other thread and this is doing like a matrix this is doing let's say here matrix multiply which is let's say super slow so this thread is running the matrix multiply while holding the lock this other executor thread is trying to run the second future tries to take the lock and blocks right because this is a standard library mutex so it blocks and at this point because it doesn't yield any other futures on that other executor thread also don't get to run as you're holding up a lot more work so that would be another instance where you would want it to be an async aware mutex so that this other executor thread its future would yield when it fails to take the lock and let other futures run in this thread at the same time while this matrix multiply is finishing but in the case where the critical section is short and there are no await points it's totally fine to use a standard library mutex and often you might want to because they can be significantly faster to acquire and release which could be really important if you have if you have operations and you need to do a lot or where there's potentially a lot of contention where a mutex might do a lot better than an async aware mutex okay so at the tail end here there's a question of like how do you detect these kind of problems if there's like blocking or something is running for a long time or you might have a cancellation you didn't expect there aren't currently any good tools I know there's some work on like a tokyo console which is basically it sort of hooks into the executor and figures out how long has it been since a given future yielded how long has it been since the last time a future was retried so that there's like some of that kind of monitoring that that could point out these problems it's not complete yet it's not something that's really tied to tokyo either I mean this one is written for tokyo but you could imagine instrumenting any executor in the same way of sort of noticing when particular patterns show up like a future hasn't yielded for say more than a second that seems like a problem and you could highlight that to the user oh yeah and it looks like clippy has some lints for this too can you elaborate on the conceptual difference between a thread spawn and a tokyo spawn a tokyo spawn gives the future it has passed to the executor for the executor to run whenever it wishes concurrently with other futures a thread spawn spawns a new operating system thread that will run in parallel with everything else in your program and is outside of the executor's control a thread spawn also does not take a future it takes a closure so if you wanted to await a future inside of a spawned thread you would need to create your own executor inside of there conversely if you tokyo spawn something you're not guaranteed to have your own thread and so you do need to be co-op like you have to cooperatively schedule if you use tokyo spawn you have to have yield points because otherwise you might block the executor if you do a thread spawn that's not a problem because the operating system is able to preemptively disrupt you basically think of it as threads are not cooperatively scheduled and so they can do blocking operations whereas tokyo spawn or just like futures tasks that you spawn are are cooperatively scheduled and therefore need to yield in order to let all the futures in the system continue running okay i think those are all the things i wanted to touch on for async and await and you'll notice that we haven't talked much about the lower levels right like we haven't really looked at the future trait any of like pin or context or waker all of the bits and pieces that executors go through those are valuable things to know about but they're not really that relevant to trying to understand just how do i use async so hopefully this gave you a good survey if there are questions now at the tail end of like the again these higher level bits of what what should my mental model be for async await what are the intuitions what are the sort of higher level techniques now's a good time to ask them and i'll spend some time just answering some questions here at the at the end when i await inside a future will the current future get scheduled on the same thread when it resumes to start progress or can it get scheduled on another thread when a future yields it it just yields to whatever awaited it right so imagine that you just wrote something that was like it takes a impulse future so imagine that i did this if inside of f you await and then you end up yielding you just go back up to b and it's up to be what happens at this point right b in this case is chosen to await and so it's just going to keep it's going to because you can't make progress it can't make progress it's going to yield and eventually it's going to yield all the way up to the executor at that point the executor just takes that future and sticks it back onto the the sort of job queue of futures which is handled in general by all of the worker threads at least in a multi-threaded executor like tokyo's default one but but this depends on your executor there are executors that are single threaded and if you have a single threaded executor it would execute on that same thread because there are no other threads in tokyo you're not guaranteed that it's the same thread you could imagine that there are or there are ways to say only run me on one thread in different executors like in tokyo there's like a spawn local which gives you some of these guarantees and you could also imagine that you had like a instead of b doing an f dot await it could do like a loop f dot poll i said i wasn't going to mention poll but i will poll is the way to sort of check progress on a future so you can imagine that b doesn't actually use the await keyword at all so it never yields it just loops you in a busy loop if it does this then f will get to run again immediately and it will be on the same thread because there's no yield here so there's no opportunity for the executor to reschedule you on a different thread but in general this won't be the case so in general all the way up the stack it's going to be a waits all the way up to the executor which does have the option of rescheduling you on a different thread in general the at least i know tokyo tries to keep you on keep a future on the same thread because it generally helps with cash locality and such but it doesn't guarantee it if you really wanted a future to not be sent across threads you would just not implement send for it of course that makes it a lot harder to work with that future but that would be the way that you would enforce that statically will be your favorite method to get an async stack trace that is to show the async call graph up to a certain point so i don't have a great way to do this it is true that if you if you print just a regular stack trace inside of an async inside of a future you will get the the call trace going up to the executor and usually that can help the problem really stems from spawning right if i spawn this future then and then let's say i panicked right here then this panic would not show main this may be a bad example um let me have a async fn foo and it does a tokyo spawn of an async that panics to illustrate the problem this panic will not include foo uh because this is a future that gets put onto the executor's job queue as sort of a top level future to to await and then the executor one of the executor worker threads is going to pick up the future and await it and at that point foo is no longer involved all foo did was like put the future on the queue um but when that executor thread awaits this future and it panics the back trace for that panic will only say the executor pulled this future so it'll point you at this future but that trace won't include foo um i don't have a great way to solve this problem i know that with um with something like tracing so there's a if you haven't looked at tracing it's great um so the tracing library is a sort of logging system that lets you emit events and then have subscribers to pick up those events um and one really neat thing that exists is uh tracing futures no that's not what i meant to do and also sorry that it's bright so tracing futures uh let me see if there's a good example of this is um you can take a future import this trait uh call dot instrument and then give it a a sort of signifier that indicates the surrounding scope so in this case what we would do is something like spawn async and then dot instrument like in foo it's not quite what you would write but but you get the idea so this produces a future uh instrument takes a future and returns a future so this is still a future and then that whole future gets spawned um and now even though the panic isn't what you would do uh here if you emitted like a tracing uh tracing error for example this said oops um this tracing error would include this in its sort of event path um and so that gives you a way to sort of trace it back to where that future was originally spawned um and so that's one way to to get at this that that i've had some luck with uh but i don't have a good sort of general purpose non-instrumented solution sadly um uh best practices call async code from synchronous code try not to do it it's really hard to do right um you can use something like futures block on uh the the downside of block on is that you don't really get this cooperative scheduling you can use like you might end up with your program well you run into some of the problems like a particular future might be using io resources from tokyo which the futures executor doesn't know how to execute and so you get a runtime panic that's one case you can get into um if you just like blindly try to block on futures um basically and then the other problem is you don't give the caller control over your user of your library control over execution um you're in basically you're enforcing an asynchronous runtime on them rather than letting them choose the runtime which tends to make people sad um because whatever like imagine that um maybe one good way to get at it is imagine that i'm writing an asynchronous application and my asynchronous application calls your synchronous library uh and i use like spawn blocking or something and then your synchronous library creates an asynchronous runtime in order to block on a future internally so now i have a nested asynchronous runtime that causes all sorts of problems um sometimes it's runtime panics uh sometimes the two are incompatible as you get up with like other runtime panics sometimes just a performance problem where now you spawn twice the number of threads that don't really cooperatively schedule with one another um it's just it's just a nightmare uh i would say that if you have asynchronous operations internally expose them as asynchronous and then leave it to the user to choose how to make them synchronous um so basically don't if you can avoid it um are there use cases where you want to use threads instead of futures there are so um anything that's very compute heavy asynchronous doesn't really add very much and and tends to sort of get in the way because all of your operations have to be marked as blocking and then it just gets more annoying to write them because you need to pass them around as closures um so there i would use something like rayon instead if it's very compute heavy if you don't really have i o um uh if you have a program that yeah if you have programs that don't really do i o there's no real reason to use async um it is true that in general writing non async code tends to make at least simple code or or streamline code or single execution code easier to read give make gives you better compiler errors um the borrow checker generally works better if you're not in async context the back traces work better so if you're writing just sort of straight line code like you're writing a simple command line tool or like a converter or something like i probably wouldn't bother with async because it does make your task a little harder if you're writing something where you think that it might be i o heavy in the future or you might want to do like uh handle multiple different types of events like this is where select is really useful and emulating the same pattern and threaded code is a little bit of a pain um then i would lean away from threading but but if you really just don't need the mechanisms that async provides then then use normal threading um uh tokyo tokyo by default doesn't bring in any runtime uh and then there's there are feature flags for enabling i o for enabling or like file i o enabling networking enabling timers enabling the single threaded runtime enabling the multi-threaded runtime so in general you you opt into the different features that you want um if you call tokyo spawn you're not on the tokyo runtime you get a panic all right i think that's where we're gonna end the stream two and a half hours that seems about right for async wait um i hope that was useful i hope you feel like you have a better intuition for for what's going on under the hood and some of the pitfalls um and if if you're really curious about how this is actually accomplished um i recommend you go uh you can look at my older video about sort of how async really works um and in the video on how pin works uh some of that is going to be a little bit more technical but but it is maybe helpful knowledge uh if you really end up digging into the details here but hopefully this has given you the the high level intuition and knowledge you need in order to be productive with async wait at least that's my hope um great thank you all and i will see you next time which will hopefully be another hazard pointer stream so long everyone