 All right. Hi everyone. Finally, it looks like we'll start on time. How about that? Just do a sound check first because last time we had some sound trouble. Can you all hear me? Like just type something random in the chat just so I can see. Oh, fantastic. All right. So there are a bunch of people who are on the stream. I think that weren't here before so I'll just like briefly recap where we were. Great. Thanks. I got it. So I've just as a sort of useful point I've started a patreon just where I'll post whenever there are upcoming videos for those of you who don't generally follow Twitch. I'll also keep tweeting so if you're interested you can follow me here. So basically the place we ended up last time or where this whole stream started was we're building this crate called Tsunami and Tsunami is a way to run jobs, especially short jobs and benchmarks on EC2 and we have this issue where we're trying to spin up a bunch of VMs and connect to all of them, run some setup routine and then run a benchmark on them. And currently the way we do that is we sort of we spawn a thread pool. Each thread in the pool connects to one server and this is really silly because the threads aren't really doing anything. So there's no reason to have all these threads sitting around. Of course the way we should solve this is by using futures using asynchronous I.O. You sort of connect to all the machines and then whenever one of them is ready to connect then you send some commands to it and you can do this essentially just from one thread. The problem is that the SSH library that we're using under the hood currently is entirely synchronous. We're using this lib SSH2 and lib SSH2 is fully synchronous, which means that we can't do this. The threads that send commands to the servers have to block every time they send a command and wait for the response and that's why we need all these threads in the first place. And so someone observed towards the end of last stream that we can use this thing called thrush. So thrush is an SSH client and server implementation of SSH, the SSH protocol. In particular what's neat about it is that it is fully asynchronous. So if we look at the client connect future. So you can connect and you get a future back that represents the connection to the machine and once you have that connection, then you have a handle that's both async read and async write. Async read and async write are the ways you express asynchronous communication with some stream. So you can imagine you have an asynchronous network stream for example just about to mention that great. Yeah, so someone had the same thought last time of why don't you just use this? Now it turns out that it's actually not entirely straightforward to just like take a synchronous library and just make it asynchronous because you need to deal with the fact that now you can try things, you can try doing a read for example, and then the read doesn't do anything and your thread doesn't block. So now you need to handle this event loop that we're gonna get into with how to handle futures eventually resolving. And so we'll look at all of that today. Basically, what the plan is to cover today is to first build a nicer SSH abstraction on top of what thrush provides. So thrush is fairly low level and we'll get back into that slightly later. It's a fairly low level library as we want to build something on top of it that's sort of nicer where you can issue a command and then you can like read the standard out of that command and send commands a standard in. And then hopefully towards the end of the stream once we finish that SSH binding, then we'll use that in Tsunami to give us a fully asynchronous tsunami. So that will also mean that we're gonna take advantage of the new release of Risotto which is now entirely futures-based. So if we look at EC2, notice how all of these things now return these Risotto futures. So the entire sort of EC2 API create is now also fully futures-supported. And so in theory at least we should be able to make Tsunami as well fully asynchronous. But step one is gonna be making this fairly low-level thrush library into something higher level. And this essentially means that we're gonna implement a bunch of future stuff ourselves and hopefully that'll be interesting. Now I have cheated a little bit this stream in that usually I don't sort of think about and design and plan the implementation ahead of time. I tried to do that during the stream because I think it might be useful to view people viewing. In this case, I did take a look at thrush beforehand just to see sort of what we're up against and thrush is a little bit weird. So we're gonna dive into this now. The way thrush works is when you have a connection you register it with a handler. So notice H parameter. So a handler is sort of like a bunch of callbacks that you're going to get from the connection. So you connect to the server and then you have a way of sort of sending things to the server. But anything that the server tells you back is handled entirely through upcalls. So in this case, for example, it can call the data function on the handler that you registered. And this will tell you that there's data available on some particular channel. And so this isn't really how futures work, right? The intention of futures is that you send a command and then you get back a future that eventually resolves to the value that you asked for in the command that you sent. Whereas in this case, you fire off a command and then later on some other function is going to be called somewhere. And we want to tie that together into a nice futures-based API, which is not what this crate currently provides. In fact, the reason I know about this is in preparation of the stream, I messaged one of the authors of this crate and was like, hey, this seems a little weird. Why it's not really futures. And he pointed out that it's a pretty thin layer about the SSH protocol and then mentioned that, hey, I mean, you could write a layer on top that provides nicer abstractions. And that is exactly what we're going to do. So we're going to dive right into it. One thing that I would recommend if you're curious about future stuff and like to read more than listen to me code is Aaron Turan has started writing this rust book on asynchronous rust stuff. It's here. It's probably linked to if you just go to his website. It's, as you can see, not really complete yet. But even the chapters have been written so far actually really good. And so I recommend you take a look at this if you are not at all familiar with futures or as you try to build more advanced things, you can follow this through and you can see he gets into fairly detailed things or will get into fairly detailed things after a while. Okay, so the place we're going to start is we have to pick a name for this SSH crate, which is always its own kind of fun. Let's see. So we're going to start a new library. How exciting. And we're going to call it what async SSH maybe. Yeah, let's do that async SSH. And of course, it's going to depend on so I don't know if you're aware, but Tokyo and futures are going through a lot of changes these days. The basic structure of them is the same, but the API is changing. And so we're getting Tokyo. Tokyo used to be lots of lots of other crates like Tokyo Core, Tokyo IO, Tokyo Proto, etc. They're all merging into this crate called Tokyo, just Tokyo, 0.1, and that just happened. And then there's also the futures crate is turning into future 0.2, which has a new API, but it hasn't really been released yet. And so what we're going to do in this stream is we're going to be using the old Tokyo and futures stuff because the model is basically the same and all of that is stable and fully released, whereas futures 0.2, for example, is still alpha. And then we'll probably do a later stream where we upgrade to future 0.3 and the Tokyo crate, that transformation should be fairly straightforward, but I wanted to keep that self-contained because it is sort of like its own problem. Okay, so we're going to use futures 0.1. We're going to use Thrush, a little bit unfortunate name, 0.19, and we're going to use Tokyo Core, Tokyo Core, 0.1. Actually, we might not need Tokyo Core. And we're going to need Tokyo IO. So Tokyo IO, we only really need for the async read and async write traits. We don't need any of the other stuff that it provides. In the new release of Tokyo and futures, the async read and async write traits are put in the futures crate, which makes a lot more sense because if you have some type that is asynchronous can resolve, that doesn't really have to do specifically with IO, even though read and write are IO operations. But Tokyo IO is more about how do you do asynchronous things on file descriptor and Linux, for example, or network streams. So Tokyo IO is more an implementation thing. Okay, so we're going to need Tokyo IO, 0.1. Great. And then here we're going to have extra crate, Tokyo IO, we're going to have futures, and we're going to have Thrush. No test yet, unfortunately. Okay, so the basic API design that we want for this crate is going to be somewhat similar to the API for the synchronous SSH crates, except that we want all the things to return futures. So you can generally think of this as the synchronous API is sort of the nicest API to work with, right? Or in this case, is a fairly thin wrapper over the C API, but that API is decently well designed. And so the idea is you create some stream, you establish an SSH session on that stream with a handshake and some authentication. And then once you have a session open, then you can open a bunch of channels. So you have a single SSH session to the server. And on that session, you can spawn multiple channels. Each channel you can sort of think of as a window to the server. So on each channel, you can spawn some commands and its inputs and read its outputs, and the commands are sort of separate. And when all the channels have gone away, you can close the session. So if you try to close the session while there are still channels open, we would have to figure out what happens in that case. But the basic idea is you have a session, you call like channel open, or in this case, channel session, it gives you channel back on that channel, you can execute command, you can write to it, you can read from it. And that's basically what we'll provide. So there are a lot of details to the SSH protocol, we will not write a complete implementation, because that would be silly. But we want an API that looks a little bit like this, except with futures. And so what we're going to do is actually the same thing as we did in the we did when we started Tsunamis, we're going to start out by having an examples file. I've generally found this to be really useful, because it's a way of keeping yourself focused on what you want to end up with. So if you remember from the Tsunami crate, what we actually did was we wrote an example file of how we wanted things to work. And then we didn't change that file basically all the way through developing the crate. And at the end, the example just worked. That's sort of what I want to get to with this. We might not get quite there because the API here is a little bit more uncertain, but we'll see. So examples, we're going to call it LS, because that's all it's going to do. So how do we want this to work? Well, we sort of want it to look like this. So we'll start with there and then modify it a little. Specifically, we want to be able to do something like do we want this to be on? So the original SSH API has this notion of you make a connection to the server and then you give that connection to the SSH library and it will establish an SSH session on top of that. Part of the reason for this is for testing the other so that you can have other transport layers like you could do something like SSH over like HTTPS or over DNS or like you would never do any of these things. But in theory, you could with this kind of interface. In fact, let's check what is supported by thrush. So if you have a thrush thing, you connect. So if we want to create a connection, would we have to give it new and config and a stream and a handler? Okay, so we can actually provide the similar kind of API. So let's do that then. So we'll, we'll essentially have a sort of new or a connect method that takes some existing stream that you've already set up like a TCP stream and then we will create an SSH connection on top of that. So in that way, we'll actually end up with a very similar setup to the the synchronous SSH crates. So in this case, this would not be standard net, but instead would probably be my OTCP stream. So there's going to be a we're going to connect to some port. We're going to establish a session. So we'll do basic SSH import the session type, we will probably also have a session type. Again, our goal is almost to mimic the same API. Now notice that the big difference is going to be here is we're going to connect and that's going to give us a future of the connection. So we'll use to your point x-ray futures. We'll use futures future. And so the connection is actually going to give us a future that is going to resolve into the connection to the server. So instead of having this old code that used to just like synchronously connect and then unwrap and then create a session and then unwrap and then handshake and then unwrap, we're going to then do this as a sequence of and thens. So in this case, we're going to do a connection. And once we have the connection, then with that connection, we will do session new. In fact, let's just do even better session new. So this syntax is sort of a short way of writing connection session new connection. So instead of writing this, we can just write, okay, so we're going to connect, then we're going to establish a session on top of that TCP connection. Once that connection has been established, then now we have an SSH session. With that SSH session, we're going to do, what else does it want us to do here? Authenticate. Yeah, so if we look at the original thrush example, where's the thing? Where does that? Yeah, so it connects. So that's sort of the similar. What's that syntax called? I don't know. But it's not really special syntax. You should think of this as and then takes an fn, probably just an fn, maybe an fn once. And one thing that implements fn once is if you have a closure, that will implement fn once. But in this case, a function pointer also implements fn once. And this is essentially a reference to a function, right? And so this also implements that trait. And that's why it can be used in this argument position. Okay, so the underlying thrush stuff is you connect and then with the connect, you call authenticate key or some other authenticate method, and that gives you a session. So we're going to do the same kind of thing. In fact, arguably, no, we'll have the client do the authentication, I guess. Okay, so this is going to call session dot, I guess we if we really wanted to mimic the API, then we'll use something similar to what this uses. Where's the session? Yeah, so on session here, there's user auth password user auth agent and user auth various things. Whereas in thrush, they're called authenticate, whatever, we can really choose what we want. I think in this case, the authenticate names are better. So in this case, we're going to do authenticate. It's a good question, authenticate, let's call it key, that's fine. We're going to be connecting to EC2. So in this case, this is one of those things where obviously the username here would be selected in this example, I'm going to be connecting to an EC2 server that I'm going to set up. Fluent interfaces, I don't think that's what it's called, but it's a good name. Okay, so EC2 users is going to be the user just because that's a machine that I can message into. And we're gonna have to give it a key file. Oh, that's a good point. So what does that take? That takes a load secret key. What can I give it? I sort of want to do this in the simplest possible way. So authenticate key, wait, I want authenticate key future, which, okay, authenticate key future, and that takes a self a user and a public key. Okay, what's a public key? Let's say you can parse, wait, what? That doesn't seem right. You can't actually authenticate with just the public key. Oh, it gives you an authenticate. That seems really weird. Okay, I think we just want authenticate key. I don't know what that function is called, authenticate key futures. It seems to do something completely different, because authenticate key takes a key pair. This is what we really want. All right, so we're going to take a key pair. We're gonna have to generate one of these. Why is this showing no fields? It's a little bit silly. How do I get a key pair? Generate one, I don't really want to generate one. Open SSLP. That's very weird. How do they propose that we do this? So the example does what? It reads a file. It goes load secret key. What is load secret key? I think it's lying to me. Unless there actually is a load secret key here, but I doubt it. Does anyone see a load secret key here that we can use easily? All right, fine. We will eventually need a key pair. It's not terribly important. So we're going to have some key here. And that's going to give us an authenticated session. So we're going to do this. And then with that authenticated session, so now we've basically done these things, and then session. This is going to all end up with one future that's going to do some stuff. In our case, we want the session to do this channel session function name is weird. Just do channel open or something. So a channel open. And that's going to give us an end of a channel. And then on that channel, we're going to exec LS. Yeah, okay. So we're going to exec LS. And what's that going to give us back is the question that's just going to give us the channel back. And so on that channel, we now want to read the output of LS, right? This looks like the making of callback hell. So it's actually, you shouldn't really think of these as callbacks in quite the same way. And the reason is because these, these, okay, you're right in the sense that you're like defining a bunch of closures that are going to be called in sequence. So the way this is actually going to end up looking like is something like await, right? So once we get await support in the language, then in theory, you could await this and then you could say let session is equal to wait, whatever. This is just us building up that same computation, but without keyword support, you sort of need to you need to say what the sequence of events is going to be. But in theory, the language can make this a lot nicer with syntax. We like someone telling a story that's getting more and more ridiculous. This happened and then, yeah, that's true. It does read that way. But also that adds dramatically effect, right? Okay, so the question is what do we want with the result of LS? Well, in our case, one of the things that's nice, once you have an async read or an async write, is you can use Tokyo IO read to end, for example, which takes anything that's an async read on a buffer, and it gives you a future that will resolve into having read all the stuff. So this is one of the ways in which you use async reads. Similarly, there's write all for async write. So we're going to have this one of these. So this is great. So with this channel that is now executed LS, arguably, this channel sort of needs to be if we wanted to really use the type system to the full to its full extent, the type of this channel should be like, un-executed channel or something, right? Because you can't you couldn't read directly from this channel without running something on it first, right? So presumably, exec is going to resolve into some channel that we record is already usable. So we're going to go read to end of the channel. And in some buff. So that gives a future that will resolve once you've read to the end. So what this entire future in theory should give us is something that connects over, connects over TCP, establish an SSH session, does a authenticate key on top of that session, and then opens a channel executes LS reads the output of LS, and then the future would resolve. And notice here that actually, arguably, this is probably going to give us the channel back as well. Maybe. Yeah, so it gives us the channel back. This is pretty common. I think in our case, we want only the data. We don't really care about the channel anymore. When using crypto back, instead of the back, the non swappable erase on drop buffers. So I think the crypto back, we can use a crypto back for the key. I don't think we need to use a crypto back for the, for the data that we read. In fact, I don't think we can. I think, yeah, read 10 takes a back, specifically a back. But for the key pair, you're totally right that we could use a crypto back. That's probably the right thing to do. All right, so what this is going to give us and notice how this code reads fairly similar to the synchronous code that we saw in SSH, right? So this is the original code. Let me keep in this part. Right. So this is the synchronous part. And this is what it would look like in our async world. The difference, of course, is that we would have to do course Tokyo core reactor. So the stuff I'm typing now is just like the wrapping stuff that you need in Tokyo to execute a future. Much of this will go away in the new release of Tokyo. Or rather, it'll be a default. You can sort of say to execute something in the current thread. So we're going to do cordon run. She arguably we could just call wait on the future instead. But let's do this. So this is going to be so we're going to run LS out. I'm going to assume that it doesn't crash. And then we're going to do the same as the synchronous as a example, I guess, print that. And then, oh, actually, this isn't a sorry, what's that crypto? The crypto back should probably be used as a back. No, I don't think crypto back can be used as a back. So in particular, it can probably D ref into a back. But in this case, notice that the read to end takes ownership of the vector. Right. So you couldn't give a crypto back to this, you would have to convert it into a back. So here's another neat thing that you can do. What if we wanted the, actually, there are a couple of things we'll have to deal with here. One is when you execute command remotely, then you get sort of the standard out and the standard error output of that channel, right? So LS might print some stuff to standard error, it might print some stuff to standard out. And currently, if our channel exec just gives a channel that you can read, then does that give you your standard error or standard output or both or whatever, we'll have to deal with that at some point. I think for the purposes of this API, we want this to work. And then maybe what we do is like, on channel, there's a, like a channel dot split or something that gives you two future or two async streams, one that gives you standard error, one that gives you standard out. And if you just read it, then you get maybe the two merged, we can decide this later. Similarly, with the channel that you get back, you might want to get the exit, exit status of that command. Like, let's imagine that you wanted to see, I guess, LS is a bad example. But like imagine that we ran bin false here, right, which is a command, a remote command that just exits with an exit status of one, indicating an error or false or something. Then you want to be able to read what that exit status is, like imagine you ran like cargo run or cargo build on the other side, you wanted to know whether that succeeded or failed. And that means you will need to be able to get the exit status back. And you see in the synchronous API, that's done here by calling exit status. And so probably what we would do is you would do something like this. So we would say channel dot exit status, which would be give you a future that gives you the exit status whenever it's ready. And then we would map that to include the data. So this would now be status plus data. Yeah, it's a little awkward. But yeah, so the way to read this is execute LS read until the LS is finished. Take the channel on the data that you now have, find the exit status of the channel, and then include the data with that status in the output. So this is going to be a future that resolves into a tuple of the exit status and the data that we read. And so in this case, this would be status and data. So we would print, if we wanted to do the same thing as what we get in the synchronous case, we would do something like, right. So if you compare the asynchronous and the synchronous code, they actually like they look different, but they're not the they're not that different in terms of like the things that you do, and also not that different in terms of length. Right. So I think we're decently happy with that. Let's aim for an API that looks roughly like this. Unless anyone has any like severe objections. So in order to do this, we're going to need a sessions right, it's going to be public, it's going to be great. We're also going to need a channel struct. So this is probably going to be like an open channel. If you have an open channel, then all you can really do on it is call exec. And then there's going to be a channel struct. Actually, maybe this is, maybe this is channel and this is like a working channel. It's not terribly important. Fine. Let's just leave it as up for now. Okay. And then what else are we going to need? We're going to need a bunch of futures. We'll get to those in a second. Exist status is probably just going to be a number. All right. Yeah. So this is the basic layout that we have a session we know is going to be essentially just a thrush connection. Right. Thrush doesn't have a session like the original one does. Yeah. So a connection DRFs into a session, you should think of a connection as a session with the handler. So the handler is the thing that defines all the callbacks, like what happens if you get data. So in our case, a session is going to have a connection that's going to be a thrush client connection. Connection has a bunch of fields. In our case, the session that we have is going to have some stream that it's operating over. Here is called R. I wanted to go S because it's a stream. Let's read and write. That's going to have to implement async read and async write. So we're going to read async write. So this is going to take an async read and async write and the handler. So notice how the connection is over some kind of stream and then a handler, the handler, this is this callback thing that we're going to have to implement. We want to use our own here, right? Because we are going to have to decide what happens when data comes in. Essentially, we're going to trigger futures to resolve whenever we get data. And so in this case, we're only generic over the stream. And then the handler, I think, is going to be self. This is probably going to come back to... Okay, let's think about that in a second. We could have this be a separate thing. Okay, so on session, what kind of methods do we want? So we want a new... So one thing I really want here is implied bounce. How do you get the search bar at the bottom? Oh, for Firefox? That was a bit of a pain. So I had to write this stuff. Basically, would you be comfortable splitting your screen the black to white to black background? It's a bit blinding. Sure, I can probably do that. Let's do it this way, this way, that way. Sure. Is that better? I can do that. Yeah. Okay, so one thing I really want here is the rust implied trade bounce. Yeah, so this is landed now. So maybe I can get away with this. It's not stable yet. So I sort of want this to all stick to stable. Essentially, here, because the struct declares that s has to be async read and async write, currently, you sort of have to do... You have to redeclare async read plus async write in every input block, whereas implied trade balance are going to essentially do away with that. All right, so on a session, we want new. So new is going to take some async on a stream, and it's going to essentially establish an ssh session on top of that. The webcam hides his code. I think that's okay with the browser being on the right, you can now see the code. It doesn't mean that it'll hide parts of the web page, but I don't think that's really a problem. Okay, so this is going to give you some kind of a session future. So let's do it. This is going to give you a session future of s. And that's really all we're going to have on ssh. No, so what this session future is going to resolve to is we're going to have a struct session future, and we're going to impel future for session future. And that's going to resolve into a session of typus. The error is going to be a, which means that this is going to have to be async read and async write. So we're going to have a different poll here. We'll get back to that later. So the idea here is that you have some existing TCP connection or something, and you want to establish an ssh session on it, you call new, that gives you a future that's going to resolve into that session having been established. And then once you have a session, you're going to authenticate, which is going to give you something like a pubfn authenticate key, which is going to generally the pattern you see with futures is that most of the methods consume self in the call. And then when the future resolves, it gives you back self. And that's probably what we're going to do in this case. Part of the reason we're going to do that here is because the, so thrush is a little bit weird in that, and actually the original SSH2 as well, in that if you have a connection, you can't do anything on it until you've authenticated, but that's not reflected in the types at all, right? So you can call like channel open on a session on a connection that hasn't been authenticated yet. So what we're going to do is we're going to have like an authenticated session type and all the real methods are going to be implemented on that. And so the way we're going to do this is the session future you get back from this is going to be sort of an unauthenticated session. So actually maybe that's where you should call it, but let's not do that for now. So authenticate key is going to take an unauthenticated session and a username and a key. What do we determine key had to be a thrush? Oh, thrush keys. Oh, this is where we get them. Great. So we're going to need this crate to this 0.9. So it's going to take a thrush keys key pair. Yep. And it's going to give you back a session future. But this is like an authenticated session future of us. And then so we're going to have a pub struct. Actually, let's just copy this. So we're going to have an authenticated session future, authenticated session future, which gives you an authenticated session. Arguably, we could flip these names and call the other one unauthenticated because most of the time you will have a session. So actually, let's do that. It's probably better. It's called a new session, maybe fresh new new is short and nice. So this is going to be a new session future. A new session future s gives you a new session. This gives you a is a session future that gives you a session. That's much nicer. The session. This gives you a session. The new. So if you constructed over a TCP stream, you're going to get a new session future. And then when you call authenticate key, it's going to give you a session future, which is then really an authenticated session. Right. Yeah, we're going to have to want to split this into files at some point. Okay, so authenticate key is going to give you a future that resolves into an authenticated session. And then on an authenticated session, what can you do? So if you have a session, which is then authenticated, what we really want is on that session, you can call for example, say, right. So at least for now, all that's going to be on session is going to be pubfn exec. No, sorry, channel open. That's where the channel is coming. Channel open, which consumes self, maybe we can think about this. Actually, maybe what it does is it takes an RSE, but we'll deal with that later. So this gives you a channel future. Over us. And so the question is, what is the channel future? Well, so you see the song and dance, you end up going through with futures a lot is you have to have a type for the future. And then you have to have a type for the thing that the future results to one thing that's going to be pretty nice is once we get impulse future, I could just hear say, or infiltrate, I mean, we could say something like this, right, which is a lot nicer. Arguably, should we just I really want to infiltrate, but actually, here's what we'll do. It's a little bit sad, but we're going to box all of these. Yeah, because this saves us from writing a bunch of these symbols would be a lot nicer when writing the library too. Yeah, exactly. So the docs as well are really annoying. If you have all these future types lying around, whereas if things just say it returns a future, that's all you really need to know. Right. So this is going to return a box future where the item is a session is a new session. The authenticate key is going to return a box future item where the item is session and the error we don't know yet. It's here to a channel open gives you eventually a channel s. Why is my no package name? Yeah, and RC made it went on a rant on on Twitter the other day how all crates should use dash because cargo knows how to convert them to underscores. But there's still a bunch of crates that have underscore in their name, which is unnecessary. There we go. Auto code formatting so much nicer. Okay, so channel open is just going to give you a channel. And once you sorry, it's going to give you an open channel. Open channel is also going to be generic over s. This is going to haunt us for so long. Let's see what that means. What's the value of the interim new session? Why not go straight from session plus key to authenticated session? Okay, so the reason we could have a shortcut for that like new with key. But the intention is that there are many ways to authenticate an SSH session. Like imagine you might want to be able to authenticate with username password instead of public key, right? And so in that case, it's nice to have this new session thing, and then you can choose how to authenticate it. And regardless of how you authenticate it, you end up with a session that is authenticated. Is that distinction makes sense? This is similar to what the other SSH libraries do. But we you're totally right, though, that this is a little bit awkward to work with. So one thing we could do is we could have that on a session, there's like a new with key function, right, that takes a stream and a username and a key and just resolves directly into an authenticated session. We could totally do that. But let's leave that for a little bit later. Okay, so if we have an open channel, so notice that currently we haven't really done anything about futures, right? We haven't implemented any of the logic that we're going to need, we're just like flushing out the API to match the example. This I found that building the library this way sort of outside in helps a lot just to keep track of which things are going to be hard and which things are not. And also so that you don't get lost in the weeds with and then you end up with a terrible API. So an opened channel is only really going to have one method and that's exec, it's going to consume self. And again, it's going to give you a box future where the item is channel. And the error is something great. And then we have to channel which is our sort of ultimate thing and channel in theory won't actually have any methods. Oh, that's not true. It's going to have exit status. Exit status, that's going to consume self. It's going to give you a box future item is like a U32 or something. I don't remember exactly. Actually, we should check connection. Handler, exit status, U32, great. Okay, so it's going to have you a future that gives you 32. Note also that we don't return the channel when you ask for the exit status, arguably we sort of could. But we're going to just not do that for now. The one thing that we will want though, is if you open a channel and you consume the session, you really want to get back the session in addition to the open channel, right? So this is a pattern you'll see a lot in in future space code is that you have some function that consumes self. And when the future resolves, it gives you back the self and also the value. Usually you want to return the self in the error as well. So if something goes wrong, the user code knows that that call went wrong, but they can still keep using that self. The other way to get around this is for us to essentially take RC self, because that way you can just clone a session pretty easily, which might be what we end up doing here. Well, let's leave that for a little bit later. Crucially, the thing that we're going to want is we're going to want to implement async read for channel. And we're going to have to implement async write for channel. And this is where things get tricky. So async read and async write are actually just marker traits. So if we look at, oh, microphone is in the way. Let's go here. So if you look at async read, it doesn't actually have any methods that we need to implement, right? Notice how all of these methods are implemented. In fact, it's a marker that an IO object is non-blocking. Essentially it's saying that you have implemented read, but with the following semantics. If you return okay, then you read that much data. That's fine. If you're doing an error, then in general, that means it was an error. But the exception is if you return an error whose kind is wood block, then a, it means that you couldn't progress. But perhaps most importantly, this part, right? It means that you have done something that will mean that in the future, the task is going to resolve. So in order to understand what this means, let's talk a little bit just about how futures work internally. What happens when you, when you work with a future is that every now and again, or I should say at some point, something will call poll on that future. So some executor is going to call poll on that future. And what poll does, or rather an executor calls a task that calls poll, but this is not important. Poll will be called on that future. So if we look at, oh, that's annoying. Futures, this one. So if we look at future, notice that future has one method that we need to implement. So this poll method is sort of the core of most of the future stuff you see. It has a lot of, a lot of code in it. But the basic idea is that a call to the implementation of poll should not generally block. What it should do is it should either immediately return some stuff if it's available, like resolve the future if it can. And if it can't, it should return like not ready. So notice how this poll thing is basically a result that is either an error or an async, where async is either ready and a value. So saying the future completed or not ready. If you return not ready, then you need to ensure that poll will be called again sometime in the future when the data is ready. Otherwise, you'd never get the real value out. And so the way you do this is that, where is this? Yeah. So if not ready is returned, then you need to somehow register. Basically, what's going to happen is if you return not ready, your caller is going to mark that task as not being able to continue yet. And something needs to mark it as ready again. And so what you're saying when you return not ready is that I have scheduled something for the future such that I will be marked as ready. So in our case, what that will mean is that, or in the case of read here, is that when you call read, so we're going to have to implement this method. When you call read on this particular type, if it returns woodblock, then at some point in the future, something will happen. In our case, some data will be sent by the server to the client that will cause me to mark myself as ready again. And at some point in the future after that, poll will be called again, and we can return that data. So in our case, we will implement this marker trait. We will also use the IO prelude, and then we will impole read for channel. We will also have to implement write for channel. So if you look at, so that was async read, if you look at async write, it basically has the exact same thing, except also a shutdown method. A shutdown method, which is basically a way, so if you look at it here, shutdown is a way to say that I'm not going to write any more data. This is useful for TCP streams, for example. But let's just ignore that for now. The important thing is just like async read, async write is an implementation of write that follows a certain spec. And in our case, we have to implement the following two methods that are from IO write. And we're going to have to implement them in a non-blocking fashion, right? Okay, so the question is, and this is going to be essentially the core of everything we do today, is going to be how do we do this? And how do we do this? And so I think what we're going to do is we're first going to deal with reads, and then with writes. The basic sort of issue that we run into here with reads is there's no way for us to really read from the underlying SSH connection, right? So this brings us back to the very beginning where I pointed out that the way a connection works in thrush is you register this handler, and methods will just be called on this handler when stuff is available. So essentially, our read is going to call poll on the underlying SSH session. And poll on that underlying SSH session is going to do some stuff, maybe call into this handler, which may call data, maybe for an entirely different channel when we're polling, and then read needs to extract that data somehow. So the way to think about this is that we're going to have to have the session keep track of the data it's received for every channel. And then we're going to have the channels drive the SSH connection forward until such time as some data is available for them. That sounds convoluted, but let's see how it actually plays out. Ooh, we can get rid of these futures now. That's great. This goes away. This goes away. So let's do these things in order, specifically session is going to have to store. Well, let's see. So I think every channel is going to need a reference to the session. The reason for this is imagine that someone tries to do a read from one of the SSH channels. That channel can't really do anything normally, right? It needs to have access to the SSH session so that it can drive the SSH session forward. Driving the SSH session forward is basically causing it to do some underlying stream reads and writes, and hopefully some data is available for that channel. If it isn't, then we mark ourselves as blocked. So every channel is going to need a pointer to the session. And what this means is that we will probably want all RLR channels to contain a reference to the underlying connection, which basically means they need a reference to session. You're going to see this a lot in futures code, where the futures that you spawn off all need some way to refer back to the thing they respond off from. So think of like a MySQL session with the same problem, right? If you have a future based MySQL thing, the futures you spawn off need to be able to point back, like if you spin off a future that's like a prepared statement or something, it needs to be able to refer back to the underlying MySQL connection. We could here choose to use ARC if we really wanted to. If we wanted to enable you to use channels from different threads, we could have like an ARC mutex of the underlying channel. In our case, I don't think we really want that. It's unclear that you are going to do CPU intensive stuff. So you want like on the different channels. So you want to be able to pull them entirely independently. Maybe that's something we do in the future. But for now, let's do this the straightforward single threaded way, which is more sort of in spirit with asynchronous rust code in general as well. So the channel is going to have a RC to the session, which means that we need an S, which means that this has to be generic or S. And because S is in session, it needs to be async read and async write want implied bounds. Okay, we're using RC. The basic plan here makes sense that we're going to have the channels sort of drive them drive the connection forward. And the connection will sort of trigger any channels that are ready whenever it receives data, I guess we're about to find out. All right. So a channel has a reference back to the session and also has a channel ID, which is a rush. What is this type? Oh, that's awkward. Where is client ID? Client? Is there a channel ID? What? Can I not refer to a channel ID? That seems unfortunate. What is this type? And why is it not linked anywhere? I guess we're going to search for it. Why is this hidden from the docs? That makes absolutely no sense. Okay, well, we're going to need one of these anyway. So the channel ID is just going to be a channel ID. I wonder why. I wonder why that's marked as hidden. It's like someone intentionally chose to hide it. I'm not entirely sure why. Maybe they wanted to hide the inner field. The inner field is not public anyway. Oh, well. Fine. Doesn't really matter. So we have a, I guess we have an ID. We have a reference to the session. We might need some other stuff for like buffering reads and writes, but let's deal with that later. And so the basic structure for read is going to be something like we're going to do, this is going to have to be a ref cell. So specifically the underlying session, we're going to have to mutate, right? We're going to have to say it's going to keep some buffers of data for every channel. So if you call poll or read on one channel, that channel needs to be able to modify the buffers in the session. And so the way we do this is with the ref cell, which gives us interior mutability. So I really want to never have to give you, except for traits, maybe overly hopeful, automatic imports. Someone write it. All right. So what read is going to do is it's going to it's going to call. Hmm. It's a good question. So it's going to call poll in the underlying session. So we're going to do session dot mute poll. So sorry to just to give you some context here, the oops client machine being so. So this is a little bit weird, but the entire connection in thrush implements future. And we'll just like eventually resolve. And we'll resolve, I don't like, this is not particularly well documented in thrush, but the basic idea is that as long as you drive this future forward, it's going to call up to the handler, read from the underlying threads and whatever, but you need to drive this future. Otherwise, nothing happens. And so the way we're going to do this is we're going to call poll for the underlying stream and we're going to see what happens. So I think it's basically guaranteed not to this is basically guaranteed not to happen. I think all we really want to do is this, we want to call poll. And of course, if if poll finished here, so use futures, yeah, futures async. So if it returns ready, then there's nothing like we are guaranteed to get nothing more from this channel. And so at that point, I guess we return okay. This means that the server just went away. Which means that we return zero. Yeah, okay, so yeah, the the future resolves if the connection is if the server is close to connection, in which case we want all the reads from our channel to just return, there's no more data. Actually, that's not even really true. The way this is going to work is we're gonna have to drive the underlying ssh connection. But we want to like, we don't know whether driving it generates more data for us. And so I think what we have to do is we have to get a mutable reference to the session. We have to check whether there's any data in that session available for us. So you do something like match. Okay, let's just use some pseudo syntax for now. State for if there's some data D, then we're gonna like copy into buff from D. Again, this is not real rust. I'm just like sketching out what it is that we're gonna end up doing. It's closed, we're gonna do some other stuff. And then, so if there is some state available, if there is some buffered state available from this channel, then we do whatever that status tells us to do. If there is not, then we need to pull the underlying session to try to drive the connection forward. If it finished, then all we can do is like return that the connection closed, which means that your read must have finished. I guess technically, it's a little bit weird actually for the connection to resolve without error. But okay, we'll just mark that as the read finished with no errors and no data. If it returns not ready, then this means that it may have done some stuff. I guess here it may also have done some stuff. See, let's do this. We're gonna do results. And then we're gonna say if the result is an error, then there's nothing we can really do. Then we have to return an IO error to other I guess. It's a little bit unclear what we do here, but I think we basically do this. If it's okay, then we need to check the state again. And then if there's still no state for us, so in this case, this would return. And this would return. And if there's no state for us, even after pulling the underlying stream, then we know that now the stream knows about our channel wanting to read. And so it will spin us up. If some data arrives, at least that's the goal. And so here we can just return woodblock. Woodblock with, what can I put here? Like this maybe? And I don't think we need the loop. This is basic flow, makes sense. Again, note that this is not actually real Rust code yet. We don't have the state for you can't index this way. But the basic idea is that we're gonna, if someone chose to call read, it's either because data is available, like we've been notified that data is available, or it's because data will be available. In fact, arguably, we could pull regardless. Is the other thing we can do? Let's do that. We could always call pull. It's always safe to call pull. We're gonna pull the future, which may pull in some more state, fill some more buffers and whatnot. Then we're gonna, if she even if it's an error, it's a good question. Does poll require us to not call it again after, after it returns an error? Are you not allowed to pull again? Once the future is finished, it is considered a contract error to continue polling. Okay, so we're not allowed to do that. That's a little awkward. I think this would have to do something like Error with and like keep track of what that error was. Yeah. So if an error happened during poll, then we keep track of the fact that it's errors so that we can return that error from the other so that we can so that all the other channels know to terminate. So in this case, actually, we want to wake up all the other channel. So here we're gonna do something like wake up all other channels. Because at this point, because there's going to be no more data, there are going to be no more events because the connection like terminated. So we want all the other channels to also call read so that they can return that error as well. I think this might require that the error is clone, which is going to cause us some pain. No, no, I think this is fine. Okay, so if it has aired, then we keep track of that and we wake up all the other channels. And then we check whether there's data for us. So I think what we can do here is we can do dot remove. So we're going to take any state that's available for us. So this would be any buffers exit status. I guess there's no really exit status here. So if there was data available, so I guess we're going to end up with some enum here of channel state. That channel state is either going to be there's some data available for this channel, or it's going to be this chat may actually maybe it's just data. I think actually the channel state is going to be that holds some data. So if there's some stuff available for us, and if there's not something else, okay, so if there's some state available for this channel. So this means that the data callback has been called on the connection with a channel ID that matches this one, then we want to sort of copy as many biases as we can. What's the thing for copying slices? Great. Okay. Arguably. So in this case, then we're going to do buff dot I don't really want clone from slice. I want copy from slice copy from slice. We're going to copy from I sort of want to think that only copies whatever is the least end. So remember, buff here could be shorter than the amount of stuff in state. So I think data here is actually going to be a VEC of restart to read. Yeah. So the amount of the maximum number of bytes we're going to read, or the number of bytes we're going to read is going to be the min of buff dot len and state dot data dot len. I'm going to explain why I have this later. Let's have it be that way. It's whatever it's less of these two. And then we're going to copy into up to N, copy from slice state dot data. Great. Do it. Then we read that much data. So in theory here, we're going to like return. Okay, we read this many bytes, right? In reality, what we want to do here is if we just returned N now, then we just removed all the state for this channel. But there may be more bytes left in that state that might be like, we might have to call read again with a larger buffer or something, right? So in this case, I think what we do is, so this is going to have to be mute. So we're going to say state dot data is equal to do this the really stupid way first, and you're all going to be sad. And then we can have this later from state dot data dot n, sorry. And then we're going to say we're going to put back self ID and it's state. And then we're going to return, right? So the way this is, the way this is shaping up is we take the states associated with this channel, we copy out any bytes that are there, and then we put the remaining state back, right? This is a little sad, right? We don't really want to have to reallocate a new vector whenever we read data. And this is why the way to get around this is we're going to keep how far into the Vec we have read. And so we're going to just do state dot data dot I think this might the compiler might deal with me for this. I don't think you can modify elements directly in a couple. So we're going to add n to that because we just read n values from state dot data. It's going to be Len minus state dot zero. That's how many bytes are left in that buffer. And now we don't need to modify data at all. The thing we can do though, is if state dot data dot zero is equal to state dot data dot one dot Len. So if we read all the stuff that's in the vector, then now we can set state dot data dot zero, set this to be zero and a new vector. The reason we do this is otherwise you just like keep increase you never throw any of the data away even after it's been read. Because you'll just keep incrementing this first U size and whenever you get data for the channel you just keep pushing to the vector or extending the vector. And so we just keep growing until there's no more data here. We sort of detect the case where we've read all the stuff that's in the buffer and if that's the case then we can just throw away the buffer and then allocate a new one. In theory here what we could do is just set the vectors length to zero instead. Just like truncate it. That way we don't have to reallocate a vector. So maybe that's better. State dot data dot one dot truncate. Is that right? Actually we can just do clear. Yep. The borrow checker is going to yell at me here. But what distro are you using? I am using Arch Linux. Yeah so we modify the state that's been stored for this channel then we put it back. So the tricky part here is here. No this is just if there's any state available for us then we do this. If there is no state available for us then we're essentially going to have to block. Right. So if there's no so we would have blocked and this is the same as from poll returning not ready. That's essentially what the the contract on async read says is that if the error kind is would block it it's essentially the same as not ready. So in this case we need to ensure that if some data does become available then we're told why remove taking ownership of the state and then put it back. Why not that immutable borrow. Sorry sorry so we could do that. The reason I didn't want to do that is because gee that's a good point. We could the reason I originally thought to do this was if there's no more data we could just not put it back. Basically we could do like this. Does that make sense? But then I changed my mind and did the clearing instead. So now you're totally right. We can just do we'd have to use the entry API but that's fine. So imagine that if there's is no entry in this map for this channel ID right. I guess we could we could make it so that we guarantee that there's always an entry there. Yeah that's a good point. So let's just do this. So the reason okay so the reason I wanted to do this is because we need to be able to distinguish between there is no data available for this channel and it has been closed and there is data available for the there is no data available for this channel but there might be in the future because those are different right. One of them signifies that the stream is closed. The other means that the stream it just doesn't currently have any data. So in one case we would return would block in the other case we would not. And so the way to distinguish this in this case would be all right I guess we can't really do that can we? Let's see. So I think the way to distinguish these would be we'd have a finished flag and now now you're totally right. Now we do get mute. We get a state which is a mutable reference to the state. We do some things and now the thing we need to be aware of is if there's no more data left and sorry if if we read no data and there is and the channel has been closed then we return okay zero right. So this indicates the stream has ended. No it's a good question it's a good question and in fact it caught a bug that I was about to make so that's good. So in fact here we actually now know that there's a state so we could do it. So what we're going to do is when you establish a channel we're going to put an entry into that state and we're just not going to remove that entry ever. Might be sad later but I think for now it's fine and so that means that we know that there's always an entry here so we could say no entry for no state entry for valid channel. That way this can go away. Yeah so the idea is we read out any data that might be there. Yep we read out any data that might be there and then if we read no data and the channel is marked as closed then we return okay otherwise sorry if if n is zero then if the channel is closed then we return okay zero. Otherwise this means that we try to read data we read no data and the channel is not closed and so this is the case where we would have blocked. Otherwise we do we just return okay. Which I guess is the same as saying if n is okay and not state .finished then we would block otherwise we just return okay and yep and now of course the thing we need to be careful about is if we return would block we need to ensure that the the current tasks of the task of reading will be resumed at some point in the future and so we're going to have to do something like uh state .notify. That's going to be in okay so um this gets back into the whole like task system which is how futures are driven so think of this as um we need to do something I think it's like futures future current task something we'll find out exactly what but this is basically a way for us to tell someone else who happens to notice that this data for this channel that they're that this task should be woken up. We're gonna have to maintain that later so we'll see how that works out. Okay so let's uh find the futures docs um and look at futures task which gives you a task yeah so oops so we're gonna store an optional reference to futures task um very uh this language looks cryptic I can't believe this is the all-time most like program language uh so this code is particularly cryptic because futures are still a little bit of a mess um it's not most code is not quite this bad very soon the beginners serve for the obvious question but you use something like cargo watch to live build your files I do not I have not yet tried to run this code um in general when I when I build things myself I will mostly use cargo run manually um cargo watch doesn't really give me anything because I get um compile errors I usually just run cargo check it's like efficient enough I don't I haven't generally found myself wanting to have it constantly be compiled okay so the intention is that uh if we would have blocked we need to store sort of a reference to our self uh current so that if data ever comes in we notify the appropriate threads uh so what we're gonna do okay so the question is who's gonna wake up this task right in the imagine that we're someone calls read on us uh there is no data available but the channel also isn't closed as we store a reference to our self and then we return woodblock who's gonna wake us up right in the beginning there's like there is no one who's gonna keep calling read right the intention of the whole future system is that um something outside is gonna mark things as no longer being blocking but in our case there is no other thing the only things there are are reads and so what we really want here is um if anything calls poll on the underlying connection actually this is gonna be pretty weird yeah so the the concern here is that if we call poll here for a read on one channel that could totally end up reading for a totally different channel because remember all poll does for a connection is that it um it just like reads from whatever underlying ssh connection it has and then it dispatches to the handler and the handler decodes whatever channel id is in there and there's no guarantee that that red channel id is the same one as for the channel we're currently reading from it could be from a totally different one in which case we need to unblock that other channel um and so i think the way we're gonna end up doing this is um it's gonna be a little bit awkward to do that here because imagine for a second that uh there are there's no data being sent by the server right so in that case um all of our reads would return woodblock and what would ever trigger anything to wake up any of those futures right think of this from the context of whatever thread is executing this code right it called read on all the channels it knows about and at that point all of them said they would block and now there are no things to call read on or to call poll on essentially we needed to keep calling poll for us to eventually get data it's a little awkward um the way to do this actually uh it's a little bit sad but okay so the way to do this is to use um tokyo core is what i want do i have a tokyo core here no that's too bad um essentially what we want is something to just continuously uh try to read things from the connection right from the overall association connection we just want that to spin and we don't really care about what it ends up um resolving us so the way we do this is we use spawn uh so if you have a handle to uh sort of a tokyo executor it's not really an executor it's a core but think of it as something that executes futures if you have a handle you can spawn a task and what that means is that future is just like always going to be pulled independently of everything else and i think that's what we're going to end up doing here which is a little bit awkward because it means that the session you construct yeah that's pretty awkward we're gonna have to spawn that we're gonna have to spawn the connection let's see so we're gonna have a handle to some executor here tokyo core reactor handle uh so we're gonna take a handle and what we're going to do is hmm it's a very good question we basically want this on the connection don't we yeah i think this is going to be slightly different so whenever you authenticate a session once the session has been authenticated then uh so we're gonna have a session i guess here um when that session is created it's gonna also need to get a handle to the current executor um yourself and once we have the session we know that we're gonna have to keep pulling the underlying future the the connection future because if we don't if we don't keep pulling it it's not going to call up to our handler it's not going to give us any events so we want that to just continuously go on in the background um and so the way we're going to do that is we are going to call handle the spawn and we're going to take self dot we're just going to keep pulling that yeah uh what are we gonna have to do what map error gee no let's do that okay we're gonna have to figure out exactly what that works but essentially the the handler is going to be i guess we can just implement handle right now the handler that gets all these callbacks is going to walk all of the futures that have been registered so all the channel states look for anything whenever it receives data we'll see whether anything is waiting on that data but checking the notify and if it is then it will notify the appropriate channel right so up here we're going to implement where's this thrush handle uh let's thrush keys handler so we're going to implement thrush client for some type i don't really know what i call it it's um it's going to have to implement a bunch of things um i don't think we actually need all of these in fact i will use this stuff from the original example it's a useful proxy um yep um oh did not like that what am i going to check what have i lied about there we go that's better uh let's see so the handler this is what has all the callbacks um so there's check server key we're just going to blindly trust any server for now uh channel open confirmation i don't care about i'm happy to use the equal documentation oh sure yeah fine data is the one that we sort of care the most about right so here we know that whatever this x is it's in fact the x that's going to contain the channel states right because the x is going to have to walk these so it's going to contain the state for it's going to be a hash map hash map from the rush client id sorry channel id to a channel state it's gonna what else do we have here we have a erred width which is a actually no i think we'll just just skip that just make it a option unit or something for now um so when we implement this we're going to trust all of those that's fine we're gonna this returns a session for x uh so oh i guess really this is going to be an rc ref cell isn't it it's going to be some underlying like uh channels state we're probably going to call it something else um and then we're going to have this x this is the pattern that i said we're gonna see a bunch where we have an rc ref cell of channel state channels state right because this state is also going to have to be in the session so that you can establish new channels right um so we'll figure out names for these later it's currently they're terrible all right so when data comes in for a channel what are we going to do well we are going to say uh i think i want the other method too i think i want channel open computation as well i know that's fine okay so when you get one of these then we're going to do self dot state for dot get new channel dot expect uh dot beta for unknown channel because we basically know that um we know that we're going to register all the channels whenever we open them so we're going to have some state we're going to have access to this underlying state for that channel and then we got some data right so because we got some data we're going to set state dot these things okay so state dot data dot one which is the vector is going to be extended with the incoming data that we got uh it's not finished we do have to notify potentially so if state dot notify if let's um state dot notify dot take so we want to make sure that we only notify the task once because at this point like if this happens that means that there's data for that channel so if there's a thread waiting on it if there's a task waiting on it then we want to notify that task um so in our case what's the no where's the thing futures on a task you can call notify great so we just do task dot notify and so notice how this is actually fairly straightforward right we look up the state for the channel we add any buffer add to the buffer of data that's now been read by that channel and then we uh notify the task that is waiting and then we just say we're done great um the other things that we're going to care about here are um uh we care if a channel closes and we care if we get uh an end of file for a channel so we're going to implement those methods too so if a channel uh if a channel closes then what we want to do is we again want to get the state um so one question is what's the difference between an end of file and a close for a channel I think a close is implicitly an end of file so we're going to set finish equals true we also do want to wake up the task because imagine there might be some reader who read there were no bytes and now there are still no bytes but it would return okay zero because the stream has not been closed a close is polite an end of file is not I see wait isn't an end of file also something that server can choose to send on a given channel or is end of file like the entire stream ended no I think end of file is something you can send on the channel too um anyway okay so the I think the handling for these are is basically going to be the same I think the difference is if a channel is closed you can no longer send on it either so this will probably matter this is like the server said this channel went away um and so you can't send to it anymore either you can reuse the channel after an end of file yeah so after an end of file you can keep writing but after close you cannot keep writing so this is something we'll have to keep track of later uh given that we're currently just doing read I don't think it matters so we're going to do this um so end of file and closer currently basically the same I don't think you can exit twice on the same channel I don't think that's true I'm I could be wrong but I feel like in the ssh protocol spec that's not true but let's see regardless um are there any other things we care about here I don't oh exit status yeah are you believe we care about exit signal too but let's just ignore that for now oh these are of course all thrush things here that's not a client session that's a thrush client session this is also a thrush client session this is a thrush keys public key who knows I wish there was a default I guess there is a default for this that just denies all access to the zone all right um first client session first client session first client session okay so if we get an exit status then I think all we really want to do in that case is just set the exit status I don't think we even want to wake up the the task actually we will we will want to wake up once we implement the exit status future but let's leave that alone for now so we're just going to set exit status equals to exit status and then say like to do uh wake up exit status and then we finished where have I syntax errored all right let's see so now we've implemented this handler and this means that um we now at least know what the type here is going to be it's going to be x well let's give this a better name okay so so y is going to be a session state x is going to be a session state ref and we're going to implement this for a session state ref the reason for this is because a channel essentially contains a session state ref great so now we know what this is this is a session state ref and this is a session state ref fantastic yeah so the idea here is let's just walk through the the things sort of top to bottom again when you initially establish a new session what you do is you first construct a stream of some sort and then with that stream we're going to do a future that's going to establish a new session right so let's write this code now just because it's been sitting empty for a while um that code is essentially just uh the same as what client connect does and I think there's a client connect future if I remember correctly client connect future right I guess maybe just wait I don't actually need a connect I want a connection new that's what I want so this is going to be a thrush client connection new it's going to take a config of some kind I don't know what config is client ID limits the simple and default great our default uh and then we're going to pass in the stream we're going to pass in the handler so this is where we're going to have to create a new session state uh so state is going to be see the sessions date this should implement defaults I think we can even derive default here yep uh so we're going to create a new session state for this session that we're creating we're going to pass that state in here uh timeout who knows what timeout is timeout is an option timeout uh I don't think we care about a timeout for this so we're going to do none this is something that we may want to let the the users configure at some point but just for the basic API I think this is better we're going to have to box this until we get infiltrate um now authenticate key oh right so that when this future resolves I guess we can map um that's going to give us back a connection right so this result gives you a cell when you asynchronous so I guess there's no async version of that oh I guess the authenticate is really the thing that matters what does weight does so there's a what does session do session disconnect no no no this is important okay so so this isn't actually a future then it's a little awkward but it's fine so I guess this just gives you a new session just gives you a self a result itself over a rush error yep so we're gonna make the connection be this so that's going to be what we get back for connecting and then once we have connected so now we have a connection that we're going to end up establishing a connection on but that connection we want to wrap in a new session in fact I think we also wanted to have can I get the handler from this I can okay great so here uh we just do this instead this dog map new session I guess this goes here great so new is just going to take the stream and it's going to wrap it in a an SSH connection thing it doesn't actually do anything yet until we call authenticate um beautiful all right so that's new that gives you a new session a new session is basically just the connection nothing else you can even aida expand constructors yep the this this the syntax here is so nice it only works for tuples though a tuple constructor with a single element because um imagine the new session had two items in it like a u32 or something here and then even if this returned a tuple of two elements this would not work because this map is really over just a single value of type something right and the constructor is this so it only works when there's a single element tuple constructor but it is pretty nice uh you could of course declare a new or something if you wanted to okay so authenticate key uh this is where we're going to get it's going to get a little bit more interesting because authenticate I hope returns a future right authenticate is a future great yeah so here we're going to call so remember new session just has a single field self dot zero uh dot authenticate sorry authenticate key with the given user and the given key uh and then that's going to give us back we're gonna whenever that future resolves it's going to give us basically just going to give us a connection again right so if you look here authenticate keeps you a future that's an authenticate authenticate implements a future that gives you a connection so we just this just rounds back to a connection um but that connection we want to wrap in a session because now it's been established right uh arguably session is also mostly uninteresting session I think can also just be a tuple struct probably yeah screw that so this just maps with session beautiful so if you establish a new session then all you can do is authenticate and then you end up with a session right and now the question is what happens with a session oh right session this is going to have to be new uh this is very good all cool right so this is where we run into the sort of weird position where we now have an establish as a session right and we need to have some way of driving the connection forward so that all the callbacks on our handler gets called like receiving data and such so that we end up waking up all the channels that are sleeping but that means that we need to spawn something so spawn is sort of a way of telling tokyo hey if you have nothing else to pull then try pulling this thing it's sort of telling you about a poll without explicitly doing one if that sort of makes sense where it was sort of without the user having requested to wait for one to complete and that's what we're gonna have to do here so I think specifically a session oh yeah I think new session uh that's really awkward this is going to be a lot better actually in uh the new tokyo because in the new tokyo you can get a handle to the the current reactor let's see if we can do this actually in the current tokyo hmm like is there a way for me to get at the the reactor that is currently executing I feel like there might be but it might be on no that's too bad because in the new tokyo there it's like if we look at so the new tokyo crate is it's just called tokyo so in tokyo you now have reactor where is the oh maybe it's not in this maybe it's in prelude no spawn I don't really want it to be a thread pool though actually that's a good question I'm pretty sure there's a way of getting the current there we go current thread yeah so you can call this thing in the in the new tokyo you can call current thread and that gives you a handle to sort of like the currently executing thread that you can spawn things on so given a current thread you can then call spawn which is really what we would like to do here because currently we need to we're gonna need to have the user pass in a handle to the tokyo reactor they are using which is pretty awkward I don't think there's a way to do this in tokyo IO sorry in tokyo core unless I'm missing something no that's too bad okay that's fine so what this is going to mean is that when you create a new session you're gonna have to also give in a handle to a reactor which means that this is no longer a tuple structure which is a little sad it's gonna have the underlying connection and it's going to have the new tokyo replacement for both tokyo IO and tokyo so currently there is so the new tokyo is a replacement for tokyo IO and tokyo core although only sort of for tokyo IO the idea is that the new tokyo crate should be like most of the things you need for doing asynchronous IO and I think that includes all of the IO stuff this is gonna have to do a also a handle to a reactor now this is gonna give us a we're gonna return a new this connection that we get back into a new session of see and handle notice here we're using shorthand struct syntax so we don't you don't have to do this if the names of the variable and the key are the same you can do this okay all right so here we're gonna have to do a new session we want to extract the client in the handle separately then we're gonna call see on that map it with a move of the resulting connection we're gonna do the same thing session see great so now when we get down to this point oh actually I want this to be it's a good question I think this should be new new see handle and the new is gonna take one of these guys and a handle it's gonna spawn the connection I don't know what exclamation patreon does but my patreon is if you were curious about it is sorry someone posted in the chat something that looks like patreon this is my patreon and so here I'll post whenever there are new videos and such so you might want to subscribe to that yeah so the intention here is that we want something to keep running the connection in the background but we do also still need the connection handle to the connection so that we can keep like creating new channels for example kind of interesting this connection implement clone that's interesting I will probably not teach you to code from scratch I don't think that's I don't think I'm the right person to do that yeah this is a good question like we need to have the connection be pulled in the background but at the same time we need to continue to have a handle to the connection so that we can spawn new channels if we just spawn it entirely huh okay I think that the way we're gonna have to do this given that connection does not implement clone that sort of implies to me the connection internally doesn't have an RC or something so we basically need to wrap the connection in an RC ref cell future of our own that we can call pull on but okay sure so okay so what we're gonna do is we're gonna have a struct it's gonna be like a connection polar future thing it's gonna take an s which is an async read and an async write that's really awkward but okay that's gonna hold the connection arguably can be a tuple struct it's gonna hold an RC ref cell I wonder why the connection has to be consumed in order to do things like opening a session it's a little bit sad it's a little bit of an awkward pattern because the problem is that means that we can't also keep pulling it in the background um so that means we can't really spawn the connection because the because the connection actually requires you to consume self for every call hmm so that won't work hmm which actually means that we might not need this handle we're gonna have to find some other way of we're gonna have to find some other way of driving the connection if there's currently no traffic that's a very good question let me think about that for a second I think the way to do this would be to have we essentially need to force the users of our API to do the same thing as what um thrush is forcing us to do which is keep pulling some thing some one thing it's a little sad ideally the connection this connection would have a way to give us a tokyo handle that it would then spawn it's like underlying um sort of reading and writing from but given that it doesn't I think we're gonna have to force our users to also pull session which is a little bit awkward it does mean however that the handle can go away but it does make me a little sad uh so now a new session is just a tuple again new session this can now just be session because session uh no so I don't actually want to access the underlying network socket what I want is I want um I want the user the user of this library or in this case the user of the thrush connection to not have to uh do anything not have to pull the connection constantly or rather it's fine for it to be pulled but currently I need to ensure that there's always something that is pulling connection ideally what connection would do is it would just spawn itself onto a tokyo reactor so that all I need to so okay so think of this as here imagine that the user has a bunch of channels right they have a session they spawn some channels and now they just have like three channel objects or three channel things they no longer really have the session they don't care about the session they just have these three channels they want to read from all those three channels so they call like uh the join three or select or whatever the the appropriate futures call is right what that means is when you call core dot run using tokyo you call core dot run and you give it a future and it will run that future until the future finished right in our case that will run the future that is our channels right all those three channels might send some data to the server the server is gonna like do some stuff so it doesn't we we try to read and nothing happens so all the three channels that are the only things that are being pulled by that tokyo reactor all those three channels are sort of done right and at this point there's nothing pulling the connection so I think what we do is um how do we do that um I think we'll we're gonna have to force the user to pull session which is like a little bit awkward but I guess we'll have to deal with it okay so we don't really need new anymore um this means that we're going to have to implement future for session you know what this is even gonna resolve to I guess nothing I guess this will sort of be the same as the implementation of future for connection which is one of those uh error is that's the error we currently declare for our handler great so the problem is I can't spawn the connection uh because if I spawn the connection I have no reference to it anymore and give it channels um no not really because the thing okay so the the proposal here is to spawn a thing that uh has a bunch of sync channels or in my case I I actually want unsync but but has channels has rust channels to the ssh channels and sort of notifies them the problem is that uh calling any of the methods on connection consumes the connection and I don't have a way of cloning a connection if I did this wouldn't be an issue right um but the problem is I can't really spawn this because then I would not have no way of calling methods on it so I think what we're going to do is we're going to implement future for session which is going to lose poll here oh if I not implement a poll future's poll we're yet future okay so polling for that is just going to call poll on the underlying thing I guess this just means what we're all we're really doing here is we're uh check your patreon okay I don't think this is terribly important but oh cool thanks thanks a lot that's really great appreciate it um all right let's see so yeah I think this all this really does is it forces the the same API on the users of our crate as what the thrush crate is making us do it basically forces them to they can spawn the they can spawn the session future themselves if they want um so they can think of this as uh the the session they get back they can either use it to create more channels or once they have created all the channels they want to then they can spawn it off using handle dot spawn um and at that point all of their channels will eventually succeed in fact I think this is going to work pretty nicely it's it's still a little bit awkward um all it really means is that this session actually what does channel open return in thrush so channel open gives us a channel open which is a future that gives us the connection and the channel ID so we would have to make this return session channel have this do that just like keep passing along the session just read to end yeah so basically what we're gonna do is we're gonna get we're going to keep passing the session along so that at the end what the user gets back is uh the sort of session future yeah I think basically what we end up with is this so we're gonna establish the session we're gonna authenticate once the channel has been opened we know that we don't really need the session anymore so at this point we're gonna do handle dot spawn we're gonna spawn the session so now the session is guaranteed to be uh running in the background um and then now we can call channel dot exec that gives us the channel that's gonna do read to end which is gonna give us all in the data yeah I think this is basically the difference this this causes to the API right this is uh we now require that the user once they want to wait for a channel they need to spawn the session uh or they need to pull the session like it's up to them how they do it but but the session needs to continue being pulled which is a fairly awkward API but it will work in this particular case okay so where were we right so now the handler the handler is going to keep being pulled it's going to keep getting events which means that it should keep waking up our channels um I guess the other thing we'll want to do is if there's an error let's see so for handler how do we how are we notified about errors I guess channel close and channel end of file yeah okay so so there won't really be this um what was this error what do we call this error field erred width yeah so now our channels actually no longer have to pull um in fact all they will all that we do is they have a ref to the session state which is where we keep track of all the buffers um but they don't actually need a reference to the underlying ssh session they just need to they need to have a pointer to the same state that the handler is updating so that the two can coordinate right so in this case they have a session they have a reference to the session state and they're a reference to the channel ID if we pull the channel all it really does is it checks whether there's some stuff in the buffer for it and if there is then it returns that data otherwise it sort of stores the fact that it wants to be notified in the future right which is this line here um so it keeps track of its current task and that task will be woken up by the handler whenever it gets data or whenever the channel closes so I'm not going to keep checking my patreon I really appreciate it but I'm not going to keep checking my patreon during the stream um all right so I think that means that our read is going to work that's kind of neat I guess the one thing is when you open a channel you need to xx so xx is the thing that we're missing um so this open channel I guess needs to have a reference to the underlying state oh channel is no longer parameterized by s that's great okay so open channel uh is an open channel is going to have the same as a channel really so arguably we could have these just be type aliases for the same underlying type I'm sorry I'm not going to teach you to code um so if you there's no point in keeping asking um so yeah so you see this is the same thing we had with session right so new session and session are really the same type but sort of with a one of them has been established and the other hasn't so actually the real way to do this would be um we would have so instead of having these be different types they could be a different second type parameter so this is often done with you have like uh two unit types so like uh like new and established and then we would have session and then you have some some trait which is like session state which has no field you implement session state for new you implement session state for established and then you say that session is generic over the s that's the stream we already have and over a session state so you would do this and then you would have impulse that dictate they are only available in a certain state right so you could say uh impulse s whereas this is the stream and it will only work on new then you will have the new method or the sorry you would have authenticate key right and it would return a session s established and similarly the channel open open is only available if the ghost state in a sense is established right and the user can't change the second parameter except by calling the appropriate methods so this is a very common pattern you'll see but it does require a bunch of this extra override just to get this extra trait this extra type in place and ideally you should uh sort of seal this trait so that other use so that users of your library can't implement this trait themselves because then they might like invent new session states and whatnot um but this is a very common pattern I don't think I want to do that now I think I'm fine having new session and session be different types because they're sufficiently simple open channel we can do the same thing with essentially having like a channel type that's generic over a channel state maybe we'll do that later in fact maybe we'll do that in the conversion to Tokyo but for now we're gonna just leave it the way it is so an open channel all you can do on it is really call exec um and exec is on uh this we don't need this later this we don't need this we don't need or this or this or this is cleaning up a little all right so on connection uh where is it yeah so on connection uh you can call wait channel open in fact how do we call exec that's a good question exec huh interesting so I guess we don't actually know when the program has started executing that's kind of interesting uh what's a sealed trait oh yeah um this is if you look at the Rust API guidelines so these are super useful in general um and you look at future proofing this is a thing you could do it's essentially a way of making a trait that no one outside of the trait can implement so you essentially make it depend on a trait that is not available outside that's in a private module it's a little bit of a hack but it essentially means that you can modify you can add methods to this trait without it mattering and you can guarantee that no one else can implement that trait in fact maybe not maybe we can't do that maybe we can just do this with a private trait actually you might not need to seal it sealing is so that um no no sealing is what we would do in general this is really nick uh I found another trick that I mean this is unrelated to all of this but the uh so this is also really neat this is some of the one of the things that they're doing with the Tokyo and future is great to ensure that if you have code written for the old futures or old Tokyo stuff it will still work in code that uses the newer Tokyo and future stuff so this trick is also really cool but it's sort of unrelated to what we're currently doing the API guidelines are super useful all right so exec exec exec don't need to have this yeah so so it's a little bit interesting that exec just seems to be like some it just seems like a method that doesn't give me anything back so I guess we don't really know when the function has been executed or has started executing rather because there doesn't seem to be a oh channel open failure we might need to deal with um yeah none of these signal the start of an exec so I guess we just like call exec and then are happy maybe and this happens to target session so this works because connection derefs the session right oh interesting all the session types just take mute self and self so we could huh so this means that we could implement uh maybe maybe we not maybe we don't need to do this trick um so notice that uh on connection channel open does consume self but on session channel open does not consume self but it's synchronous some immediately if the connection is authenticated yeah I think this is the way we'd get around that is we would call channel open session and then sort of deal with the channel the channel opening confirmation in the handler ourselves and that way we could spawn the connection because we wouldn't need any of the methods on connection except made data this data also on session yes okay so in theory we could do that uh but let's leave that for later let's do the basics first um okay so exec oh that means exec does need a reference to the connection so channel open is going to take some mute self and it's going to do oh we didn't implement channel we should do that first probably uh that's going to be self dot channel open session actually self dot zero dot open session dot map so that gives us a channel sorry that gives us a session and a channel uh in particular look at what this is so channel open is a future that gives us a connection and a channel ID this is going to give us a channel ID uh and so once we have the channel we want that to turn into a future and that future is going to give us an we we want a session and an open channel we have a session and a channel ID so all we really need to do here is we need to make the channel as an open channel that's going to have oh yeah so here we run into this issue again of we're going to return the session but we also want the open channel to have a reference to the session so that it can call exec but we need to return to an owned version of the session that's pretty awkward so I think what we're going to do is actually uh open exec we're going to call this we're going to not have open channel in the middle it's a little bit sad but essentially we're running into the same issue of connection not being cloned and not and having all its methods consume self um this is one of the reasons why often uh future-based types and rust especially things like connections are cloned usually by having an rc ref cell inside them uh okay so instead we're going to have command which takes a string and that's going to give you back a channel so we call channel open session gives you a session and a channel ID then we're going to call uh session dot exec this is where the sort of awkward part goes that exec is a function of connection well of of session but through connection uh so session dot exec what else does it take you want to reply I don't know what want reply does I wish more of these arguments were like documented but uh it's true we probably want to reply right uh the command is this and that doesn't return anything so now that we've called exec now that we've called exec the channel should be fully operational so we can now return this a channel which holds I guess this shouldn't really be called session it should be called state so the channel that we construct has a reference to oops to the state which we can get through session dot handle which we know is cloned right so remember how session the handler is a session state ref and a session state ref is just an rc ref cell which is cloned because of the rc and so therefore we can clone that here to give another rc to the channel um and then in addition to the state there's going to be the id which is the channel id like so actually this is going to come back to bite us for sending data over a channel we're going to run into the same issue so maybe the argument here is that we need to just use the session methods I think we're going to have to it's not great okay let's deal with that later so channel so we give back a session and a channel and uh this is compile sort of tokyo is also okay so that gives us a channel exit status uh we're gonna not implement quite yet right where we haven't implemented yet so currently it is just async right async read I mean this is gonna be futures I think we're gonna want futures async maybe not uh 134 futures we don't use handle anymore that's true uh this is a session state wait have we missed a method I guess let's do the other ones first uh thrush keys isn't that all it took am I misremembering something so handler is given a public key so uh this is a I think maybe futures is one of those things where we'll do use futures future seems like something we're gonna see how this works wrong number of type arguments expected no type arguments oh that's right channel is not generic anymore what method is it missing future sign what is future sign the example doesn't have a future sign but I guess apparently we need one just gonna be futures finished wait that was really weird future sign so it's gonna be whatever this returns uh but what does the default implementation do how can this well I guess it's just gonna be this this is here can I just do this why does the example for the crate not show a working example this is why we have doc tests wait like ah latest version nope latest version also does not have this that's for calling an agent the default does nothing yeah the problem is I have to specify it I don't know what type to specify here just a future finished works all right let's see 179 is this not implemented error so crypto back huh the rush keys where can I find crypto back so crypto back just like directly in this yes great so this no field state for 37 oh right so um remember how session so the handler is a session state ref um which is just all these pointers to the session state and we're going to mutate that session state so we're gonna have to borrow we're gonna have to borrow the session state while we do this updating right so we're gonna do state it's gonna be state dot the bar the state for on the borrow and we're gonna run into this on all of these right so this goes here here channel d line 40 that's because this is a hash map so it takes a borrow of the key that's a little bit sad this hash hash maps not allowing owned keys especially for copy types is pretty annoying all right so handler is not implemented for session state that's true this is gonna have to be okay so the problem here was when we create a new session this we give it a session state right which gives us a connection session state but we've only to define handler for the session state ref so we need to make this a session should get us pretty far uh 1.2 that's no longer necessary so we must be self dot zero what else do we have uh 131 so this is saying no method map found for that isn't channel channel open session on a connection returns a future right or am I missing something um let's see connection uh channel open session gives us this which is gives us a future of a connection and a channel ID I see that's only partially true this is we need to re-wrap this as a session or we really get back from this as a connection right c connection c connection c and then we need to reconstruct the session from that no method authenticate found wait I can only call authenticate key on a tcp session why only where tcp oh this is for shutdown oh I remember looking at this yeah so notice how most of these other functions are implemented for any stream that implements async read and async write and for any handler type but the authenticate methods require that the stream is also tcp where tcp is a trait that's defined just in thrush that defines a shutdown method shutdown is just a way to shut down the tcp connection it's a little bit weird for this bound to only be applied there but I guess we will have to deal with that so this is going to be where s implements thrush not ssh but tcp this much tight expected box found math oh right because we don't have able future yet we need to box all of these box this to every time 125 uh this is actually gonna return a thrush error is that true handler error currently our handler doesn't return any errors so that's fine we'll probably end up using the failure crate here as well for managing errors uh explicitly why is there any requirement for that to be static are there static requirements on connection no right I hope not no no so why does this require static oh it's for user no users should not be tied to this it would us need to be static for this to work or rather why does it not need to be static here presumably the oh does authenticate take a lifetime no there's no lifetime here wait this is really weird why is it requiring me to make a static huh I don't know all right let's deal with the others first um it's saying that map isn't found for channel open because but really but channel open is a future wasn't this the thing we just looked at oh it's yeah why can't I map on that let's give us some more space um for channel open the method map exists but it was but channel open does not implement future fairly sure that's not true so what is the bounce for oh it's tcp again uh this only works if it is also tcp I guess technically I could do the same thing here as I did there just avoid that so that should allow me to use it as a future um no method handle found for connection well that sounds true because it's handler oh right this has to be ours okay so this is a this is something you often run into when using rcs is you if you have a reference to an rc and you want to clone it then if you call just dot clone it will um as in this case for example it will just clone the reference to the rc which is not what you want with an rc you generally when you clone it you want a new rc right the points to the same underlying thing and so the way you get around this is you use rc colon colon clone here so if you look at this rc it has uh colon colon clone should have colon colon clone yeah so you use use a universal function calling syntax to explicitly say that hey I want to clone the rc I didn't want to clone the the reference itself so that should clone that expected rc ic actually we're running into a slightly different thing here which is uh this isn't clone now this is clone session channel oh this is the handle error thing yeah so this actually doesn't have no error it has uh thrush now we're getting pretty close parenthesized parameters may only be used with a trait 155 I can't call pole this is tcp again isn't it plus uh thrush tcp I wonder if it's really necessary for tcp to be for all these methods oh that's sad but all right uh we can't do why self dot state dot oh I guess the problem here is we really want to call borrow mute on the ref cell whereas borrow mute isn't a mesh method on state ref I think what we really do here is just uh infool deref so this is something I've really wanted to rest for a long time is a way to like derive deref for wrapper types at least I know there are other people who also want it so we may end up getting it at some point but for now we'll have to do uh rc ref cell session state deref takes a self gives a self item and it's just going to be self dot zero and now the borrow mute should work fine incorrectly syntax what wait what did I do oh how about that to use the implementator it hurts uh more info about the annoying tcp thing at a link let's go to that link see where that takes us let's subscribe to that and uh wait what did he say Tokyo I heard it news you see link above I guess this one oh I see so I guess this is coming at some point then okay that makes sense fixed in future 0.2 nice uh all the more reason for us all to use 0.2 as quickly as possible um why can't I do this because it requires a reference to channel ID okay how about now now lifetime issues 147 let's see uh the channel oh the command yeah so the problem here is the command is a reference to a string and we want to use it inside the future which is not called immediately right it's once the channel has been opened then we will need command which actually means that this future that we return has a lifetime tied to the lifetime of the command so this is going to be a future with this lifetime and that should be fine still complaining about the lifetime of the stream which I find super weird okay and 204 this is complaining about error oh what's a valid IO error type empty string I guess we I don't know what they want me to return here because it's a little bit silly like this is uh there's sort of in futures or in async Tokyo there's sort of abusing the fact that you can have a read method the returns would block and sort of indicate that as being the same as poll not ready but the problem is you need to construct an IO error which needs to take an error I guess in this case a static string is probably fine uh meta question what's the use case for line numbers to be based on cursor location uh oh so it's uh because in Vim you can do like delete the next 20 lines this lets me very quickly see where I would delete to so imagine that I like want to delete from here to the end of the function I can very quickly see that that would be 18 lines so I can do 18 dd that's the reason it's relative but I could totally see that being really annoying in uh in pure programming because you don't know you can't refer to lines the line number is shown in the bottom right let's see yeah so we need to give the string there that's fine because string implements error for whatever reason this business though why is it that the stream needs to be static or specifically why is it the stream needs to be static given that it's not necessary for the thrush connection to be static it's really quite odd I wonder if it's just that we need to do this but I don't think that should be necessary I mean arguably I'm fine with the stream being static just seems a little bit odd but sure I'm okay with that if it makes the compiler happy all right oh others uh ext is not used that's so ext so this is a whole other business where we need to decide what to do with standard error I think for now we're going to do something like if ext dot is none which is standard up then this is going to be standard error the standard error can probably follow a similar kind of pattern uh unused variable state oh right this is already up to do so we're just gonna wait no what do you mean state is unused oh right the return this is unused right now what uh bunch of mutability line 48 this is the mute when I call borrow mute and the resulting thing not being mute that it seems like something I would never want more of the same cannot move out of self because self is borrowed yeah so non-life school lifetimes would fix this issue instead yes to create a scope where that borrow lasts for want non-life school lifetimes so badly closure may outlive the current function but it borrows command it does not borrow command it moves the reference of command 160 c needs to be mutable 206 s needs to be mutable now the same thing 214 cannot move out of borrowed context let's see oh that's right um state here is just a reference so it's sort of like okay so so the thing here is we have a reference to a tuple right where the reference oh actually we can make this not be a tuple anymore we have a reference to a channel state this can be this needs to be state dot data start to state dot data start plus state dot data start um yeah so we ever we just have a reference to the to the channel state and we're trying to overwrite I just actually know this is now just fine because now there's not a tuple anymore state dot data start is zero this is still complaining about things oh there's definitely one more thing we have to do and that is when you open a channel we need to record its initial channel state so specifically uh hey no worries uh yeah I'll post it afterwards I'll tweet out the the youtube url and stuff and then people can come watch so all the video will be available after yeah so so remember how in our implementation of read for channel we assume that a channel state exists for every channel which means that we need to create it when the channel is initially opened so we do that here by saying something like uh state is going to be actually state handler is going to be c dot handler dot clone and then we're going to do state dot borrow mute uh dot state four and then we're going to insert channel id and some channel state I guess this is likely just going to be default a channel state can this just derive default I think no uh now it can what's a default for bool it's a good question uh should be false I think is it a woman default the debunk default that's okay false great I guess it's probably said here too yeah okay so now I think we can derive default here so channels oh data start nope fine uh input default for channel state default gives you a self and gives you a channel state of data start it's going to be zero and then everything else is going to be their default values what else are we missing remove this comma what it's not pointing to a comma oh that comma keep three extends the data that's fine uh exit status should be some wait what you're not allowed to do this I'm pretty sure you're allowed to do this probably not all right like new uh finished is false initially uh notify is none because no one is blocking on it exit status is unknown initially I did mean none you're right holy cow the library compiles now notice there's still a bunch of things we're missing right like you currently can't write to a channel in fact writing is going to be a little bit annoying um you can't uh get the exit status yet in fact that's something we will also want but like at least you should be able to read which also means that our example in theory should work uh do we need we do need my own so data dependencies uh my dogs dot rs my oh 0.6 which there was a way to just like say current and I would fill it in the first time I found it to compile a bunch of times and let's uh ignore the exit status for now I would just give us the data I just want to compile the example once uh can I do examples great uh line 20 oh that's right we needed to find a way to get the key what's the way thrush keys private key or sorry key pair how do I get one of these uh maybe from the root secret key but but in what format though I have a pen encoded private key but none of these things say oh pen but can it decode one thanks uh I don't know how do I let me check this key real quick uh it is yeah but load secret key is that sufficient that should be and more super okay sure let's try that so we're gonna do let key is I guess actually but uh we're gonna do thrush keys so we're gonna do thrush keys load secret key uh the path is gonna be this file and it's not gonna be a password and it's gonna parse right oh I guess we have to spin up a VM uh let's see launch an instance launch this is gonna be easy one of those uh the smallest you got wait that key all right instance let's uh see what you can do so we're gonna ssh to this place expected socket at it really they're not uh hey my oh you can do better than this right where's your net connect do I really have to they're not uh I have to call parse really dot parse and then dot unwrap and then a ref to that line 24 is it not just oh it's new session new now that's pretty awkward I think this method should be on session even though it doesn't give you one this gives you a new session session new don't they all give me oh that one doesn't give me a handler error can I make a handler error from a thrush error I wonder um a handler error well sort of is that a standard oh great so I can it's just a little bit awkward but I can do dot map error thrush handler error great uh this is gonna return a handler error as well even though it never can see this is one of the things I uh really dislike with futures is you get really big types expected a new session yeah it's because uh you can turn an IO error so the problem with um when you chain things with and then um it assumes that the error type stays the same which is of course not necessarily true um because in this case we start out with a with a myo error like if if this fails it gives you well it gives you an IO error but if this fails it gives you a thrush error and so what how what type should the error of this thing be should it be a myo error or a thrush error and so it gets sad um the real question is can I make this an IO yeah okay so this is maybe a little bit ugly but we're gonna do map error uh thrush oh that means I have to we're gonna want to have a slightly nicer API for this so that all the users of this library don't also need to depend on thrush and thrush keys so we'll probably do some re-export um in fact this is gonna have to be a handler error just make it a we can map twice uh error IO and then we're gonna map that in a handler error async read does not implement for myo tcp stream what that there's no way that's true I guess maybe we should maybe there's a type in tokyo IO directly I don't think so wait why can't I just use a myo here feel like that's not true uh docs.rs slash myo oh maybe the version is too new maybe I need to use one of the older versions tcp stream doesn't just implement async read and async write am I being silly wait let's see tokyo futures my tokyo myo tcp stream why can I not use that there pretty sure this should just be usable right here but maybe I'm wrong these pieces are fine listener thought incoming but that's tokyo oh is this in tokyo core oh fine so I don't need myo so we're gonna instead use tokyo tokyo core tcp I guess we can just do this here tokyo core net tcp stream and it presumably has the same api yeah kind of in a handle connect using this handle yeah so this is the kind of pattern you saw with futures a lot um back before before the new tokyo crate basically everything had this kind of api where you also pass in a handle and the reason is so that it can spawn futures onto that handle that deal with things like io in the background um and so this is the kind of stuff that I would have liked to see in in thrush as well is if it spawned some of its sort of background work essentially the things that do up calls to the handler spawn that onto the given handle and then of course in the new tokyo you just use the current thread or the current handle um handler error now we're almost there uh no method channel open that's true because this method is really what do we call it open exec and we still want to do that and then we want to return the channel yeah so this is again this is where we have to spawn um okay so the the reason we've run into a bit of an annoying part with needing to spawn is because the session that we spin up um once we've made a channel we can't just check the channel right ideally sort of what we want here is like a core dot run and then we give it a channel and then we whatever we get back is the data but we can't actually currently do that because imagine that the channel has no data then you would just keep pulling the channel but what you really need to pull is the session which is the underlying thrush connection which would then cause the handler to be run and so we need to spawn the session onto the handle so that pulling channel will give you those events and hence drive the channel forward it's now containing a 33 uh future is not implemented for async channel is that true oh i guess that is true i guess this should really be a map and this is going to be a yeah so the channel is going to be this and then the data is going to be core dot run read to end of the channel right so okay so the flow here is we connect to the server we map the errors because we need to we establish an ssh session on that connection then we authenticate on that session so now we have an authenticated session then we create a channel on that session and return that channel i guess this gives us back the channel and the data and currently we don't really care about the channel anymore at that point we should just be able to do this and we can't print evacuate so we will do so essentially remember read to end just give us the bytes that we're going to interpret it as a string and then hopefully we should actually get the output of ls on the server it's all i want uh what do we think do we think it's going to work the first time we try it probably not right ls i'm hopeful what do we think is it going to work server is running has the key has all the ssh stuff this is the first time we try to actually run the code so you know it generally doesn't work the first time around but it would be really fun if it did it is true that in many cases with rust i find that if it compiles then it does the right thing it's definitely not always true this is the same kind of claim that people make make and made about haskell um that because the types are so good you don't make mistakes i don't think rust is as good and it's also like not generally true but i definitely run into fewer like stupid errors i think then in python for example if only compiling was faster crashed that's annoying uh well it failed okay it failed because it couldn't parse the address to the server it's a pretty good error uh if i have to say so myself the better error than many alternatives could have been um came from 22 yeah why can't it parse this am i being stupid there uh should be a totally valid it should be allowed to do that just like parse should do the right thing but maybe it needs to be can it not be a host name or something i mean i guess i can use the ip address instead how about now yeah but i didn't think it should be necessary to give the ip address i thought it would sort of implicitly called you know i did something it didn't crash are there no files in the home directory do we do ls-la that is ridiculous we have fully asynchronous i over ssh working the first time we compile and run it i think we did pretty well that was um that was unexpected um okay well that's kind of cool um let's like get at this working ls okay um i guess we'll fix exits that is then sorry i'm a little amazed that that worked um so how do we want this to work like we ideally just want this right that's right it compiles a wrench of it immediately it's all done it doesn't matter that like things are hard coded it's all good um basically ideally what i want is i want the client the code that a user of this library to write to be about half the length of this right because currently this is a somewhat awkward api like needing to know that you need to do this is really awkward but again i think we can get around this by using the um by directly using the thrush session instead of the thrush connection once it's been authenticated um let's so exit status is what we're doing right uh exit status so exit status is going to return a future yes indeed um but in particular let's see actually this is a really weird future it's like a okay we're gonna have an like an exit status future and it's gonna have almost no well i guess it's gonna have the same fields as channel i guess it really just is a channel okay so we're gonna have a uh session state graph we're gonna have a channel id and exit status is just gonna give you an exit status future which is really just self it's really just a clone of self this is where having variant types would be nice um two socket adders oh yeah i remember this from a while ago yeah i'm fine with using ip for now in a sense it doesn't really matter right because all our library takes is and it's just a tcp stream or any stream really and actually we might want to implement some testing on top of this and we could do that pretty easily but um because we're generic over the stream but it means that this connection code is all in the end users code anyway so we don't particularly care about it um so we will implement oh this would be our first implementation of poll great that fits well so we're going to implement future for exit status uh it's gonna result to a u32 it's error it's gonna be uh it can't error it can fail to get an exit status but that's about it um let's do futures so we're gonna use and now we're gonna also need poll and async and basically it's interesting because this is basically going to be the same as read is uh just a little bit simpler are you believe you should have done this first it would have been easier um we're gonna look up the state uh we do need to borrow it mutably we don't need any of this code all we need is if the state if the exit status is known then we return okay of the exit status otherwise i guess this is going to be async uh ready this otherwise we're going to return not ready and something this has to this has to uh queue for a later wake up um it's interesting because uh we're gonna run into this issue with read as well that there currently isn't really a way to signal an error to the individual channels basically this is saying that the channels cannot error which might actually be the thing we want uh if the underlying connection errors then the reads all just return zero but they don't really return errors anymore um but they will they will finish i think yeah but there is no notion of error i guess no that's not quite true um here if exit status is is none and state is finished that is an error that means that the uh the connection went away and we did not get an exit status so in that case we do return an error but for reads we don't really have a similar kind of thing uh we and in part this is because the our handler doesn't get notified about there being an error right it's only notified about events on the ssh channel but like neither are closed nor an end of file is really an error and there's no callback for some other thing i did caused an error which is what we would need in order to signal that error back on all the channels so i think for now it's fine for the channels to just always succeed um and just when there's an error they just return end of file which seems roughly okay um for the exit status if there's one set we give it if the stream is finished and we don't have an exit status then we return an error otherwise we do this and here we need to ensure that we get woken up um this is a little interesting because um here we want to be woken up i think this wants to be woken up whenever state is finished or if exit status is set so this is a slightly different wake-up state from the readers um i think this is going to be read notify exit notify so this thing is going to set read notify i wonder if we ever lose so i don't think we do i'm wondering whether read can be ever be called from two different tasks i think this is right okay so here we're going to set uh state dot exit notify to be some uh futures task current i also wonder are they here exit notify is unknown i don't think this is important um but i think we uh i don't know actually what happens if you try to notify a task that has like gone away you could imagine them being freed or something let's just be nice citizens and if we if someone calls read and we return successfully then there is no reason to notify this task again i don't think this should matter but okay um so for the handler now let's see if you get data uh if you get data then you we don't need to notify the exit if it closes we do because that sets finish so we do need to notify exit uh if we get end of file that sets finished so that also has to do the same thing um and if we get exit status then we need to notify the exit task we do not need to notify the readers right i think that should do it uh i guess we're and we're printing the status there so let's see what that does isn't it called exit notify my messing this up exit notify do this can't leak private type that is totally true compiler good job oh yeah the other thing we'll want to do here is uh set missing docs do some documentation work i think the difference between this stream and the previous ones is that i won't actually do um the documentation stuff on stream probably i'll do that offline um oh i'm not up on error i wonder whether the exit status is set later yeah i wonder when we get the exit signal specifically what i wonder what might happen is you get end of file then you get exit status then you get close but end of file sets finished for us which triggers exit notify so i think really what we want for these is end of and then have it just like not be called finished because it's not really accurate um and then now it is not clear that exit status yeah exit status if you get end of file that does not mean that you should exit but if it's closed yeah that's the way to do it so we want to track whether it's closed and whether end of file has been reached separately because you would if you get end of file and then you get the exit status and then you get closed then there's a state in between where where the channel is not yet closed so you don't want to error on exit status because you could still receive it um so i think this is what we want and i guess there's now a question of whether closed can be set without in the file i don't think so okay so in exit status it is only if only if it's closed do we error and then up here close is going to set end of file and close to true and it's going to notify exit end of file is going to set end of file to true but not closed and will not notify exit because it could notify exit but it's not clear that it matters yeah exit status zero so i guess now what we do is um ooh let's be extra fancy here and do that command is going to be equal to this and we're going to do actually we're going to not move command just so we can choose what command to run as an argument to the file hello okay so that worked fine and then the question is if we run like bin false we get exit status one and no output great so this means that now we have exit status working uh and exit status so the question now i think is whether we want to deal with the standard error or whether we want to try to get rid of the handle dot spawn you guys have a preference we can try either i think getting rid of handle dot spawn will be a little bit harder but it might be instructive okay well i want to get rid of handle dot spawn so let's get rid of handle dot spawn um oh fudge nuts that's only true if it's only true if session can be pulled hmm that's kind of awkward actually no uh we can still call poll handle spawn please all right so we're doing handle dot spawn okay so so the trick let's see uh the reason we're running into trouble is because uh the connection is owned which means that we can't have multiple pointers to it that's basically the issue um and the reason we need multiple pointers to it is because we sort of we want one that just ensures the connection keeps being pulled in the background while you're trying to pull other things uh but we also want to continue having a reference to it so that we can do things like open new channels um i think the way we're going to have to get around this is we're going to wrap it in an rc ref cell so that we can have multiple pointers to it uh we're going to spawn one thing that has a copy of it and that copy is going to um just call poll because poll takes a mute self no it does not consume self um and then for the other methods which is for us currently is basically just channel open session for channel open session instead of using the one that's on connection we will implement it ourselves using the one that's on uh sorry instead of using the one that's on channel on connection we will use the one that's on uh session which takes a mute self uh now we of course then need to implement the logic for waiting for confirmed ourselves but in theory that shouldn't be pretty straightforward yeah this just gives you a future that my guess is just waits what's first round it's a good question i feel like we should just try and see what happens but the essentially i don't know whether session is something we're supposed to deal with ourselves like i i don't know if it's okay for us to use the methods down here immediately by the confirmed the corresponding channel is there like a channel there's a channel open which is a future that resolves into one is there something else as well like a channel the channel here seems oh channel confirmed has the other side confirmed the channel okay that seems unhelpful um i think we basically just need to look at the implementation of channel open this does a bunch of stuff so the poll here basically just keeps polling until until channel is open is true so the question is when is channel is open sector true because what i really want to do is i notice that on handler there's a channel open confirmation so what i'm imagining is that when this is called on the handler like right after this is called on the handler then we can start using the channel right that's how i imagined that that works which would mean that all our channel open future would do is it would check it would we would basically keep a similar kind of tracker maybe even in channel state saying like ready and if it is not ready then it will sort of register its task for a wake up and then in the implementation of uh channel open confirmation we will set ready to true and then trigger that task so this is a similar kind of thing we did for reads right um and that i and then we will just resolve the future whenever confirmed has been set to true but i'm a little bit worried about this channel open future doing other things i don't know what this first round thing is see here the connection the first thing it does when you try to open a connection is a board read there's a board read something we have access to i don't think so no it doesn't look like it um i assume this is like something along the lines of the connection is currently or the the connection is generally in a state of waiting for stuff from the server and this is a way to tell it hey look uh stop trying to read stuff from the server because we're trying to send something um but it's a little bit unfortunate because uh we can't call this method let me think about that sorry i just have to go to the bathroom i'll be back okay yeah so the question is what does a board read do because these other things don't seem important it like ensures that it calls pole in the underlying connection oh so it does pull the underlying connection yeah this is basically it getting around spawning is that the channel open sort of has access to the underlying pole and just like can keep calling that this would this would also be unnecessary if the if uh thrush itself spawned the underlying connection okay so i think all the rest of this i don't know what this is reading businesses c dot is reading so it seems like the connection can be sort of in a state where it is reading and that's what that abort read does too so we're gonna have to figure out what that is all right so let's um it's a little bit annoying that the code isn't like easily available connection so is so it is public so we can't call it it's just hidden what this does but seems like we have to call it's a little bit unfortunate that we have to reach this far into the internals makes me a little bit sad the problem of course is if like if imagine that this implementation changed our implementation also has to change to match that but i guess in those cases the abort read method would probably go away but okay so let's uh get rid of that so what we're going to do now is we are i mean this is essentially us re-implementing the connection part of thrush and just using the session stuff but all right um when you create a new stream you are going to have to also give a handle which is going to be a tokyo reactor handle um we're going to take the connection that we get so this is now a session at this point we want to disconnection actually d ref mute as well because otherwise we have a bit of an issue uh because we're going to need mutable references to the session it implements d ref target session and implements d ref mute great okay um so in that case we should be able to make a let's just use a type alias here this is going to be uh takes an s and it's one of these guys but specifically it is an rc ref cell of that and that's what session is now going to contain so we're going to have one c that we're going to spawn uh i guess we're we're just going to do rc this is the connection this is the thrush connection right so we're going to wrap that in an rc new and ref cell uh we're going to take one of them and we're going to spawn it and i think it's this should be spawnable because you can call pole uh might i think a future is required to oh let's see what happens okay so we're going to thread spawn c dot com this is not actually going to work we're probably going to have to implement future for a wrapper type um it's not thread spawn but handle dot spawn um and then we are going to give back a this we're going to map out the c and now when you try to call wait we don't actually want to do this here we want to do this uh yeah we don't actually want to do this here we want to do this when you make a session not when you make a new session so here i think we're just going to take the handle i'm just gonna do c handle dot clone not do this yet yeah let's do this later okay so this means that new session is going to go back to having a connection or a c i guess um and a handle it's going to be a tokyo reactor handle and so now this is going to map the c into a new session c handle nice handle is going to be handle dot clone um yeah and then fn from u is going to take a new session no it's going to take a uh it's a little bit awkward for this to be called make but all right uh so this is going to take a existing thrush connection and a handle wrap the connection in rc ref cell so that we can spawn one copy and keep another um and that other is what we're going to use when we later open xx okay so make is going to take the uh connection that way this can hold a c and this can hold this so this takes a c s and a handle and gives a self and it wraps it and then returns a session so this is of course initially not going to compile it all in part because i'm being silly now authenticate key is going to not just map the session it's going to take the connection it's going to extract the handle it's going to call authenticate key on the connection move in the handle and when it creates the session uh it will make that from the resulting connection and the handle uh and then when we get one we do this and now open xx is probably not going to work handle doesn't work because handle not found and also unused import yeah so the issue now of course is uh that down here we're trying to spawn uh a clone of an rc ref cell connection which doesn't implement future we know it implements future but it sort of needs to implement future by borrow muting the thing under it which won't work so what we're going to have to do is we're going to have to make a rough that's going to be like a uh a shareable connect takes an s which is an async read and an async write and just source the c of s stores an rc ref cell implement future for that and luckily this is pretty straightforward because all it really has to do is borrow the inner uh now note that we need to be careful here because borrow mute is like runtime borrow checking right um and so this wouldn't work if the thing if the when we call pole on the underlying connection that pole ends up trying to borrow the connection mutably right because then you would end up with two mutable references at the same time uh and so this basically means that our handler is not allowed to have a um a c s it is not okay for it to have a c s at all like it it's not allowed to keep a reference to the connection or it could but it would never be able to use it because all of our handler methods are called in the context of polling the connection which means that we're holding a mutable borrow to the connection i think this would be fine because i don't think any of these need access to the connection so i think we're fine uh but it's just worth keeping in mind what we call this session state ref session state ref handler cannot uh okay so now uh this shareable connection should be portable which means that we can now spawn it uh it can't i guess really this should be uh this is now a shareable connection we also want to this is similar to the other case where we have a um a sort of ref where we we want to be able to get to the session given a shareable connection actually let's not do that sorry uh okay so we now have a connection that we can spawn that is a future that will um that will keep polling the underlying connection but we also still have a pointer to the connection itself which means that in theory we should be able to do open exit of course now if you try to check this it's gonna yell at us because oh this is the tcp stuff again we only implement this if this is also oh that's right i forgot about this um so derive clone is a little bit stupid sometimes derive clone um when you have a type that's generic over some type parameters will only implement clone when s is when the type parameters are also themselves clone so it will generate something like implement s clone for shareable connection s where s is the existing trade bounds plus clone however in our case we do not actually need s to be cloned for shareable connection to be cloned the rc takes care of that right so we want to generate this input um which derive clone will not do for us so we need to do it ourselves we'll just be uh shareable connection self dot zero dot clone because we need to be able to spawn it even better so remember when we spawn a thing it needs to um a spawned future needs to not produce any errors or any output because it will just like be tossed into the void right um but now that we implement future ourselves here what we can do is we can do uh is we can keep track of the error so that we can return it from all the channels if an error does indeed occur so finally we got the thing that we wanted originally which is here um if pole finishes so it exits with like okay then we return okay if it exits with error e so this would be a handler error then what we do is we do self dot zero dot borrow dot uh what does handler require dot handler mute the handler is already an rc so borrow is fine handler is fine so remember this is really a session state right so this is state and then state is going to be state dot borrow mute which gives us a mutable borrow to the the underlying session state and on that state we can now say error width is sum of error here error width is going to be a option of thrush handler error of no error and then we're going to swallow that error right because now we've sort of stored it um and so we're now the the spawned future what it really will do is it will it will pull the connection and if the connection returned an error when we pulled it we'll store that error and then pretend like it didn't happen this is important because now we have the error in the session state which means that reads and exits that is and such can look at that error if we wanted them to do this now needs to implement a 205 session c right so this will now actually hold an i guess a shareable connection and now it's going to complain because yeah so now we get into the funny business here of open exec now uh channel open session the one we're using either we need to use channel open session on here which is what we were using which consumes self and gives us back this nice future thing or in our case now we're going to call a channel open session on the underlying session which is not a future just gives us a channel id and then we need to deal with it ourselves um and so the compiler is now saying hey there is no method channel open session the reason for that of course is that this is now just uh this is now an rc ref cell right so if we do a session it's going to be this dot for me i'm going to say session is session dot dereft right which is going to give us the underlying session we don't really care about the connection anymore because there are no um really all that useful methods here oh channel closes another we need to deal with we don't write to any channel so it's fine for now but if we this does mean that if we currently were to try to execute say the cap command it will never exit because we don't send a no file from our side uh okay so session is going to be dereft and now we're going to do now we're going to be using the channel open session on session so we're going to say channel id is session dot channel open session this does need to become a future at some point oh right that's what we're doing so handler we're going to have to implement channel open confirmation so here uh we're going to have to like uh notify channel open which means that down here what we really want to do is we want to make a channel open future which is a thing we haven't defined yet um and that's going to keep track of this channel id and probably also needs the state would be my guess uh so state is going to be session dot handler dot clone so channel open session is going to give us a channel id immediately so if we look at look at the method um channel open session uh return some immediately if the connection is authenticated we know that it's already authenticated and then it will only become uh usable later so this will return immediately it might return an error though so i guess we will do futures failed i think that should work okay so now we have a channel open future and oh that's right we did all sorts of weird things here didn't we this doesn't mean that open exec no longer needs to return the session it can just return a future channel because it just takes a rec self right so what this enables is that you can call open exec and that gives you that gives you one future that's going to give you a channel you call it again that gives you a second one i guess this does actually require mute self because you can't open two channels literally at the same time i guess given that this type isn't send is not an issue but just for correctness sake you are sort of modifying the session when you add a channel to it right this gives us one of these and then i think we don't even need the i think this command needs to go here now i think we just do box new on this and then we're gonna have to have this um we're gonna have to have a pub struct channel open future it's gonna have to take a lifetime for the command and it takes basically the same thing as all these other things done here and then we're gonna have to implement future for it and now we're not gonna need it for session anymore so that's kind of nice um channel open future does not need all the s stuff um it can't even return an error i think no it can this would be error width right so this would be the same thing as we have above um this thing it's going to be very similar to the channel stuff that we did for all the other things right it's going to be we're going to check whether the upstream state has changed for this id i guess actually we'll end up putting this state in immediately yeah so here we immediately construct state for this new channel it's going to be default by initially um this can go away and we're going to pull down here so in poll we check the state um we set connect to notify to none um if state.confirmed so this means that now the callback up here has worked out right like if this callback has been made for this channel um then we can return ready and in particular we want to return a ready of channel uh where state is oh that's awkward it's still self dot state except it we have it borrowed mutably which is a little awkward non-licical lifetimes would fix this um and id is going to be self dot id if is there um handler like channel open failure we need that one too so channel open failure so it's also going to have to notify us about this um so this might error depending on what the state of the channel that we opened is otherwise we're going to do the same thing as we did the all the other ones which is going to be uh open not notify and not ready so this means that a channel state now has an open notify you're starting to see a pattern here open notify it's going to have a I guess really we should group these someone so this should go here these are all related this is related and this is related and this should be open state it's going to be an option uh well it's sort of like an option option which is a little weird it's like it's some if we got a response and the inner is some if there was an error so I guess really it's more like an option result nothing and a rush which of course initially is close to false uh there's none they just started that open notify so here we're going to do we're going to be open state dot take if we got some okay then that means that great the channel is ready if we got some error e that means that we got an error uh and I guess this is this could be a handler but let's just ignore that for now there's going to be a channel open failure um and if we got none that means we're not yet ready and so that pattern makes sense it's like basically the same pattern as we had for for exit oh and I guess once this is the case here we sort of want to exec again so the question is whether we're allowed to call that from here so remember how we can't have this have a we can't have things in handler have a reference to the connection because we're going to mutably borrow the connection when we're pulling it the question is whether in this poll can we have a reference to the connection I think we can the reason we need one is to so that we can call exec I'm pretty sure we can because this poll is not called from within this poll so I think we should be fine I think this means that this is going to have to have a session as well which is going to be a shared connection which means it has to be generic or s so that if this happens we do uh session self dot session dot borrow dot and and what does how does exec not return anything I think that's so weird self dot id true this and then it returns that hey uh this takes an s that's true asian greed plus asian right 45 is going to return no longer one of these but instead a channel open feature on this may no longer need tcp 267 zero p is channel id and it also needs what was the other thing we found it also needs a session self dot zero it's a good question we could have yeah so so the observation here is that open exec sort of almost synchronously calls uh channel open session which returns a result which will return some immediately I think this is meant to say okay um and we know that it's authenticated so I think we can unwrap here sessions this should no longer be a box yay dot zero dot zero I don't think you're allowed to do that yeah I don't think this what I think this needs to be like this or something this has to this has to do the same thing as all of these things are doing which is cut the state for that channel and then it has to set state dot open state is some okay in this case because we got a confirmation and then it needs to notify the open waiters and then failure needs to do sort of the same except it needs to store an error of the reason oh this is a description in language we're just ignoring those for now I guess we're ignoring this one too unmatched types 76 oh this needs to also return this business oh right this has to be we specifically want the one for the session not for the connection 290 this is not allowed to return that that's false this should return all right borough checker tell me what I did wrong uh can't move self that's because we don't have no lexical lifetimes yet so we need to do this and borrowed value does not live long enough just need to keep the lifetimes alive and now it's going to give me the one that is annoying to fix which is that here we've taken okay so this is in the poll of channel open future where we sort of take the state oh can we get around this maybe so the problem is here we have a reference to the channel state and then at the same time we're trying to move out of that state right because so we we have this borough here that's into the state and then down here we're trying to move out of self dot state it turns out though that we can get around this by okay so in theory we haven't actually changed anything except that you should now no longer need to do the spawning business the example won't compile because now needs a handle so it's going to be given a connection and it needs to call new with a connection and a handle all right and the open exec here gives us a has an error type that's different from the one that like it's no longer a handler error right this is where we would probably pull out the failure crate and then make just all the errors be error because they should all be uncommon in this case I think what we'll do is we will instead just do make this a handler error and then we're going to here do just wrap it in an error what are the different error types here again actually that's a good question keys error can I actually reuse any of these that's a little weird what is what does channel open give the channel open thing handler error how does it turn an error into a handler error that's what I want to know apparently it just doesn't so it ignores the thing that's given to the handler the actual error description we get well in that case we are going to make it an IO because IO is easy to fudge with it's going to be an error kind of other and it's going to wrap me except we're not allowed to do that because it this error type does not implement where's it channel open failure does not implement standard error so there's no way for me to expose this error type inside of a handler error okay I'm going to make you all really mad at me and do this not worth the time that's an error right okay so in our example I'm 27 yeah that's right this no longer needs to map away anything can just do this let's see if this does the right thing oh we forgot to do the board read stuff so this was like oh I guess we have the code here right so there's a first round field and first round has to be true and then here this has to be like a if self first round and then the connection session I think so and channel is open should just like clearly be true here but I guess we can we can double check so so s is one of these we're gonna do like an almost like an assert yes s.channel is open channel.id self.session and I can't borrow mute because session.zero so now we're like sort of doing the same thing as this is doing I just like don't believe that this is necessary it's really weird this like is reading I don't actually know how to deal with that yeah because this seems to like not do the right thing at all hmm that's a little sad because it also suggests that it's not that it doesn't get where we want it to see how far it gets actually oh I guess it's a little bit tricky for us to do that but in theory it should at least get to here trying to open is this roughly makes sense I know we're are pretty deep in the weeds here but I did tell you that some of this is gonna get a little bit ugly because it does get to that you shouldn't normally have to abort read you sure because that's what this code over here is doing I mean I'm very happy if I don't have to abort read but essentially we're writing our own implementation of channel open right because we sort of have to yeah I mean the the the thing to think about for Tokyo in futures is is that it's it's just to get the model right in your head so that's why the the article by Aaron Turin is one that's useful to read if you want to get up on the basics as we're not having to call this normally I agree but we are actually we're re-implementing channel open because the only way to get a channel open from thrush is to have a to consume connection to consume a connection whereas all we have is a ref mute like we have a mutable reference to a connection I need to make one so we do actually need to implement this ourselves no that's fine we are again we've like done a pretty deep dive here so I think the right thing for us to do is to call abort read here I don't think this should be necessary it's like checking whether it's open because we only return it once the confirmation has come in right so once this method has been called on the handler when that that method has been called on the handler that's when we return the channel so the question is it's still necessary to check these two and I don't think that's the case but this code I'm not super familiar with like this is thrush code right like deep thrush code the more concerning part actually is that it's um is that this just hangs so do we get a channel at all is the other question we do get a channel this log is three misspelled it do I need like this I don't think I need that this oh I need to invent it logger that's why so we did get a channel so it's stuck somewhere else specifically it gets oh so that actually sort of suggests that things are working it just suggests that the spawn isn't working show that screen again yes I can am I hearing clear mx clear switches so I think these are clears actually no they might be browns this is the um the filco ninja but are they clear that's a good question switch it's definitely not black brown yeah that makes brown uh sorry it's telling you it started to wait for packets yes which is sort of what we wanted to do however um yeah the concern is that we now have a channel open right like the we did in fact this this future resolved which means we did get the channel back um but we aren't able to read from it so this means that this read never returned something useful which probably means that the underlying session isn't being pulled so I'll be here uh I know the abort read happens earlier the abort read is uh the abort read happens uh when we the first time we pulled the channel open future yeah so see here the issue is that the ssh connection is not being pulled after we got the channel hmm this almost suggests to me that the hmm I wonder if it's already gotten the day it's just like print out some things yeah exactly but we do get the confirmation right because the channel open future does resolve so we open the channel pull the ssh connection for a bit then we get the channel back but after that there are no more calls to pull on the ssh connection do I need to do something to like re-enable reads after this has been opened basically I enter this right like uh this reading it's only considered open if C is reading so then why is it not being pulled anymore this does explicitly call pull but only if it doesn't get a channel we do get the channel we basically go through this loop like basically once I could check I guess but um once this returns so it really is the read then that fails so let's just check that read is actually called so we should see we should see it say not yet at least yeah this is almost like the the ssh connection is not being marked as runnable once the channel has opened I wonder whether uh the channel open needs to explicitly call pull I don't think it should be necessary but let's see okay so so for the um channel open when we get here so many times this goes through that's a little interesting yeah once it gets the channel the last time it doesn't pull the connection anymore so I almost wonder whether we like need to do a pull here but it shouldn't be necessary but I guess we can try it we would have to draw to do this after you get the channel the only thing to do is to start using it no but I'm trying to read from it I'm not trying to send anything right I'm opening the channel and then reading from it so there shouldn't be a need to do like this calls exec right so I do do something with it so when this is exec when this is exec like doesn't do anything but I don't see why that would be the case oh if I don't call pull the exec doesn't send anything right so that's probably why s.exec because I'm only using the session here this probably um so I'm calling exec on the session but there's nothing that triggers that to be sent over the wire so that's totally why so this needs to so I wonder whether this needs to call pull sorry self dot uh session which is all sorts of weird because what if this errors I guess it has to be this and this requires tcd oh that's right and this I guess really all it needs to do is wake up the the task is really what it should do that should be sufficient for pull to be called eventually so really what we're saying here is something like this is going to be a struct now it's going to be a a connection it's going to have a c it's also going to have a it's going to be a option yeah that's zero dot borrow dot c dot so this is going to be the connection is going to be this so basically we need to keep track of which task is responsible for pulling because given that we spawned it and then we need to sort of poke that task once we exit so this is I guess this is the the weird part about using session is that session is just like queuing things and doesn't do any work so we would do something like c dot task is some futures task current and then we would pull c dot c or something right because now down here uh connection and this will now be I guess s dot s dot task dot unwrap dot I think so yeah so so now um and then it's going to complain about lots of things any same three handler c seven type connection cannot be derelict and this is now going to be session dot zero borrow me dot c connection c these kinds of like mechanical changes are c dot handler 37 I guess this could just have a thrush but sure why not 320 finally yeah the problem here is oh I guess now we don't need to do this anymore because we're not calling pole directly here right so in theory this should now sort of um ready up the thread that manages the connection so the the thing that we spawn there we go so basically this this pokes the thread that's responsible for doing all the IO on the thread um so it pokes that thread after we call exec so that it actually sends that exit command it is a little bit weird um that we have to do that like in a sense I wish that exec would cause those notifications to happen but I guess it's not terribly surprising that it doesn't uh okay so that's nice now we got exactly the API we wanted right it's pretty neat in theory now we should be able to do writes pretty easily actually but all this should come oh and yeah so so the other thing that's uh that we should think about here is the errors uh that we now store so because we ended up choosing to do this slightly differently now um any error that happens during the connection uh we store and so the question is where do we return that error uh so this is the resort channels yeah so session state has this error with which is while we're polling the main loop of the connection so the things that does IO what if an error happens there then what happens um currently that error isn't returned from anywhere but arguably it should be the question is just where because currently oh I guess here's what we'll do we'll have um we'll have a method on session that you can ask for an error I think this is actually the the API that we want specifically um you can have a bunch of channels and if some of them return prematurely like return and you weren't expecting them to return um then you would call this method and you would get what the error was I think that's actually the API we wanted so this means that on a session uh there's now a method that you can call uh that's like uh I guess which gives you an option chandler error which itself dot last error uh no a session just has a shareable connection so borrow mute dot a shareable connection has a connection which has a C oh I guess really what we need here is the state don't we um it's no longer sufficient to just store hmm well I guess you can get to it in a really roundabout way so we can we can uh say that the the connection is going to be self dot zero dot borrow and then the handler is going to the handler is going to be connection dot handler the state is going to be is it true self dot zero dot borrow yeah but the real connection is going to be uh connection dot no actually a handler is going to be connection dot c dot handler the state is in the handler so we're going to have to do handler dot borrow dot zero dot borrow mute so that gives us a state through the handler uh and then we will take I think maybe time us this up probably uh shareable connection zero dot c dot borrow did you mean zero probably oh yeah I did we should probably not have as many new types this is the problem with the new type pattern so this is a new state which returns last error yeah so now in theory if there was an error on the connection so if all your reads suddenly like returned early but they all returned okay right they just returned incomplete results you can now ask the session what went wrong so that seems nice and then I guess now we're doing writing that seems like actually oh let's see it pointed it's getting pretty late um okay let's see how hard writing will be and then I may have to cut this at some point okay so we're going to implement async write for channel somehow uh actually async write for channel we can probably do pretty easily right isn't async for channel just uh in session there's a thing I saw a thing on session where you at uh connection there's a channel close which I think is what we'll want to call except we can't actually call it because all we have is a session so we're really going to call it's something like channel is there a channel close here really I guess channel end of file is really what we'll send I don't know if that's what shutdown is intended for in async write but that's my guess um okay so we're implementing write for channel so write for channel is going to have flush and write uh the layout is going to be fairly similar than as to reads but um we're going to have to do let's see that's a very good question we're going to be calling data aren't we yeah we're going to be calling this and remember that actually this is going to run into the same thing as we had previously of um of we will have to notify the task that maintains the connection right so this is the same thing that we found out that we had to do for channel open it's actually really convenient that we found out because we're going to have to do the same for writes um luckily a write is fairly straightforward I think so for a write uh what we really do is we um we need to get at the session and the session oh yeah let's suggest the channel does indeed need to have a reference to the shared connection bearable connection nope not there there's there's channel which means that this does indeed need to have the stream this is so that the channel has the ability to uh send data it needs to have access to the underlying session so that it can access the session buffers essentially with the sending pipeline um so a channel is going to have a con this means that now it's going to be able to do self dot con dot uh zero dot borrow mute so this is going to be the connection um and then on that connection we're going to call specifically on the session that we get under there we're going to call data with the self dot id with no specific channel and with some amount of data which is going to be buff in fact I guess write never blocks here just a little weird oh I guess the blocking would actually happen in flush it's maybe a little weird I don't know how that's intended to work because here you could imagine that a writer just like keeps writing oh I guess actually n being zero implies that it would block yeah so if n is zero and buff and and not buff dot is empty uh then this is really an okay sorry and this is a woodblock so this would be a we're gonna need a write notify otherwise we just happily return an okay there are no other things that can happen here uh yeah exactly but we need to apply so the observation is that data never blocks it fills a buffer instead but we do need to apply some back pressure so presumably there's some window on the data connection or something right so it could be the data uh yeah this could presumably return zero to indicate that it didn't read any bytes or it's not willing to put any more bytes in the window right and so if it put no bytes in the window that's sort of the same as saying a right woodblock that's that's how we're gonna interpret it here if we didn't do this if we just returned okay zero then um the writer would probably just spin calling this method right um and so we would rather uh sort of say we would have blocked and then notify this task instead when space is available in the window which I assume that we learn here window adjusted right I guess maybe it's not window adjusted so a way for me to tell the data has been sent I feel like it probably would be window adjusted because okay if this doesn't trigger whenever data is sent uh then we will have to just return okay zero here okay let's leave that as a revisit later yeah so in a sense this is about uh keeping track of the window to avoid saturation right it's that otherwise the client would just like keep pushing data if nothing's pushing back I guess the other way to deal with this would just be to indeed just return okay zero I don't think we want to do that um this does however have to uh if we did write some bytes then this needs to poke the um the thread again the sending thread so where's our channel open future over here so this is going to do something like oh I guess we already have the con so it's going to be you could write a flush plus pull the flush this would try to send something to the socket yeah I guess that's one way to do that I sort of want to avoid explicitly calling poll but I guess I could and then see whether that would block um yeah so so the suggestion is that we uh here maybe call poll on connection um see if it would block but that brings us into some other issues too of like now we're calling poll inside of an asynchronous write um it also means that we would be um we might be writing data for an entirely different connection so it blocking isn't that interesting yeah we'll have to see exactly what we do here it might be that okay zero is the right thing to do um okay and then I guess flush here is going to do something like is there a flush call I assume so but the flush here as you observe is like sort of weird it flushes the clear text buffer into the encryption buffer the question is can I also do a flush I guess the issue is that I there isn't really a way to flush a particular channel because if there's like a single thing that keeps track of all the buffers that is going to write then flush would mean send all the data to all of the channels because I don't really have control over the individual um channel buffers so this one's a little awkward actually yeah in fact that one's a little bit weird it's also like I don't know how to wake these up again if the handler isn't notified that there's now space on the connection yeah the back pressure here is tricky I don't know how we're gonna do that yeah it's gonna be a little bit tough I think um what we're actually gonna do here is uh I think I'm gonna there's a single buffer for the entire connection yeah exactly so this would mean that there isn't really a notion of flushing a single channel a flush would flush everything um and I don't think that's what we want um so the stream is getting pretty long I think uh and I think this will actually take a while to fix so I think we're gonna cut things here um and then we can uh yeah I guess we'll do another session where we deal with where we deal with writes uh I think writes is the biggest thing remaining I guess writes in like standard error which should be pretty straightforward um and then we'll do a session after that or maybe in the same one depending on how long write ends up taking um we can do a session on uh integrating this into which was the original intention integrating this into tsunamis so the tsunami becomes asynchronous um but that's sort of one step further down the line uh and then converting to Tokyo and futures the the new Tokyo crate and the future 0.2 that should be a very straightforward transformation uh we might even do that as a part of the next stream because hopefully writes should take a lot less time than doing all the rest of this great because the rest of this great was a lot about setting up the infrastructure for the crate right um so yeah let's see the intended way is the data flush and then connection poll yeah it's going to be a little bit tough to do that here but we might be able to I mean in theory we can call connection poll from here it's just uh we'll have to be a little bit careful about the boroughs but I think we can make that work okay in any case I think we'll stop it there and then we'll do uh writes and writes and standard error and Tokyo upstream Tokyo and future 0.2 in a new video and then in the third probably will end up actually upgrading tsunami um yeah I think that's the plan um I'll post this video to to YouTube sometime in not too long hopefully once I sort of get home um and then sorry once I get to a place that has a decent network connection and then once that's been done I will post the twitter post the patreon uh and then I'll announce upcoming streams on twitter and patreon too um thanks for joining in it's been fun bye