 Alright. Hello everyone. This is part three of the Tokyo Zookeeper adventure that we're on together. Just to mention it again because I do it every time, I'm John. I do a bunch of these open source streams. This is one of the more advanced, this is like a later part of a more advanced stream. You can find them all on my Patreon page or on my Twitter page or if you look at YouTube, my account there, there's a playlist of all the different Rust videos that I've made. What we're doing in this particular instance is we're building Tokyo Zookeeper. So the idea is to build an asynchronous crate that allows you access to an Apache Zookeeper instance. So Zookeeper you can think of sort of as a key value store, except it's highly, highly consistent. You can have multiple servers running and they will all give consistent responses. We're writing this in Rust. I recommend that if you have not seen the first parts of this video, then you go look at the original or the YouTube playlist and look at the previous parts before you watch this one, otherwise this one is unlikely to make much sense. And so from this point forward, I'll assume that you've at least seen parts of the previous videos and that you're somewhat familiar with Rust. If at any point you have questions during the stream, then feel free to ask them in chat and I'll try to monitor that on the side and answer them as we go. Do you know how to get Rust 2018 async functions working with Tokyo? You can't quite yet because Tokyo is still built with future 0.1. There's a bit of discussion about exactly how this is going to pan out because a bunch of modifications are needed to bring it up to speed and it's going to be backwards and compatible in some weird way. So you can't currently use async functions with Tokyo, sadly, but hopefully in not too long. So the place we left off last time was that we have basically all of Tokyo working like we can interact with a running Zookeeper instance. The API is not particularly exciting so far. I don't remember if we pushed this to DocsRS. But the API is pretty straightforward. You can call exists, you can delete, you can insert. I don't think we implemented get, so that's the first thing we'll implement today and then we'll add support for watches that are not the global watcher. So if you remember from previously, in fact if we look at the documentation for the synchronous Zookeeper client that exists today. So notice that for basically all of the methods, there is an exist thing that takes a Boolean of whether to watch it or not and if this is true then a watch event is sent to the global watcher that was passed in when you initially connect to Zookeeper or create the Zookeeper client. There's also an existW, so notice that this is also the case for get children, get data and various other methods where you pass in a watcher and this watcher will be notified if this thing changed. So you have a choice of either making it the Boolean one or the or sorry either make it the global watcher that's notified or no watcher of course or some watcher, some one-shot watcher that you give it. So adding watcher, this style of watchers is something we'll also add today and then hopefully we'll also get to the ability to connect to multiple Zookeeper servers which is not currently something we support. To build on that here, I find this sorry for the suddenly white background, I found a FAQ for Zookeeper that talks about how you should handle things like connection loss and session expired and especially how that interacts with multiple servers so that might be something that comes in handy. It's a little annoying actually to test this because Zookeeper doesn't really it's a little annoying to run multiple instances on one machine so we'll see if we get to this and if we get to it whether we get to test it and if we haven't tested it then arguably it doesn't work even if we think it does. There's also the Zookeeper Java API that we're going to be referring to a bunch. If you look at the documentation here you notice that it has a similar pattern to the rust crate in that there's an exists that has a boolean and this then exists to take a watcher. Here there's also a synchronous version and an asynchronous version. For us all of these will be asynchronous. As we'll take a look at that someone also pointed out after the last stream they had some questions about whether we're breaking linearizable semantics given that we're using an asynchronous implementation. So linearizable semantics the idea is that if you perform well there are many definitions of this but the general idea of the property is that if you perform one operation and you perform another operation later in time they should be they should be visible in that order too. So if you do something to Zookeeper and then you do something else to Zookeeper strictly later in time like after the first one completed then you should make sure that all your reads should be consistent with the operations having happened in that order and this person has a question about whether this also applies when we have asynchronous implementations. You can read the discussion here we haven't quite come to a resolution yet because I'm not quite sure where they believe the the issue arises but this might also be something worth keeping an eye on. I also found the Java implementation that has a lot of the client code. I haven't had a chance to read through it yet but there are some interesting comments here that we can we can look at in terms of processing things in order. We probably won't do that in this stream but this might be a good way to sort of compare the Java and Rust implementations and see that we do things correctly. So if you recall from last time there was some discussion about how when you get disconnected which servers you choose to connect to that the client library has some like thundering herd prevention and so with some of this we might be able to lift from the Java library but it that might become relevant today but it's unclear. But the place we're going to start is to implement get data because if you can't get data then what are you even really doing with Zookeeper? So let's pull up our library here so for Zookeeper remember we have connect which gives you a new Zookeeper instance. We added create, we added exists and we added delete. We don't yet have a get. Now we believe the get should be pretty straightforward to add so let's just check that now. It'll also be a good way just refresh our memory of how the system works. So we're gonna have a I don't know if I want to call it get data. Maybe. So this get children is also a useful operation as Zookeeper. We might want both of these. Get children gives you the names of or the names of all the nodes that are underneath you. So in Zookeeper you essentially construct a hierarchy of nodes similar to sort of a file system almost and so you can get the children of a given node and get data gives you the contents of a given node. And so let's start off with get children why not. I think they'll basically end up being the same. Get children. Oh yeah the other thing we'll do today just because it's going to be necessary in order for this library to be even remotely usable is we'll try to incorporate a bunch of the documentation from the Java code into this. It shouldn't be too hard it's just important to do and we'll hopefully while doing that also observe if there are any mistakes we've made along the way or if our API doesn't actually conform to what the Zookeeper API is supposed to be. All right so get children. We're going to ignore this watch parameter for now. So the way we're going to end up doing watch we've already added it to exists as a Boolean because remember we tested this last time. I think what we'll want to do is actually remove it from exists and then have a pub exists watch which takes like a little unclear how we'll do this but it'll take something like a self and a watcher which is going to be an option watcher and so the idea here is that if you want to use the global watcher you would call exists watch with none but that's a discussion for slightly later in the stream. Forget children though the only parameter then this path because we're going to ignore watchers for now. So similar to many of the other API endpoints and like all the other ones we're going to return something that is a future that contains both self so the client itself and some kind of result in our case of course the result is going to be a string so a list of all of the child nodes names and an error get children which we haven't added yet so if you remember this pattern from last time the item of the stream is the client and result in the sense of whether the given operation succeeded or failed so this this vac here this okay value of this result is notice the item of the future so this is the future resolved correctly and the operation gave you an okay or the operation gave you a an error right so this would be you try to delete a node that doesn't exist for example the error case so if the future results with an error what that means is there was a protocol error protocol lever level error of some kind and those we want to expose separate separately from just your operation failed if there's a protocol level error then it might be your connection has been severed it might be that like it's unclear that you can keep using the same client you might have to start a new one whereas if only the operation failed then you only really care about how that operation failed so if we look back at error notice here that for delete for example we only expose specifically the errors that are possible to get the delete so that the user can match of them as necessary so let's see so forget children we get the client back and a list of children there's going to be some kind of error for create children children similar to all the other errors these are also going to derive all these nice things and we'll have to figure out exactly what kind of errors can occur when you do not get children what kind of errors can occur when you get children and for as for what get children does the exact definition is pretty straightforward if you recall the way that we issue requests is we have this packetizer that runs as a spawned future in the background and we have a channel to it that we on that channel we send a request and a response channel and so what the packetizer does is it's in this loop where it sends request out on the wire receives responses asynchronously and sends on all the stored stored reply channels whenever the response comes in and so in this case if you look at proto-mod we have this in cure which is what's given to the client the in queue method takes a request and returns something that is a future that will resolve into the result of that request and internally all it does is it creates a one-shot channel and sends both the request and the transmit end of that channel to the other side and then returns the response end which will be resolved whenever the response comes in so in our case what we want to send here as we want to send a get children request and we're gonna do the same thing as we've done in the past here where we'll lift this from here so get children is the opcode so now we're gonna have to go through all of these all these steps of adding all the various enum variants for this operation so here for requests that can currently be a connect exists the leader create and now of course it can also be a get children request we don't exactly know what that contains yet in fact we should probably find out get children so a get children request is a string and bool so string and bool just as in the same thing exists is the reason it's a string and bool is because it also has this watch flag which we will always set to zero when we send it currently because we are not also doing watchers quite yet for these operations so it will really just be the path and watch which will just set to zero and it's gonna start warning us that we haven't implemented all these will do those with compiler warnings because it's nicer here we'll set watch to be zero because we don't want to watch to get children the response we get back that's a good question so get children response looks like it is just a VEC of string here so that just reads an I-32 and those things okay so just like we added a variant to request we'll also have to add one to response in this case response here it's gonna be a children actually just do this because that might be usable for other methods too and we already have this well maybe we don't have that yet hmm so we need a way to parse vectors that come back it sort of looks from this implementation as though the only thing that gets a vector back is get children but that sounds a little weird so I think what we'll do is we'll implement read from for VEC string and here again we can just sort of lift this this implementation because they're all basically the same like the protocol level is the same right so in this case we want to read an I-32 which is going to be the the number of things I'll create a vector with that capacity read that many strings and then return okay great because now down here if what we sent was a request response then the response is going to be a strings and then a reader no back well how do we want to do this back string from read from right so the idea is that if we get in a if we get in a or if the thing we sent to the server was a request then parse the response as a back of strings using the read from trait that we just implemented and then wrap that in a strings response expected VEC found that doesn't sound right oh oh I updated my nightly not to wait for that for a little bit that's interesting why so apparently it doesn't like this because oh this should be reader still complains maybe this but I think that should be necessary expected mute found you wait oh right right for for so a the read trait is only implemented for mutable pointers to slices so that as you read you can change how far into it you've read whereas reader here is the buff which is just a slice of u8s now no variant named request yes this should be get children so if the request was a get children is what it should have said then parse responses a vector of strings now that should be happy we don't need version here that can go away and now we know the response should be a strings children then resolve to children we get something else got non strings response to get children this is one of those things where we've mentioned this in past reams to but we're in this unfortunate place where the response we get is also an enum and even though we know that we can't get anything that's not a strings response to get children we have to handle that case because we need to exhaustively match as what we do here is what if we get a response that is not of the correct type we just panic in theory we could return like a protocol level error here right like resolve this into an error and that might actually be the right thing to do come to think of it yeah in fact maybe we should just do that I don't really like having unreachables that aren't actually unreachable they should be unreachable right in the common case we don't really want panics in our code so I think what we'll do is change this to a bail so bail comes from the failure crate and is basically a return error format error and then whatever you give it we're format errors I think that creates a failure error so in our case we'll do this so this is gonna be so did not like that what did I do wrong oh hmm this should be okay so this and basically just changing all of the all of the unreachables to be bails so they won't actually panic the user will get to the user will get to choose how they want to handle that kind of error so I think that's all of them okay so in response to get children we know that we should get a strings and we handle that case we get something else that was not an error then it must have been the wrong return type what other responses can we get to get children is the real question where did we used to get this not here not there where is it here so get children is the one we want to look at so let's see not sorted no guarantees provided no note if the path does not exist okay so no note is something we can get right so that's something we want to let the user match on because it's an error that can totally come up which is not terribly surprising this is the case for most of the zookeeper operations what else can we get looks like that's actually the only one right so not empty is not an error we can get bad version is not one we can get node exist won't be a problem no children for ephemeral apparently can't be raised invalid ACL doesn't apply so the only thing you can really get to get children is no node doesn't implement display that is not true anymore and so any other error is unexpected right any error that is not no node is not one that you should be that it should be possible to get with children and so at this point it's unclear whether this should even be a result it could just be an option like so then this would be an okay some and this would be an okay none and then we don't need this error right so it's a result it's a it's an enum with only one variant where the variant does not contain any information which is basically the same as an option so what we're saying here is if you try to get children of a node doesn't that does not exist you would get none which seems totally reasonable and that should be basically it what is it complaining about here did I mess up syntax somewhere it did but where here this is missing a thing there we go all right so we can add this to our ever-growing test case down here of saying so in theory we would create a node then under foo so bar yes we'll do that after all the exist calls then we'll do this right so we've in our test we first create a slash foo we check that it exists and whatnot and now we're creating a bar under foo with some different contents and then we want to check both well actually we don't really care about the exists as much what we do however care about is all that should have created foo bar and then we want to do get children of foo right and we would inspect what we get back children and we want to assert that children is a sum that bar it's a good question exactly what it'll return but in theory that should work fine and then we have to delete bar I mean we could also test that deleting foo does not succeed so in zookeeper you're not allowed to delete a node that has children and so in this case we would expect to get an error back and that error is going to be a delete not empty right so we're basically trying to work we check that foo indeed got the new bar child we try to delete foo we should get an error says it's not empty we then delete foo bar which should work and then we should delete foo which should work so if I now keep her tea probably not compile I think we have all the components we need could be wrong let's see 203 it's complaining that it can't compare a reference to an option with an option how about now okay so here this is one of those places where rust pattern matching saves us we forgot to say how to encode a create request and we happen to know that a create request is serialize exactly the same as a exist request sorry I get children request is exactly the same as an exist request right they both just take a path and watch flag and then we want to well this one's a little tricky because we want to write a different off code it might be cleaner to just have them be separate so I guess let's do that it's like unclear there's a big advantage to sharing those two lines when we also need to add a match on itself again to write out the correct off code now what does it complain about 159 off code it doesn't know what the mapping from a request to an off code is we can tell it that pretty easily let's see what happens well that worked so in theory let's see if we can find this so up here this is remember we're doing a lot of logging internally that we'll probably want to turn off to or actually that might be a good idea what we could do is use the s log crate or some other logging crate and actually have all of this be proper logging that might make our lives a lot easier so we create bar and then this is a request for food she gets a response this looks a lot like bar in ASCII so that would be my guess that that is bar so then we handle the get children and then we delete foo which gives the error then we delete again foo bar okay great so that seems to just work and notice how little we had to do like this is extremely straightforward so let's then try to add I guess get data I'm a little bit tempted to just call it get but let's stick with get data for now what is in what's the this calls it get data okay and oh interesting okay so get data just take the path in a watch similar to the other ones but also takes a stat that's interesting I'm not sure why a stat is passed in it doesn't make any sense to me like why is that here not a Boolean I guess it returns byte and so this is like a reference to a stat that will be updated okay so there's a so the way to think about this is there are two variants of get data one is get just the contents of the node and one is get the contents of the node and it's sort of meta information so let's look at here get data request huh get data request just takes a path on a watch just like the others get data response always returns the stat okay so it sounds a little bit like it is just a more convenient API that seems odd I guess here another question is whether where's the here so forget data what kind of errors can we get only no node okay so option is still going to work pretty well in this case though what you're going to get back is a Vek of U8 right here is one of those places where it's a little sad that we can't just like slice out the U8 that we got off the wire we have to actually copy all the data it's a little sad the fact that there's a stat though means that sort of in theory this is also this I don't remember where we put stat and types okay so it's sort of a tuple of the bytes that are contained within the node and the meta information about the node and we can the basically what they've the decision they've made in the Java API is to have two different methods one that you call if you want just the bytes and one that you call if you want both I wonder what the rust one has get data just returns the top all yeah I think we'll do the same there's no reason for us to remove the stat so I think we'll just do that and this is going to be data so again notice how straightforward these are now to implement because we spent so much time in part one especially building a nice API for the packetizer and that's paying off now right so the response should be a get data response which is going to contain bytes and and stat and we're going to return bytes so non data response to get data so these are all basically the same right like in response to a get data we would expect to get a get data response if we don't then something is wrong on the wire in terms of errors we can get no node that's an expected result any other error is unexpected and we propagate that as a protocol level error and then we'll have to do the same thing here so there's now a new request type which is get data also takes a path in a string we might as well also add this this point it might actually be worth given it looks like a lot of these are actually the same because like this is going to be get data is going to look exactly the same get children I think we'll just do this after all change my mind given that get data is also the same and so the way we do this is we match on self I guess all code is equal to this and then actually we already have a method that does that so we can just do here self dot off right think so and now that will be the same for all of them right so get data get children exists all have the same contents so their serialization is the same and we call self opcode to get the opcode to write out which does have to vary between the requests no that looks good of course we have to map get data to get data some sort of fine we now know that there's a new type of response which is a get data response which has bytes actually I guess this is going to be a bytes instead it's going to be a struct bytes which is going to be a vec of u8 just the bite contents of the file and a stat and then down here here get data and now of course the question is how is the response structured get that response it has a data first and then the stat and a vacuate my guess is a vacuate is something that we can parse pretty efficiently in fact we already have a thing for doing that right so that's this that we implemented a while ago where read buffer just gives you a vec of u8 and so we're going to do bytes is reader dot read buffer right and stat is that read from reader so we've noticed that the order is important even though we could do this like they're sort of the same in terms of defining the struct but we actually do have to read out first the buffer then the stat because that's the order they are in in the response and this should be a get uses a get data response beautiful so notice that that's all we had to do now we have get data in theory right and now we're gonna try to test that by calling get data on foo and that's gonna return us a bytes in a stat and we want to cert that the bytes actually I guess this is gonna be a response and we're gonna rest is unrappable as in we don't expect to get none we get expect to get some well actually we can even do better than this I think we can do the data length is actually the data is gonna be this right and then we're gonna assert equal the response to we're gonna assert that rest well we're gonna make sure it's some so this unwrap would fail which is what we want we expect it to be some and then we're gonna check that rest of zero which is bytes is the same as data and we're gonna assert that rest of one which is the stat dot data length is data dot right and then I guess we could do the same thing with down here with bar not that we expect that to be any different we might as well the test for it let's see how that does fantastic so now we have get data notice how straightforward that was I promised last stream that would it would be straightforward ad and needed was straightforward ad we look at here this is the response we get back right so somewhere around here yeah this is this is hello bar right nine characters H E L L O space B a R and then the rest here is the stat and similarly up here somewhere we should have get data for foo which gets the response here 11 characters hello space of 32 a space and the world great so now we have get data and get children great let's commit that while we're at it all right so now the time is to add more sophisticated watchers now there are a couple of ways we could do this we could add a separate version of each of these methods where that separate function you pass in an optional watcher and that like that's totally something we could do it makes me a little sad because it means that you end up with a pattern like this where you for every method you have both that method and that method with an underscore w or some other similar kind of similar kind of pattern one thing that I like doing instead is to have something like this something like this so we're gonna have some some auxiliary data structure it's gonna be this and watcher or and watch which is gonna contain a zookeeper and a watcher and on and watch we add all the same methods that we have on the main structure so there's gonna be a lot of the same for delete and we'll have the same for get data right so the reason for writing the pattern this way is now if a user wants to not watch something they'll do like they'll have those zookeeper and do get data and they give some path right so that's all fine if they want to start watching that then they will do I guess the exact naming of this will be a little weird but they could do something like this right where this is a watcher we're seeing a trade watchable or something like that with some default methods it's extensible for all the interactions not entirely sure what you're thinking so the problem we have is that the well we could change the return type of all the methods to be something that the user can then call like dot and watch on the problem is that if they don't call and watch like we wouldn't know when to issue the request is the issue right so if imagine that get data returned like a watch or not we call it whatever we want and then now the user like stores this in an X right the question is when does the request get sent because we can't send the request yet because we don't know whether or not it should be watched so the user if they wanted to watch would do something like and watch whatever right and at that point now we could send the request at that point we could totally send the request but what if the user doesn't want to add a watcher right they would have to do something like X dot send or something which is a little awkward I wouldn't want them to have to do that and then the send is what would return the the future that they then wait on right which would lead to real really weird patterns like dot wait right whereas in this case if we have the watch proceed get data then now this is still going to return the future just like in the API above and if you don't want to do the watcher the pattern is the same right get data just returns you a future and so that's why I don't want this to be a property of the return value of get data but rather by the time you call get data it is fully determined whether or not you have a watcher of course this is also something you could do with this right if get if there was a get data W then this has the same semantics that the moment get data W is called you know whether or not there's a watcher right so I think there's gonna be a watch which is you want to use the global watcher well I think actually this needs to not be parameterized over W so this is gonna be a none we're gonna have to box this up somehow right so the way this is actually going to work behind the scenes on the protocol level is zookeeper doesn't know about the fact that you might have many watchers she's gonna get really awkward so the way the way watchers work in zookeeper if you want to have more than just the global watcher that's notified of all events is that you just keep track of the paths so if for if a notification comes in for a path you check for any watcher that may be watching that path and then you send to all of them which is a little weird because it because here the watch doesn't know the path yet oh but it it will know in the path though because the those methods will be called on and watch right right so here once we get here we have the path and so then we could register it so the the basic idea is we're going to register with the global watcher the fact that there's now an additional watcher they should notify for this given path so we're gonna have to figure out the pattern I think this is how we want this to be set up hmm yeah the the way this is gonna have to work is something like ooh actually here's what we'll do we'll find a better name for later so we don't want to re-implement all these methods either right and so what I want to do is have this be something like and watch with like watcher none dot get data and then I want all the implementation of all these methods to only be in the watcher because otherwise we have to duplicate them between the two right which is annoying and so I think though for us to do that we're gonna need an enum that has something like how would you like to watch this call and the the options are like none global or particular we could call it local but it's not really local so there's a global watcher there's no watcher as in you don't want to watch the call and there's custom and custom I think is going to be I don't think we want watch settings to be parameterized so there's it's sort of tempting to do this right and make a generic over F the problem then is what do you set F to if you're giving it a number of global is we don't have any trait bounce on it it might be okay sure we'll try it and see what happens I think this is gonna get really weird but I guess we'll find out so that takes a watch settings W and now the idea would be that if you call watch it's gonna create an and what I don't really want it to be called and watch because the user is gonna see one of these right like when they call watch they're gonna get back some type oh actually no that's fine so they will only ever see it in the context of watching they won't see that it's being used for exists so I think this is okay so that gives you an and watch where zookeeper is self and watcher is like watch settings none sorry global right so this means that if the user does something like zk.watch.getdata of foo yeah then the global watcher will be notified of that I think that that looks pretty nice right and then the alternative for watch width is gonna be something like watch width and they're gonna give some f that does like a print line f or something it's not terribly important dot get data foo then now if if the data foo changes then this closure will be called and also the global watcher is I think always called not sure actually I think it is always called we should check this exactly what the watcher protocol here is watches one watch when the data is chained yep well this isn't really answer the question okay server doesn't really say a client will never see a change for which it has a set watch until it first sees the watch event that's fine watch is ordered is what's the one-time triggers that's fine the same watch of this register for an exist and I get data the watch object will only be invoked once with the delete and notification for the file okay so they are only sent once so that if you have well it doesn't quite answer the question either but I think what this means is if the same watch of this registered for two calls then it will only be invoked once well that's sort of interesting I guess it depends whether we're expecting these to be closures or whether expecting these to be some kind of like global objects so for watch sorry for watch with are we expecting users to call like self dot watch with like event and then some closure dot get data get data foo or are we expecting them to watch with and then pass in like a custom watcher right where custom watcher is reused across many of these I think it's the former because you can encapsulate the latter pattern in the former but not the other way around why should you could do the other way around too I think we want to do it this way and then the user can choose whatever they want to do and so in that case the question then becomes if we get a watch event we notify all of the watchers for that path so that's fine do we also notify the global watcher I think so but I don't have a good answer for that I think we do let's assume that we do all right so this takes a this makes one of those that's fine this makes a watch settings custom watcher is not going to be public this is what all the other methods are going to call they're going to call I guess well I just want to shortcut for these methods are all going to have to do something like and watch none dot exists right and I just want to shortcut for the inner part here which I think we're just going to call like no watch we'll find a better better name for it this now has to be parameterized by the W I think it's a little awkward now let's not do that these don't have to be high performance anyway in which case this can be a box new where W where W is an FN ones from well that's a good question what's our watcher type again it just gives a watched event so the FN ones is given a watched event which hopefully is clone otherwise we have a problem watch event is clone all right and it returns nothing right so we're basically saying the user gives us a closure that's going to be given the event that happened and can do whatever it wants with it no watch does nothing so and exists is just going to be self dot no watch dot exists path and then remove the exists implementation into the and watch exists right and now this is going to be the watch here is going to be if let watch settings none is equal to self dot W then zero otherwise one so if we're not watching then give a zero otherwise give a one apart from that the implementation is the same right and then we'll do the same for all the other methods so for delete will be this delete and version these go down actually no I guess I guess delete doesn't really have this because you can't watch a delete so I think this you know hoist that up here might want to make these two different impulse blocks actually just for our own sanity sake so we'll have one impulse block of things that can't be watched and one of things that can be watched so exists as that get children does this so notice how these are all basically the same right and the condition under which they watch is also the same delete here then I guess get data is the last one so get data will be the same data here right so now we have at least the is not W anymore and this is not F anymore and this is I guess a box FN ones watch event to nothing probably has to be send but what's exactly for a game or no this is a programming this is not rust the game this is rust the programming language this confusion comes out roughly twice every stream let's see right so this does not take a watch settings these will have to be self dot zk so of course this only sort of works because currently if we set watch to one all that will happen is that the global watch will be triggered we will put go language too much right so so currently this will only ever trigger the global watcher because we're not storing this custom watcher anywhere so what we're going to want is something like FN register watcher that's going to take a path and a watch settings path is going to be a string I guess we want to make this really convenient we do this I'll explain why in a second so this will turn zk this was going to have to do something to register W with whatever dispatches events right so that's all well and good this is then going to do something like let zk is self register watcher path and so this doesn't need to take a W so this is the reason why I wanted this to be why I wanted this to return zookeeper because it means that we can do the the destructuring here because otherwise here would have to do something like register this and then we'd have to give self dot W but we want to move self dot W out of self and then I guess we could use zk here still be fine but this just reads a lot cleaner and so this will now use zk I guess zk dot connection and then we'll do the same for all the others right so all of them are gonna somehow register the watcher we haven't defined how yet and then they're gonna issue the request this is complaining for some reason this is complaining for some reason not sure why that is complaining expected type and watch fountain oh right this does have to right right self here is a different type self here is and watch but we really wanted to return back to the zoo keeper type can't use self anymore in the ones that are on and watch so this is gonna be zk zk right so the question now is how are we gonna register these watchers so if you remember the way the global watchers set up is when you first connect you're given a future that will resolve into a tuple of a zookeeper client and a stream and that stream is overall watched events and that stream is given to us by the packetizer I think maybe I'm wrong yeah I am wrong the stream is actually generated here so really what we want to keep track of is like instead of having these instead of just directly exposing the stream we want to sort of interpose on the global event channel and any events that match paths that we've stored we want to dispatch to that too not entirely clear how we're gonna do that though huh that is a very good question because basically somehow whenever we receive on this global event stream we want that notification to also go elsewhere the alternative of course is instead of doing this in zookeeper we do it in packetizer so this this registration that we do why is this complaining W does not this no longer take watch so that's nice we wanted to get rid of this this has to do something like this also has to return whether or not it should true for now we need to have something to match on here right so this is gonna be something if watch then one of my zero and that's gonna be the same for all of them because we've we've moved self the w in when we did the register path so this is arguably the reason not to do that but and this is gonna be this and watch and same up here all right yeah so the thing we could do here is we could send the watcher to the packetizer and have the packetizer do the work of also dispatching to other things I think that might be the easiest way to go actually so we would do something like ZK dot connection oh that's a sink isn't it the incur so what I was thinking was on and cure to have a method that's not in queue but like send watcher or something that would send that on to the onto the the channel that the that the packetizer is listening on however the stuff we send there has to take this form right so we would sort of have to we'd have to have some kind of enum that is the kind of stuff you could send it's probably the nicest way to do it though so we'll just do that so it's gonna be enum task it's going to ask so it's gonna be request which is gonna take a request and response right so that's gonna be that or a task can be an ad watcher just then take a path and a actually we don't even have to register the watcher if it's for the global one because the global one will always be notified regardless so I think this is gonna be something like if let well actually let's just match on W so if we get a watch settings none then we just return ZK and true and false right we don't even want to add a set watch to one if we get a global then we can just return true all we really need to do is that's the watch flight to true and then the the event will be propagated correctly it is only if we get a custom that we need to do some stuff and then at the end also return and then also return true so that we set the flag to be to be one right so I think this is what we're gonna have to do and then we're gonna have to figure out what to do in between here and the idea that we're working with is that we're gonna do something like send or add watcher path to string and W right so we're gonna send to the the packetizer the watcher and the path that we want to register that watcher for and so add watchers gonna take a path string and a like notify so instead of the other way we could go about this is instead of passing a closure that gets called like a callback we could have it return a future instead so currently we're gonna end up with this like fn ones watch event to nothing but like currently we're gonna end up with this right and notify is gonna be called as the watcher the alternative would be that when you call when you call watch or actually when you call watch with what it returns is the thing you're gonna call a method on and also the future that will resolve when that thing tracker triggers that might be better so how would we even express this right so the proposal is that this returns this and error I guess is nothing error is yeah is nothing so instead of it taking a watcher it's just gonna do this that does look a lot nicer and sort of more inspired with the asynchronous world the question is what to call it because I sort of want this to be called like notify and this to be called watch but that's also a little odd right because this is just saying like also notify the global watcher whenever something happens this is give me a watcher so maybe with the watcher and this is just watch it's all a little bit awkward I don't quite know how to reconcile these but the the basic idea is gonna be we create a one-shot do we have one shot somewhere here okay so we're gonna use futures sync one shot so this is gonna create a new one-shot channel and the custom is gonna contain the transmit end and then we're gonna return AW and RX like so right so now what the what the user gets back is is both the things that are gonna call the method on and the future that will resolve when the watch event triggers I think that's what we want it's good it's complaining about something though oh it's probably the error type is something else the error type is up here somewhere futures canceled which sounds about right right like the the error of that future is that the future was canceled which I think is basically what we want it does make the API a little bit more awkward because now you need to you can't just do like dot with watcher dot get data right because you're gonna have to remove the future that got returned but I think this is still what we want and now this actually makes this a lot easier because now we know that this is a one-shot sender watched event right and now this is a TX this is not a box anymore this is a one-shot sender watched event like so and then cure is now gonna have down here it's gonna have another method that's gonna be add watcher it's gonna take a path that's a string and a TX which is this one-shot business right it's not gonna return anything it is just going to do an unbounded send of task at watcher so I guess it's a little awkward that this unwraps currently and this of course now has to be a task so the things that we the channel that we have to the to the packetizer is now different right because now we're sending in an enum instead of instead of just a tuple like we were before so this is gonna be a request where the end response is gonna be TX so in theory everybody's happy except for whatever reason and this is complaining because this is now a task expected type now this is now a sender of tasks and it's complaining about that because it does not implement debug that's fine we can implement debug back we can now we can't implement clone but we can implement debug so now the packetizer actually has a channel here that is not this but instead it gets task right and so now the main loop of the packetizer or is our thing that reads things from the channel see it reads from the timer poll in queue RX poll this no longer gives an item TX instead so now it can get either a task request which has a request and a response in which case we want to give back the request and the response just like before but in addition to this it can now get an ad watcher which has a path and a TX so the question is what should I do here right so this is this is the packetizer gets told to add a new watcher what does that even mean the way we're gonna implement this is basically it's gonna keep a track of all the path paths that it's supposed to send things on and then it's gonna send on now I'll deal with that in a second okay so this is gonna do something like self dot watchers dot I guess entry path dot or insert so we're gonna have watchers are gonna be some kind of map from paths to from paths to a list of watchers to notify right and then we're gonna have to check that whenever we send out on the watcher right so this default watcher where do we end up using default watcher is on here this is an active packetizer isn't it yeah so I think watchers is gonna have to be a field here watchers is gonna be a make it a hash map for now it might we might want it to be a try eventually but for now it's fine it's gonna be a VEC of one-shot senders of watched event right so the idea here is this is custom registered watchers and so here watchers is gonna be nothing by default and then the the setup that we want is that whenever we're whenever we receive an event so down here remember how xid equals one is a watch event that gets sent to us from the server then here in fact we already have it to do for it do you do unit testing there are very few tests in this library currently in part that's because zookeeper is not particularly well built for testing but that's totally something that should be added I agree I have not been focusing on testing in this particular stream some of the earlier streams have some more focus on how to do testing right so here really what we want is it's gonna be a little so what is in and watched event path nice so if E dot path I guess here's really what we want to do watchers is self dot watchers dot get mute E dot path right custom so in this case there are custom watchers for this path and so we need to notify them and so here for W and watchers in fact might just want to remove this that's not gonna be correct but let's leave it like this just for the for the time being so the idea is that we're just gonna loop over all the watchers and for all of them we're gonna do W dot and then clone of the watch right so there's the basic pattern here makes sense that we have this like map of path to watchers and whenever we get an event if we find that we have any watchers for that path and we also notify those watchers of the change right I think this should in theory be all right this is actually gonna be on the active packetizer it's also a little awkward because we're gonna need when we transition from active packetizer to reconnecting we also sort of want to notify well dropping them is fine dropping them is just gonna cancel them which is basically the behavior we want FN1 is not implemented for back 54 here too we just ignore if there are errors so here one-shot senders are also not asynchronous in futures land so if you send on a one-shot sender it will never block which is really neat because it means that we can do it in this loop and it won't actually be a problem it won't cause us to go to sleep or anything you can see that if we go to documentation for future 0.1 we look at sync one-shot sender send and the only case in which we get an error is if the other side was dropped and that's why we just ignore the response ignore the case where the receiver has been right because we don't actually that doesn't matter to us we still we still want to continue processing and there's no where we can send the error so we just drop it now is it happy all right and now what we want to test is let's just see that the that the test that we had already will still pass right so remember we have this setup where we we watch exists right so we're gonna change this to be zk dot what do we call the thing watch yeah watch that exists this is also gonna be watch that exists and are there any other we do this nope I think that's all of them now done here just to see that the watch events still work the way we expect them to when when we're using the global watcher there's no big reason why it shouldn't but it's nice to test and now we want to test that the the localized stuff works too so we're gonna do this then we're gonna do well I guess where's our end then this thing so we're then gonna call down here actually let's do it the other way around it's gonna be a little bit easier so we're gonna get a future and an exist watcher from calling call this with watcher these names aren't great exists right so keep remember how oh sorry with the watcher food dot exists and then ZK I guess we expect this to be that ZK underscore ZK watch exists so I don't know if you follow that but the with watcher returns us to futures right it returns us one future or it really returns us an end then or the an end watch which is gonna be this few tier and that is where we continue calling exists and it also returns this other future that will resolve when this watcher triggers and we expect that to trigger here I guess we check all the watchers at the end currently but right here we're right here rather we expect that exists to have resolved where's the place we get watch events down here somewhere yeah so the moment we do the create we expect the exists W to resolve and so we're gonna check that right here that that is not no deleted but no created I don't know if that if you follow that that was a little bit fast but the basic idea is that when we after we first connect what we do now is we set up an exists with a custom watcher and that it initially exists should return nothing for food and then we set up an exists with a with a global watcher as well and that should also return none then we create that should trigger both the global and the local watcher and then we check that the that the local watcher indeed has resolved right before we continue and then we go on with all the rest of the stuff this did not like that error is failure right this so this entire path expects that the error types are failure errors but for us the the error type here is canceled right like the the event loop exited and you got no signal and so I hear I think we just want unreachable might as well format error got or exists W failed 21 that needs to go here can't compare watch event an option watch event who is also right see how that fares beautiful okay so this means that the the watcher we set up also correctly works right so this assertion worked correctly which means that the watch future we got back here indeed got triggered and the global watcher also indeed got triggered so that's great that means that the basic setup works right so but I don't want to commit it just yet because the current implementation actually has a bug that I just didn't tell you about and the reason for this is business so notice that there are a bunch of different types of watchers right so there's created deleted changed and child now because triggers in zookeeper are one-shot imagine that you call get data on some path and you call exists on that same path and then a created event comes in for that path if that happens the in the current setup that we have the created event would trigger get data or would trigger get children right which is wrong that should not that should not actually happen because get data can't be triggered by a created and so we actually need our map of watchers to keep track of to keep track of the event type that's being requested to and I believe that if you look at rust zookeeper that is basically the same thing it does here it has this watch type why is that different from watched event type how are these different yeah so that's why the the synchronous rust zookeeper has this enum called watch type is precisely to track these kind of changes and so we could test this we get out a check for this too by doing I guess a food to actually it's a little annoying to do hmm it's just a little awkward to chain these with watcher calls but not impossible the reason it's annoying to chain is because let's say that down here I wanted to do a zk dot with zk dot with watcher right this is gonna give me a the sort of the real future that I want to continue chaining and the eventually thing for when the watcher I guess we call this watcher right and then I sort of want to return watcher here but sorry return few tier alright fuse dot exists or whatever call I want to make right the problem now is of course I've lost watcher I have no way to carry it along and so that means that you have to annotate all of your futures with this like dot map move zk zk watcher right all of these calls are gonna have to chain the watcher along until the point where you need it or you have to stick it in like a data structure you have access to somewhere which is also a little bit annoying so I think I don't think we're gonna do anything in particular with that now but I wonder so what I what I was hoping to do is add a add a part to this test where it will add something like a get data watcher and check that that data watcher does not get triggered actually looking at them again though is that even possible here so a changed event okay so I guess that would actually be the thing that get children can only be triggered by a deleted event but in our current setup a change event would trigger get children similarly a child event would trigger a get data right which just seems wrong so we do have to keep track of this type all right might as well just do it where's my here so in where we want to Steven live I think we want this to live in it's a good question actually proto maybe the proto is already pretty large I guess it'll be in proto that's fine right so when we add a watcher down here register watcher is going to take a path and a what to call it E type it's gonna be a watch what do we call it a watch type like so and when you add the watcher you're gonna have to send the E type long to and now in proto a task contains an E type I guess this really should be W type given that it's watch type so this takes a W type which is a watch type here when you add a watcher you also take a W type which is a watch type and include that when you send it to the packetizer we're basically just chaining this along right and then add watcher is gonna push and that is now going to be a TX and a W type so watchers so this is now a VEC of this of watch type and now when we walk this we can't actually remove what we're going to want to do is watchers watchers watchers here what we're going to want to do is get mute of this and then we're gonna do something like watchers dot retain and we're only gonna retain the ones we remove so where the where the watch type matches so retain is a method on a VEC that iterates over all of them and if you return true then the element is kept in the VEC otherwise the element is removed actually that's not gonna work either because retain does not get to own the thing it's also a little awkward hmm I think we're gonna do something like we have to do for I in zero to watchers dot Len if I is greater than or equal watchers of Len then break you'll see why it's because we have to so we have to mutate the collection while we iterate over it and retain is normally the way you'd want to do this however retain has the problem that it doesn't give you you can't own the reference that it gives you easily just a little sad I sort of want something that's like retain that gives you a thing and you return an option that thing thing to decide whether or not it stays but that's not currently what it does and so here we're going to do is we have to match on the watchers I and watchers I dot one and the actual event we got so that is a watched event which is where which is event type E dot event right so we're gonna match on the the pair of the event to this watcher is waiting for or the kind of kind of watch this is and the event that we actually got to see what the two match up and then we're only going to remove the ones where it actually matches right so here let's say that we get watch type so if we have watched a child and any just write these out watch type data this seems like it's missing one oh I see yeah so right so this is it's a get children watch which should only be triggered on deleted or on child so this should be on watched event type delete or deleted or on was the other one that get children can trigger on child event pull up this where's watched event type node children changed so node children changed either of those will trigger so that would be true any other thing with child will not trigger if it is an exists then a watched event type children changed it's going to not trigger and any other type will trigger and for data it is deleted and changed are the only two so for data deleted and no no data changed these will return true and anything else with data is going to be false right so now we have the exact cases for when for which type of events trigger which kind of watchers and then if triggers then we send then I guess we're going to do something like so we need to the problem is in order to send we need to own the sender which means we need to remove it from watchers so swap move the current one so swap remove is take this thing and swap it with the last thing in watchers and then remove the last thing the reason you want to do that is if you try to remove something in the middle of the vector you have to shift all the things that come after it down and that's pretty inefficient whereas swap remove just gets rid of that problem so that gives us the W and this watcher is no longer active and then we send otherwise well otherwise we don't really do anything like it stays in watchers and so the idea here is that we're gonna keep walking until we get to the end of the array or the end of the list of watchers the reason we need this condition is because this range when we initially compute watchers actually we just do while make sure we don't walk past the end given swap remove right so the initially this range is gonna be if we just had this be zero to watchers all Len and didn't have this condition then it would compute watchers all Len is the beginning of the loop right but then as we swap remove we're shortening the vector so it's now looping to let's say that the vector starts out being this long and so that's what the range is set to and then as we move through it we remove a bunch of stuff from the end so we're walking this and like the vector is only this long so we walk to here which is all fine but then we start walking this space because initially it was this long but here all those items have been removed and the program would crash because that's why we have to recheck the length on every iteration I did not like that watched event type so I guess this will be watched event type complaining about something else too but I don't know exactly oh this and now of course if watchers is empty then we want to remove it which there isn't really a nice way to do it might also require NLL which is a little sad so down here we're gonna do something like so the the only we sort of want to remove if there are no watchers left for a given path then we want to remove that entry from the map just so we don't take take up that additional space then true then removed is true and then down here if remove then self-watcher is removed we don't care about whether it got removed well it should always get removed try to remove watcher that didn't exist can't swap remove causes to skip a watcher because we never look at the watcher that was lost no that shouldn't be a problem because remember we keep iterating until we're past the end of the vector so if something gets oh you're right we need to recheck the current index oh yeah you're right you're totally right in fact totally right I think this has to be watchers dog Len to zero I don't know if I'm allowed to make a range that way let's do this the old-fashioned way while I is as I size while I is less is greater than equal to zero then do this you're totally right that's a good catch I think this should be right then so we start at the end and we walk backwards through the array it might be flipped for you I'm not sure so we walk backwards through the array and swap things to the end to remove them so then the thing that we swap into the current location we have already processed because it was at the end and so therefore it's correct to continue moving back yeah I think you're right very not found oh is it just exists instead of exists and this it's complaining because no matching scent because there we don't care about the great see what that oh hello oh watch type should be this should be clone copy debug partially oh hey you're back five 34 W type right we need to extract that from this truck up here cannot find watch type right so this has to use ZK error and watch type so we even have it in scope two or one right and these now have to say what type of operation they are specifically it's a watch type exists this is a watch type child and this is a watch type data whoo oh failed that's a fail to enqueue new request cancelled panicked the length is one but the index is one that sounds like the while loop was not actually correct 373 what oh this file 373 yep that is correct this has to be minus one let's try that instead still crash is it expected whoo oh cuz it's cuz we ran it previously this is one of the annoyances of testing things that are persistent because our test failed halfway through so it it created the node and so now when we run the test again the test assumes that the node does not exist at the beginning great fantastic all right so now just one thing that we might want to do it probably doesn't really matter but the we could make the path the the watchers list to be a try so that it's a prefix tree instead of a hash map if you have a lot of paths they're similar this might save you a bunch of storage I don't think it's terribly important for this so we're gonna do at custom watchers good job team we have custom watchers I'll push in case people want to see the alright I tried to write a Travis file it's a little bit annoying it turns out running zookeeper on Travis is non-trivial okay so the question is where what do we want to tackle next I think we want to add some documentation to this because it's getting pretty large luckily we have the advantage that there's a bunch of documentation that's been written for the Java version and most of that should just straightforward apply to our implementation as well so let us start by adding deny missing docs we also want to deny missing debug in full and missing in her attribute is not permitted following in powder attribute these are all outer attributes oh unknown limp last missing in full it's a called missing debug in full it's whatever this thing is yeah but what is the actual oh it really is this okay I think there's one for clone to or maybe it's copy actually oh right these all need to come first great so the top level create docs I don't want to write quite yet the zookeeper create docs here we could use the new external docs feature where you can include documentation from external documents into your documentation directly it's pretty neat it just landed basically you can do it's like doc include and then some path and then it just that becomes the doc string of this particular struct I don't actually want to do that in this case because I think we don't want this verbatim we want at least some changes to it let's format this little the one of the advantages to doing this so of taking the documentation from a different library rather than or from in this case the sort of original library and then modifying it as opposed to just writing our own is that as we read it if there's something there that we find that we've missed we'll detect that if we just write our own we'll just document what our library does which is not necessarily what the library should be doing okay let us not use them quite that directly a connection to zookeeper interactions and zookeeper are performed by calling the methods of a zookeeper and we sort of want to say so remember how a while ago we also derived clone for zookeeper because you can have you can totally have multiple zookeeper instances or you could have multiple things that send to the packetizer and so we will sort of want to say that here I call them the methods of a zookeeper instance all clones of the same zookeeper use the same underlying connection isn't it fun also in the in the Java one they have to document that things are thread safe unless otherwise noted in Rust this is expressed in the type system the client was on hard as they are vertically we don't currently let the user set the certain timeout value so that's not terribly important the zookeeper instance will then no longer be usable and all futures will resolve with a protocol level to make further the application was created new zookeeper instance so this is totally true for us right like if the connection if the connection to the server goes away then the logic we wrote in part two ensures that we try to reconnect to the server if the session expires then if we look at proto mod remember how we have this like sort of nested a structure now where the packetizer has a state that is either connected or reconnecting and so if we where's the place where we try to connect down here somewhere yeah so if we pull the underlying state so this is trying to drive it forward and we get an error and we were connected then we try to connect again right we send another connect request and if we get an error while we're trying to reconnect then we just fail so here I guess this is something that we right sorry here this bit so if we get an error while we're in the connecting phase like the reconnecting phase then we currently just returned that and we give up now technically here we should do things like if we get a session expiry error then we should let everyone know or if we get a session expiry error then we should then we should return like we do now but if we don't get a section of spire if we just try to connect to a server again it fails we should probably just retry and try a different server that's hopefully something we'll get to today as well but for now we just exit which will cause the packetizer to exit which will call all futures to resolve with an error which is the same thing we're promising here we might want to leave a note here that we will not auto connect is your service server the client interface it fails in other words it's not automatic try to connect yep and then here we sort of want to add a note that so this is saying that we're trying to auto connect which is true if I guess what we could do here is add like a section on limitations up here that might be a better way to go about it we want to say that at least for the time being multi-server deployments are multi-server connections are not supported also errors during reconnects our client does not recover from errors during reconnects such as session expiry right so that's totally something that we sort of would have to fix it shouldn't be all that difficult right so it's basically in this code in in the packetizer we're here you would have to check what kind of error you've got and if the error was a session expiry then you indeed want to return the error if it was not a session expiry then you just try to enter the connected state again right you just try to connect again you're gonna have to remember like the the zxid from last time and what not but that's something that should be doable secure APIs are either asynchronous or synchronous that is not true in our implementation and the watches other successful secure API calls oh actually that's a good point this is also something we don't handle so this is one of the reasons why this exercise is useful if a zookeeper API call fails then the no watch should be added whereas currently for us if it fails that's not something we do anything about hmm that's a little awkward basically the watch is not actually added unless the call succeeds in fact there's a race condition that we've introduced now so we add the watch or the watcher I don't know why they refer to them as both watches and watchers we added the moment we send the request whereas in reality we should add it once the response comes back because you could imagine that we that we send the request and add the watcher as we currently do some other call modifies the node we trigger our watcher even though and then the response to the original request that would have added the watcher comes back and now the watcher has been triggered but the watcher should only be triggered after the response the client gets back so I think here actually we're definitely making a mistake so let's leave some space here and deal with that specifically the observation here is that I think what this has to do is with the watcher I think all of these have to return just an end watch yeah I think the API will actually want is instead of this just have them all return a width watcher and I guess is a custom it's not going to take a TX so custom is not going to take anything and then what we'll do is we'll have this return a zookeeper and an impulse well we're not allowed to use nested impulse futures but this will be a one-shot receiver watched event actually won't even be that it'll be this right so if it succeeded if the operation succeeded then you get back the thing that will tell you about the watch event I think that's what it's gonna have to be this is gonna make some other things a little bit more complicated but same thing here that is going to be and similarly it's gonna be a little bit annoying for the global watcher this API is not pretty it's really not pretty I'm gonna have to find some way around that yeah that's not great mmm I don't know I don't have a good sense for how to tidy up this it basically means that we need to send the the transmit end of the watcher along with the request to the packetizer and then the packetizer when it sees the response it adds the watcher so it needs to sort of buffer up all the watchers that it's going to add that's a little weird I guess this just means that we need we're just gonna have in queue be the oh here yeah okay here's how it's gonna be so exists it's gonna take a path a string right a path is a string and a I think we're gonna move the whole watcher business the whole watch settings and move that up to here so a watch or yeah so watch is is still gonna be a one-shot a one-shot sender of a watch event so and then we're gonna have exist and get children and and get data events instead of including this watch you wait they're gonna include a watch that's gonna be a use it's gonna be a watch so if we get children it's gonna be the same for path the forget data the reason we want to include the watch with the request is because when we in queue the request that's when the packetizer needs to remember that when the response comes back it has to install this watcher doesn't implement debug sure it does now we don't need task anymore because all the things we send are just going to be requests actually this is not bad this is not bad this is gonna simplify the code a little bit so the in-cure is still just gonna take a request it's gonna give you a yeah it's gonna give you a response add watcher goes away this now contains so we bring in-cure back to include just a request and a one-shot receiver of response is that what it used to have whatever what do we use seven-task result response in ZK error so and this is now just gonna send it use the same which is just a tuple this now receives the same thing it used to receive which is just this tuple again here so the plan now is that the packetizer it's still gonna have this one-shot sender watch type that's all fine but it's also gonna have something like a pending watchers which is gonna be a map from XID to to a watcher that it has yet to install to watcher to add when okay right so that the idea here is if an operation with that XID succeeded then add this watcher and so pending watchers here and then down here where do we have our XID is one so that business is all still correct the difference down here is if the XID is something else down here if let some watcher is self pending watchers dot remove XID right so if there is a if we were if we're supposed to add a watcher when this request completes then we do that here and that will be pretty straightforward it'll just be watchers dot insert we don't have a path so pending watchers also needs to include the path because we need to insert it into we need to record the path in watchers when we do eventually add it so then that will be W dot zero insert W two it's not very nice code but it'll do the trick right so what we're doing is if we get something with this XID then we extract this string use that as the key in this map and then we insert the latter two things so these two these two and we insert those into that back I guess this is gonna be entry this and this is gonna be default watchers no pending watchers and it's just going to insert oh it doesn't know the XID yet doesn't need to this is just gonna be now we always get the first response and so we in queue that with that XID and then we're gonna match on the request and get to do that probably not I have to match on the item and if the item is a request get data right so get data does have a watch property the watch is a watch custom so then we need to add that depending and same for get children that has a custom and same for exists that has a custom right so the observation now is we instead just look at the request and see whether that had a custom watcher in that case we will do a we will have to do AP dot pending watchers dot insert self that XID which is the XID of the request and now we're gonna have to steal the watcher and we we need to own the watcher in order to stick it into here so we're gonna have to do a man place watch with a watch this has to be global the reason we have to so the reason we have to swap out the watch is we need to own it so that we can insert it into pending watchers and the reason it's okay to replace it with watch global is because from this point forward the only thing the watch field of the request is going to be used for is whether to send a zero or one as the value for watch and global would cause it to be set to one and none would cause it to be set to zero and we can't set it to custom because custom well we could have this contained an option but that's also a little weird so we just set it to global which causes it to be set to one as we could document that here set to global so that watch will be sent as one way so now we're gonna insert into that the I guess we're gonna need the path so we're path dog two string and W and the watch type data and then we'll have to do the same for the other two is also gonna be path is also gonna be path notice that these are all basically the same the only difference is what they what the watch type is so here we could pull the same trick as we did in the other place where we in request serialization where we match on all of them first and then just do a second match for this maybe that is nicer sure let's do that so it's this or this or this and then W type is gonna be match item cast get data is gonna give a watch type of data get children is gonna give one of child and exist is gonna give of exist anything else is unreachable and then this will use W type and then these can go away anything else we don't deal with whoo alright what is it complaining about pattern does not mention the other fields don't care about the other fields what else is it complaining about pattern does not mention fields path watch missing fields but it's very confused why does it say that 528 why not a lot to do this thought I was a lot to do this like I want to bind to watch but only if it's of this type pretty sure I'm allowed to do that but I guess I could just do this and then if let watch type custom sure apparently that's okay don't know why it's why it's suddenly okay with it now no variant custom found for oh this was just watch right plenty here no that's just old stuff undeclare type of module watch settings 224 live 224 right register watcher is no longer necessary now in fact none of these are oh no they are this has to be a watch is the only thing that's important so this will be self dot zk and this will be watch is self dot watch and then this just has to use this and watch right so the only difference is that we're instead of just sending a zero one for a watch we're sending along the entire watch thing that we want right and then I guess we're gonna have to TXRX is we have to create the channel that will actually be used it's actually a little awkward how that's gonna work let's leave that aside for now self dot zk this is gonna self watch this is gonna be self dot zk so we we need to when we create the end watch in order to create a watch we have to create the transmit end of a one-shot as we'll create the receive and well we have to stick the receive and somewhere so that we only return it when the thing resolves which would be down here so this is where this is now in Rx I guess actually I totally be down here zk no it doesn't need to be here we only want to give back the Rx and I guess technically this should be an option but this API is really awkward right because you only get back a receiver if you asked for a receiver I worry that we're like abstracting too much over the over the fact that this these may or may not be watched maybe the better API really is one where you you're only exposed to this if you hmm that's pretty awkward so my worry here is that well actually this receiver will only be sent on if you used with the watcher because if you use watch you also get an end watch but then it will not be sent on because we'll be sending on the global watcher so it's almost like we want this to be a watch globally struct that is really just an end watch but we hide the we hide things from the return type this would be a with watcher and this would be a so here we can just map out the watcher right so in all these methods the user wouldn't see the sort of option receiver at all so this is an option Rx so this means that the exist type in here would have to be off yeah I don't know how to how to make this API nicer except by having lots of methods right you would totally imagine that you have like just like in the original API you have a bool that's watch on all of them and then you have a separate version of them to give you a watcher it could be that we really just want to replicate these maybe they are different enough that here's what we can do fix well map exists this is going to help a lot actually don't know why do you think of this so if we move this out right then now exists is really straightforward right because this is just going to be this method now right so now it doesn't really hurt to replicate exists and so now we could have a separate one for each of these and then the return type is just not a problem right so now it'll be this and this will be watch none yeah I think that's the way to go why is it right so this will now just be self-dot connection and then we'll do the same for all the others right so we'll have a map get children we have to set a return type but I'll do that in the second because now we can just use these and build on top of them right so and this will then be here and then get children response and then what's the last get data response yeah this is gonna be much nicer so now the methods that we have that do not add any kind of watcher are just straightforward here sorry I'm just moving a bunch of code around it should be clear once these are in place okay so the things we're looking at now are the the three sort of watchable methods directly on sue keepers so this is without watching them at all right so in this case watch should be watch none for all of them and the return type for all of them should be this this doesn't have to be a future at all even does it no I think this just returns one of these a result of that well sort of that's actually a little awkward I think I made that sillier than it needed to be this for exists it's really just a stat right I think that's the type exists as it's an option stat it's an option vex string and that's an option next step okay so this is an option string so great why is it saying that's not okay expected close the limiter that is true now it's coming associated type bindings are not allowed here that is true it in fact has to do this in fact I'm oh that's the one place damn I was unnecessary didn't copy it to the one place where the signature was actually correct here should be a vex u8 and a step these can actually be important all right so these top methods now why is it this should be so so the reason I wanted to move them like this oh what is this cannot fight type response should be proto response this should be result proto response zk error I think that's the type of R so yep so for these these top methods the implementation is now a very straightforward and then what we could do is if then we don't need no watch anymore there's now with watch this could really just return a watch globally that just wraps self and then just calls these methods again right so watch globally it's just a wrapper around self just a new type around self that has these exact same methods so pub struct watch globally just has a zookeeper inside Intel inside right and that implementation has the same methods in fact with the almost the exact same implementation that's not what I'm going to do this the only difference is that instead of a watch none it's a watch global and instead of self it will be self dot zero give you a zookeeper this is a really awkward way to work around adding a boolean argument but I think it results in code that's nicer to read right like if you think about it all we're doing here the only thing we're doing is to change the we could do this even better we could watch boole if watch then watch global just to make us all a little bit happier is this w path balls because now this implementation could be self dot zero dot exists w true even less duplication and then this would be get children w watch pool so notice that behind the scenes we sort of have the same API as the the synchronous crate really but we're just trying to expose it in a slightly nicer ways using what are what essentially amounts to combinators so they're the w methods are not public the public methods all just take path now get children w false and the one in here is going to be get children w true and the same forget date it's gonna be this use this just like the other one and it's gonna be a one without a w here who in fact we can actually make this even better so I'm gonna do that in a second but so this is self get W's exact same pattern for all of them and down here this is just gonna be get data w so the question of course now is how do we get the actual what is it two parameters right the question is how we add the one set of custom watchers and we can do that a little bit nicer by by instead of it having it take a bull having it take an option actually having it take a watch I guess is really what it should do so this takes a watch and then just uses that directly here and then this would just pass in watch none this takes a watch that in here false it with this takes a watch so watch none right so we now have this sort of helper building on helper building on helper I guess in theory we don't we didn't need to extract the maps anymore with this pattern don't know why I didn't think of this before so this just returns a good watcher so a watch globally is pretty straightforward and in fact a with watcher is almost as straightforward the only different in the with watcher is that it also includes the one-shot receiver right for the watch event that is the only difference these are now tuples right so if the operation succeeded then you get both back something you can wait on for the result and you get back the the actual result of the operation you did and now these alright these all have to be watch global these all have to be they have to make a one-shot right one-shot channel and then they do something like watch custom tx and then we're gonna have to hear dot map if R is okay yeah so R here is basically a result where this is the okay value and this is the error actually no R here is just that tuple right so map is only called on the okay of the future which in this case is this from the underlying exists w right the item is this which so we're mapping on one of those so I think what we want to do is let zk is r.0 and then let V so the value is going to be r.1 if it is none well I guess it's just going to be R one dot map that's going to be a stat and we're going to make that be move RX and then we're going to return fact becomes even more straightforward it's just this I don't know if that makes sense but we're calling the underlying exists w right which is is just takes any watch it doesn't really care and then on the result that we get back we map the option that normally just contains a stat when you call exists w and we put in the RX when it succeeded if it did not exceed it just gets dropped in which case the the we never give out the receiver either so it's not really a problem right so here we whenever you want to use something with a watcher we create the one-shot channel we send the transmitter down the pipe as a request when the response comes in the response will be generated here if the response is an okay which we know because map only is trios on okay and the option is some which we know but this map then we give out the RX otherwise we don't give out the RX and it's just dropped and nothing happens nice so the same pattern we can basically use for all of these I think so this is just going to be this instead and this is going to be a little bit different just because it's already a couple so this will be white since that and we're gonna make that just a three tuple instead of a nested tuple like so I guess instead of s we should call this C for children it's not terribly important and now all the stuff down here can all go away and now of course the API here changed a little in that the API is this dot with watcher dot exists and this is going to give us a yeah this is gonna be a little bit annoying right because here we're gonna get the RX back here and we're gonna we sort of want to eventually receive on this RX right do how do we pass the RX forward specifically we need the this is the watcher right we need the watcher to be passed forward to down here well sorry to down here so this is now what used to exist of course the way to do that is we inspect and then here we actually have to this is where it gets annoying where we have to map it forward yes we could be a little bit nicer here and this so that this becomes a little bit less annoying right so notice we're just like change this is what we've done this a bunch before with futures to we sort of end up having to chain values along the futures because we get the exists w like all the way up here and then we need to access it all the way down here and so we need to make sure that all of the futures were resolved in between carry that value along so here we get it in and then we have to make sure that we pass it along down here w this ignores it this is w and uses the exist w let's see what oh that's a lot a lot of errors there proto mod 547 right these have a bunch of fields that we do not care about watch is private I'm 29 is it really make it private no it does have to be public great watch type also only has to be upgrade 29 don't need watch that is true 69 expected receiver found sender oh right and similarly I guess this should be a sender alright we're getting there slowly but surely 425 remove xid has to be a ref 549 ambiguous associated that is the weirdest error I've seen for that before expected sender found watch a little awkward we need to extract the actual watcher so this will be a watch right it won't be the the inner thing of a watch custom so we need to extract that out and we know that we can't possibly get here because we just observed that it was watch West 737 right and this now has to be right so the now that we have this watch type when we serialize that onto the wire then all we really want is we want to write out a one if we want the thing to be watched and zero if it does not which is pretty straightforward it's just if let watch none myself then it's a zero and in every other case it's one request 132 right this now has to be a watch it used to be a copy type right it used to be a 540 cannot borrow a field of immutable binding is mutable and now we're back to missing documentation which will just so this is how much work can end up being by just following the documentation of the official implementation just to check that your implementation actually matches because that's what we were doing when this started all right let's see this should just be zookeeper expected option hmm that's a good point so for exists you actually get the watcher even if the operation did not succeed this is not true for get children or for or forget data because the only way they can fail is if the node does not exist but for exists the watcher is still added because you can learn that the node was added I don't think this is true for get data it's a very good question let's see what the Java doc say so for get children with a watcher and the call is successful no exception is thrown and an exception is thrown if the no node exists is this not true for exists thing yeah so exists does not throw an exception it just returns null so therefore the watcher is added about get data get data throws an exception if it does not exist okay so it's it's only for exists that this is the case specifically for with watcher on exists the one-shot receiver is outside of the option is all this means that here you always get the RX if the operation succeeded sorry even if the operation failed there's a protocol error you obviously don't get one see how that turns out 361 right so now you always get that and this is not an instant it's not great that's not great so that sounds like we didn't get a notification somewhere so let's see about request exists so where is this even created so it's handling the response to node created and at that point that's where we block for exists w so this suggests that the dispatcher never actually sends out the signal here so something is definitely wrong right so it does get the watcher right it gets both the custom watcher up here and the global watcher down here then it gets the event down here but it never sends to the custom watcher so the question is why huh so let's see in the packetizer exists so I guess this is here watcher for XID this on this path for the W type and I guess pending watchers we want to see whenever that gets triggered for XID this turned into real watcher I guess this is XID let's just see what that gives us oh we're gonna have to delete the note again let's see what happens so here let's see added so it did add it for XID one on slash foo for exists and they response XID one without code exists this doesn't seem like it's it doesn't do anything in in return to this so that's probably our problem so for whatever reason this doesn't trigger because it prints this right it prints that the it prints that it got the response to XID one we know that oh it does produce an error huh yeah that's why but that's still a little awkward right so this means that we sort of end up installing this thing I guess this is like if error dot is none it's sort of a special case right so normally if an operation fails we don't want to to turn the pending watcher into a real watcher the exception to this is if the is if the operation was an exists and the result was no node that's pretty awkward I wonder what other are there other errors that we can get for an exists call a map exists response so map exists response only handles the case of no no right hmm like either there's an error either there is no error or the opcode I don't actually know what the nicest way to do this is it's I sort of want to do like if the opcode is equal to opcode exists and the error is equal to ZK error no node right like that is the only case in which you want to add the watcher normally watchers are or I guess watches are only added for successful operations the exception to this is if an exists call fails with but it's telling me that I can't compare ZK errors that is definitely true because here and partially please and now it should be okay right really why not have all code here oh it's request and we need to delete because the test failed halfway through right great now we're back we are back let's just see that the progression here works the way we expected so it creates the session so that's all fine it adds a pending watcher right and then when it gets the response it turns it into a real watcher even though the exists call failed here it gets a watcher that's global but it is not at a pending watcher nor does it add a normal watcher and then when it gets this foo it just notifies the global watcher which is still will be expected to do nice I think we're all good so let's see actually I do want to get rid of these these map things because they're no longer needed so map exists response here it's gonna are so that goes away map get children response goes away and with first trying to add a watcher from the pending ones you do not leak the watchers for the unsuccessful requests let me see if I parse that correctly with first trying to add a watcher from the pending ones yeah you do not leak the watchers for the unsuccessful requests well so it gets gets removed from pending watchers right and then if this condition if these conditions are not true it doesn't get inserted into watchers and therefore it will be dropped which is the correct thing to do right because in this case it should never be used by any of the consumers anyway like the RX should also be dropped in this case so I'm not entirely sure what you think is with this I think this is right but I've been wrong in the past you first remove it from the pending ones on a successful request yes it always gets removed so remember it pending watchers are tied to an X ID right which there will only be one of we should only get one response from an HX ID so I think this should be right alright so let's commit this before we continue our crazy path of docs here that's a formatting change oh so fine we want this to be only register watches once request has succeeded back to the docs now that this statement is true again some successful zookeeper API calls can be watches on the data nodes in the zookeeper server other calls can trigger those watches once I watch it triggered yep server the interruption oh I see you just wanted to point out that no that's fine so you pointed out that in the the code is better now because in the past it would leak watchers yeah that's true the other reason why it's better now is because it also doesn't have to create one-shot channels that will never be used the old code sort of had to create them regardless because of the abstraction that is now gone so I agree with you I think this is much better okay so this is slightly different for us we don't actually send that so this is something that we'll want to add but don't currently have I also want to add some examples here it seems a little bit weird for some of this documentation to be on zookeeper yeah I think some of this I want to be top level documentation in fact probably what we want is something like this right the zookeeper programmers guide is sort of like this is the kind of stuff that or even the getting started guide or the overview right it's almost like well this stuff is almost like what I want to bookkeeper what is bookkeeper overview this overview right it's like these kind of things I want to be in the in the docs but I think like I will probably do some tidying up of this after I finish the stream because moving text around is not particularly interesting to watch it's more that I want us to add it and do a parsing of the text to see whether there are things that we haven't implemented right which we've already found one right all right so back to this so I guess well I guess this connect which would be what new in Java I don't know if there's a documentation for the constructor sure let's see whether any of this is true for us yeah so we we certainly don't have CH root yet which will want this is similar to what the the synchronous Rust Zookeeper implementation has where like it will if you add this this CH root argument then any path you get back will have that prefix removed for you so you don't need to deal with it it's a very convenient feature that we just haven't implemented because it's not particularly interesting but definitely something that we would want to add going forward and also of course currently we only support a single server to create a zookeeper instance connect to a zookeeper server instance yep I guess here we don't actually take a watcher argument right we just return see when the connection when the session is established a zookeeper instance is returned along with a that will notifications of any changes in client state well I guess it really is state because it's server state as well this notification no it's not important for us so this is a part of that that we don't yet have that we may get time to do in the stream it's a little unclear again that we ended up in the the rattle of of watchers we'll see if the connection to the server fails the client will automatically try to reconnection fail so this is the code we wrote last time so only if reconnection fails is an error return to the client requests that are in flight during a disconnect may fail and have to be retried handshake is private so that's fine all right let's look at create you guess this one the ace your greatest version of create great thanks good docs create a node with a given path the node data will be the given data and node ACL will be the given ACL that seems unhelpful as its contents let's not be overly verbose in docs the flags argument do we have a flags argument no the mode argument specifies whether there will be created as a femoral not specifies additional options for the newly created if mode is set to a femoral what do we call this mode do we remember oh we also we need to do a bunch of work on this ACL stuff we currently just copy that straight out of zookeeper and i think this can also be tidied up a lot in particular these lazy statics things should just not be necessary um so let's see create mode if mode is set to a femoral or i guess here it also applies if they're ephemeral sequential the node will be removed by zookeeper automatically when the session is associated with the creation of the node expires uh if mode is set to persistent sequential or ephemeral sequential the actual path given path plus the suffix i where i is the current sequential number of the node the sequence number is always fixed lengths of 10 digits that's not controlled by us so that's fine so this is going to be one the uh i guess we'll document elsewhere let the final the returned uh the newly created nodes full name is returned uh when the future result the future is resolved what else we uh if a node with the same actual path already exists in zookeeper uh then exists if a node with the same actual path already exists in zookeeper this is returned no exceptions in rust so that's nice uh note that since a different actual path this is a different actual path is used for each invocation of creating sequential nodes with the same path argument uh calls uh with sequential modes will never return node exists uh if you return async not ready once does your future not get pulled anymore um so not ready does not actually impact whether you get pulled again in general not ready just tells all it does is tell the um the thing that runs the future that the future is not yet ready it does not say like when to wake up that future again or when to pull it again in fact normally it will never be pulled again until something notifies that future and so this is why you need to make sure that if you have a future you pull all the underlying futures um so that eventually they will like pull a network socket or something that that zookeeper or sorry zookeeper that tokyo or whatever drives your futures knows to wake things up again so for example if you had um if you had to implement a future that just like all it did was return uh immediately return async ready it will be called once and never again you need something to so this is what notifications are for to notify that something has to be woken up again uh or create uh let's see other or i guess it's weird to say is return for futures right uh the return future resolves with an error of yeah that's better uh if the parent resist the return future resolves to uh sins ephemeral nodes cannot have children uh if the parent know the given path is ephemeral uh or i guess ephemeral nodes cannot have children in zookeeper therefore if the parent know the answer is ephemeral the returned future resolves to in that case it will be error create this is the one that has the funky name no children for ephemeral all watches left on the node of the viewing path by exists and get data api halls in which the parent node by get children api halls if a node is created successfully the zookeeper server will trigger watches on the path wait isn't the same the same thing this operation if successful will trigger all the watches left on the node of the given pathway exists get data api halls i don't think that's true i don't think get data will be triggered wasn't this what we looked at so or semantics of watches yeah created should not trigger get data i think that's false if a node is created successfully the zookeeper server will trigger the watches on the path left by exists calls and the watch is in the parent of the node by get children calls wait so that this documentation is wrong these two sentences are contradictory and i think only the second one is true so a raised larger than this will cause a keeper exception to be thrown is this an error that we don't handle like is there a too large thing here i think we looked at this last time doesn't look like it maybe like a bad arguments or something let's just leave it like that for now it's a little under specified but we'll live all right so it looks like we have create working correctly um so let's look at the lead uh where's the delete delete async version thanks how do we feel about this one delete the node with the given path the calls succeed if such a node exists and the given version matches the nodes version if the given version is minus one it matches any nodes versions uh i don't think we actually need to include any of that information because that's already documented on our error struct right so i think that's still fine like this is not because this is not java right like it's not like you might randomly get exceptions thrown so i think we're gonna remove that too that's good this operation is successful trigger all watches on the node of the given path left by exists api calls and the watches on the parent node left by get children calls that also seems correct great uh these we're gonna have to do a little bit later because they're a little bit special but exists we might as well right now so exists and this is exists without a watcher so it's this one thanks does this have a special doxia probably all right so this one i return the stat node the given path there's a space on the first sentence of the delete method do i okay uh you missed a space ah good catch thanks i think the deletes version could be none instead of minus one oh yeah no you're totally right if the given version is none matches any node versions it is sort of true that like you could also give some minus one it's just that none is the right you're totally right uh return the set to be given a path if it exists well and i guess here what we'll really do is we'll take the same docs and put them on watch globally for exists if no errors a watch will be left the watch will be triggered by successful operation that creates or deletes the node or set that creates or deletes the node comma or sets the data on the node wait really changing the data triggers and exists that's really weird i don't know why that would be the case oh i guess it changes the stat is what they're thinking uh great so that means that here it is really just uh or none if the node does not exist which i guess we probably want down here too if no errors occur all right uh get children is we might as well add the this one as well actually i guess realistically the watch will be triggered by system and a notification will be sent to the default to the walk to the global watcher and then here the documentation for exists with the watcher is going to be and a notification and the included one shot receiver will be notified great all right so get children get children up here turn the list of the children of the node of the given this is terrible english turn the list return the names of the children of the node with the given path better uh watch can't be true the return the returned list of children is not sorted uh or none if the node does not exist i guess this we're gonna put down on here too and then we're gonna remove the stuff about watches down there this is now if no errors occur why is this all passive voice a watch is left on the node and is triggered by a successful operation that deletes the node with the given path or comma or creates or deletes a child of that node i guess the watch is triggered uh and well the phrasing here's a little bit there are more paths that missed the ticks well let's see probably uh really oh down here probably yeah well i'm still trying to find the right way to freeze this let's look at exists first if no errors occur a watch is left on the node with a given path the watch is triggered by a successful operation the creature deletes the node or sets the data or sets the nodes data triggered by any successful operation that creates or sets the nodes data uh when the watch triggers an event is sent to the global watcher stream the watch is triggered by when the watch triggers an event is sent to the global watcher stream great and then of course the same will be forget children and this should now be the watch is triggered by by any successful operation i really think that should be any any the watch is triggered by any successful and causes the included one to do to resolve okay so here's the watch the watch triggered by so the same thing any successful operation and in turn causes the included one shot saver to resolve okay i think that's decently clear all right what else we have get data get data this one return the data and the stat of the node at the given path great much better um where were we get data right uh there are no watches here so this is just or none if it does not exist this gets moved down to get data then this phrasing is basically the same if no errors occur a watch is left on the node at the given path the watch is triggered by any successful operation that sets data on the node that sets the nodes data or deletes it and then we add this bit and you watch triggers an event to send to the global watcher stream and that error goes away and then get data down here it gets the same docs except that it does not trigger on the global stream it instead does this fantastic all right what have we missed probably a bunch of things this is uh per operation zoo keeper error types oh it's really reads good so far okay that's great i think it's certainly better than the java version we'll see whether it actually ends up being good uh error an error that occurs i want to phrase this as it's an error that occurs because of the request and not because of like the protocol or the server uh a reason uh it's like reasons i don't want to call it reasons either it's like uh errors that may cause a delete opera delete request to fail oh do i need to document all the variants no target the target node does not exist exists uh the target node has a different version than was specified by the call to delete with these several little uh the target node has child nodes and therefore cannot be deleted i also want some of these to have links to the java docs or to the zoo keeper web docs um but that seems like something that's slightly less important uh the target node already exists i guess a node with the given path already exists no node exists with the given target target node is fine uh the parent node well how do we even want to phrase this i guess this is uh the parent node of the given path does not exist the parent node of the given path is ephemeral and cannot have children the given acl is invalid it's so unfortunate that invalid acl does not give you further information it seems like a seems like a mistake oh i guess these should derive well certainly debug and i guess in theory clone although i guess there's no reason not to have it derive clone it's just a little weird uh clone it but zookeeper cannot oh you can't debug zookeeper that's uh proto error that should definitely derive both copy and clone arguably also like hash ord partial ord oh we're getting pretty close uh oh no error bad version really i need to document every field uh the the expected node version minus one indicates i guess here it basically has to not be minus one because you could never get this error if you expected version minus one because it matches any version so there's no reason to include it there um we also need to document this um so i guess this is a proxy for operations that well this is a proxy for zookeeper uh that adds watches for any uh triggered operations for any for initiated operations triggered watches produce events on the global event global watchers stream and then i guess for with the watcher that adds watches that adds non-global watches produce events on uh triggered watches or i guess events from triggered watches uh are yielded through returned one-shot channels all events are also produced on the global watchers stream and what's left well we're almost there fn watch so this is uh perform the following operation uh i guess the next chained operation uh or add a watch on the node targeted add a watch for the next chained operation and just call it a global watch and i guess we could say see uh it's not even necessary add a watcher add a watch for the next chained operation and return a future for any received event uh along with the operations results successful wow successful result whoo all right so now we have docs add lots of docs we don't really have uh top-level docs yet though let's just do this for now we'll obviously want it to be much more expensive there are probably still some things we want to tidy up i don't think i want to start multi-server connections in part because i don't have a good way to test it chroot would be a great thing for someone to add um i don't think it's particularly interesting to watch me implement it because it's mostly just like cutting strings in various places um let's see what else do we have here i guess we could add all these to do's office limitations to be like the right thing to do or we could add them's issues really there's a better thing to do so actually let's do that while we're here issues new issue give send connection events on global watchers from i guess i want these it's a nice way to just like track things that still need to be done uh so there's that there is support uh connecting to multiple well support pools of zookeeper servers more than one yeah we also don't currently distinguish between um an expired session and just an error during our connection and technically those are very different in the zookeeper world uh among those lines also handle uh session expiry differently from errors during what else we uh right support chroot yes we can call it a feature i guess i should tag these two these are great if you just want to like start playing around with the code from java dogs note that this uh needs to affect all paths on the code base um including uh watched events just learning fields in watched event so we probably have other to do's too let's see what do we have to do test this nice yeah that's true we don't really have tests for connection failure in part because they're a little tricky to do i think we basically have to mock the network which is totally something we could do it's just a little bit annoying to get set up uh i don't think realistically that's going to change anymore uh yeah that is true actually this is something that we don't say specifically we sort of need to say here somewhere that all of these future require you to run them on a tokyo runtime because we use things like timers right so you can't just like call dot wait on this and expect it to work uh so i think we'll do that up here that all operations that all future uh the futures in this create expect to be running under a tokyo runtime in the common case you cannot simply uh resolve you cannot resolve them solely using doll wait because doll wait wouldn't run a runtime uh what else do we uh okay so this has now been done uh connect here direct oh that's right so now when we added the zookeeper transport trait in the previous stream we now have this weird setup right where connect only call calls like tcp stream connect directly and then does the handshake but now we can just do that directly in packetize renew because zookeeper transport has this connect method directly and it's sort of unfortunate for this to be tied up in the in the outer connect because it means that this outer connect can't be generic right we sort of want this to be generic over s um where this is like s adder and i guess s would be zookeeper transport adder or something i don't remember exactly what it was uh did i i committed this already right yeah i mean we could try that um so in theory that just takes any adder i don't think we actually want to do this right now but yeah i think i think this is definitely an addition that we want to add so usually support non pcp connections and handshakes because packetizer is fully generic over the underlying transport but zookeeper is not but it only only really requires that for connect uh so we should uh we should use the zookeeper transport trait to bring all the network connectivity stuff things uh into packetizer new including the handshake uh because the handshake can be uh then be reused for session re-establishment so if you recall from the last stream uh establishment if you recall from the last stream uh in the session re-establishment stuff we have to create a new one of these like request connect objects which is basically what handshake does already and it's sort of unfortunate for this to be happening in two places and so that's also something that should be uh would be nice to fix up um what else do we have that said to do i think this one can go away uh that we can go away because we know an issue for it that error i don't know if we even need oh maybe we should do logging how do we feel about logging do we want to do like uh just like do a pass and get rid of all the print lines and debug stuff and add proper logging throughout the thing would that be interesting to watch i mean if if you think so write it in the chat um just finishing the pass through for to dos um or if there's something else you want to see then uh now's the time to shout out to dos to get issues yeah do it all right then let's do it so let's get rid of some of these i guess we do want to add links to some of these later it's it's not terribly important for now i'll probably do a pass on the documentation after i finish the stream because we've now done the important thing which is walk through and see that we actually conform to what the Java client says that we should do um but there's a bunch of other tidying up that we can do and you can watch the commits on the repo if you want um if you want to see what things i end up doing later not sure whether i get to it today but we'll see um so for logging there are a bunch of rust logging libraries one that i like really uh quite a lot is uh slog i don't know how you pronounce that slog slog slog sounds good uh because it has multiple logging levels which log also does but it also has um it's customizable in terms of the output format uh so you can have uh like there's like a colored terminal logger for example this is really handy so let's do that uh slog is like one point something two point something three point three point two i know uh extra and create slog and what we want that's a good question so the the basic setup of slog is you have what's known as a drain or a logger um and if you look at the logger where's logger logger um so a logger like you can create new loggers pretty easily um but the crucial thing that you want is there's a bunch of bunch of macros that are basically up here like crit debug error info and log that are the different logging levels um and if you have a logger you can always create a new nested logger with some values preset so this is a really nice way to make nested loggers where the where you have some logger that always includes data from the parent logger so you can create a hierarchy of loggers um loggers are also cloned which is really nice so what we want to do here is well they're cloned of the underlying drain so a logger is connected to a drain and the drain is sort of where you output stuff um and so what we're gonna say in our case is we're gonna use the default drain because we might as well uh and we're gonna say that a zookeeper has a logger which is a slogger slogger why not we're also gonna have our proto mod contain a this file has a lot of stuff now should be broken up some more new issue break up source proto mod into smaller files it's getting ridiculous it's a great issue in the description probably going to end up hating myself for that but it's fine all right so we want on paketizer where's our paketizer not an active paketizer but on paketizer itself where's paketizer there we go the logger uh is a slog logger and yes we want to use oh and we also need to bring the logging macros in and that awkward so we sort of want the user to be able to specify the logger but default to something that's just a drain so um sorry to clarify um there's a there's a logger that is called is a drain i think it's no where's the discard so a logger discard is just the thing that you log to and nothing happens um and so it makes a lot of sense for that to be the default logger in our case i think we just do log slog logger there's an argument to new so and i guess exit logger is log.clone because now finally we can do info well actually no this is an error exit logger the other thing that's nice about slog is you can have tagged fields i don't actually use this right now but you can do things like at the end do like who that will be printed nicely and formatted if you have structured output and whatnot for this though we actually just want the the print out right and then i think what we want here is for active packetizer to also carry a logger around probably just so we don't have to oh we can just pass it in each time there there's some there's some advantage to having each packetizer have its own logger because then we could set fields for every logger like for every packetizer i mean to say that like like whenever an active packetizer prints it also includes like its iteration id or something so that we we can tell if we go from reading the log output from one active packetizer to the log output from a different one but i think we'll we'll probably leave that alone for now it's probably not terribly important um so specifically pull pull it's gonna be logger and take one and take one of these and we'll get back to actually making that work it does mean the packetizer new down here third argument it's getting long so we sort of want a like a connect with logger this is where you get a builder right um so i guess we do that so pub struct zookeeper builder it's gonna have i guess session timeout is the other thing that we the session timeout we currently just like set to zero and in theory the user should be allowed to specify that so session timeout um what is an i32 probably i guess this should really be uh because we have a good language time duration uh and what was the other thing we want to write like a logger and we want to ample default for zookeeper builder default nice catching a live stream always better to watch it live i think unclear actually uh you tell me so the session timeout is going to be zero by default and the logger is going to be s log uh discard i think that's how you normally do it uh the docs for this are not always great like there's a bunch of like here features we have but i don't really care about the features i want to see how to use it you got to do this all right so this is how we create a discard logger so the default is that we don't do any logging and then zookeeper builder it's going to be a this i guess we'll probably move the handshake too because this might as well just be on builder um and this i guess can be cloned it's unclear that you want to be able to clone it but you might as well have it be all of those useful things um yep so connectus is going to do that it's going to move self handshake handshake is going to also take self and it's going to get the adder and then here this is going to be self the session timeout and i guess here we'll get to do our first debug self log well self dot is a logger or log logger um about to perform handshake great um packetizer new is going to be given the logger and i guess here uh this is going to be a connection log is self logger dot clone and that's going to be here because we probably will want to trace that so if you enable trace output you want to be able to see the you want to be able to see the response to connect um handshakes are not important enough that we want to always show them so you should think of info as info is shown in release mode debug is shown in debug mode and trace is only shown if you turn it on um so now i guess we might still want to with default parameters uh see zookeeper builder so this is going to be zookeeper builder default dot connect adder so and now we don't need handshake there anymore in theory we could store a logger inside zookeeper too i guess we already decided we were going to nice uh in that case how does it oh this does not return a self this returns a zookeeper so we're going to make this bp log and then we're going to have here we're going to put in the the logger itself right so we have one logger that we give away to the packetizer which we're going to spawn up in the background and then we have another logger that we're going to keep for the zookeeper instance right and now here we're going to do something like trace self dot logger uh create and path is here probably don't want to log the data probably want to include mode uh it also has this nice thing where you can prefix something with a question mark to use the the debug the debug output of that value so we have that we have delete version here exists path and this will just be watch i guess because we'll want to see that as well and we don't need to log in the the sort of the the leaf things because they all end up calling these uh children and get data i don't think there's that much left in this file nope and so now the question is what we want to log in proto mod so here we now have the logger inside the packetizer let's start at the bottom of the packetizer um where is new right so that just that's really just packetizer pull that's sort of the main loop these we sort of need to decide how noisy we want debug output to be i think things like this are probably trace because that's not terribly important right there are other things that will be important like i guess you don't actually print that much here the connection response is definitely a trace about the handshake again so this there's probably an info here where we want to say like uh down here info um connection lost reconnecting and then here we want to include the session id and the last zx id because it may be relevant we could include the password here too but it's not terribly important also i want rust to start formatting macros even though i know macros are hard to format okay and then i guess the rest of this really goes on in the state polling so self.logger i wonder if that matters the name is slog slog it's this great or are you asking the oh the yeah yeah this is logging great the poll so this is given a logger but crucially this i guess is given a logger and it calls this with the logger for reconnecting i guess we do need to include the logger so where does reconnecting's poll go oh actually that doesn't need to do that at all great so the packetizer's poll in queue that's gonna have to take the logger logger's just like end up everywhere uh so this takes a logger that's a mute slog logger let's see uh this is definitely a trace i mean so got a request could be a debug actually uh and because this is in queue not really yep in queuing requests uh this is i mean trace at best because it could be debug but no it's probably trace um great wait why does it complain so eprint line where do us do we have one of these right so active packetizer that's the one we want so for active packetizer we call poll uh i think this poll read is not terribly important this is gonna be given the logger i guess this is trace sending heartbeat oh i guess that's why i guess we do need the the trace here to know that we're about to do a poll read to distinguish it from doing a poll write down here and we want to pass the logger to a poll write too and then this should be debug logger because it's useful to know when the packetizer exits and now i guess poll write poll write takes a logger again slog logger well it's a mute slog logger and it will do this is gonna be a trace resetting heartbeat timer so this is where we reset the heartbeat timer because we wrote something on the wire heartbeat is since last inactive since last wire traffic uh this is debug and then poll read which is gonna be where most of the logging where most of the logging will probably go oh i see here we have a bunch of things just to even figure out where request boundaries are i think these we probably don't even want to make logging statements could have this be trace so we could do trace logger the other thing that's nice about slog is it compiles out any logging statements that are below the threshold of the current build mode so for example if you're compiling in release mode all trace and debug statements will not even be compiled so like this computation of inland for example will not be computed uh here the bail is sufficient this is probably a debug server closed connection this is not important this is definitely a trace right so this is updated zxid from this is a trace watcher events i think we also want to trace but notice how very few of these are actually debug right so here i guess that's another good question this is going to be something like waiting customer potentially right because some of them might be for a different for a different event type this is the fix we did earlier probably don't want to notify on every watch this is definitely a trace unclear that it's even important enough to uh but probably so here this is where we get a response to a user request handling server response so here we certainly want to put out the xid and the off code the bigger question is whether we also want to print out the uh the contents of the response i think we might actually but what did we make uh yeah so in queuing requests as a debug so i feel like the response should then also be a debug i guess one question is whether we want to include the body of the response in the debug output two probably well this is definitely a trace that is a very good question i guess here not turn here we probably want to include the error and again because trace things are compiled out it's not a problem for us to include some like it's not even expensive but just to include some additional information here right um what else do we want let's see i do wonder whether we actually want to print out the response but it's a little annoying to do so um because it's not parsed until down here i guess we could just move this whole thing down right handling server error response which would just be e and then down here we would do the same for handling server response so that works pretty well that gives us basically i think the debug response we want because here we might want to make this info like this is an operation that failed so it seems info worthy um i don't think we need time out uh we may want to log the fact that we've set a timeout though negotiated session timeout and that values timeout and what do we decide it was millisecond and i think that's most of it do we still have any print lines here nope do we have any here got through all futures that's a part of testing probably not going to compile because i probably missed something out in fact i know that the session timeout this it should be lost uh time duration uh i want sub that's awful i want just into millies and this is still not something that's here so you need to compute it manually which is pretty annoying so it's this dot sx times the thousand plus seems unhelpful 610 format argument must be a string little right these should not be like this this should be xid is this it should be path is this and it should be w type is this logger ooh use of undeclared time well we do need time it's true you're missing two logger in the macro oh yeah sorry there's a slight delay in the stream uh and so i by the time i saw your comment i'd already fixed it but thanks uh this is just going to be self logger 77 oh i guess right so the crucially the thing we need here is we need the ability to set these values otherwise it doesn't really work this is going to be a pub fn set timeout which is going to be a t can this just take a move itself there there's a lot of discussion about exactly how the builder pattern should work out there's an argument for the best thing is if your builder can be this um that's a little tough for us or sorry if your pattern can be this uh and then your build can be this um in our case that's a little awkward so i don't really want to do it um because it would require that the pointer is static which is all sorts of mess well messed up so that people can chain builders and instead i think we're just going to take commute self for now something we could optimize but so session timeout is t and similarly we want to set logger where l is an s log logger because now in down here uh builder is zookeeper builder default and then we're going to do builder dot set logger and now we're going to steal one of these s log loggers specifically where is it you have to pull in a bunch of crates which is a little sad so this is going to be uh only in debug mode or only in test load actually so we're going to have dev dependencies and those are going to be s log async and s log term because now we can use the where is it this business to make our logger this is going to be logger i guess we can just do this and then this becomes builder dot connect if you aren't doing anything special why are you using setters rather than modifying the struct fields or if i missed out on something um i don't like to make fields pub whether that's a good thing or a bad thing is unclear um but the the test like i don't really want to mark these two fields as pub right and theoretically i could i guess so that you can read them out as well i just have like a an aversion to doing so the other reason is because it's easier well yeah i mean it's a good question um i guess one of the reasons to do this is because it means that i can imagine that later i wanted to change session time out to like just hold an i32 directly right that i could still present the same external api of taking your duration but then just store it as an i32 internally if i give make the fields pub then i'm i'm committing to this being the context of builder the other reason i don't want to do it is because um it's totally reasonable for us to want to add more fields to build or later that would then be a backwards incompatible change because if these fields are public it means that people might be matching on zookeeper builder which i don't want them to do wait why is it complaining about this oh yeah that's wrong so still gonna yell at me somewhere but uh found u32 so i would want this to be as i32 and i want this to be as i32 oh it's like log or not send or something wait this has never been a problem before what did i change yeah so i mean it's uh it's a good question um in java you definitely do getters and setters everywhere i often don't even do getters we're in the builder pattern specifically i only do setters not for any good reason it's just usually the setters are all you need i do like not exposing the fields directly because it means that you can change them i think that's generally a good pattern and especially in a in a language like rust where you have pattern matching too on top of this right um i don't know why it's saying that this has changed it's gotta be that logger is not send oh no it is not it is because this has to be longer and then it has to move in this so that's fine and then it's complaining in 707 oh actually it's really this but this needs to just be log how does it complain 492 oh 492 right this is because i need to use as log grain opcode 461 really use of moved value opcode but opcode should be copy why did it not warn me about that now we're almost there 643 cannot borrow it mutably more than once at a time where is it already borrowed as whole nq does not need to take a logger it already has the logger itself and send down here oh this should probably say the xid it's gonna enqueue it with builder that allows creating more customized allows customizing options for zookeeper connections and these of course a session expiry time out see that's just like an unhelpful comment arguably there should be a link here to something like uh where's the faq that i found here really they don't have direct links to this it seems actually i guess in theory it should be here right to link to sessions that's a lot more helpful logging instance to use uh the default time out is dictated by the server uh defaults are always something that like should be documented more than aren't by default all logging is disabled and i guess also i sort of want to link to the docs see also documentation that's not that you are all right how about now let's see what it gives us that's beautiful beautiful so notice now it's not include including any of the trace output so what i like to do is there's a the way to turn that on is by saying features is a max level max debug level trace i think so i'd like to have that line commented out so that it's easy to add again uh oh what's it called slog what's the name of the feature max level trip max level trace really do like seeing other people program in rust as i get to see very different styles to programming as i'd say mine is quite different but then again the only sizable rust project i have is my msc project so i don't really have to worry about other people ever using it i mean that's true um i do think that they're seeing other people code is like interesting in its own way uh right so here notice here there's now a bunch of trace output right and all of that is normally just hidden oh that's interesting dropping messages due to channel overflow wow so i don't think i've seen that before really i feel like there's a better way to do this but it's not important enough for me to care uh this means that we have all the tracing that we need so i'll i'll pop this back to this um it's a little annoying actually that it's not easy for users of this library to then turn on max level trace for s log and i don't know how to fix that i think i asked the developer at some point and i didn't have a good idea all right so all of this to say that we now have logging add support for logging great good push also let's do a cargo publish it's like version 0.1 oh it might complain about all sorts of other things because this file i guess let's see what kind of things we're missing so we need this a synchronous client library for interacting with apache zoo documentation is going to be a docs on our rs tokyo zookeeper i don't think we even need to set that anymore actually oh i guess we'll have to figure out what keywords are going to be things like uh zookeeper tokyo asynchronous i also found out recently about craze.io slash category slugs which is basically the page i wanted last time because we want categories api bindings what else it's not really a database arguably it's it is a database because it's a key value store but it's like ignore that for now network programming i i guess i think that's true um arguably not web programming i think the game is just included as many as possible well sort of there's actually an upper limit you can't publish if you if there are more than seven or something i ran into this unintentionally um we don't have trials yet i did not try to categorize it with all the categories but just wouldn't uh didn't let me i guess there's still we still don't really have any top level craped implementation documentation but that's something i'll add later uh commitam better cargo automo oh i guess i also need to do this push and then we do cargo publish uh allow dirty so that doesn't complain about my travesty umo file have they mentioned that category stuff publicly i've never heard of that part of the site before now i know i hadn't heard of it before either um but there's a pull request just landed that means that it will now be listed somewhere here list of category slugs it's now under docs it did not used to be it was added like in the last week but it has apparently existed for a very long time and just like no one knew about it uh all right docs.rs slash tokyo zookeeper probably hasn't been built yet but in theory should be there pretty soon ooh types of chunk of memory i'll take faster that's pretty cool though i think we're in a pretty good place now um what i'll do is also uh go through the github issues and tag them for like things that are beginner friendly i think many of them should be pretty straightforward to like dive into and try to implement yourself if you'd like want a challenge um and i'm happy to review pull request in general while we wait for this to finish um i think we'll basically finish up here i don't think there's too much more i wanted to cover um multi server connections and sadly we did not really get time for that it's also a little bit tricky to test because i didn't find a good efficient way of running many servers in one machine and i get weighed that's not super annoying so that's still something that's like a relatively major to do but it's it's also somewhat uninteresting from the asynchronous part of things so i think at this point i probably won't do more live streams on tokyo zookeeper unless there are questions i'll still continue fiddling with the crate but but i'll probably do that more in my own time um and so if you have ideas for other things you'd like to see me do then reach out either in chat now on twitter on patreon any of the other places like i love getting ideas for this kind of stuff the last thing i toyed with was doing some kind of standard library data structures so we did a hash map a little while ago using like a linked hash map which has turned out to be quite popular as we could do something like that that's sort of a little bit more introductory rust but also pretty interesting or we could do some other crate that's asynchronous there's a bunch of other ideas there so if you have ideas feel free to reach out or email me for that matter oh this is not building uh i'll be gone for the next like four weeks probably and so it'll be a while until the next stream but all the recordings for the past streams are online so you could watch those if you haven't already i'll post this as soon as we're um as soon as i finish recording i want to there we go tokyo zookeeper we have a thing that's out that's fantastic i think we might want to clean up some of the structs like maybe put some of the like acl and stat under a module or something let's see how this generally looks so decently respectable decently respectable i guess here there's a bunch of documentation we haven't done because the stat and acl we've just copied from the synchronous zookeeper stuff um but it's a good start all right uh thanks for hanging out with me and uh and writing writing zookeeper stuff it's been fun um if you uh yeah i didn't mention this if you have like feedback about the streams as well then i'd love to hear that there've been people who pointed out that uh some of the pages i visit are too white which actually makes a lot of sense like i'm switching back and forth between dark and light windows um or like some people wanted larger font sizes um any of those kind of things just reach out um and i'll try to do my best thanks for watching and uh feel free to try to address things in the repo if you want i'm happy to review pull requests all right bye everyone