 Welcome back. It's time for another stream as as always I'm John I do a bunch of rusts like live programming and educational content and I try to for all of these videos give you a say as to what you want to see and this time we decided we're going to do another open source contribution stream so we've done one of these in the past where we did some contributions to projects where we tried to sort of tidy up a lot of async code this time I wanted to focus more on rust tooling because I think people are a little bit afraid of trying to contribute to the rust core tools because they think they're overly complicated or fancy and scary in some sense word whereas in reality they're just open source projects like all the others and so in the stream we're going to make a contribution to cargo and we're going to make a contribution to rust up now I've made a contribution to rust up before but I've never made she's a true I don't think I've made contributions to cargo before I also had a bunch of people when I mentioned this on Twitter asked me whether we could do a converting a crate from future 0.1 to standard future and so we're going to do that as well which order we do these in is I don't really mind too much we can order them whichever your preferences as always I'm John yeah we the one thing I want to mention before we start is I want to say thank you to all the people who bought me stuff it's been fantastic like now someone complained about last time the tapping for my keys you could hear through the microphone and then someone bought me a shock mount which now means the theory that shouldn't happen anymore so there's a Amazon wish list that's pinned on my Twitter where you can buy me stuff for the stream or for life I'm not allowed to have income in the US so this is my work around for not being able to have a patron or something like that alright without further ado let's get started so oh yeah you can if you want to if you want to vote on what our next stream should be about there's this website here it I can paste it into chat as well for those who are in chat so this page lets you basically vote on what you want the next stream to be and we even did a stream on building the site it uses like ranked choice voting so it's pretty cool you can vote on multiple things and rank them and this is why we're now doing an open source contribution stream alright so do we want to do cargo first rust up first or converting old futures to new futures preferences I'll give chat this is this is the maybe one of the few advantages to watching live is you can ask questions and you can suggest order of things and what to focus on let's see here cargo first cargo cargo cargo futures futures futures alright I see mostly cargo so let's do cargo first alright let's see how this goes so usually when you want to make a contribution to a project there are many ways you can go about it the easiest one is usually to go to the project page go to issues and then look for labels and then usually you might want to look for things like easy e help wanted e mentor there might even be things like good first issue in my particular case I've decided to not do that because I have an issue that I want to get fixed and so I'm just going to fix it I filed this issue about a week ago and it's been bothering me for a while so I figured let's just use this chance to fix this problem so this problem relies on a particular environment and variable you can set when you run cargo and I don't know how many people know about this but if you set the environment variable cargo target dur then any time cargo compile stuff it's going to put it in that directory instead of in like a target directory in the current directory right so instead of ending up with target directories all over the place you end up with a single target directory that you can choose where is and this is really handy because it also means that you can share compile artifacts so let's say create a and create B both depend on create C if you can compile create a and then compile create B create B doesn't have to recompile create C because it's already stored in that shared target directory and so this is actually really handy for reducing the amount of stuff you have in your system and reducing the amount of duplicate effort and compilation you have to do and so I have this set and I have it set in a slightly weird way in that my target dur is a sim link so the reason for this is I have a script that I run on all my machines when they boot up which looks for a sim link in my home directory called dot cargo target and if that exists then they set the cargo target dur to be that value to be that path otherwise it doesn't set it and then I can use that sim link on different systems to sim link it to a directory on a different disk for example or on temp or something else the problem that arises and what this issue describes is that if you run cargo clean which is supposed to delete your target directory if cargo if you have cargo target dur set or if target is a sim link then it will only remove the sim link it won't remove the contents inside of that directory so this is pretty pretty annoying let's see if I can give you an example of this so let's do something like let's make this a little larger and let's do cargo new libfoo oh man foobar great not enough temporary names okay so if I do cargo build then you see there's no target directory here and this is because I have cargo target so I have cargo target dur set to this path I can unset it and if I do if I do this be then you see now there's a target directory here and if I look inside target there's a bunch of stuff in there that is related to the current project that I built where this gets and if I run cargo clean if I run cargo clean you'll see that the target directory now goes away which is what you want now where this gets weird is if I make some like my target and then I ln my target to target if you now look at the directory listing target is now a sim link into my target alright so now I'm going to run cargo build and you'll see now that if I ls target or rather if I ls my target then the stuff is in there but if I now run cargo clean notice how the the target sim link goes away but my target is still there and inside of my target there's still all the stuff so cargo clean just removed the sim link it didn't actually clean anything and this is annoying because it means that a not only does nothing get deleted but also my sim link on away and so now suddenly it's my cargo target or one gets set and this is all sorts of annoying so this is a relatively small issue in cargo but it's one that's just been bothering me for a while so instead of just complaining on it and an issue let's just fix it let's see yeah exactly so the thinking here is that we want to instead of just deleting the top level sim link what we want to do is detect if it's a sim link then delete everything below it otherwise you can just delete the top level thing I know that if we look at removed or is that the function from the standard library removed or all right so I think this is what cargo currently uses but notice that it does not follow symbolic links and just removes the symbolic link right so this is the behavior that cargo exhibits and this is the behavior that we don't want how do we keep track of what we own in the symbolic directory if it's shared so cargo doesn't actually care if you run cargo clean this is going to delete the entire target directory it doesn't check of the stuff inside of it which like arguably is a problem in and of itself but it's not something that we're going to be solving in in this stream probably because it's a larger issue all right so now we know what the problem that we're trying to solve is cargo conveniently I already have cargo checked out what you'll see is that the way I have this setup is I have to get remotes I have one called origin which points to my fork and then I upstream which points to the upstream cargo and the way I do this check out actually I can just show you is that I do get clone and then I clone the upstream thing right and then I rename origin to upstream so this way the master branch locally for me still tracks the master from upstream not from my fork and then I do get remote at origin and then I add my fork because now when I push new branches origin they will appear in my fork but I can still pull in master to get the upstream master so the question now becomes okay we now have a checkout of cargo now what right where do we start well let's look at what's in the cargo directory so good places to start are usually the read me and also files like contributing architecture might be handy we might need that later to figure out where we're going to make this change but let's start with the read me so the read me talks about installing cargo compiling from source that seems promising okay so it looks like it's just like cargo build we don't actually need release because we're not going to be running this in a performance sensitive way the fact that they use cargo build makes me think that we could just run cargo test if we're going to run the test suite so let's look at contributing working on issues yeah so here they talk about things like easy and mentor like I mentioned before opening new issues as a cargo discord where you could discuss issues it must compile on stable rust okay that's fine fork and pull that's fine basic steps for cargo create a branch code style that's fine commit as you go include tests we're going to do that make sure cargo test passes okay so this makes me think that cargo test is all we need cross compiles we can probably disable so we probably want to I think we probably want to just ignore those okay so in theory we should be able to run something like cargo test and so that I don't break the stream I'm going to run it with only one thread you could also add your own fork is origin only for pushing the upstream is origin for fetch that's true I think it's just it's just easier for me to think about master like I'm never going to push to master anyway so I just want master to be tracking upstream anyway and then I just want any branch I make to track origin okay so the test suite seems to be running working so at least now we know that we can compile cargo I'm not actually going to run all of these it seems relatively unimportant the question now becomes how does clean run and there are many ways we could look at this the easiest one might just be to search for clean see what we find okay so there's a bunch of stuff in docs bin cargo commands this seems pretty promising alright so there's a clean CLI that seems very promising and there's a mod in here called clean what does clean do okay so now we found where clean happens oh yeah someone's asking you can see whenever I type commands in Vim you can see them down in the bottom right corner here and someone was asking what does double double code plus why do I don't think there was a T oh so T lets you select until a particular character so if I do so the double code plus means copy to the system clipboard why means copy like the I set the destination to plus that's as a double tick is set the destination of whatever you do plus is the system clipboard why is copy T is until the following character and something like semicolon is going to be copy everything until semicolon to the system clipboard okay so we have cargo clean let's see what it does clean options huh ops clean cargo ops okay so it looks like what we want is maybe cargo ops cargo clean great alright so here we see these clean options and I wonder whether we want actually let me start a new branch for us let's call it delete through sim link actually no let's call it clean through sim link we might want to add an option actually that let's actually it's unclear I wonder whether we want an option to say delete through sim links or don't delete through sim links but when would you ever want it to only delete the sim link there might be cases so that would be the argument for adding an option I think we're going to not add an option to begin with and then take it from there what if you want to copy anything up to a sequence of characters rather than a single character so then instead of T you can use slash which is search there's also there are plugins for adding additional nouns as they're called in vim such as like do a search for a double character which is usually s but I find that I rarely need them alright so clean so packages build artifacts okay so it's interesting because many of these are not a problem right so if you have doc set then it's going to look at doc inside the target so if the target directory is a sim link then doc will not be a sim link doc will be a directory inside the sim link so this is fine this too like this looks in the if a profile is specified then it's going to look inside the target directory so the target directory being a sim link is not going to be a problem yeah whereas this is the real thing right if you haven't specified what to clean so with cargo clean you can do things like dash p and give the name of a crate for example you can give like doc to clean only doc so there are a bunch of options you can give but if you haven't given any options it just deletes the whole target directory it looks like all of them call this rmrf function so that's a very good question one thing we could do here is yeah so I think maybe the fix here is going to be just changing rmrf to check whether something is a sim link and if it's a sim link then we want to then we want to delete all the stuff inside of the sim link rather than the sim link itself also one thing I want to point out notice how quickly we got to this point right we had a bug in cargo and now we're editing cargo source code and like it's using clap it's just a normal Rust program that you can make changes to and it might even be that this change is going to be local enough that it's just inside of this function in cargo clean we'll also want to write a test for this although in theory that shouldn't be too hard I think here so this uses FS metadata why do they unwrap or false instead of is some or is okay they can't because is dur this is an option bool and so they want the bool if the bool is there isn't the correct fix to resolve the sim link and then run removed or all in the resolved path the sim link would be orphaned but I think it's closer to the behavior without sim link what if it's a sim link to a sim link yeah that's a good question so the proposal here is maybe the thing to do is if path is a sim link then resolve the sim link and then basically retry because that way you would remove the target of the sim link so there are two considerations here one is would this fix my issue and the second is what is the correct behavior I agree with you that resolving the sim link and then deleting the target of the sim link so in our case of the bring that up in the case of our example here right where we had target so if I run now ran cargo clean what that would do is it would delete delete my target but I don't actually know what that would do okay so let's let's try to emulate this right if what we did was just delete the target of the sim link rather than its contents and leave the directory in place that would be equivalent to running this this is the state you would be in and notice that this sim link is now dark dangling let's see what happens if we now do yeah so notice how this now fails because it doesn't try to create the target directory this is also somewhat important imagine that the the shared target directory is on like some shared drive where you aren't allowed to create directories so someone else created it for you with the necessary permissions for you you don't actually want to delete that directory because you wouldn't be able you might not be allowed to delete it because it's in a directory someone else owns but also if you were able to delete it you might not be able to create it again so I think we actually might want to delete the contents and not the directory itself which is a little weird it's almost as though the behavior should always be to just delete the contents I don't think we need a max dep well maybe so it doesn't iterate forever I think we should delete the target of the sim link yeah exactly so is there a reason not to just empty the directory in all cases rather than deleting it yeah so that's what I'm wondering whether what it I'm wondering whether that's what it should do it doesn't mean that you're left with a bunch of empty target directories but it's a little bit tempting to just delete the contents the other question is if we delete just the contents yeah I think that that might be what we need we could have a special a special case for if none of it was a sim link then just remove the top level just so we don't regress in the current behavior so maybe that's the way to go we will also need tests for this of course okay so the in order to do any of this the first thing we're going to need is to know whether something is a sim link so they call this fs metadata right which returns one of these this function will traverse symbolic links well that's unhelpful oh man alright so we need to find some way to determine whether something is a symbolic link which is going to be this thing yeah I think this is so we're going to have to make this be sim link metadata and I assume here that there's something like is link what are these oh no extension traits for different operating systems so there's not a single is link it's also weird because on windows they're only hard links oh file type though maybe oh aha is sim link great alright so what we're going to have to do is to figure out whether it's a sim link is going to be m as ref that's fine map and that's going to this is D ref is that why they're able to yeah file type dot is sim link right unwrap or false and now if it's a sim link then we want to do something right you see the pattern what's the downside of removing the top level directory if someone created a directory for you can't you just create the top level directory inside this top level no so the argument is someone created a target directory for me to use in a directory that they control and I do not right and so if that directory gets deleted I can't create it again because I don't own the parent directory if it gets deleted now there's an argument that the removal would fail but that's still a problem that means cargo clean would always error so I think what really what we wanted to do is only ever delete the contents and leave the directory intact and the exception might be if target is in the current directory or something like that yeah you're right so if we assume that the directory is owned by someone else then we wouldn't be able to delete it but that would mean the cargo clean would fail whereas it seems weird for it to fail if it just if it could delete everything except for the top level if it could leave it an empty top level that doesn't seem like to be an error right at least to me yeah so the question here I think is really one of actually there's an argument here that we don't even need to detect whether it's a sim link I think really what we want to do here is to change the behavior here where if it's a directory then remove all its contents without removing the directory itself unfortunately that's not entirely straightforward where does paths come from join paths normalize path they're all they already test for whether something is a sim link and if so it's interesting because it's implemented so that it specifically will just delete the sim link which is not really what we want I wonder why this code is here let's find out alright it's gonna get bright here for a second where is this source cargo util paths source cargo util paths and then we're gonna use blame great feature now let's look for where's our remove path here alright I want older versions of this file apparently older versions did not have that okay fine so this is probably what it added this remove business yeah so why did Alex add this specifically why is the sim link clause here this suggests that he really wants us to just delete the sim link that's a good question it's interesting because this actually does walk the directory so this is the code path that we want in theory one way to fix is just to comment this out or perhaps even better to say here if it's a if it's not a sim link then removed her that so this way if it's a sim link it's gonna delete all the contents or actually regardless of what it is if it's a directory it's gonna delete all of its contents and then if it wasn't a sim link then it's also gonna remove the top level don't you need to check whether it's a sim link because it is you cannot treat it as a directory when you're recursively deleting a directory you can do things like read there on a sim link that should work fine at least I believe that to be the case that's a good question I'm fairly sure that's true interesting I feel like this is what you want right and now we don't even need the sim link change this doesn't have to change at all this does mean that we've now changed behavior for anywhere that tries to delete files through that removed her right and then this now should still be this the only question is whether reader works on sim links but I believe that it does why not follow the default default behavior of RM and Unix follow sim links and lead all the folders including the top level folder so the reason for this is as I said I don't want it to delete my sim link so RM and Unix if you delete a sim link only the sim link goes away and that's specifically the behavior that I don't want because it means that my cargo target directory cannot be a sim link because it's going to go away each time which sort of ruins the point of having it be a sim link in the first place do you need to change sim link yeah exactly so we did that removed her all recurses and now won't delete nested sim link makes it a directory non-empty when trying to delete that's not true oh I see this one I see what you mean so the problem is being pointed out is this function is recursive so here if it finds something that is a directory then this might be a sim link and now this sim link won't be removed because we just made it not remove sim links and therefore this removed error will fail because the directory isn't empty so the question really comes down to do we want it to preserve do we always want it to preserve all sim links including nested sim links right like if my target directory is a sim link or in fact even just ignore that imagine my target directory is not a sim link but it contains a sim link like the debug directory in there is a sim link to somewhere like for example I have debug and release beyond different disks or whatever like they're both sim links then the way we've currently written this is those would both be left behind they wouldn't be removed because they're sim links but when we try to remove the top level directory which isn't a sim link then this call would fail I think we want to preserve all sim links I just really don't think we want to I really don't think you ever want to delete the sim links but I could be wrong I think in some sense I'm only concerned with the top level behavior at the moment sim linking deeper down in the target tree seems less likely but let's look at where else this uses removed all ok so this uses fs interesting that's awkward so different things use different versions of it like some of them use fs wait there's also a crate called removed her all oh man look this uses removed her all the crate what we could do is add like one version that's the base case here once that's the recursive version and only delete the top level but you see the debug and release sim links it makes sense to keep the contents would be removed which feels like that's what intended with cargo clean that's my thinking as well that you never want to delete the sim links the only question then becomes do we ever want to remove the directory and I think we do otherwise we're just going to leave tons of empty trees down the place ok here's a proposal um it's a little bit sneaky but what if we do removed her and if it fails we just ignore the failure right removed her will only succeed if there's nothing inside of it I think that might actually be what we want to do we allow this call to fail because if there are sim links left within the directory uh deleting the directory would fail we also don't care about that failure it's uh keeping track of whether we did this intentionally would be pretty painful so we just let the OS do the checking for us you could ignore not empty errors oh yeah that's a good idea what's the am I blind I don't think there's an empty that we can easily get at yeah I think that just gets races other so I don't think we can actually detect we could write that here in theory if error kind not empty was available to us then that would uh then we could be less um let's just say actually let's do if error kind not empty was available we could only ignore the relevant error but sadly it is not yeah so the problem is we can't actually get at that specific error I think it gets races other but keep in mind that if someone like the permission errors will still come up when they when cargo tries to delete the stuff below right this is we're only ignoring the error for removing the top level directory right we're not ignoring any of the other errors well what we could do is just look for other yeah so if I don't have permission to delete the target directory but I have permission to delete the contents I don't think cargo clean should error I think that's fine it should leave the top level directory in place so that I can keep reusing it yeah I think it would show as other I think you're right other with the code 39 but the thing is I wanted to also not show me permission errors so this this is the example of if I have a target directory in a place that I don't control like someone made the target directory for me on like a shared drive or something I don't want it to try to remove because it would fail they would get permission error and I don't want it to remove that directory and I don't wanted to treat that as an error I know that so let me type that it's a good point so let's type it up note that we intentionally do not propagate other errors permission denied in particular because the top level directory may be in a shared directory the user does not have right permissions to in which case we want to leave the target there so we can keep using it in the future actually that's poorly phrased know that we intentionally do not propagate other errors because the top level directory may be in a shared directory the user does not have right permission to in that case the user will want to leave the target there so they can keep using it in the future and will not expect removal error for that directory well keep in mind that you will still get permission errors right like all the other removals or the first of the other removals will still raise a permission error this is only ignoring it for removing the top level dirt right so I think this is still the behavior that we want permission denied will still be raised when trying to remove the contents of the directory so this shouldn't be hiding errors that are relevant to the user so I think this is actually the behavior that we want all right so now the question is how do we test this well what do we see in the tests directory is there a particular test suite for clean that would be nice aha test suite clean test suite clean want to be for removing any dirt within the hierarchy you're right that it's any empty directory in the hierarchy but removing any files you'll still get this error so I feel like you will see the error like it seems like it would be a really weird case if only removing directories would have errors and you want to know about it right I agree with you it feels a little weird but I think it's the right thing okay so now let's just first make sure that we didn't break something um let's do cargo uh oh man why is that this is what I want test suite clean and I want uh clean see what that gives us so notice I haven't added a test for us yet I first want to see that making this change doesn't break anything else it probably will actually there's probably a test that checks for whether no target should be removed it shouldn't break anything okay all of those passed let's look at build script clean git registry clean verbose okay I think we add this pretty early on cargo clean leaves simlinks oh there's a test support package what does it do lib cargo is a variety of integration tests that I see with the cargo binary okay it creates a sandbox that's fine the real question is can I use it to make a simlink that's really all I want to know simlink and tests test suite package simlink interesting okay so we're gonna need this thing right this simlink supported business um because it says here we can only run a test that uses simlinks if simlinks are available to us right so we're gonna stick this in here what else does it do t is here macro okay so cargo test support has a macro unclear what it does macro is t oh it's just an unwrap okay that seems relatively less important the best thing would probably be that you only delete the files that are from your project out of the simlink directory or at least as an option but I don't know if that's possible without changing run build check and so on I don't think we can do that because cargo doesn't really track those files and adding that would be a much larger effort um yeah so so holger cargo clean will still error like the way we change it now it will still error whenever it tries to remove any file it's only for removing the the empty directories that will ignore the error for um so what does this do to actually add a simlink it just simlinks them great so in theory we can just do the same thing um so we'll create a project that's fine um why is there a foo rs that's fine we can leave foo rs there um and what we want to do is we want to now I need to remember the order of arguments for simlink source and destination the destination path will be a symbolic link pointing to the source path so this is going to be the link target and this is going to be a real target which also means we're going to have to make a directory here somehow um is there a way for us to test support project project uh get paths publish and registry all right so where this project comes from some down here oh simlink builder that seems handy that's perfect okay so we have a simlink builder that's something I want um okay so then maybe we don't need these and then what we want to do is simlink builder let's look for someone who uses that no one uses that great wait it's also not pub that's unhelpful oh great there's a simlink on project wait this is fantastic all right fine dot simlink dir and order is destination and source wait that's all sorts of confusing but okay what what is the it's simlinking but that's the opposite of what this does the first argument is source oh man why okay so destination here is passes the first argument which is destination here which is used as the source okay so we need to flip these in our head so okay the first argument is the first argument it's just that they're named differently so we still want real target to point to I think we want this I also think we want real target debug to point to real debug and now oh won't the strategy of leaving simlinks cause issues with any simlinks generated by cargo itself maybe it might very well be the cargo generate simlinks but then I don't think we're moving them as a problem I mean the test pass that's a good indicator you are right though I don't know I don't know it might be that we can detect cargo simlinks at least but I'm not sure alright so what do we want to do well we want to check that builder join target actually it's not even do that we want to make sure that the build directory what is the build directly is that just going to be target great so we want the build directory to still exist in this case cause it was a simlink we want the target target debug to also still exist because that was a simlink what else do we want but what we want though is we don't want join so this is where it's going to get fishy which is what inside of that what can we look at inside of this I guess build we don't want build to exist inside of there this is cargo clean leaves simlinks leaves leaves simlinks that's fails why is that path was erroneously implicitly accepted for binary foo that's fine we don't need it to be a binary can't fight foo bin specify bin path wait does project require us to actually set a library seems unlikely oh basic bin manifest that's not what I want I want basic lib manifest basic I want basic manifests and I want it to have name and version sure which I guess means that there was also what was it called target debug there was also something else down here which was example path to an example built as a library okay that's fine see what that does so what are we doing well so what we're doing is we are basically there's an issue in cargo that's been bothering me for a little while and we're trying to fix it which is that if your target directory is a simlink it removes the simlink but not the contents fail to create directory really why did it fail to create directory I wonder whether cargo doesn't support making debug be a simlink 846 okay that's relatively unhelpful well actually no it fails and run but which run build or clean clean line 35 fails and build yeah I think it might not support this which seems like also an issue that someone should fix fail to create directory target debug file exists no we're not cleaning up after ourselves that's probably why was that what this got as well never mind this is fine the problem now is when we run the test and then run the test again the test doesn't clean up after itself because these things still exist but clean doesn't actually clean everything um so the question is how does this do its cleaning no although these don't clean I have this sneaking suspicion that uh alright let's see if I'm right clean tests clean targets what does clean targets do I wonder whether this does actually I wonder whether the testing script actually uses cargo clean to clean and therefore it doesn't clean up after this project because the sim links are still there um which like is fine it's just uh p dot build I guess uh target debug dir this should be not actually sure where those files would be where does it look let's just remove that cargo integration test and then try to run it again ooh that's true uh major won't fail if directory already exists but if it's a sim link it will yeah I just set up your cargo target dir variable I set it in my um in my bash setup script what linux distro this is arch linux file exists this makes me think that cargo really doesn't like if the debug directory is a sim link I guess one way to really find this out is to do this there's no way to have a per test function temporary directory gets removed not to use in the test once again um that would be nice although I want to see whether this is really the issue yeah here it fails again even though now debug isn't a sim link so something is fishy in this pond alright what about now now there are no sim links so now in theory the test should just fail the arch linux wiki is actually really good makes it a lot easier to install yeah oh now okay huh so it's really the target directory being a sim link it doesn't like which makes me think that it doesn't actually it's the build that fails why does the build fail it's a very good question I feel like we've somehow messed up our environment um I want to see the path here again and see if I can find out what's going on so let's just sort of cd over to here we are removing the target of the sim link this is what we didn't want to do because it breaks the sim link and that's probably what it's complaining about it might also not like that this is also called foo but that shouldn't really be a problem alright okay so this is a problem in our code in particular I think what we need to do is a clean hmm why is it removing the target at all some interesting stuff right there uh I have the feeling the clean should remove everything as it does by default if you want sim links you should have an environment where everyone created when run sorry I don't want cargo to create sim links I wanted to not remove them if they're there the sim link there actually should create the repository your sim link your sim linking to though perhaps it only creates the sim link it looked an awful lot like it pretty sure it creates it as well sim link builder um new builder build or file in sim links it calls mc and mc yeah mc does make there a p so it should create the target and then sim links hmm and you see the sim link is created correctly as well right that is the path we expected to point to um so it exists actually here's what I want to do um hmm hang on that's a good point this is the state because the build fails right we're still in line 35 is the one that's failing I assume uh 35 yeah so because what fails is the build this means that this is the directory layout that um that cargo is working with so that does make it seem an awful lot like the target directory isn't created yeah you're right you're right make your p only creates the target parents how weird like why is that I mean that's fine I guess we can always just create um I guess we can just create the directory ourselves but how weird that it doesn't create the target of the sim link that that's that seems to make new dear kind of useless uh like okay who uses sim link uh that's not helpful uh it's used only by test suite package properly it seems wrong okay it's used in one place and they're the target it doesn't matter if the target exists I kind of want to change this to create the directory itself as well oh it's because they don't know whether the I think if self source uh is there then it should do this actually I don't know if that's even a thing that I can do no probably what that seems incredibly false alright well I guess we have to create the thing ourselves fine fine we can do that uh that just means that we need to go back to our test suite and here we're gonna have to say before build runs we're going to have to do why can't it find make dear it's called creator see there's create dear all but that's fine like if this it calls make dear peer it's it's all fine we'll just use this so we'll do create dear p dot uh router join uh real target unwrap test so we can unwrap and debug and now in theory we don't need this I think um yeah uh Daniel they call it on the parent I see that I'm just wondering why but it might not matter let's see how this does oh I probably still need to do this isn't it called root dear am I missing something apparently I am uh it's just called root could not this is another I've been seeing lately and I don't know why it happens or I have to do bizarre um no no I understand why they have to create the parents first I'm wondering why they don't also make the directory file already exists I mean I'm sure it does but we're line 35 again wait now real target already exists what all right if they say so because it's reusing the directory layout we don't care if it errors actually no we do care if it errors we have to do I guess uh isn't there a file exists I want a handy way to do that but apparently not basically we we only want to create it if it doesn't already exist so I guess if let air is this uh if E dot kind is error kind uh it's just file exists right I forget already exists yeah it's real sad that this like needs to be this verbose I mean needs to is a strong word um let's see how that works the only reason for sim link is because of windows yeah I guess my argument is more that if it's a directory which they already know then they should also create the directory that's being created as pointing to and they don't and I'm not sure why um it would be more consistent behavior if in n you have a target set as a path then during build cargo should create sim link to that instead of dir so that's not actually the problem either is the problem is when you run cargo clean what should it remove and I'm arguing that should not remove my sim links because I placed them there for a reason um failed ooh okay so now we're getting a failure at the assertion on line 51 so that seems like a better place to go uh we're getting an assertion failure that the target debug oh okay so clean correctly did not remove our top level directory so that's good uh but it did remove whatever that it did remove the let me just go there over here see what's in target ooh okay so it did remove the sim link in there so why did it do that that is a good question does that even if we start from scratch why can't we clean up after test instead of putting if logic check for the tests um so that doesn't work because if the test fails the code after that removes wouldn't be run otherwise it would be attractive to just remove it at the end of the test it's a little weird that they have this setup where the the build directories are cleaned each time uh right uh now I want to see what's in here um target still points there but what was inside what was inside real debug was not removed so this suggests that real debug was treated as a um actually I kind of now want to just do this uh actually this to see that we in fact get what we expected here does that the layout of these directories are what we expect yeah alright so there is a debug and it is a sim link um so the question now is uh it still think I'd remove everything at the beginning of a test and not have to care for last test run artifact but maybe I'm missing something um or remove at the beginning of the test could remove at the beginning of the test that seems fine um so you're arguing that we should do removed er well it can't actually be removed er but it can be removed er all and then we would have to ignore errors from this and ignore errors from this in some sense I'm agreeing with you that this testing infrastructure seems weird to me it's more that I don't want to change cargo's testing infrastructure as a part of this PR that seems excessive um it's only true we can do this though that will help clean up a little bit okay so the sim links seem to be set up in the right way um and after build things seem to be correct um but then after clean the debug sim link goes away so that should suggest that there really is an issue in our code somewhere um let's just give ourselves something to work with uh a curse I guess it's actually this um you have to do this I think right see what this gives us yeah I mean there are many times when I want to fix a bunch of things but I need to constrain myself to only make the change that's sort of relevant here right let's see what this prints so well that didn't print anything at all why this is probably an artifact of um how the tests are run that's kind of awkward boobar because what it really does is it like compiles cargo and then runs cargo at least I think that's what it does which makes running the test a little weird but I don't know why I would just swallow our output unless this code is just like not called at all but that seems weird too well then that's particularly unhelpful let's look at test suites uh lib lib test suite what's even in test suite what does cargo test actually do and where does cargo test come from so a lot of give me a this maybe test macro right what does it do it's just gonna run my thing but hmm it's a very good question why is my output not visible at all even if we go here those helpful print messages right I think it's just totally suppressing our output which makes debugging a little bit more annoying um it's also not trivial to attach gdb here because it goes through a bunch of levels of indirection but we can try alright so we're not getting prints so we're gonna have to break out some gdb so let's do what does it actually give me here uh running recompile it yeah see it builds the cargo binary that's my concern that it ends up um running this as a binary for us uh which is the reason we don't get the output alright so uh this ends up running this so you give me that with cd exact command let's run this command and now run this I want to break at paths.rs line 291 yeah so it really spawns a new process test support source lib alright fine test support source lib support source lib alright so what does fm clean aha run output okay so test suite clean uh can I run output of this please I don't need the uh is there an interactive step by sub debugger for us gdb works fine um I live code on both youtube and twitch it's live on both uh yeah I have rr as well it's really handy the problem here is the processes are a little bit annoying why can't I just process builder hmm I'm in the mood for cheating verify checks okay okay I just want I just wanted to give me the process output do it please um you should rather use exec with output return the output of the command instead of matching against it where oh is this instead of run well so that's on xx but it doesn't that doesn't really help me from where the it's not a function on the project see here we cheated alright so it recurses into debug then it removes recurses into depths removes removes depths that's fine this is the line that's the problem um this rmdir why is that rmdir even run um so remember that here this rmdir is only supposed to be run if p isn't a sim link but this path is definitely a sim link at least that was the intention so unless that gets removed somewhere else further up which I don't think it does interesting oh unless cargo recreates that directory that could very well be um that this is cargo being like I don't want my debug thing to be a piece it could totally be that cargo will just like blow away debug if it's a if it's a sim link seems like it's own kind of problem that we probably won't fix in the same run so when it chooses to rmdir this well that's relatively unhelpful huh seems like it's just a normal it thinks it's just a normal directory how do we want to even deal with that like why is this why does it think the debug isn't a sim link I don't know what we want to do there um in fact let's look further up here so we already checked before right we checked that debug was in fact a sim link pretty sure we checked that we can we can check it again just in case like at this point just to like be paranoid it's not well that certainly explains some things that's because this has to happen before that otherwise we overwrite the sim link which won't work ah but we can't I think we might have to do this let's take a look at that ooh this now fails because not found here clean line 31 creator all I think we need here although creating these should just not be an issue oh I see why it's an issue it's an issue here because this creates something inside of real target which means the real target must exist so really the solution here is going to be this is really what we have to do we first have to remove the real target so they're creating it just fine I guess we can remove real debug too that doesn't really matter now this line is going to create real target and it's going to create the parent of real debug and then we have to create real debug here I think that should do it yeah that's why I changed it to p dot root probably doesn't exist until you call build so that's why that's why I do this is just a removal it doesn't actually need to create it doesn't actually matter we ignore the error this is to remove it in case the in case the test previously failed midway through so now what happens if we do this now it's a sim link so now if we remove this return it's crossed rm on that which is better but not quite right so now it's no longer hitting this case but it is hitting this case that's probably because this sif entry file type is sim link ignore don't delete sim links well the problem here is we want to recurse into sim links why is that a very good question please try to create context as you go what do you mean context as you go okay so now debug is in fact a sim link but because it's not a directory it doesn't get recursed into my guess is that this is because reader does a sim link metadata instead of a normal metadata so I think what we need here is dot sim link metadata we want to if the sim link is pointing to a directory then we want to recurse into it recurse link else ignore sim link I entered midstream and I'm unclear what you're doing what problem you're focusing on I'm happy to give like a recap where we are in theory we're almost done with the cargo bit I'm hesitant to keep repeating what we're doing because for the people who are watching like that would be a little bit jarring I think what I'll do is because this feature is now almost done the testers passed I'll do a recap when we're done which should be in like my guess is 5-10 minutes yeah so the the observation here is that entry when you do read there all the entries you get essentially run sim link metadata not metadata so the file type we get for sim links is not directory but sim link whereas if it's a sim link to a directory we want to recurse into it so that's hence this thing which if it's a sim link then we look at the metadata of the target and see if it's a directory so this would be we still want to actually I don't think that's even necessary see that this works again aren't you opening up cargo for getting caught in endless sim link loops this way yeah the previous one would just rm sim links so it wouldn't get caught so that's a good point we could do this by just keeping like a I see what you mean the easiest way to deal with this is a recursion modifier so we just add a number here that you count down for each step but if it reaches zero you go like recursion limit reached there isn't really any other way to detect sim link loops in this way if we want the sim links to not go away it's a good question maybe the way to deal with this is to say a recursion limit and then give I don't know 256 and here do so this is going to in fact maybe we only decrement the recursion limit so this is really like a follow link limit because that way we can say we can make this a decent amount smaller like we probably aren't more than 16 sim links deep and then we do if follow link limit is zero too many chain sim links we're probably in a sim link loop and so in this case we can return well that's a good question target results I think what we might do here is something like an IO other it's unclear actually maybe I always forget what the IO and we'll do something like caught in sim link loop and this would be standard IO IO error kind probably other probably other and this probably needs an into be my guess I agree with you it probably needs a separate test expected one parameter moved her all what is it complaining about oh I see this needs to call these mismatched type that's fine this is going to be why does the into not do that we might need a from instead suppose somebody creates a link outside of target and target for some reason some new integration test previously the directory was safe now it's gone that's true now there can be links outside of the target directory whereas previously there couldn't be I think that's almost necessary for this feature to work the way I want it to so what I'm going to do is submit it and then mention it in the PR it's a good observation alright and then let's add to the test suite leaves sim links cargo clean detects sim link loops and now how do we want to do this we probably we don't actually need any of these instead what we're going to do is just link so this oops we're just going to link target debug to target which obviously is not okay this at least in theory cargo clean detects content sim link loop shouldn't be the error text because just your guess as to what would happen instead there should be max sim links follows yeah you're right would not remove build directory let's see so that is back here in fact in fact really what this should do is we should have format error here I think it's going to be shouldn't really be in the IO area either which is max sim link well this is also kind of awkward because we don't actually have the original number here sim link recursion depth sorry and this failed where could not remove build directory because of that error which is what we wanted it to be right we want is there nothing here that tries and sees whether it fails that's odd we really want this to be like a run but we want to run that checks whether it fails my guess is there's a we want width status 101 right and with standard error where is the web line where is our actual error message maybe it doesn't print it with error contains and that should contain our error message from up here and none of these tests let's see what that gives us failed why huh this is just like not print internal errors that seems relatively unhelpful um can I do this so it very clearly is printed I want to make sure that we oh is it maybe printed to standard out that seems weird or does contains require that to be the entire line maybe the given contiguous lines where is I think there's some like spacing that needs to be there we see the actual output says could not remove build directory caused by simulink recursion depth limit reached but see how there's like two spaces at the beginning that seems so brittle though but let's see if that does the trick one of the things you'll see is that the yeah okay so the process here is going to be I'm going to submit this as a PR and then it might very well be that the cargo maintainers come back and say like this is not okay we don't want this feature at all or like things like this test they come with better proposals for how to implement it um what I wanted to do here is keep the diffus small as possible so this is why we haven't fixed a bunch of the testing infrastructure stuff that we've seen um and so you see the diffus actually pretty small all right so let's do arguably we should have committed as we went here so we should have committed probably before we added the um uh the recursion limit but here instead we're just going to commit um just let me check that I don't have any weird print lines or something in here I don't think so so this is make sure that we tag the issue that was issue number 7510 uh so I'm going to prefix the message with clean to get this relates to cargo clean do not remove sim links only contents uh previously cargo clean would remove sim links uh in the target directory but not the contents of those sim links this meant that uh large amounts of uh clean in some cases like when the user sets their cargo target dir to a sim link uh did not clean any files it also meant uh it also uh causes surprising behavior where um a build following a clean might fail uh where a build following a clean uh might place build artifacts artifacts uh in the wrong location because because clean removed sim links the user expected to be there uh yeah my editor does cargo format my font is I think you can do both artifacts and artifacts uh this is the notosans mono font note that this commit does introduce some security risk um before this change cargo clean would never leave the current directory would leave the target directory whereas now that it follows sim links it might it might recurse recurse into directories beyond the target directory uh an active or uh the decision that this is okay should be uh we should decide whether we think this is okay fix the 7510 and then we push origin clean through sim link clean through sim link great screen is gonna be bright now um and now we're gonna file this as a PR compare and pull request we have a pull request alright so pull request one is out I suggest that you watch this PR if you want to hear I'll put the link in chat uh you watch this PR if you want to see like what the follow up process is like right so what we did is just the first step and usually after you submit a PR they're then gonna be reviews of that PR um follow up changes um maybe additional tests a bunch of other things that might happen um great okay so that was PR one uh now on to the next project right so I'm gonna CD out of all this so that was the first thing we wanted to do congratulations us now the question is what do we want to do next so um I want there's a contribution I want to make to rust up uh and there's a contribution I want to make to a project where we're gonna move from future 0.1 to standard future which would you like to see first my hope is that we'll have the time to do both so would you rather see more rust tooling or would you rather see futures actually um I'll be back in a second I just realized one thing we forgot to put in the PR description is what the PR changed it said what the behavior was previously but it didn't say what the new behavior was um let's see what does chat say ah futures futures rust tooling congrats and good luck future future future rust up future future rust up rust up futures futures see I could put up a BRB screen but it's annoying like screen management in OBS is annoying because you don't get it through the right hand here um if I did that would be a lot easier but I don't um tooling futures rust up rust up rust up futures okay I think we're gonna do futures first um the futures change is at least in theory more straightforward because it's mostly a mechanical change um and then we'll do the rust up one afterwards um for rust up there are actually two different issues and I don't know which one to choose yet um but that let's do the futures change so um I've done a lot of streams on like async stuff on futures on pin all of that stuff but um one question that keeps coming back to me is uh if I'm on the old future stuff and I want to go to the new one like how do I do that don't just tell me about how it works but like show me the actual process so that's what we're gonna do and so I looked around a little bit and asked for suggestions for uh crates that are using futures 0.1 uh and that aren't too large so that we could pretty easily move it to standard future um and so I ended up with this crate uh by someone who actually watches this stream pretty frequently um so this is an implementation of the beanstalk d um protocol uh and it was written for futures 0.1 and we're gonna port it to use standard future um and you'll see that the the process we go through is fairly applicable to any project at least that's the hope um I don't think I have a fork of this yet oh I do great how about that so we're gonna do the same process of others I'm gonna clone this apparently I already had that let's just blow that away because I don't know how old that thing is I'm gonna rename origin to upstream like before great alright so the question now becomes where do you start? just like with always um and we're gonna start by looking at cargo tumble so generally um whenever you want to do bigger changes like this in Rust doing it in sort of a compiler assisted fashion is really handy is what we're gonna do here is that we're gonna just start changing the types to the new types of wood bricks so this is gonna become 05 alpha 1 um we're gonna use Tokyo 02 alpha 6 uh future 03 alpha 19 think those are the most recent versions and then let's just run cargo update and see what happens uh we're gonna have to rust up uh override set beta oh yeah there's not a futures crate yet this is gonna be futures core preview so the futures crate there is a futures preview that has all the stuff we won't actually need all of futures preview so instead we're gonna bring in core and util are probably the ones we're gonna need so core has traits like stream um util is uh has a little bit of things like channel selection all of the extension traits for future and for stream um so core is just like just if you want to implement the stream trait you would use that uh whereas util has all of the like um map okay map error and then like all of that stuff is in util alright let's do a cargo check this is probably this is obviously going to fail in a bunch of ways because none of the types are the same um but at least it'll kick off the build process uh what's today's stream about today's stream is an open source contribution stream where we're gonna take on a bunch of open source projects and try to make contributions to them uh we previously did one on we did a contribution to cargo uh now we're doing taking a crate that uses futures 0.1 and moving it to standard future um and then next we'll do we'll try to fix an issue in rustup uh can't find crate for futures that seems like it's obviously true actually does this still use uh this is not on the latest edition so i'm gonna go back a little and then i'm gonna run cargo fix edition so cargo fix edition uh is you i just want to move the crate to the 2018 edition because it's nicer um and we're gonna make a standard future branch and this now did some rewrites now we can say edition is 2018 in here and that should in theory work variant is never constructed that seems fine um what else do i want i also want there's a worn rust 2018 idioms i forget what this lint is called rust 2018 idioms great um so this this lint is used for like things that you should be doing in the 2018 edition that you couldn't in the in the 2015 edition so let's just fix these so this this um the failure one is handy because it uses macro use uh 56 this is another change that happened in the 2018 edition where um if you have types that have a lifetime parameter uh it's optional to write out that lifetime parameter but in the 2018 edition you're supposed to do it to indicate that this type has a lifetime parameter it's not static um and you can do tick underscore to say i don't care what this lifetime is um and then let's see where else proto request uh 87 same thing here um the external crates we already dealt with and the variant never used cargo format uh i think i'm also gonna do a uh rust format about toml and say addition equals 2018 move to 2018 edition uh we install uh can you do the cargo toml thingy where you install crates future score preview but you call the future score in your rs files um so dash preview uh the the all the futures preview crates are their names are without the preview suffix automatically you don't need to do anything uh i don't think fixed edition did work because you didn't change it in the cargo toml did it uh cargo fixed edition you need to run without changing your cargo toml first um so your cargo toml should not say addition 2018 um and then you're in fact it will it will tell you this if you try to change cargo toml and then run cargo fixed edition it will say nope you've already changed your addition uh which rust edition version do you prefer to work with futures uh you basically need the 2018 edition there there is no 2019 edition but the 2018 edition is where you get async await and these types of things um alright so now we can go back to this um and run cargo check and it's gonna complain about a lot of things um and so we're just gonna start at source lib uh and look at where it complains so first it complains about tokyo prelude uh which makes sense because the tokyo prelude you basically don't need anymore maybe missing a crate tokyo no why is it saying wait what it's lying to me oh right edition 2018 great let's try that instead great bunch of errors uh lib alright 139 it says susan type item not found for future that is right so there are two things we can do here um the most straightforward one is that future no longer has two associated types so we could just say output result this that's sort of the most straightforward change in reality we don't really want to do any of this like imple future business anymore because we have async functions so we can just make this an async function uh that just returns the result nothing more to it and now we can do something like um let connection is this await uh and then we're gonna return okay setup C see how much easier that was um my guess is there are more of these these dot comments will not work correctly this should say this has to be like this this has to be like this and this has to be like this and there need to be no line otherwise they won't actually work correctly uh the additions are for rust format um it isn't necessary if you run cargo format but if you run rust format which rust.vim will do sometimes then it doesn't pick up from cargo tumble it's a little unfortunate um we can do the same thing here right so you see here put returns this in fact here there are two things we can change first we're gonna have to change this to be output but also notice that this is a pretty common pattern that you see in um 0.1 libraries which is they consume self and then they return self um and the reason being that they need to like mutate self in some way but with the old futures this wasn't really easy to do here though we can just say make this an async function say that it takes mute self and that it just returns I guess this it's a little weird but that's what it says I so I've never actually used this library so I don't know what it does this is this is in some sense a blind port um what's important to note here is that when we put the async keyword before function what that means is the return type is a future that produces this functions return type right and crucially that future will be associated with the lifetime of this borrow right so us taking mute self here is fine it just means that the future you get back is tied to the lifetime of that borrow of self and we'll see how that impacts things later now here you see the insides of this of course we this is now an async block and so we can use a weight if we didn't want to use a weight here notice that this uses and then you can still use and then so let me pull up futures util preview uh so let's look at the future trade if you look at future the new future trade the one that's in the standard library it just has an output right there's no item and error anymore and when you pull it just gives you that of course the change is that you might still have futures that produce results right that basically futures like an error um and there are some convenience methods for dealing with that and they call this try future and they have the same for try stream so try future is a trait that is implemented for anything that's a future um where the futures output is a result and so try future is basically like if you were previously depending on a future that could error you can just change use try future instead um and then you'll notice that the uh under uh under future there's future x so this is the extension trade for future which has all the normal combinators you're used to right but you'll see that this doesn't have an end then it has a then right because if you have a future and it resolves and then you want to do something you you call then but it doesn't have an end then because it has no notion of succeeding and then doing something else um similarly the map function here is a map regardless of whether the underlying thing succeeded or failed if you have a future that can succeed or fail you would have a try future so you'd look at try future x and that has a map okay map error and then or else like all the things you're used to from the old futures that could have errors um and so we we would be able to use this if we just imported that trait but look at how much nicer it is if we don't do that and we say that con is this await and then just that right we no longer have to do this this callback business because we can just write it in a straight async await form and this is where this is of course where async await shines um we can do this for all of these right so this is going to take a mute self it's going to return a I don't know why result is used here it seems weird for this to use the result type um con is this thought await um and then we do the same thing ah same thing here this is what I mean by this being a relatively um mechanical change is that you don't really need to think while making these changes even though I just said uh while saying that um and there because generally this is it's just transforming code I'm pretty sure you could write a tool that just did this at least for most of the common cases uh the result for item used was used there because there are method specific method specific errors and protocol specific errors which I wanted to surface ah okay so what threw me off then was pro I see I see is that here for example this just says consumer whereas I guess this is like consumer errors or something um doesn't matter too much what I want what my goal is for basically all the PRs we do today is to make the minimal change and this this should often be what you want to do in PRs the less you can change at a time the easier it is to review the PR if I did a major rewrite of this crate like beyond just the mechanical changes it would be much harder to review that PR instead what's usually a good idea is um make self-contained changes for your PRs so this PR will only be about moving to async await and then we can do a later PR which cleans up in these types should we need it right but but that shouldn't be something that um that we bundle up with the main change we're making uh it's kind of easy to do these changes on code that use simple future often but how about crates that have own future types so um for crates that have their own future types um often you could do the exact same transformation the manual implementation of the future um you just turn into a an async block it's the same thing and if they don't that's fine too right um you could imagine that this returned like a instead of this it returned like a my super future because why not and then in here this used to create a my super future and return that well if you're in an async block you can still create that and you just do it so and and then turn this back into this so the the same transformation still applies uh I don't think they want it either I think they I think result result is really what was intended here so we have one of the we have the author of the library is in chat and is uh is pointing out that this was written when they were pretty new to rust uh and that's fine we we were all there um I also don't think this code is bad it's just like it was just an odd pattern to see result result um but I think the explanation makes sense for why why that's what you wanted um now this this library is probably more regular than what you would normally expect to see right you see all the methods here are basically all the same like the transformation is the same um we'll see whether that's the case when we get to like deeper parts of the library I don't know this should now be result and this should be con is that await this I forgot to change this should say mute self did I miss that anywhere else yeah this should say mute self great mute self arguably I should just make a at this point I should just do it like a vim macro for this but in fact I wonder let me try to see if I can do a vim macro for this so for those of you unfamiliar with vim macros in vim you can record a sequence of key presses and then you can replay that recording um so let's see qa word forward insert async escape percent to go to the end of the block forward to the eye change to the result escape right to go to the result percent to go to the end of the result right to go there delete to the equal sign and replace the equal side with a comma crap I forgot mute self you're right uh back oh what is it it's it's shift f for backward to character end parentheses percent append to line mute and then rust format will take care of that for me percent to get back there twice okay uh so that transforms the header um and then we want if they're all as regular as this and we want one line down zero to the start of line let con equals percent to go to the end of the send append to line await question mark semicolon escape line down delete insert above escape percent line down double delete uh now let's see whether it worked seems pretty promising it just didn't delete this line but apart from that I did what I wanted I think oh but this is this function is different um oh yeah you definitely spend more time constructing the macro but also of course now we're like almost at the end of the files it's unclear that it's worth it um and of course this function looks different so uh alright let's try this one yeah didn't have to do any work what about this one uh not quite for some reason I didn't like this one I really didn't like this one alright let's try recording it again just for fun uh QB alright async mute percent percent forward to the I change to the R in the results escape percent to the end of results uh delete to the equal sign replace it with a comma line down zero let con equals escape percent to the end of send append await line down delete insert line above just because we want it to be pretty uh line down percent to end line down delete end of Q no then now of course there are no more methods damn it uh oh well we tried um uh no Sebastian we're we're roughly halfway through um Alex no I'm not using a plugin for percent that's just what percent does it does it goes to matching brackets I kind of want to I kind of want to undo now and like retry my macro um okay let's see how that worked well no method named into future line one one four uh so handle response is the macro that's called in a bunch of these and we don't need this to be into future anymore why was it even into future in the first place huh okay so this is kind of interesting we may have misread what handle response does so oh no it handles an okay as long as the connection from send comes back okay response is going to act on it and con here I see okay so this is going to turn into an async block actually it doesn't even need to be an async block what am I talking about why is this an into future so that means that con is a result uh I don't think it needs to be into future anymore I think this can just do match on input and if it's okay and in fact there might be an opportunity for us to improve this quite a lot so this send business previously used to consume the connection in some way I guess uh whereas now I think this certainly doesn't need to produce any these anymore right I think it just needs to produce whatever this is and the error no longer needs to be this the con is no longer going to be included so this is really just going to be that not sure why error turns into okay oh because it's an internal error okay alright so let's see what that does ooh I did something silly didn't I uh this no longer needs that this matches expression type nothing that seems false what is the type of con really where's my assignment to con frame oops that looked like a problem this definitely should not say what it does ignore I think we actually accidentally got too excited about our macros uh ignore no longer sends anything where's ignore ignore should send one of these really uh I guess let's look up the docs for Tokyo uh Tokyo codec and we have a framed and what do we have on framed it implements sync and stream aha okay so what this means is we really need uh back in cargo toml here so in we need to use it as a sync so we want to use sync x and sync x I think has it just send yeah great so this is the thing that we're calling over in over here right self connection is a framed uh we call send on it and that is this business and what does send give us back it gives us back a future I assume yeah just a future right mm-hmm so why is it nothing I see what's going on uh I see what's going on so okay to explain uh self dot connection is a framed uh in Tokyo world and a framed is a it's basically something that's both a sync and a stream that's implemented on top of async read and async write as what we're doing here is we're using it as a sync to send a request to the server and then we're using it as a stream to read a response back and so handle response should really wait for the thing to come back and that is no longer con right we don't care about this value anymore this con is no longer important because the return value so send gives you back a future that either errors because you couldn't send or it succeeds with no value so there's no like con isn't a thing anymore um and so anywhere where it says um handle response con what we really wanted to say is handle response oops is handle response self connection and then the macro that we have this is going to be expert dot next dot await please in theory this should say con this match expression yeah so what we're saying here is we're going to send a request and then we're going to call next which is the stream method for getting the next thing as a stream which is going to be the response from the server and then we're going to await that now in a standard future now streams return options of the output rather than like a result of option so that's why you're seeing here that we're trying to match on the result of this as a result now an option that contains the result rather than the other way around um and so this means that oops uh this clause no there's no longer an ok uh alternatively you get some ok valve in which case it's this uh or you get some air e in which case is this now this being an ok I think is because um the error is if we fail to send something yeah the outer error is if we fail to send something to the server which is like a problem with the connection the error here is we got something back but it's an error but it still seems a little weird but I think that's the semantics we want oh wait that compiles alright well that was easy we haven't ported the test yet um I'm surprised there aren't other I guess these are just like display and stuff alright so let's try cargo test of course I don't actually have beanstalk do I uh ok so this is gonna be uh port library parts beanstalk I guess I want to install it that seems fine yeah um so David it doesn't actually always return ok um because this question mark can cause it to return an error handle response will always return an ok but this question mark means that we can return an error in the sense of the outer error I still think that these should probably be collapsed um and just have the the error be um hierarchical uh but I don't really want to make that change as a part of this PR start beanstalk D so now we need to port the tests and uh luckily this is probably gonna be a lot easier um I guess stop in stock D if it's gonna start it on its own um so here what we really want to be able to do is just use async here as well and it turns out that we can when you use tokyo you can use tokyo tests you can make this an async fn and then you're rolling now tokyo will already create the runtime for you so you don't actually need to worry about it previously we had to create these for every test we run no longer necessary this this annotation will do that for you um now because we're in an async fn we don't need to do things like block on we can just do wait unwrap down here we can say uh this is just an await this is also just an await that we're gonna unwrap on we don't need to keep reassigning bean because this now just takes mute self um here we're gonna do response is this dot await and then we're gonna assert equal response test all of these blocks go away we don't need to check that it's not an error at the end because we're doing that as we go we no longer deal need to deal with the runtime now let's uh I would anticipate an okay handle connection but yeah let's make a change I mean I would be okay in putting the okay around but I think the current structure to not put okay around handle response um if handle response changes so that it might produce an error instead it's easier to keep it the way it is uh uh this is vim I don't use Adam I don't like to have browsers running where they don't need to be um okay so this change is basically the same thing we're just gonna await unwrap and all of this is gonna be similar kind of things so this is gonna be response doesn't actually use the response here so I think this is just an await unwrap that's all that inspect really does uh this does bean dot reserve and here we do actually want the response we want to assert that the response data is that apparently uh and then we want to peak bean at this thing we want we want to assert that response we want to assert response I think that's what that's saying uh oh no that's just an unwrap this is a weird way to write an unwrap uh this though this one's response.id we don't need the unwrap because we already unwrapped up here um the response this doesn't care about the response uh this doesn't care about the response just wants to unwrap it this seems to care about the response so you see how this code is now basically becoming almost the way that you would write it um uh if you were writing it just imperatively tell that there's async here apart from the awaits because all of these like weird closures and inspects and and thens just go away um and also it's nice that now there's no longer this tuple that's returned by every method instead you can just call the method directly goes away the response here ooh this is supposed to be an error I think right this is supposed to be oh this is where the nested uh nested okays come into play so really my guess is most of these unwraps need to be double unwraps it's a little awkward so this becomes that I see so that's why there are all these unwraps in here that's why this is an unwrap .id so really this is just this is a double unwrap this is a double unwrap all of these in fact are probably double unwraps I don't really want to add all of them we can add them later um wait unwrap see what I mean by this is uh mostly mechanical change right this can go away go away and then the rest of that can go away alright 561 uh yeah exactly so here we're starting to see like all of these are double results 581 this is a double result in fact how about we just do uh .await unwrap with .await unwrap unwrap the exception to that is this which should be air and this should check that and this also doesn't need to do that and this doesn't need to do that and they don't need to be rough and the bean needs to be mutable with here and up here see what that does um cinema made a tool to automate the transformation for commodity adjacent weight um I haven't seen any tool I feel like it should be possible to write but I haven't seen it yet uh sufficient enough is the wrong wording more powerful to the right way of saying it sufficient enough oh above them oh I see I missed some chat uh can you overload stack names I don't know what you mean by overload stack names but yes you can declare a variable with the same name as a previous variable and chattow it um this is a published crate it's rust compared to go lang in terms of performance of writability it's a much larger discussion that it doesn't really fit here um I would say they're they're both good for different things uh can you elaborate on browsers running where they don't need to be um I want my browser to be I want a single browser I want to control which one it is and I want to control all the updates to it um I don't want a program that runs an outdated browser in the background like this is why I don't use electron apps for example I would rather just have them be another tab in my browser and use a pin tab or something um I am totally happy with Vim the way it is I don't want to have any richer features I don't need the richer format um what's the reason you switched to cock.vim um it was actually because I wanted better integration with rust analyzer uh which is a little awkward to get with ncm it's also weird because I was using ncm and ale for two different things and cock just provides both of them um alright why did this not pass this did not pass because connection fused address already in use really is that so that does look an awful lot like that what who ran that is it still there? no okay it works succeeded and then consumer commands failed can I have it just not spawn for me please? spawn is just going to be true uh and then I'm just going to use that how about now apparently not failed for some other reason but spawn not found where here source lib line 600 did I miss something in the port probably that that test uh which part of the test is this this is after more data where is more data more data uh this seems like it's expecting an okay response interesting if I run just the consumer test yeah that's what I was afraid of okay so they work in isolation they don't work if you run they do work if you run both okay well I'm just going to ignore that for a second um let's not include that but let's include all the changes there and then actually do I want to include all those changes for tests to standard future I'm just going to leave that in there um and now you'll notice the next things were failed were the the dock tests and the dock tests basically need to go through the same processes we've done before um we want don't need any of the extra crate anymore we don't need tokyo prelude um we can do that that's fine um and this now becomes tokyo uh main don't need to do this anymore and we can have this be visible just so users don't get confused wait unwrap this is going to have to be mute in fact let's see how did this how did I break this down I kind of want the same breakdown in the docks it just makes it a little bit nicer to see wait unwrap it's really sad that this double unwrap has to be there use a particular tube alright you say so this so that actually shortened the example by a decent amount and especially given that we expanded out this part right arguably this shouldn't even be necessary this parse um could probably be gotten rid of but again in the interest of making fewer changes to the library at hand we're going to not do anything about it so this is now the standard setup this is going to be something like let spawns is bean reserve await unwrap unwrap do something with the response I'm going to take an artistic liberty here and do this and then bean.delete response I guess this should also be double unwrap like so let's see if it's happy about that great port dock test to standard future do you see all of this like very very straightforward mechanical changes across the board what is bean beanstalk D is like a I want to say it's like a job server the idea is that you can submit jobs to it or request jobs from it it's sort of like sidekick or factory why do you want to rewrite async ssh with ffi just to show ffi or is there any problem with thrush so when I first wrote async ssh thrush was very a very immature project as we had to hack around it a lot maybe it's better now like maybe it's fine it doesn't have to be with ffi I just haven't looked at it since I also one reason to do it with ffi is to show off ffi but it's not particularly important to me if thrush is now good enough that we could build something reasonable on top of it well we have to write this many unwraps when async await hits stable oh so the unwraps are there because of choices we made in the library the unwraps are not like async await does not require any unwraps the unwraps are just there because this library has a double result type and so we need to unwrap the outer result and unwrap the inner result but it's not related to async await I always unwrap my unwrap before I unwrap my favorite is when I get a little bit tired I start writing unwarp instead of unwrap and it sounds way cooler oh I can go for a wrap wrap sounds tasty alright it's gonna get light again people boom poor quest that's not very helpful port to standard future this PR updates the crate to standard future and the alpha based ecosystem around it tokyo alpha.6 and futures preview alpha.19 mostly mechanical changes except that all methods now take mute self instead of self and thus also do not return self anymore oops also I made a stupid which is undo that last commit for two reasons one is this extra spacing but I don't want to check in that change to the boolean yeah I did I said it was gonna get light again mostly mechanical changes except that all methods now take mute self instead of self and thus also do not return self anymore the double result wrapping is now particularly apparent and should probably be changed but that's for another PR this patch also updates the crate to the 2018 edition boom good job team we now have two PRs under our belt pretty good for a day's work so let's make it three yeah there are dark github styles but maybe one day I don't I actually prefer the light style even for docs.rs but for the stream I don't because people get sad when they're in because it's the middle of the day here but if you're in a place where you're in a dark place in the middle of the night then the stream is pretty annoying. Alright time for the third and last contribution of the day okay so for those of you who have joined only sort of midway through what we're doing is an open source contribution stream the idea is that we find Rust projects that we think we can submit a PR to and then we submit PRs to them we first did tackle an issue in cargo now we've done a port of a crate from future 0.1 to standard future and the last thing is going to be contributing to the to another part of the Rust tool chain which is the Rust up tool the thing that you usually use to install and manage your Rust installation here we have here we have two different choices of where we're going to go next the first one is this issue can you go back to the first PR there was an error during the build error during the build you say sure we can do that before we do the the Rust up one really it face all nightly interesting why does it fail all nightly these errors are unrelated to us these are nightly issues I think this is like cross compile stuff I don't think this is us I mean could be of course but it doesn't seem like these errors are our faults failures that one is us though on macOS no such file no this one no such file or directory interesting although there seem to be other errors along this line like different DER also fails with no such file or directory yeah this also fails with no such file or directory so I'm tempted to say this is unrelated to us no maybe not this also runs clean we did push it with the um we ignored that huh it's hard for me to reproduce given that it only happens on macOS like windows passes and linux passes if someone with a mac wants to try to run the test suite and like dig into it then that would be very much appreciated it seems to work elsewhere so I'm not entirely sure what's going on there I'm gonna leave that for a bit the nightly errors seem unrelated to this change they also seem to be occurring in other unrelated tests it's useful in PRs to at least follow up with a short like when you get errors like this at least let them know that you've looked at the errors and what the status is in this case if I had a mac it would be a lot easier to reproduce but I might look at it later alright so with that aside dealt with so there are two issues that we could choose to tackle for Rustup the first one is this one which is Rustup recently got the ability or the support for something called profiles and the idea for a profile is that um by what happens if you install a tool chain or rather which components get installed by default when you install a tool chain uh previously you used to get a bunch of stuff which included things like all of the standard library documentation which meant that if you download a Rustup and then try to install stable or beta or nightly or whatever it would download a lot of stuff because it had to download all the documentation um and so recently in 120 which was released just a few days ago um they landed support for this thing called profiles which is you set your profile to be minimal for example then Rustup will only install the parts of Rust that you need to compile things rather than all the sort of nice to have business which is very handy for CI scripts as an example um however you have you're only able to set your profile when you install Rustup which is nice but you also kind of want the ability to select which profile to use when you install a particular tool chain like when I install nightly I want to say install it with or without the bells and whistles um so it might be useful to be able to set this on a global stage but setting it for a particular install is attractive um so this is a very straightforward issue I don't know how hard it will be to fix it shouldn't be too bad um but this one like seems like a straightforward thing to deal with um if we want to get into something that's a little bit more complicated but also is only tangential to Rustup um there's this issue uh which is basically if you interrupt Rustup in the middle of a download um then there's a basically when Rustup has to download something from the web it downs it into a file with the partial suffix um and it just keeps appending to that file as it downloads and then if you interrupt the download it's going to try to resume from that file the problem is if that partial file either is corrupted somehow um or it actually completed but it just didn't do the move of the file then trying to continue the download is going to fail in weird ways and basically you need to manually remove the file to make it make any progress um and there's an existing PR to try to fix this that PR seems to have sort of stagnated a while back um and there's also a follow up issue that's somewhat related and a little bit easier to reproduce and so we could try to tackle this sort of group of problems um it's a it's a bigger test to undertake and it's also a little bit harder to test because it sort of requires to interact with um HTTP servers um in fact what is this okay oh this is the follow up comment um I think my preference is supporting this because it seems like such an obvious hole to fix and it's a gentler introduction to Rustup but if you particularly want to see us deal with like HTTP download issues then we could do that too so would you rather see improve profile support for installs uh or would you rather see like HTTP debugging one vote for each so far it's interesting because there's a there's a delay to both YouTube and to Twitch and so suddenly there's like a stream of messages that comes in uh HTTP HTTP Rustup I mean it's going to be Rustup regardless both of these are Rustup but one is how Rustup deals with HTTP weirdness uh and one is um how it basically fixing the profile support that's interesting so it seems like oh the test suite has a web server built in great that makes it easier um seems like Twitch prefers profiles and YouTube prefers HTTP oh yeah okay great it doesn't have a web server alright I think what we're gonna do is we're gonna seems to be about 5050 so we're gonna do the profile thing um and here's the reason why it's a little bit shorter uh which gives us more time to talk through the changes we make uh and to test them properly uh it's also something that is more important in terms of features that Rustup needs um like this is more needed than for people who interrupt downloads um it also means that the HTTP download issue you can go fix right like after we go through this like try to fix that problem and now there'll be a bunch of people who can pick up that PR for example and try to run with it so let's do that um okay so I'm gonna close down these uh and then we're gonna go here to and here I cheated a little I have a checkout already uh the other thing that's a little bit nice about that I actually really like about the Rustup codebase is that um I can build it in fact I've already built it um and then I can do this uh this business so if I run this funky command um then this lets me run Rust the run the compiled version of Rustup on my system as if it wasn't the real one um and this is handy so for example if I do this uh install uh I guess nightly 2019 0101 great um so this is now running the Rustup that I have compiled on my system rather than the system-wide Rustup and you'll see that this is now downloading things like Rustdocs ironically I'm now gonna interrupt my download um and really what I want the ability to say here is profile minimal profile minimal but that doesn't work um I can change I forget where this is is it here somewhere hmm is this where I can set like profile equals minimal nope apparently not um but generally it's when you install Rustup so when you do like the whole like curl business to which has its own problems but it's when you install Rustup that you're able to pass this profile business and I think it creates a file somewhere I forget where um I think this was in the wow I cannot type anymore Rustup120 see if I can dig up this could also just read the source code but that seems too hard having a piece of state somewhere where is the piece of state that's what I want to know profiles uh huh uh huh uh huh yeah so I can do this great and then this and notice hmm why does it still download docs that doesn't seem right have I like not pulled that shouldn't matter that's interesting okay so this seems like an issue um when I set the profile to minimal then installing something should not download docs maybe it's because I already tried to download this one I tried to download a different one what happens nope it still fetches docs we go that seems broken yeah okay so it is just setting profile minimal there okay so something is broken um the real question is do we want to fix this or do we want to fix it differently um huh oh maybe you're right maybe I do need to do uh this set profile minimal although it should just have the same effect I thought yep still download docs try a more recent nightly sure okay yeah so with a more recent nightly it does not allow no docs whereas if I did this oh default great then now it will download docs it did not like that uh unavailable clippy great yeah so now this seems to work no I'm on 120 it's um it's the nightly I was installing the 0101 that did install docs but on 1004 then it will then the profile applies correctly so I'm not sure why that is but that's the case so you see now um let me demo this again now that it actually works so if I have the default profile and I try to install say this nightly you'll see that it downloads cargo clippy rust docs etc um if I set the profile to minimal minimal profile minimal and then I try to install it then it will not download clippy it will not download docs this apparently only works for recent nightlies but that's fine um and now the question is is there a way where I can do profile here minimal right so I'm going to set my profile back to default and now I want the ability to do this yeah exactly 0101 is before profile data was available in the channel manifest so that's why the older ones didn't work um so I want the ability to run this but of course profile isn't supported on install and that's what this ticket is all about all right so we want the ability to do this with rust up how do we do that well let's start a new branch and we're going to call it fix 2004 great um where do we start well start with contributing the same thing that we did for cargo right and we've worked the project done we created a branch done we can't commit any changes because we haven't made any yet I feel like there's a 2.5 here which is make edits um yeah so we want to develop rust up itself so we're going to install it into temporary directory which is what we just did um there's cargo builds you know what I actually did I didn't do a rust up home I'm just going to share my normal install and that's fine um ah so this is a different way of doing it it's basically installing another copy of rust up I didn't want to do that it doesn't really matter um I know that this particular thing works yeah so that the here developer tricks trips and tricks that's where you see this environment variable that I'm using to trick rust up into thinking that I'm running rust up um so in theory just cargo build is all we should need which also means that in theory we should be able to run cargo tests actually let's do test that it's one those tests are slow why are the tests slow seems slower than they should be I probably like launch this in processes or something okay so the test now we have a test suite that runs great now let's see if we can actually figure out the issue so um this is part of the install path right like we want rust up install some nightly to take a profile flag so the first thing we want to do is find install um generally the easiest way to do that is look for a file called install well there's source install what does that do uh installation and upgrade of both distribution manage and local tool chains that seems exactly like what we want um run uh okay so this seems like the stuff that gets called when you try to install something but the real question becomes where is this used like how is this called in the first place so this install method see if we can find where install method comes from ooh source tool chain that seems potentially promising let's look for install here component status install alright so this all seems like sort of internal wiring was really what we want to find is where the arguments parsed so maybe let's look for clap that seems promising rust up mode source cli rust up mode that seems very promising alright and here is there something called install there is interesting so main okay this seems great so this is the this is the main that gets called when you want to run rust up as a binary with the name rust up basically you should think of this as rust up behaves differently depending on the name of the binary you're running it as and if you're running it as rust up then you're in rust up mode in fact if we look at source cli you'll see that there is like setup mode proxy mode who knows what main is okay so it looks like install just calls this update function alright so where does update come from what is update well here we have sub command install that seems promising so we want sub command install to take profile but we also know that there are other places where you can call profile and ideally we want them to work the same way so let's look for arg profile alright so setup mode also has a arg profile great we want the same things in our profile which means we need profile names and profile default name where does it get profile from gets it from over here so we need that over here too which we already do great so now we have the profile argument to install so at least there's that but now we need to parse it somewhere alright so this creates a giant list of commands for rustup and at the end it probably has to parse them somewhere interesting so this was cli so something calls cli rustup main great yeah so this is what matches on the sub command looks for install calls update so where does update come from let's see if we see an update actually do I have gd should have just that seems promising is there an fn update great so it looks like both install and update really call this update function so this is sort of a catch all for you want to change the set of tool chains or tool chains are installed and in our case what does this do values of tool chain does tool chain have a default value then values of let's see so what is tool chain even set to in install target tool chain update install it's not set to anything by default that's interesting hmm so this is I think I think this will be some with an empty vector but I'm not sure regardless it might not matter so this calls update all channels and this calls okay so what does update all channels do so where does config come from fine update all channels on config so update all channels just lists all the channels which is things like stable beta nightly and it calls install from dist on all of them and if we go back to our update it calls install from dist on all tool chains so it looks like regardless of whether you end up through this path because you haven't specified a tool chain or you end up here because you have specified a tool chain you always end up in this install from dist business and so really what we want here is we want to make sure that we thread through the name of the profile if there is one so let's look at what install from dist does install from dist so install from dist does install method dist that seems promising so this is like a giant struct or tuple of various properties about the thing you want to install I guess right so here we see components you want to install targets you want to install who knows what all the other fields are but this seems like a place where we need to include the profile that we want to be used and this calls install what does install do calls install method run alright so that means that this means that this install method thing must have a run method so install method dist what happens if you call run on it it calls update from dist so this is like a web of calls so we're just like trying to trace down to where where ultimately it's trying to install all these things so dist update from dist what was it update update from dist dist update from dist alright and that calls update from dist so down here calls try update from dist what does try update for dist do downloads and manifest so that seems promising get profile components alright get profile component seems like something we care about if profile is some alright so this is the thing we need to end up overwriting with so where does profile come from profile comes from here which comes from here which comes from here which comes from here which comes from here which comes from here which comes from exists which comes from profile alright so profile so tool chain self config get profile. Alright, so how does the config get set for install for from dist. So that would be back in Rustup mode. Tool chain. Okay, so it seems like what we want is to set is to set the profile in the. Yeah, we want to set the profile in the config that's used by the tool chain. Yeah, does this track? I think this tracks. So if we go back here, this tool chain has a config field. And I mean, we can double check that, but it should. As a tool chain has a config field. And inside config, there is these hopefully, how does profile get set here? Interesting. Oh, this just mutably modifies the profile file. That's not ideal. Because we don't actually want to change the file, we want to change this temporarily. It looks like get profile will always open the settings file. That's awkward. Okay, so basically, what we want to change here is we want the profile to be or the we want the profile to be stored in the config, either that or we want to pass an override. So that if the override is set, it uses that one. Otherwise, it uses the the one return from get profile. That might be the cleaner way to do it. If we don't want to modify the config here. Again, this is in the in the interest of keeping PR small. That said, like here, I think it would be okay for us to change to change config so that it actually stored the profile and say loaded it when it starts. And so there's a presumably like a new from nv. So this probably opens the settings file, I would guess, apparently not. But you could only imagine that this set default. Yeah, so you can imagine that here in from nv. Yeah, settings file, like right after this part, it would load the profile here. That's actually probably what we want. What else is settings file used for probably used to set a bunch of things? Yeah, interesting. Okay, so we have we have two options here. One option is for us to pass to install from dist, like a profile override. I'll be one way to do it. We'd have to do the same for update all channels down here. So maybe it's a little awkward, like we'd end up having to override a bunch of things. Actually, we can just override it in here, I think we want to change it on config, because that way we can just change the config right here. Problem is, this is read only. So we don't really have a way to override it. We could clone it and modify the clone. It's definitely a little ugly. So I assume the reason that it's read only is because you're not supposed to change the config on the fly. Because it's basically what is stored in the files. But it feels a little bit against the spirit to be adding a mutable field to config, given that it doesn't seem to have any other doesn't seem to have any mutable fields. So it feels a little bit like cheating to add one. But at the same time, like, all right, we could try it. Does this even implement clone? Oops, that is not what I wanted to do. Does not implement clone? It's also kind of awkward. Think it could implement clone? Just check that it actually can. Hmm. This is something that you run into for for submitting PRs to other people's projects is that it doesn't always make sense to you. The design decisions they've made and you want your PR to align with the design they've chosen for their application, right? As you really need to think about what is the way in which this change will match the current design. Temp config. That's a good question. It seems to be something that's like the temporary directory used for temporary stuff, it seems like, because it uses the rest of dare joined with temp. Let's see if we can make this be clone 37 arc, big 93. This is now an arc new. Okay, so that's now clone. And then I guess what we'll do is cached profile. I think it really should say just profile. And then what we want is, so that's going to be a dist profile. And then this, so get profile is really what we're going to do up here. So we're going to say, let profile is equal to that. Because then when we construct the config down here, we can now add in profile here. Then down here, we can say that this now just returns self dot profile. This is now no longer true. This cannot fail anymore. Interesting. So this previously returned an error. If settings can not be read or the profile is invalid, both of those are now instead going to cause errors when you create the config, which is probably fine. And it will still return okay default. So this I think can now go away. And now set profile also has to change because do this. So set profile has to do self dot profile. Oh, balls, we can't do that. Can we? Because set profile only takes a ref self problem here is if set profile is called, then get if you called get profile after you would not get the same result. Okay. Here's a proposed alternative strategy. Given that that won't work. We make this profile override. Then we have this be we have this set the profile override to nothing initially. And then we have that won't work either because because this still just gets a read only config. No, if we clone it, it's fine. So here's a proposal. Get profile is going to be if self if let some p is self profile, then return okay p clone, right. And then we have a with profile basically that is pubfn with profile, which takes a mute self and a profile and returns a self. And it does self profile override sets that to some and then return self. Because now what we can do in Rust up mode is say, let config equals config dot clone dot with profile. Well, actually, it has to do this. If that's some p is m value of profile, then config is config with profile p. Great. And then of course, you can do that config. See what this is. How are you able to search the source tree for file so quickly? So in my editor, I have it set up with the fuzzy finder with fcf. So control p brings up a fuzzy finder for the file tree. And so that gives me fuzzy find for files. I also have double spacebar map to switch to my previous file. So I can swap back and forth between files quickly. And then space semicolon gives me a list of previously opened buffers, and let's me fuzzy find through them. So that's my navigation flow needs something like a dist. So that's almost there. And then now we need this to parse the profile, I guess. Do you need to figure out how though, I sort of want to give the same error as get profile does when it fails, which comes from tool chain. And that just seems to error normally. Okay. And what error did that give? That gives a this error, like so. And this just profile work is profile even in scope here. Probably not. Oh, no, it is great. Let's see how that works. Okay, so that worked. Let's see if it actually works. So my profile is now default again. And if I try this set, oh, I need to rebuild. See what that gives us. Okay, that builds. Now, that did not download clippy or rust doc. And what if I do remove this? Then it gets clippy and rust ox. So that seems promising. That seems like exactly what we wanted. Right? Right. So let's try writing a test for this now. So install add dash dash profile flag to override install profile override profile sound is much better than last time thanks to ever give to you the shock mount. That's good. I'm glad. Yeah, it's because so the reason this happened was this microphone is better than the one I had like much earlier. And the boom arm is much more firmly attached to my desk, which means that the tapping sounds transferred to the mic a lot better. And the mic picks it up better. So I think the shock mount helps could use RC instead of arc. I could. The reason I used arc is first of all, the cost doesn't really matter here. Like the atomics are insignificant in this kind of workload. Like if you're working with a command line tool. And arc means that the things are now send and sync, which is nice. And also because this code base uses arcs elsewhere. And so just for consistency, we might as well use arc here as well. Install test install, huh? Package contains for profile here, really any profile here, but set up profile in tests. That looks more promising. So there's self update. Climus Climus Climus do with profile. Alright, so this does what exactly? This seems to indicate that it runs rust up in it with profile minimal. Wait, does that not install cargo? Really? Profile minimal doesn't give you cargo? That seems false. Alright, well, I mean, we could try that. Let's just copy this test to install. And then just add install with profile. Do we have a setup function over here? No, so this has a setup that does this thing. So let's copy that and see what this will do. Of course, we don't actually want to run rust up in it. So what are the other tests that do installs over here do? They do installer builder seems awfully complicated. I don't want any of these. I think this might be the wrong file. It's just like hang on to this and then find like test. Maybe just dist. Is there any install here? I don't know what all this business is, but different dist, uninstall. Just want some easy way to write a simple test, please. Fly rust up. That seems promising. Target. See, this seems more promising. This does like rust up commands. That's what I want. Remove target list target link. Do we see anything here that looks like installing something? That seems promising. Install lightly. Show tool chain. Show tool chain. Okay, this certainly seems more like the type of the type of test we want to write. I just want to find something that actually looks like something that looks for like components, right? That's really what we want. This is a handy way to I mean, this file is particularly large. But when you're submitting a PR to a project and you don't really know how to write a test, look for a test that does sort of what you want and then just copy paste it and then modify from there. Well, that's unhelpful. Let's look for like nightly. What else do we have here? These like test slash client looks pretty promising. Miss custom tool chains, search path, run install. That seems kind of promising, right? Like this has an install and then checks for installing component. That sounds like kind of what I want. Yeah, like these tests seem pretty promising. Not going to lie. Install profile. That's what we saw earlier. So really, we've returned to the file that we started at. Okay, well, how about we just try one of these? Install profile. And I feel like I want the setup from something like this one, because this seems like a nice test setup. Let's just try that. See what happens. And then let's just get rid of this business down here. All right. I don't know what this does yet. That's fine. default default stable seems fine. And then I want to run install. And I want to run install profile minimal. See, what did the install with profile it used, it uses setup simple v2. Okay. And that does not end up with cargo. Okay, so simple v2. Install that. And, oh, I guess we need to do like nightly maybe. See, this seems to do stable. But I guess, just see what happens if I do like this. All right, so we're going to do a setup. I don't maybe we don't need this. We're going to do a simple v2, which who knows what that is. And we're going to set the default to be stable. I don't know if we even need that. But sure, it's a default stable. And then we're going to try to install nightly with a minimal. And we're going to see what that does. So we're going to do cargo tests. And we're going to run install profile. And see what happens. That's an awful lot of errors. This is all the this is all the changes we made to temp config that apparently a bunch of things were relying on being able to use standard sync arc. There are other things that apparently dist install. And I guess we here to need to use standard sync. I just want to run my test. I don't care about all these other tests. And yes, I know this. There's a flag I can add to test to make it only compile my thing. But test this great, almost there. Now run my test, please. That seems promising. Running one test. All right, let's look at Ooh, okay. So it's running default stable that ran correctly and told us that we had stable installed. Oh, I see when we said default stable it installed stable. And it seemed to include like cargo rust docs rusted. Great. Then we install nightly with profile minimal. And then it downloaded rust C. Well, okay, that's sort of good because that means that it did pick up the profile. Not install. I think I know what this is. This is going to be me cheating a little probably. I know what this error is because I've seen it before. I think I'm pretty sure I need to pass this flag. And we're just going to pretend that I spend a lot of time searching to find that and then see if it works. Okay, this is getting annoying. Climbisk. I did mean climisk. All right. So install profile minimal nightly no self update that downloaded just rust C, which is what we want. Rust up version that worked rust C version that worked cargo version worked, which is not what we want. Oh, it's because we haven't said default. So that these are still run under the default toolchain because we haven't set the override. So what do these other ones do? I say we certainly just set the default to nightly after doing this. Otherwise, the following commands won't be run that way. Let's see what that does. All that passed. Right. So okay, so let's look at what this test does. I don't actually know what simple v2 does, but it's the same. I'm guessing this like sets up a manifest of some kind. And it sets up a manifest that we know from the other test we looked at, right? So the other profile test, this one that already existed, use the same setup. And when it installed with profile minimal, then it got rust up and rust C, but not cargo. And so what we're doing up here in ours is that we start with a default of stable, which has all the stuff. And then we say install nightly with a minimal profile. This no self update, I can tell you is normally rust up tries to check for self updates at the end of installs because it treats them as updates. And this is just telling it not to do that. The reason is because it's not actually installed, it's running in like testing harness. And so the self update would fail. So we start with the default stable, we install nightly with a minimal profile. So at that point, nightly is installed, but it will only have rust C. And then we switched tonightly by setting it to the default. And now cargo won't be there because for whatever reason, this manifest is set up such that cargo is not installed when you install the minimal profile. Even though I feel like the minimal profile usually includes cargo. So rust doc might have been a better example. In fact, I kind of want to see simple v2. What's in simple v2? And mock high tools, simple v2. Okay, so when it's simple v2, you have a nightly and some stuff. Okay, so what's in these releases? What is a release release build mock channel? All right. Build mock channel, build mock channel. All right, so where's the profile stuff? Okay, maybe it's related to this is extension business. No, it doesn't seem right. Well, there's no mention of profile in this file. All right, well, then I don't know. But it does the right thing. I wonder why. I wonder how it determines this profile business. I mean, this works, right? We've tested it and it does what we expect it to do. It's more that given that in reality cargo is included even in minimal installs, it seems weird to be testing that way, but it seems fine to me. It's painfully obvious to me now that I need to do a better job of structuring the install flow. Structuring the install flow and what? So this is one of the authors of Rustup. I don't know what you mean by structuring the install flow. I think it's more about making it clear what the testing flow is. But the install flow, it's true. If you want someone to edit the install flow, there was a pretty long chain, we had to go down like a pretty deep rabbit hole to figure out how anything ever gets called. Maybe one thing that would be useful is some higher level docs that explained how these pieces fit together, because there's a lot of just glue that really points everything to one place. Although maybe that is documented somewhere and I just skipped the documentation. Okay, so let's first add these disk businesses, because they are all, and then I guess this change install profile did not intend for that to be the case. Okay, we did a stupid, so I'm gonna, do you see what we did? I changed the existing test call install profile, but I copied the test call install profile to install with profile, which is the opposite of what I wanted to do. Install profile, install with profile. This should be down here with the other one. And this is the old one, which is install profile. And this is install with profile. There we go. That's more what I wanted. And I want this to be the way that it was. Don't want to make any changes to the existing test if I can avoid it. Great. And now let's see that this actually still works with profile. I meant more the vast sequence of function. Yeah, it was a pretty deep like rabbit hole to dive into. Great. Okay, so the test will pass this because we didn't change it. But we moved around. Add test for install install with dash profile. Now, what I actually want to do here is I want to be a good citizen and do this. Oops, that's not what I wanted to do actually. Squash these and then I want to say previously the installation profile, the profile to use for installation to use when installing a new tool chain had to be set manually, had to be set through Rustup profile set set profile. I forget set profile or at Rustup install time. However, often in CI contexts and otherwise, I don't know how to phrase this. Previously, the profile to use when installing a new tool chain had to be set through Rustup set profile or Rustup install had to be set globally through Rustup profile or Rustup install time. However, in CI contexts, or in some contexts, however, sometimes this is inconvenient because you only want to use a particular profile for a specific tool chain. This patch allows users to pass dash profile to Rustup install to install a tool chain to override the profile used when installing profile used for that one install. One question that this raises actually is if I update a tool chain that I installed with an overridden profile, does it now suddenly install the other components? That's a good question. Let's look at that test. Sorry, what did I mean? Rustup init time, sure. I just realized that what happens if I now do Rustup update nightly? This should do that. I don't know that it will. I think this might break. That worked. So here's my thinking. Actually, no, I'm almost positive that this will not work. So here's my thinking. If you install it with profile minimal, there's nothing recording that the profile you installed it with is minimal. And so if you later update it, how would it know to not install the other components? Maybe we'll check that they aren't already installed. I don't actually know. The reason this test passes is because it recognizes that there is no newer nightly, so we won't update. So the way we can get a simple v2. Okay, so there's a different date that's available here. So is there like a date? Wait, I can set. Yeah, here we go. Okay, so I'm going to set this here, and then I'm going to do this and see whether that fails. No such file or directory. Wait, what? Rust backtrace equals one. Rust.backtrace equals one. Why? Where? How? Change channel date. Apparently, I'm not allowed to do whatever I did there. Apparently, you can't set the date as we experienced at the start, but other people do it. So why can't I do it? Maybe I can only set it here. Oh, so these are mocked. These are not real manifests. They're mocked manifests. So the 2015 is you can ignore. It's not related to real dates. I'm going to make this expect error just to see what the output is. It's fascinating. All right, show me that output. Yeah, so it didn't actually do an update there. So why don't understand why this won't work? All right, bye, Dutch. You're here for like every stream. It's amazing. No such file or directory. At mock dist 48. That seems problematic. Why am I not allowed to do this when other things that use v2 are allowed to? Or maybe they aren't. It's because other people aren't doing that. Interesting. So I can't actually make it update. Because this, for whatever reason, won't work. I wonder why. What's in dist? Where is it getting these manifest from? Is it like generating them? That's very odd. I don't understand why there are other tests that are able to call this current dist date. But I can't update unavailable Rust C. Like what happens if I run that? Update unavailable Rust C. Yeah, that runs just fine. So why? Oh, why? Oh, why? Does mine not? This sounds like simple v2 just like doesn't do that. That's fine. I'll just do one called, I don't know, full. Full seems great. Let's do full. I wonder why. It seems odd. Huh. Well, now that works. And apparently the update, I'm going to assume that it now actually does the update. And apparently that update doesn't install any other components. Don't quite understand why. That sounds like if you run Rust update, it won't install components you didn't previously have installed. Which actually, I guess, maybe makes a lot of sense. I mean, we can leave it in as a test anyway to check for regressions in this behavior. Yeah, so it updates and then it only installed the component we had. Okay, great. So that was the only concern I had. But at least now we have a test for it. So this is going to be installing with minimal profile should only install Rust C. After an update, we should still only have the profile dictated components. All right. So I'm going to just fold that into the previous update. And we're also going to say that this fixes 2004. 2004 was broken. All right. It's going to get light again, people. I'll pull request. 2004 in 2019 with tests that talk about 2015. All right. Pull request. Excellent. So we now have another PR that you can watch. This one. Oh, I forgot to push big this PR. This was the other PR that we filed that you may want the URL to. There we go. All right. Well, we now have three PRs to three different Rust projects. I think we're probably going to stop there because I don't have an idea for another contribution and it's also getting a little bit late. So let's just recap what we did. So first we made a contribution to cargo where we changed its handling of what it does when the target directory has Simlinks in it. Then we ported a crate from future 0.1 to standard future. And then we made a change to Rustup so that you can choose which profile to install a given tool chain with at install time to override the globally set profile. It was a pretty decent amount of stuff we got done and like what four and a half hours. It's pretty good. I think we did well. Well done us. So at this point, I recommend you watch all of the PRs that we just submitted because very often there's follow-up work either because the authors or the maintainers of the crate request added tests request changes to behavior. And sometimes they might even say we don't want to fix this way. Like for the cargo change we made that change might be more controversial because as someone mentioned in chat as well, we will end up recursing outside of the target directory if there are Simlinks outside. And so these might be worth watching if you want to continue to see the process that these PRs go through. If anything, my goal with this particular stream was to just show you that supporting a project or contributing to a project that you haven't contributed to before is okay. Remember that here we just sort of dove in. We did not do any mentored issues. We just dove into the code. If you look for issues that are marked as easy, things that are marked as Hacktoberfest also work pretty well, things that are marked as mentored, then you can get more help getting started if you feel like you want that. But it isn't that daunting. Like just start. And initially a lot of what you do is just going to be copy pasting and that's fine. And then submit a PR and then people will talk to you and ask for changes. And that's sort of how the whole system works. And remember that even the Rust tools are just code, right? And you can contribute to the ecosystem a lot by improving the tools that lots of people use. Like these fixes are going to matter to people. Like especially this one to Rust up, my guess is we will see this used in like CI scripts for lots of Rust projects. And that's pretty cool. Whenever you help improve the Rust tools, you really improve the entire ecosystem. Will you tweet about the follow up if applicable? Yeah, I'll do that as well. So if as the PRs get updates, like whether basically when they get merged or closed, I'll make sure to tweet out as well to mention that this happened in stream and to point it out to you. So if you want to follow updates for upcoming streams, this is me, twitter.com slash john who. And I tweet. So I mean, I'm a normal Twitter user. I retweet things and there's stuff there that's not Rust related. But I try to make sure that whenever I have thoughts for an upcoming stream and I want to hear feedback about it, or just if you want to talk about stuff, or if just announcing upcoming streams, or announcing like interesting changes in the Rust ecosystem or to the Rust language, then I'll post it there. So feel free to follow me there if you want. There's also, so I also am part of a, actually I should mention that here. There's a Rust podcast called the Rustation Station that Ben Striegel and I have started and it's sort of a community podcast. So the idea is that if you want to make a podcast episode about Rust, even if it's just a one off, then get in touch with us and we will help you make that episode. And then we do regular episodes on what's new in each new Rust release. So we recently talked about 138. I recommend you give this a listen. We also have guests hosts almost like almost every other week where they just talk about things they think are interesting and interview people they think are interesting. So if you want to be a part of that then, or just listen to them, then I invite you to do so. And of course, if you enjoyed what I do, then please take a look at the Amazon wish list. It's like the only way I have to get anything back from this apart from just your, you being happy about being educated, which is great. That's why I do this. But if you have some extra support, I would love that. And with that, I think we're going to sign off for today. Thanks for joining me. It's always fun to program with other people. And I think this turned out pretty well. I will see you in a few weeks time. So long for well, I'll feed us and goodbye.