 Hi, folks, welcome back to another stream. This time, I kind of wanted to call this a crust of rust. I'm not sure whether it is, the boundaries a little ill-defined. The basic idea here, oh, what do I do? The basic idea here was to do a stream where we take an arbitrary crate that happens to be the one that we implemented in the previous stream, but that shouldn't be too relevant to what we're gonna do this time. And look at, okay, you've written the code. Like you have a thing that works. What do you do next? Like this is the sort of polish part of a crate, but it's not just like surface polish. It's things like setting up CI. What kind of steps do you want in your CI? Setting up prop tests maybe. What do you put in your readme? How should you structure your docs? What kind of options do you have for your docs? How do you get coverage testing? Like that sort of stuff. So in some sense, maybe you could call it hygiene, maybe polish, maybe some combination. And I'm aiming for somewhere around two hours here. So sort of cross the rust length. We'll see what we get to. I don't really have an endpoint necessarily in mind as much as the goal is to just go through things that I do when I have a crate, like this one where I have one commit. I have a readme that just is a link to a stream. And then I have the source for the crate. I don't know why this loads slowly. That's gonna be pretty frustrating. Also, I don't know why I'm not signed into GitHub. That's also gonna be frustrating. Let me go ahead and fix that right now. Oh, now I need my key. There we go. All right, let's try that again. Source. Fantastic. Yeah, so we have a bunch of code here. We also have like a little test suite that we wrote. It's not super advanced, but we have one. And the question is, what do we do next? And as we go through here, if I do anything that doesn't really make sense, then just call me out on it and I'll try to dig into more. I've done this a few times, so it might be that I rushed through something. And yeah, this is gonna be a shorter video in the sense of two hours. It might be shorter, actually. Well, we'll see how far we get. But things like if we really dive down the rabbit hole of PropTest, then it's gonna be longer. There's an argument here for maybe there should be a separate stream on PropTesting, but I think it's a pattern more so than teaching you how the PropTest crate works. And so I'm hoping that it's not really suitable for just its own stream. Great. I'll get into what PropTest is in a second. Okay, so let's start with CI. So I have here a checkout of, oh, I did roadstop update tonight. So I have a checkout of the crate here. The only diff I have is the patch that we made to cargo. This one actually now can be a git patch instead because it's landed upstream, but that's not terribly important. Oh, actually it means it won't build. All right, fine. We'll fix this up here real quick and say this is gonna be, um, nope. Just lying cargo. The reason I want this to not be a path anymore is because otherwise CI wouldn't be able to run. And then I guess we can say, we can sort of link to the PR that landed oh, public. Uh, closed. I just wanna find the PR from last time. This one, new project kind. So that's the one that landed during our previous stream. So what I'll do is I'll say it's this one. Remove once cargo for Rust 169 is released. 20, 23, oh, four, 20, which is when that release lands. And then we can just do a cargo check to see that that's actually true. Why do I have a config as well? Oh, I have two cargo config files. That's silly. Let's have just one of those. Okay, so that builds just fine. Interesting. Skipping duplicate package. Interesting. There's a lot of warnings. Someone pointed out that for the cargo Toml, why is cargo 068? So the versioning for the cargo crate is a little weird. It is the current Rust version. The current Rust version, but zero point instead of one point. Zero point instead of one point and plus one. So for Rust 1.x, the cargo version is 0.x plus one. I don't know why. I guess we could keep this in here just for our own reference. And I guess in fact, what we really should do then also is something like turn this into 70 because otherwise the patch won't get picked up. The alternative here actually is to do which commit to this land in. This landed in this commit. So if we want to be real specific here, which I think maybe we do, we say rev equals this one. And then this is gonna be 070 because it lands in Rust 69. Cargo T. This skipping duplicate package is a little weird to me. But this feels like different from what... This feels like a problem when you take a get dependency on cargo. Interesting. Let's see if this builds. So this is the first step to getting CI to run, right? Just to make sure that your package builds somewhere that's not on your own laptop. And in this case, it wouldn't in the past, right? Because I had a patch path override. And so this is just fixing that up. And hopefully this should now build without issue. Yep. And all the warnings are about this skipping duplicate package bit, which we can ignore. I think this is because the cargo project includes a bunch of test crates and those seemingly confused cargo itself here, which is weird. I would like this to not be the case, but anyway. Okay, so get commit. Avoid path or cargo patch now available upstream. On get... You can push that. Okay, so what do we do next? How do we get this to actually run any kind of CR? Well, I have a collection of CI scripts that I normally use that I have just in a separate GitHub repo. It's not something I've documented very well, but it is just like, this is where I keep all of my GitHub CI config. And if GitHub decides not to be annoying today, great. So when you configure GitHub CI, you put all your config in a .github folder. You see here I have configurations for dependabot, I have configuration for codecub, which is a code coverage. And workflows, which is all of the CI steps. You see here, I have different sort of workflows for different kinds of things that I may or may not want for a given crate. So Nostud, for example, is not going to be relevant to this particular crate. Safety is probably not going to be relevant for this particular crate. This is for things like if you want to run Miri and AddressSanitizer and ThreadSanitizer. So we won't dig into these two because we're not going to use them, but check, scheduled and test YAML, we're all going to look through. And the reason I have this in a separate repo is because I can do this. Get remote, add CI, get fetch CI. So this is basically adding another Git remote to the current Git repository, fetching that. And then I do git merge, allow unrelated CI-main. So what this is going to do is it's going to merge the history, the Git history of that other repo with the history of this repo. And normally Git disallows you from merging completely unrelated Git histories, but there's nothing that prevents you from doing so in the sort of Git data model. So by doing this, now you see it created all of these .github files. And the reason this is nice is because in the future, I can run git fetch CI. If I make changes to that other repo, I can do git fetch CI and then I can just merge from here again. And then I get normal, I get merge conflict if necessary. It'll just handle any local changes for me. And so this makes it really easy to keep the CI up to date. I could have some automation to check whether I need to do a merge from CI and stuff. That is actually an interesting CI step I could add to this, but we're not gonna ignore that from now. And if we look at these files, let's start with dependabot. So dependabot is GitHub's tool for scanning your cargo.toml and your cargo.lock to look for, essentially look for outdated dependencies or dependencies with known issues and automatically file PRs to you when anything has changed. And the default configuration for dependabot I don't always love, but this one I've found works pretty well. So the first here is you see you can opt into specific types of updates. In my case, I'm opting into update notifications for GitHub actions. You'll see this when we look at the actual scripts, but I'm using actions like there's a cargo action, for example. And those are all versioned. And so if my CI is using a version of an action that is itself outdated, then this will file a PR to update that action for me. And then I also have one for the cargo ecosystem. I set it to run daily. It runs at the root of the crate. And I wanted to ignore all dependencies if the update is a patch or minor update. And the reasoning behind this is I don't actually, because this is a library, I don't actually care about updating my lock file, updating the minor version of my dependencies because whatever I have in my lock file is not going to affect my consumers anyway. They build with their own lock file that's independent of mine. All that matters is what's in my cargo Toml. But if there's a major release, then I want to know because then I as a library author have to take action and update my crates cargo Toml to move to the new major version. And then, you know, as it says here, if you have a binary then it does matter because your build is going to include exactly that set of dependencies as you don't want to ignore patch and minor versions. Although then daily might be too much for you. So that's pretty neat. What else do we have? So CodeCov, we'll talk about coverage a little bit when we look at it later, but this is really just configuring code coverage reporting so that it's a little bit less aggressive about false negatives so that we have a higher standard rather than saying like zero to 100, we actually care that it's decently high, ignoring the test directory because I don't want coverage for tests. And then making the CodeCov comments that the CodeCov makes on PRs and stuff less verbose. And then we can look at the actual real jobs. So if we go through here and look at, let's look at check first. So you'll see here, there's a configuration at the top. This is all like GitHub CI actions, CI configuration. So there's a bunch of stuff you can put in these files but let's just walk through what's actually there. This is when does this workflow run? And I've said it should run on push to the main branch. The reason I say only the main branch here is because if I make a PR to my own repository, the way I do that is I create a new branch and then I do a PR from that branch into main. But without this rule, when I push to my branch, it'll run CI on that branch and then it'll also run CI as a result of opening the pull request. So you're doubling the amount of CI seems unnecessary. Since that I say when pushing to main or when creating a pull request is when I want this. The name of this workflow is check and the jobs to check runs are format and clippy and doc. Oh, and hack, forgot about hack and MSRV, right? I keep forgetting that I keep adding to this. So format just runs cargo format check on the stable compiler. You'll see here that it checks out the code. It installs Rust stable with the Rust format component. Then it runs cargo format check and it does this through the actions RS cargo action which configures cargo and sets up some caches and stuff. And I believe also ensures that the output of this goes back into the GitHub CI checks stuff. So this is like it will actually show up in line as warnings in the file diff view of a PR. So it's a little bit easier to spot later on. The other check I run is clippy and clippy I run both stable and beta. The reason I run formatting only on stable is because, I don't know, it seems more reasonable. It's the thing that people are gonna have installed. I don't really wanna hold them to the bar of beta. For clippy I run stable for the same reason and I run beta because partially to help test new versions of clippy and partially because clippy tends to have like a lot of new lints that come every beta. And I like to get a little bit of a heads up if something is going to start failing in the future. I've also said fail fast here because usually if something fails in one it's gonna fail in the other. Steps here are the same. You know, it's a checkout install Rust with clippy run clippy and right, it's given a GitHub token here so that it can echo back to the GitHub check API. So this suggests that I'm actually not doing this for formatting, which I think is fine. The formatting is usually obvious. If it fails, you just need to run cargo format. Doc is run separately and is run on nightly. The reason I run doc is because there are a bunch of checks when you run cargo doc that don't run when you run any other command. So if you run let's say cargo check dash dash all targets it doesn't actually do things like check that your intra doc links are correct and not dangling. To get that you actually need to run cargo doc. At least I used to be the case. I think it probably still is. And I run it on nightly because there are some really useful nightly documentation features that are commonly used on something like docs.rs for example, where people will docs.rs builds your documentation on nightly. But it also means that you can use some nice features like config doc or doc config, which means that you can mark a particular function or type or whatever in the render documentation as only being available if a certain configuration is true. Like for example, only if you're running on Linux or only if you're in x86. And so in order to be able to make use of those kinds of features I run doc on nightly and I run it without any depths because I don't wanna spend time documenting dependencies when all I'm really doing is checking that the docs work. And then cargo hack is this really neat tool for checking combinations of feature flags. So in cargo there's a requirement that your features are all additive. So that means your crate should compile with any combination of your features. The reason this requirement exists is because it's on the called feature unionization. So essentially if you have some crate x, you have two dependencies A and B. A enables one feature of x, B enables another feature of x. What cargo will do is only build x twice, sorry, only build x once with both of the features enabled. It will not build it twice once with each feature. And as a result, depending on what the dependency graph is, you might end up with any combination of feature flags. And so cargo just requires that your features are additive. And what hack does is it allows you to check a particular combination of features or a set of combinations of features. So in this case, I've told cargo hack to use the power set of the features. That means every possible combination. I wanted to run check and I wanted to check Libs and tests and that works really well. Sometimes you have to tune this if you have benchmarks or nightly only features and stuff and hack has ways to specify this. If you checked out Detonate Rust Toolchain, it doesn't really matter which one you use. I don't think it makes that much of a difference. And then the final step that I have is MSRV. So MSRV stands for Minimum Supported Rust Version. And this is essentially to figure out whether the crate continues to build with the version of Rust that we claim, let's say in the readme that it builds with. So in this case, it just defaults to 156.1 because that is the version that released the 2021 edition, which is what all new crates default to when you run cargo new. So this just seemed like a good default to have in the sort of CI template here. And it really just does the same as the other steps here, which is it checks out, installs, and then runs cargo check against MSRV. It doesn't run the full test suite, arguably it could, but this was more like, you really just wanna make sure that you still compile against that version. The test suite is somewhat less important when it comes to MSRV. So those are all the check steps that I have in there. And then for safety, we're gonna remove this one because we don't actually have any unsafe code in this crate. So this is running Miri, running the leak sanitizer, the thread sanitizer, the address sanitizer, and all of that good business. It also runs Loom, which again, doesn't matter for our particular crate. So I'm just gonna remove safety. Remove safety. Nostad is similarly, it makes sure that your crate still builds against targets that have no standard library and no allocator, or targets that have an allocator but not a standard library. Also is not relevant to us here, so we can remove it. The next thing is, let's look at scheduled. So the scheduled CI job runs, in this case, I think it runs every day. But scheduled jobs in GitHub are a little weird because they only run as long as the project is still active. But sort of conceptually, it is more correct to think of these as rolling jobs because, and the rationale here is, they might break independently of any given PR. And that's the reason you wanna run them on an ongoing basis. So the first example of this is building on Knightly. New Knightly releases come out every so often, specifically Knightly. And when they come out, they might break your crate. There might be a regression in Knightly. And so as a result, you really want to run a build on Knightly, Knightly, or at least fairly regularly. And that's indeed what this does. It checks out, installs Knightly. And then this thing is a little bit interesting. I'll get back to what this does when we talk about testing, about why I have this bit here, but essentially it runs cargo test and it runs it with dash, dash, lock. So it runs it with the lock file that you have checked in. And this little bit is just, if you don't have a lock file checked in, it generates one for you. And this is to ensure that we are actually testing Knightly and not updates to your dependencies. And then this one is updates. So this is sort of the inverse. This is saying we're still, we're gonna build on beta. And we're going to run cargo update and then see the test still succeeds, right? So the idea here being, we want to make sure that if you run with a newer version of the dependencies, then your test suite still passes. The reason this one is important is because you're consuming the crates that consume you, the crates that depend on you, they are gonna have an independent lock file from yours. So they might use newer versions of dependencies than the ones that you previously tested with. And so you want to regularly make sure that they don't get broken as a result of your dependencies getting updated. And that's what this will do. You see here also, I've specifically set this job to fail if there are any new deprecated items in my dependencies. Okay, so those are the scheduled jobs. And then finally it's the running the actual test suite. Again, you see the on here is the same for all of these. So this one runs your test suite on stable and beta. Beta here is entirely just to help test out new versions of the compiler to catch regressions and such. It has the same little bit here. And now we can actually look at this in a little bit more detail. This is GitHub CI kind of magic to say, if this file does not exist, then run cargo generate lock file. The reason I have this bit in here is because I want to be able to merge in this template. Regardless of whether the current project, the project you're merging this into already has a lock file checked in or not. And this way it'll work either way. But I do think there's a strong argument for checking in your lock file. And then this command will not run. And instead you're just gonna run cargo test dash as locked, which will test with your checked in with your locked dependencies. The reason I think we should check in lock files is because it allows people to do things like bisect your crate. They can check out an old version of your crate and it should still be able to build by virtue of using the exact dependencies that were present at the time. But this way it's not required as a part of the CI template. And then minimal is only runs on stable, but it happens to also install lightly because minimal versions is an unstable flag. And what it does is it runs cargo update with the dash Z minimal versions flag. This is an unstable flag that causes cargo updates behavior to be sort of inverted. So rather than choosing the newest version that's available for your dependencies, it chooses the oldest version that's still subject to the dependency requirements that you set in your cargo tumble. And the idea here is sort of the same as for why you wanna test with cargo update, which is you don't have control over what versions of your dependencies your consumers use. The only thing you know is that cargo when running your consumers is gonna pick some version that's compatible with what you put in your cargo tumble. That's the only thing you know. And cargo update is gonna be a way to test that if everything is the newest that still works, cargo.lock or the stuff that's in cargo lock is gonna check that whatever's in your lock file also still builds. And then this is gonna be if some consumer only got those dependencies through you and hasn't updated in ages, they'll still be able to build. This is basically to check that the specifications you have in your cargo.automobile file are accurate. So for example, let's look at our cargo tumble here. Take temp file. We could specify temp file as three, right? Because it major version three, that's the API we built against. But imagine that in our crate, one of the things that we did is make use of, let's say the, I don't know, the foo function in temp file was added in version three dot two dot one. Well, if we give three, that means that one of our dependencies might choose to use temp file three dot zero dot zero because that's compatible with the specifier. But if they did, then our crate wouldn't build because we actually require three dot two dot one. So minimal versions is a way to try to tease out these kinds of problems to ensure that your consumers don't end up in weird states. And so this is why usually I recommend that if you take a dependency, you add the full version number and not just the current major version to this precisely to ensure that you don't, or it's the real thing you should do, right? Is look back at the version list and find exactly the minimal version that you need in practice, no one can be bothered to do that, but it's better than to pick the version that is the current one, because you know that has everything you're going to use rather than trying to pick some generic minimal because it's not clear that that'll actually work. But minimal versions then is a way for you to ensure that the minimal requirements you've set in your cargo tumble are actually appropriate. There is some work, there's some annoying parts about minimal versions because it chooses minimal versions transitively, which means if one of your dependencies has an incorrectly specified minimum versions, like imagine back here, imagine temp file takes three dot three dot zero, but then it depends on, you know, foo equals three, but uses things from foo three dot two dot one. Then when we run our resolve with minimal versions, then the foo dependency, which we get transitively through temp file, we're going to choose three dot zero dot zero, that's what minimal versions tells us to, but temp file actually needs foo three dot two dot one. And so they have a bug in their cargo dot tumble, but as a result, we suffer. And so that's why minimal versions is a little bit of a pain sometimes. Like I definitely have some crates where I end up doing things like to make minimal versions happy. And then down here, I said like foo equals three dot two dot one via temp file. And so this is one way to basically force that minimal requirement because temp file hasn't specified it themselves. And it's really sad to have these, it changes the edges in your dependency graphs or cargo builds can't be quite as efficient. So it's really unfortunate that these are here, but usually it's not too bad to work around. But as a result, there's another proposal which is dash Z direct to minimal versions, which chooses the minimal versions of your direct dependencies, but the highest version of all transitive dependencies. So it's sort of a combination of these two. And I think this is a good thing to test too, but it's not, I don't think it's gonna be available on nightly until the nightly for 170 comes out or rather on beta until that comes out. On nightly it'll be available, I think, pretty soon because I think it's landed. So that might be an interesting thing to try, but I haven't played around with it yet. The next test that we run is OS check. So this is just to make sure that the crate test suite also runs on macOS and Windows using the stable compiler, nothing special here, just locked all features, all targets. And then we run coverage testing. There are a bunch of different ways to do coverage testing in Rust these days. I tend to like cargo LVM Cove. It uses the new, I guess it's not that new anymore, but the instrumented coverage. So rather than using a sampling coverage checker, like what we used to have before with something like KCov, this instruments the code when you build it and then runs the instrumented code, which logs all the lines that are executed and then you get much more accurate reporting because it's not sampling. And cargo LVM Cove is a tool that makes it easier to build your code with instrumentation for coverage and then running and collecting that coverage. So this again checks out, installs this helper binary and then runs cargo LVM Cove, which essentially just runs your test suite with the other arguments. And then it outputs it in a way that CodeCov.io understands. And my choice here of CodeCov.io is sort of arbitrary. It's one that I know I picked up years ago when I was looking initially at code coverage and it's worked fine for me ever since. So, and it's free to use for open source projects. So it's just been very easy. Okay, so that means we now have, we deleted no sudden safety and the rest of this in theory should just work. So if we do add CI bits, actually here's what I'm gonna do. I'm gonna create a new branch called CI. I'm gonna go back to main and I'm gonna reset this to Oregon origin slash main. And then I'm gonna check CI and then I'm gonna push CI to origin. This is just because I want to do a PR of this to see that the PR machinery works as well. So if we now go back here and open a PR from this one, add CI and then I'll put the link there, create pull request. And just for those in chat here, I'll put that there in case anyone's curious. So hopefully now you see this now brings in, because we did a merge with this other repo, it brings in all of the commits from that other repo as well, but that's fine. This is a one-off thing. And now you see it kicked off all these jobs and I'll be interesting to see whether these actually pass. Hopefully they should all pass, but we'll find out. I think the one that's most likely to, the two that are most likely to fail are minimal versions and minimum supported rust version because neither of those we've checked. Also, who knows about Mac OS and Windows? It's always a little bit of a coin toss, but I don't think we're doing anything particularly OS dependent. And Clippy, I don't know if we ran Clippy on this actually back in the day. Better to preserve history or do a squash merge. I preserve history for this merge. And the reason for this is because in the future, I'm gonna want to merge from Rust CI Conf again. And if I do a squash, then I'm gonna have to deal with a lot more merge conflicts because Git doesn't know about the connection, the existing connections and merge history with that other repo. So this I specifically want to be a merge. Let's see. Nightly dock works fine. Cargo update works fine. I forget whether we checked in our cargo lock. That's another good question here. We did not. So that's the other thing that I'll wanna do here is Git add cargo lock. Then dot git ignore. I don't wanna ignore cargo lock. And I wanna add my cargo lock. Check in cargo lock. So that's the next thing I'm gonna wanna do. Something else failed here. Okay, so minimum supported Rust version failed. And that's because, oh, that's interesting. cratesindex019, huh. Is there something weird about cratesindex019 that it doesn't work on 156? Fascinating. cratesindex, if we go to the repo for this cargo Tomo. Cargo Tomo. Ah, so it has a Rust version 060. But this error is very unhelpful. So what we'll do then is we'll go to our GitHub workflows, check. Oops, I mean test. No, I mean check MSRV. And this is gonna be one dot, I guess really what I wanna do here is something like show me 0190. Yeah, so it too has Rust version 160. So what we're gonna do here is 160.0 for cratesindex. Bump MSRV for cratesindex. See what else might fail. I like to just deal with, I'm committing each of these, but I like to do all of them before pushing so that we avoid having to wait for CI each time. Minimal versions, it's not terribly surprising. No version new, found for version. So this suggests that open SSL sys has an incorrect specification of its dependency on Rust C version. So if we go to can type open SSL sys, the reason I go via docs.rs is just a really easy way to get to the GitHub repo. So let's see if we find open SSL sys, it's cargoTOML. It has a dependency on build main on Rust C underscore version. Is that still true? That's the other question. If we look at build main, you know what I think? Yeah, I think that open SSL sys has moved away from using that dependency at all. Now it uses autoconfig, but we're using an older version of open SSL sys as well. So this suggests that this was a bug in an old version of open SSL sys. Open SSL is particularly annoying when it comes to this stuff. So what we can do here is cargoTree-i open SSL sys and look at what we find. Okay, so we get in open SSL sys through a bunch of different things. We get it in through curl. We get it in through curl sys and we get it in through open SSL. So there are a couple of ways that we can try to solve this problem. My first instinct here is actually just a bump for an SC minimal versions. Again, this is where it's annoying to use minimal versions is because this isn't the problem in our specification. It's a transitive problem of minimal versions. But what we can do is just pin a much newer version of open SSL. So currently you see we're using open SSL version 01045. If we go to this, nope, that's not what I wanted. Open SSL is currently, it is currently 1045. So we're using the newest version of open SSL, interestingly enough. Oh right, I haven't never mined, that's false. I need to do cargo plus nightly update that C minimal versions. And now we can do cargoTree open SSL sys, see what we get. Okay, so we're bringing an open SSL version 0100. So that's pretty old. So let's try to do a sort of binary search here. So if I do 20 and then I do cargo update and then I run cargo check, there's a build. That builds, okay, so what if I do 10? Binary search, cargo check, that builds. That's interesting. Oh, it's cause I'm not using all targets, I think. Yeah, so this breaks for a number of reasons, but let's do sort of 45 here. Update minimal versions, cargo check all targets. Okay, so we fail somewhere else too, which is Rand Isaac. So why are we getting in cargoTree Rand Isaac? Isaac brings in through, brought in through Rand 0.6.0. So let's see, what's the newest version of Rand? Well, 085, okay, so there's a much newer version of Rand. So where did we get that from? We got that in through crossbeam channel, which comes in through cargo and parking lot. All right, so let's just see what versions of 0.6 there are. Oh, these are all yanked, 065, okay, 065, check all targets. That still fails, so then let's look at Rand Isaac. And again, this is why people can't be bothered to use minimal versions because it's actually pretty annoying. 011, Surrand Isaac, 011, oops, update minimal versions. Great, okay, so then we might not need that specification for Rand here. The nice thing is once you've done this once, then it sort of fixes itself. Okay, Libgit 2 can't find range version. So the range version here is a method on, it's a method on package config, pro version, range version. Okay, package config, oops, so we're bringing in package config version 0311, and so this was clearly added in some later version than 0311, so let's do Cargo Toml, didn't need that, but package config is 0.3.26, why not? Oops, again, let's see, cargo check. I think maybe we did need Rand 06-4 here, but now it's elliptic curve. Why are we even getting that in? HKDF through P384 through Poceto through Cargo. Oh man, all right, what do we have here? Docs, HKDF, this is actually one of the worst I've seen. It's because Cargo brings in a lot of dependencies too. This is 012-3 for this one, HKDF is 0.12-3. This is another thing where someone should write a tool for this, because it's very mechanical. Yeah, this is insanely annoying, I agree. Great, oh, you're right, it was 065. And I think demonstrating how annoying this is is useful. Okay, so P384 has a problem too, P384, wait, what? Oh, 011, so what's a newer version of 011? 011.2, now note that the part of the problem here is it seems like some of these have actually been broken by newer versions of Rust maybe, either that or their dependency specifiers are again just straight up wrong. Okay, RAND AC is 0.1. That one doesn't have a newer version, okay. So RAND AC gets brought in through RAND 065. Was there a newer version of RAND on the 06 branch? There's not, okay. And I think it's gonna be, oh, so the other thing we can do here is actually just bump, do we take a dependency on RAND specifically? Yeah, just that one. So we're getting RAND through parking lot, through Crossbeam channel actually. So what we can do is instead of trying to do this, get RAND to a newer major version, which we get by getting a newer version of Crossbeam channel. So Crossbeam channel. So Crossbeam channel is 0.5 now, but let's say 0.39 instead of 0.36. So we do Crossbeam channel, 0.36. Nope, 0.39. Cross, Crossbeam. Let's see. So that got rid of the old version of parking lot, which is what brought in the old version of RAND. And then we also make sure that we don't accidentally bring it in again. So now you see the old version of RAND is just completely gone, which is what we wanted. Now let's see if it builds. This looks pretty promising. Never crossed the beans. Okay, so that builds, so we can now get rid of this. Let's see if we can also get rid of that, for example. Nope, that's still needed. Then we can do a bit of a binary search here if we actually wanna be a little bit nicer about this, because 0.11 was the one that failed. 0.26 works, so we can just sort of dig our way through here. Okay, that seemed to work. So 11 worked, so let's do 16. Let's see. Great, so that works. I don't wanna spend too much time fiddling with these. Open SSL is the same, like we could do somewhere halfway between 0 and that one. That fails, so 30. This is just so that we don't have to go all the way through the binary search, but we can at least go a little bit further, right? So 38, great. So that seems like it builds. Package config, I think these we can just leave. And then we want to get, check out cargo.lock, and then we wanna cargo check, just to have it reflect the changes we made here. No changes were needed, because everything that was in our cargo lock was at least these versions anyway. Fixes for bad minimal versions. You see, I've written that comment before. So now we can get rid of all these, and this, and then we can go back here. What else failed? Minimal versions we dealt with, coverage failed, and MSRV failed, which is fine, we can deal with that. Ah, could not find a repository. So this is for codecub, you need to, before you can start getting CI, you need to like go to log in with GitHub. This should also always open in dev. And so now I can do this, at least in theory. Really? Why? Interesting, there we go. Okay, so now it's been created. That was all that was really needed. And now that that step should work as well. So now we can do a git push, and then we see. Now I haven't added any actual approval rules here yet, which is the next thing we're gonna do. This is stuff like, what do we actually require before being able to merge something? Let's see here, okay. And then the other thing I wanna do is make sure that all of the workflows actually ran. So we go back to check here, rolling and test. Because sometimes there's like a syntax error in your YAML or something, and then it basically gets hidden from this list. And you have to go in here and see that it's actually running, but that seems like it is. Nightly now fails, why does nightly fail? On nightly, the lock file needs to be updated. Locked? Oh, interesting. Ah, all right. New lock file. So this is, I guess I didn't rerun cargo check after I checked it out and updated these dependencies. So the reason that the diffs here are things like, we now have multiple versions of FF, for example, being brought in as a result of the minimal versions that we specified. Did I leak a secret? Is there a secret in here? No, that's fine, I don't care. If you wanna take over my codecub for this one, that's fine. I think I can also just erase that after the stream, but thanks for the call out. All right, so now at least hopefully, there were three rockets and now there are only two. Makes me sad, it's not a rocket anymore. Let's see, if everything goes to plan and these should now all turn green. I wish they would just like all run in parallel and all run fast, but I suppose GitHub is like, donating CI for free for open source things. Yeah, I know, it's crazy that they just show the secret without a button to reveal. Come on, CI. No, don't wanna scroll. They're all running. Test isn't even running yet. Can I cancel that run? I wanna cancel that. That's the execution from the previous push. Same with this, I don't actually care about that one. And same with this, I don't actually care about that one. Because for open source projects, GitHub CI limits the number of jobs that can run in parallel. And so that's why some of these haven't even started yet. And so by canceling the runs for this push, as you see some of them have now been canceled, I'm freeing up more resources to run this one faster or more in parallel. Hey, now there are three rockets again. Nice. And Windows takes forever to start because they presumably have a limited number of Windows hosts. As a software engineer, doesn't this work feel like a waste of time due to the minimal benefit versus time used? I mean, if it's your free time, it's your choice, but you might get paid for your time. When you say this work, what do you mean? Do you mean doing streams or do you mean like the minimal version stuff? If it's doing streams, I have all sorts of good answers. If it's minimal versions, yeah, it's a waste of time. We should have tooling that does this for us and I would like dependencies to fix the problem. Now, is it a waste of time in the sort of grander scheme of things? Unclear, like I do think it's actually valuable to have a CI check that tests your minimal versions. And I think it can save you a lot of time down the road when your consumers come to you and say, hey, my thing doesn't build with the dependencies you specified in your cargo or the minimal version of your terminal. And especially for things like, I tried to build and it failed and it fails in really weird ways if they're using a dependency that isn't new enough. So this is a way for you to catch it before all of your consumers catch it and it's better to catch it in one place than many places. So I don't think it's wasted to have the correct minimal versions and check for it. I think the work required to get to that point is a lot of waste and we should really build tooling for it. Yeah, so that's why, if it's for the minimal work stuff, that's the reason why I think it's valuable. Clippy seems like it passed, so that's nice. Oh yeah, I think we did a pass on that in the previous stream. MSRV succeeded. So that's good. Cargo hack succeeded. Coverage, we don't know yet. Mac OS, we don't know yet. Minimal versions, we don't know yet. Tests passed on beta, so it probably passed on stable. Updated passed last time, it's unlikely it'll fail this time. Would be interested to hear the answer for streaming as well. Yeah, why wait for CI? So the reason why I think streamings or streams like this are not a waste of time is partially because I try to build things that are useful. If I stream something where I just build something completely throw away, then it's less clear that or that part of the value sort of goes away but I do do those streams too. If you take the Wordle stream, for example, it doesn't matter. Like it's not a thing that needed to be built. It was just fun to build. But then I think the other value with streams is that at least my hope is that they're educational. As a part of going through this, even just the stuff that we've done so far, hopefully there was a bunch of stuff there that some subset of the audience for each thing haven't seen before, haven't been exposed to, haven't thought of. And so it's a way to try to teach many things all at once by showing the real development process. And I think that's hugely valuable. I think it has a long-term impact. It has a broad sort of fan out in terms of who it teaches what. And I think it teaches things that are otherwise hard to pick up, like it takes a long time to pick up a lot of these small nuances, things to think of in CI. So at least my hope here is that these streams are actually have a pretty meaningful impact and are definitely not a waste of time. Me spending two or three hours for a short-ish stream and then as a result, having what is currently 189 people and possibly more when they watch afterwards spend the same amount of time but hopefully they are learning new stuff as part of that time, seems to me like a pretty reasonable trade-off. Like my time is valuable but it's not as valuable as 200 other people's same amount of time. Windows failed. I have to hate Windows so much. Okay, Windows failed because, oh, I've seen this before. Where did I fix this? Positories, sort by last updated. Flurry, Ward search, IMAP, Inferno, Github. This one doesn't even have Github yet. Yeah, found it. So on Windows, you have to do this little thing and really I should just add this to my standard setup but essentially under steps, you have to do this which is really, if you're on Windows then you need to specifically install OpenSSL. Install OpenSSL for Windows. Now for many projects, this doesn't matter because you just aren't taking dependency on OpenSSL in the library but when you do, you run into this one. And Mac OS, I think is probably gonna pass. The reason I don't wanna push this yet is when I push, it's gonna restart all the CI. So it's a little tempting actually to merge this and then deal with it later. Let's see. Oh, we got coverage too, I think. So in theory, there should be a comment coming in from CodeCov here anytime soon. Let's look what it says. Does it have coverage in for now? Right, so it only has for the CI branch yet because we haven't merged and coverage here is about 68%, so that's not great. So we have terrible testing of index. What's in index? That doesn't seem right. Oh, it's because we never have a test that tests features or dependencies. That's what it's telling us and that's totally right. Like that's a thing that we really should have a test for. Mac OS passed, great. So now what I'll do is I will get push and then when this kicks off, assuming the YAML is still valid. So this is where I'm just gonna go here, go back, test. Yep, so it picked up all the jobs and then we're just gonna go here and merge. Fine, I'll leave that and go for merge. So now, delete branch. Boom, we have CI. And then the question becomes, okay, what do we put in the readme? Because readme's are hard. And I think the last place I did this was here. Yeah. Look at the readme here. I tend to reuse this a lot, which is here. Main, pull on main. And then for all of this, it is replace furry with crates index transit. That's what we called it, right? A cargo index. Really it should maybe be crates index transit now that I think about it, but fine. And it's the main branch, not the master branch. Badges. So this is a crates.io badge, which you want to show what version. A docs link, which really goes to docs.rs. Dependency list, which is this thing that just shows what your specifiers are and whether they may be problematic. My guess is we're gonna get PRs from Dependabot here pretty soon about, well, cargo is gonna be a little weird. I don't know whether we get a PR for that, but for cross beam channel and for P384. Unfortunately though, we can't actually update these because they're there for minimal versions, but we'll see whether it comes in and then we can ignore them. And then codecub should upload once the codecub step of this actually completes. All right, so that means we now have CI. We have the badges in the readme. Where do we go next? Well, there are a couple of paths we can go. One is on writing documentation, which I kinda don't wanna spend a lot of time on writing documentation here, even though it's valuable, but it's also kind of straightforward. So I think what I wanna do next is look at improving the test suite. So for testing, there are a bunch of different ways to write tests and it's not as though you should only do it in one way, but one way to write tests is something called property-based testing. And there are a couple of tools that you have in Rust for doing them. Two of the main contenders are PropTest and QuickCheck. QuickCheck is made by Bernsushi by Andrew Gallant. And it's really good. The, I think the recommendation these days is to generally favor PropTest. The reason for that is because QuickCheck is, it has a little bit of a simpler model of how to do property-based testing. It's faster as a result of having a simpler model, but it's not clear that it's better along meaningful axes. If you look at the, sorry, let me see if I can make this be dark. It's interesting. Why don't they have the little, it's usually a little icon. No, I don't want to print this book. There we go. Rust, Cole, Navy, IU, IU's looks nice. So they have an entry here on PropTest versus QuickCheck. And I think, you know, there's a, you can read this on your own time if you want to. I'll put it in chat too. But basically they're saying PropTest is, it has a more elaborate way of doing property-based testing, but the real value in QuickCheck is that it's faster because like generating the complex values in PropTest can be much, much slower. But we're gonna start with PropTest here and I won't give you a full introduction to what property testing is or all of the nuances of it, but the basic premise here is rather than writing a, rather than writing individual test cases, you write a, you essentially write the pattern for test cases. So if we dive into an example, you know, here they have a parse date function. So it takes a string, it gives you back the components of the string, of the date in the string. And, you know, normally you might write a test like this, we use parse individual dates and you see whether or not they return something reasonable. Unfortunately though, this means that you have to sort of hand code a list of dates. You also have to come up with the patterns and if there are patterns you didn't think of, you're not gonna be testing them. The other way to do this is with a PropTest or property-based testing where you say for example here, for any S that in this case is a regular expression, for any string that matches this regular expression, call parse date and don't even check its result, we just wanna check that it doesn't panic. So here, what this is gonna do is it's essentially gonna generate a bunch of different strings that match this particular pattern, run it through parse date, see that it doesn't panic. And then if it does find a panic, it'll tell you what input it gave and what actually happened. You see here, it ran 102, it ran that test 102 times of different inputs before it got to a panic. The other thing you can do is you can do this. So for example here, they say for, I want all the S's, so all the strings that match to this pattern. So these are things that should be valid dates or at least should be parsable as dates. And here you wanna check that the result is actually some. It is like fuzz testing, like property-based testing is in some sense a structured form of fuzz testing. And you're right, it's also not deterministic. Now they have some mechanisms for trying to make it a little bit more deterministic, but it is exploring an infinite space. Or in this case, it's not quite infinite, but it is very large. And so realistically, it has bounds on how long it keeps running for and you can configure that. But the idea here is that just straight up fuzz testing, we actually give arbitrary inputs is a little bit less useful. And instead you want to be able to express constraints on the inputs to more efficiently explore the space or more intelligently explore the space. But it really is fuzz testing. It's a sort of subcategory of fuzz testing. And then they have some, I'm not gonna dive into all the details, but they have some mechanisms that allow you to save the state of past runs to ensure that you hit the same ones if you run the test suite again. There's a bunch of that kind of mechanism in here. But so in this case, so yeah, so running prop testing can take a long time just like fuzzing. And in practice, what you do is you limit the number of steps that it actually runs. I think what we would do is set prop testing up to run as a separate CI job, but it is slower. Like it does take longer. There's no doubt about that. But it also much, it tests your program in a much, much better way. And the question becomes, how do you generate these inputs? And if you generate them from relatively small domains, then your testing is gonna be, you might actually be able to fully explore the space. And then it allows you to do things like actual ranges. So instead of just saying any arbitrary date or any arbitrary string, we're actually gonna generate a bunch of valid potential inputs. And again, it doesn't fully generate all of these that tries to semi-intelligently explore the space here. And then we can do whatever we want inside to test the things work correctly. And again, it's gonna run a bunch of inputs and tell you what input it might fail for. I don't think I would use this to test performance. I mean, you could, but it's... Okay, so the way you could use this to test performance is it sort of randomly explores the input space. And as a result, performance benchmarks often end up weird because you use the same input over and over in a loop. And this is gonna force you to handle multiple different inputs. The challenge is even just generating the inputs takes a bunch of cycles. This is one of the reasons why fast testing can be slow and prop testing in particular is because you have to generate inputs that actually match the constraints, which is not true for fast testing, which is just purely random. And as a result, if you try to do benchmarking based on this, it gets a little weird because you're probably gonna benchmark the prop testing suite just as much or the harness just as much as you're gonna test the actual line code. You see here, for this case, it generates a bunch of things that match this pattern. And you see, it's not just a straight list. Like it doesn't do a linear scan. It tries to be more intelligent than that. And it ultimately refines down to this test case fails. And there's a bunch more in here. We're not gonna get to all of it, but I wanted to see if we could get some some kind of basic prop test setup for this. So let's look at the tests that we currently have. So we have this round trip function, which is, it constructs a cargo project for you. It expects that you're gonna make some modifications to the cargo tumble. Then it packages that up and then runs it through all of the different steps where it converts it, sees that it's convertible into a normalized manifest, see that it's convertible into a crate version that can be published, that it can be turned into things that are compatible with these other crates. And ultimately you can create an index entry. So it just does this full round trip. And currently our round trips are super simple. This one doesn't modify the tree that you get from cargo new at all. This one, we didn't even implement, but the idea here was to add dependencies. And I think we could actually do this with prop tests. And I'll show you in a second how to do it. So if we wanted to add prop tests to this, the first thing we're gonna have to do is where is the, they're missing a crucial step up here, which is adding the dependency. Ah, there we go. So as per our previous discussion here, they claim that you wanna do 1.0.0. I don't know if I believe them. I'm gonna go ahead and make that 1.0. And now if we go back here, we use the prop test prelude. Now, I wonder here, whether we even want this to be in the same file. This is what I'm gonna do. I'm gonna make this pub. You're gonna hate this. If I remember correctly here. Why? That's interesting. I think I need to go ahead and do this. Might just be confused. Interesting. Check out cargo lock. Huh. I don't know why it's now complaining about these. I guess it's because we added prop test. But I'm not, this isn't running with minimal versions. That shouldn't make a difference. Doesn't even build with the latest versions of everything. Found create bytes, compiled by an incompatible version of rust. Compiled to our rusty beta. Interesting. Oh, my environment variable is no longer being set. That's why. So it feels about that. Now I specifically didn't wanna run cargo clean because I have a kind of weird target directory setup. Although, apparently it wasn't using it anyway. So it wouldn't have made a difference. So that looks like it builds. Check out cargo lock. Cargo check. That's very strange. Looks like we got into some weird fingerprinting issue here. All right. Well, that seemed to now be working. And with, all right. That builds. Now, the reason I did this is because I want via cargo to just be a standard integration tests file where we have like just imperative tests. And then I want the prop tests to be separate so that we can choose whether or not to run them. We might even ignore this whole file. Now the challenge is this round trip function. I actually want to be usable in both of these. So what I do is I make the via cargo thing a module that I then use something from. I might, actually this might not work the way I wanted to because it might end up running those tests twice. There are other ways to do this but the auto detection the cargo does for test suites is actually a little bit annoying here. You can turn it off but let's see what we get out of here. So many dependencies. Oh, you know why? I think I know why. It's because, no, not that one. It's because I changed my cargo config to include this. And I think that caused it to get confused when trying to reuse old artifacts. So I think cargo test is actually now going to run the unit test and via cargo twice. Yeah. See it runs it via here as well. All right, fine. So what we'll do then is we'll make a sub directory called something like util and then we will copy tests via to tests util around trip mod.rs. And then we remove the unit test from here and then we go back to via cargo. We say mod util and we use util on trip and then we can get rid of all these. And then we do the same in prop tests. And now let's see what we get. Why does it complain that this is dead? Oh, because it thinks this is a main, that's fine. We'll do allow dead code. Okay, so back to prop test. So now that we have this and it only runs our test once and the warnings are all from prop test. Oh, balls. Okay, so you see how it prints the output from cargo as a part of this test. So that's another thing that we're gonna wanna fix here. And the fix for that is actually a little bit annoying. It's probably here that what we want is, so when we invoke cargo as a library to do things like builds, what it's gonna do is it's gonna print to standard out and standard error. But it does so in a way where it prints directly rather than through the print line machinery, which means that it doesn't get captured by the mechanism that Rust uses to capture test output. I thought there was a way to do this. So if we look at the docs for cargo and we find shell in here, shell is the thing that it uses to sort of emulate the output location. But what we want here is to probably do it from right, I suppose. So if we do from right, I forget whether, this makes me wonder actually what the Rust test suite uses for this. No, from right. Okay, it just uses a box new vec. I see, it uses a, yeah, I guess we can just use a file here instead. So what this is effectively gonna do is just drop all output, which might be annoying. We might actually want that output, but I think in this case it's rare that we'll want the output from cargo itself. That's fine. All right, so back to round trip. So what we want here, we're gonna start from one of the examples that they give here. And if we go back to via cargo, we can actually steal this whole test. So what we want here is, you know, this is the modify the workspace before packaging and this is check the final, check the output structs or check the various structs, the various transit structs. Great. So the question here is, what are we actually prop testing over for this? And this becomes a question of, well, what are we actually interested in varying? It could be something like features, right? We wanna make sure that as you go through all of the steps, regardless of what the feature is named, it'll just get passed through appropriately. That doesn't seem all that interesting to prop test. If we go back to dot create, look at what we have here. So these are sort of the kind of things that we might expect to do prop testing over, right? Are the fields that we know are gonna be read in by our thing. And so this is like stuff that might be in cargo Tommel. So it's the name of the package. We could prop test over the name of the package. It seems unlikely that the contents of the name matters. It's just an arbitrary string that's passed through. It's not really parsed. The dependency specification is though, right? So here we could do things like, we wanna make sure that when dependencies are added, the appropriate, when dependencies are added, the appropriate like version specifier, optionality and stuff ends up being passed through. So that seems like a reasonable thing to do. And we also want to prop test over whether it's dependencies, dev dependencies or build dependencies. So I think that's actually the thing we're gonna prop test over here is, this is gonna take a, probably a vector of dependencies, I think is what we want. So let me go back to the strategy here. So you can do things like, where's the prop compose? Here, no, generating filtering, generating enums. Where's the prop for getting a vector? Higher order strategies, there we go. So here, the kind of inputs that we want for this is depths. But really, I wanna see if we can do this in a better way because, all right, this might work actually. Because what we want here is a function that generates dependencies. And so it's gonna generate a, I guess it'll actually just generate a string, like a vector of string. Where that string is really a tommel specifier, right? And so what we want is something, and we could specify that just by regular expression, but it feels more appropriate to generate the structured representation and then turning it into a string. Now, what I'm curious about is the right way, ah-ha, canonical any, right. So I think what we want here is something like, struct dependency. And the dependency can be enum kind, can be normal, build or dev. An dependency has a kind, it has a name, it has a version, and it has a feature. Is there anything else we have? It has an optionality. I guess we can actually go back and look here. What else gets pulled in? It has an index field, which is also arguably something we wanna include here. Public is still nightly, default features, package. Arguably this is just, we wanna generate arbitrary for this. We want to be able to generate an arbitrary dependency. So in fact, I think we can just do that. So maybe that's the thing we're gonna do instead is to have a way to generate a normalized manifest. I think that's actually what we want. Right, so there's a trait called arbitrary. And arbitrary is also used in Quick Check and it's essentially a way to say, it's a trait version of saying, generate me an arbitrary version of this struct. And we can use the prop-test-derive. It's experimental, but it seems pretty reasonable for us to start with this. So what we would do is go over to, oh, but we can't use that in a unit test, right? So the temptation here is to do derive arbitrary. The problem is if we derive arbitrary, then now we have to take a dev dependency, no, a normal dependency on arbitrary for this crate, which doesn't really feel right. The other thing we could do is we could do a config atter, test-derive-arbitrary. But this doesn't work either because test is only set for the library crate when the library is being unit tested. When you're running integration tests like prop-test, then the library will be compiled not in test. And so this won't work. So instead what I think is we do actually have to replicate this, which isn't the end of the world, but it suggests that maybe instead of this, we should just do this. We should say struct dependency should just hold a cargo index transit dot crate, normalize manifest. So really what we want here is a, we wanna say that a dependency is a combination of kind and normalized manifest. Of string is fine. Oh, I forgot we had all these things. Oh no, actually we want a dependency. We can do this. And I guess actually I think we have a definition of dependency somewhere too. We have dot crate and I think it's an index. What we have, which is not an index, it's in publish maybe then. Yeah, in publish dependency kind. There we go. And then what we want to do is we wanna implement arbitrary for dependency. Now implementing arbitrary is a little bit annoying. Oh, do they have a helper for this? So this is the kind of stuff that we want, right? We want an arbitrary, maybe we don't need arbitrary here. Maybe we want a ArbDep, right? That's what they call this. So something like ArbDep and it implements a strategy for generating a dependency. Oops. And what we want it to generate is, it's not quite gonna be that because it's gonna be, we want a compose. So this is for generating recursive things, which we don't actually need. But I do think what we need here is, where's the compound strategies? No, prop compose. That sounds like what we want. Yes, so prop compose. So an arbitrary dep kind is gonna be any in kind in zero U8 to three. And it generates a publish dependency kind. So this is just, how do you generate an arbitrary dependency kind? Zero is gonna be publish. Arguably we could just replace this now with this. That's gonna be a dependency kind dev one and two. So normal build and dev. And then this is gonna panic. So now we have a way to generate an arbitrary dependency kind. And then we're gonna do the same for arbitrary dependency, I guess listing, which is to generate this second part. So that's gonna generate a dot create dependency string and we'll figure that out in a second. And it's not gonna take a kind, but it's gonna take some other stuff. And then we're gonna say that to compose a arbitrary dep, which is ultimately what we get to, we want kind in, what are they write this? Yeah, you just call the function. So our dep kind and listing in our dep listing. And that's gonna give you back a dependency. And this could just gonna be dependency of kind and listing. And we can throw a debug here. That seems fine. Arb dep, it's fine multiple times. No, not anymore. And we'll still have to figure out exactly how to write this depths thing that generates a vector of these, but that's fine. And to generate a listing, what do we actually need to generate a dot create dependency, fill the struct fields. Okay, so here we're gonna take version which is gonna be, the version specifier is a regular expression. And it's something like, I guess really here we're trying to have a regex for semantic versioning specifiers, right? So we'll do, it can be a caret or an equals, but that's optional. Then it's followed by a zero to nine, any number of, any number of, but at least one. And then it's followed by a actual dot, followed by zero to nine of at least one, followed by potentially like build specifiers and stuff, which I suppose we could include. So there's a dash followed by, I think arbitrary ASCII characters, but I think it is limited to ASCII. So that's an optional one. And then same with build meta information. So this is like, if there's a pre-release specifier, this is, if there's a build meta, and maybe this is like too extreme, but we'll start there. So that's the version. The name is just any string really. And maybe we're gonna regret making it any string because it's not actually right. It's, it is A to Z, zero to nine dash and underscore. I don't think capital letters are allowed. And I think it has to start with a letter, but it's allowed to be a single character. Okay, so this is gonna be the name. Ooh, no, it's not. Right, we actually need to generate a name for this as well. Remember, so the way that these are structured is actually that the name of the dependency is separate from the dependency specifier. So this is the dependency specifier and there also needs to be a string, which is the name. And so this is really the package. And then when we do arbitrary depth, it separately has to dictate a name. And there's an interesting case of if the name is equal to the listing, that actually changes our logic. We're not currently encoding this. That might be an interesting thing to try to encode. So version is a version. So version is actually gonna be semver version rec parse. And there's a way here in prop to specifically say prop assume. So we can do rec is equal to this. This is basically a way to say that if it's not a valid semantic versioning number, then we're not gonna bring it in at all. So we're gonna assume that rec.is okay. And then rec is rec.unwrap. Put package instead of version. Oh, you're right. Thank you. So version takes rec, package takes package. It's also gonna take optional in... It's a good question. How do you specify bools here? Doesn't really say, does it? There's gotta be an easier way than saying it has to be... Oh, we can use any, any bool. And say with default features. So optional is gonna be optional. Default features is gonna be default features. Public for now, we're gonna set to true. Actually, we're gonna set to none. This, I guess optional is gonna be option of this. So that's fine, package is gonna be... Oh, that's interesting. I guess we can do this arbitrary package and say it's gonna take this and it's gonna return an option string. That's not really what I want. Okay, I wonder if there's a nice way to do... Actually, I guess we can do it this way. We can say package is gonna be an option string. And so it's either gonna be none or it's gonna be, I guess, like this. Assuming this is what they want this to look like. Prop map S. And this is gonna be some S. I think that's what they're after. And that way this can go away. Package is gonna be our package. And so that way we're actually testing now both the case where the package is set to something that's different from the name of the crate and the case where it's not set, which is what we wanted. Why do I need to wrap this in okay? That doesn't seem right. Features for now, we can just leave as none. We're gonna wanna generate this eventually, but for now we'll make it simple and same with target, it's gonna be none. Now it's complaining that this has to generate okay. And now it complains that it expected to be a closure that returns dependency string, but it returns result of that. Actually this doesn't need to be prop compose is the reason I think. Or maybe it does prop compose. Interesting, I don't know why this one is being unhelpful. Cause I feel like this is supposed to generate this. Oh, but then it can't use prop assume. What does it say under filtering? Oh, you can only use prop assume inside of prop tests. That's why. But we can use it locally without, no, I see. So we're gonna have to do dot prop filter. S, semver, version rec, parse, s is okay. And that way this can go away. Is it gonna let me do that? I expected two arguments to prop filter. Not a valid semver rec. All right, but it might be, is there a filter map? There is a filter map. This pictures are found. So s dot, s dot and then this, okay. That's also none of what I want. I think I'm just gonna leave this as this for now and just use the unwrap. We can't use any dependency kind because dependency kind doesn't implement arbitrary and we don't want to take the dependency to make it implement arbitrary. Okay, so now we have arbitrary depth and for ARB depths, we wanna say that it takes. Right, so any, I think we now should be able to say ARB depth, zero to, I don't know. We don't, I don't think we actually need all that many. Oops, and I don't think we need to map over it. Oh, we do actually, we wanna prop map D and we wanna turn it into a string which is gonna be something like format, nope, but it returns string. I don't want that. I want, maybe it's just map. Vic strategy is not an iterator. Okay, prop turns string. Actually, this is gonna be easier if we say that ARB depth stir takes a depth which is gonna be any depth and returns a string and it just formats that and then this now is just ARB depth stir. That doesn't work either because the, we wanna branch with section of the file we put in depending on the kind. Maybe we ignore kind for now, that's fine. We can deal with this. So this is gonna be something like the name. We can use a raw string here. Make it a little bit nicer to write. It's gonna be equal to, turning this into a string is actually gonna be a little bit annoying, right? Because we want something like depth dot name or depth dot zero, right? Which is the name. We're gonna ignore depth dot one, which is the kind. And then we're gonna say, and then we kind of wanna test the case where there's no map here. But that's actually kind of annoying to do. So instead, I think what we wanna do is something like if let some depth dot version then s dot push stir format. Oh, this is way grosser than I wanted it to be. Version equals, one of the expressions fields, oh, depth dot two, no version. Oh yeah, we can write, you're right. We can do write instead into s, mute s, which is marginally better. But then we have to use std format right for that to work. The version is always there. And then we're gonna have it ultimately output s. So this would actually, this gets us to something that's kind of working. Now the to do here is also write out other props, including getting comma, right? I just wanna see that the basic infrastructure we have works. We're not gonna do this. We're gonna take depths, which is gonna be in our depths. And then here, we're gonna do ctoml is to open, read to string p dot join cargo toml. So we're gonna read in the existing toml. This is one of the reasons why this project starts to get me really slow, because it operates on the file system as well for every test case. And then we're gonna do push a new line. Remember that the cargo toml, the cargo new generates has a, it already has an empty dependency list. So we should be able to then do four depth in depths. Ctoml, I guess we can write to ctoml. We don't even need to do that. We can do this depth. And here we'll use standard format write as well. And then we'll do std right to p dot join cargo toml. And we'll write back out the modified toml. And then we still need to do this check later on, but that's fine. Let's see what happens. I just wanna see here if we do prop test. And there's a warning here. We don't use self. There's also a warning saying online 69. Right, we don't unwrap this. And I guess here we could for our own sanity print out what it actually does. So what is the list of depths that is trying to inject? Well, it does something, file failure persistence. That's fine. It doesn't find librs or mainrs. Oh, actually I wonder whether the prop tests are supposed to not be integration test, but actually be test binaries. Oh, I guess it is. Not entirely clear where they intend that this should go. They have a main, but they don't actually say where, like they have a main function here, which is a little interesting. Let's find, I don't think that matters, but it does say that my regular expression isn't valid. Repetition, repetition missing. In this pattern, it might be this dash actually. So it's complaining about my thing here. I'm gonna simplify it a bunch and see if it still complains. It does still complain. Value of minor version exceeds U64 max. This just, right, okay. So we probably here wanna say instead of this being arbitrary number, realistically versions numbers are gonna be between not very many digits and same here, probably between one and three. See where we get to with this. Well, this is interesting. Certainly seeing some panics here. Could not parse a equals version equals, okay. So that's correct. So that means our printing is wrong, which is correct. This should be a double quote around this. Let's see. Okay, it ran for a little bit longer. Wow. Check out this list of dependencies. So this is all the stuff it tried. So here's a crate named R underscore underscore dash underscore O underscore six I underscore underscore zero Y dash. That's the name of the crate. And we take a dependency on that on version two. Sweet. Thanks. Great. Although this is what we asked for. And it failed on this one because invalid leading zero in minor version number. Ah, yes, okay. So this is our regular expression again where it actually needs to be one through nine and then maybe zero through nine. And we could do this with zero one instead. And this too has to be, it has to start with a non zero. Ah, that's also not true because it can just be zero. So it has to be either zero or one through nine followed by any number. And same here, this has to be either zero or one through nine followed by zero to nine one to two times. Ah, regular expressions are the best, man. It's running for longer now. Okay, it's doing stuff. Let's have it not capture the output. Okay, so it's generating some like reasonable dependency specifiers here. And remember, this is running through the full round trip test. So this is turning it into all of the intermediate formats. Now, of course, there are a bunch of things we're not doing at the moment. So for example, we're not writing out all the other properties. So here we're gonna say, if let some x, I don't actually remember what's in here, depth.2.public, oops, what else do we have? Oops, what else do we have? We have optional public. We're not currently writing out. So optional, frustrating default features. What else do we generate? Optional default features and package. So let's do those first. Then we're gonna write out optional equals boolean. Same with default features. It's gonna be default features equals that. And then this is gonna be for the package. And that's gonna be package equals. So we're gonna quote this one. And for all of these, because we know version comes first, we're gonna add a comma at the beginning of them. So now if we run this, let's see what we get. So we just halted for a second here. Look at this dependency specifier it generated. Version equals carrot 0.0. Optional equals true, default features equal true. And we can go further here too. For this, it doesn't have to be those. It can also be less than. It can also be greater than. It can also be greater than or equal. So let's say it can be one of these or it can be less than or equal or it can be greater than or equal. Technically it can also have commas in it. But we're just gonna ignore that now. Now what's interesting is it seems to, it biases these little bits. It seems like it tries to like pick every branch equally often and stuff. But I don't think that's too bad. Option equals false, default features equal true. Package equals this other random string that's not the same as that random string. Nice. And what's interesting here too is that, this is actually testing. Remember how if you watch the previous stream, you'll remember that there's a bunch of logic around whether package is set, how that gets transferred between the different like representations. And here like we're gonna generate all of them. Now currently we're just checking that this doesn't panic. Because down here, we're writing out the dependencies but we're not actually checking that this is present in the ultimate thing that we end up with. So there's a bit here missing which is in the index, we want to check that for depth in index dot. Now I don't remember what the dot dependencies dot iter. The regx for minor versions is wrong because it should be zero, it should be one to nine. Oh right, yeah, this should be zero or two. Yep, that's right. So we want to loop over the dependencies that are actually listed in the in there. And for each one, we want to make sure that final depth, input depth in depths, this scan could perhaps be more efficient. But what we wanna check here is that the dependency that ended up in the final index entry is actually one of the ones that we gave in the input. And this is gonna be check because what we want to do here is panic. If we ever get here. So we wanna see if final depth dot zero, which is the name, no, if the input depth dot zero is equal to the final depth dot name, then we wanna break out her. Realistically, that's not actually, we need the full check here. No field zero on string. Right, I think I actually want this as a helper. I think I don't want this to generate a vector of string. I think I want this to generate a vector of dependencies. And then I want this to just be a helper function. Just depth, depth to Toml. And it takes a dependency and returns a string. Because realistically, that's what we've built here. There's nothing about that function that's actually about prop tests. And then here we're gonna write out depth to Toml of depth. And then down here, this was check. So this is just checking that, this is just checking the name. It's not checking that any other property is maintained. Let's just see that that isn't wrong. Like 71, and I guess this can be here. And this is gonna be something like here. All right, let's see. Great, okay, so this at least suggests that for all of the final dependencies are in the input dependencies, are one of the input dependencies. We actually wanna check that they match, but there's also the inverse here, which we have to check, which is numfound is zero. Then here, also this should continue check. Numfound plus equals one, and when we get down here, right, this should be a panic. And this should be if numfound not equal to depth.len, or we can just do cert equal, numfound input. So the assertion here is that, no, this is depth. The assertion here is that we actually found all of the input dependencies in the output dependencies. And again, we're not actually checking that these are correct beyond just the name is present. Let's see what we get. Okay, so this is just running through now, and my guess is it would have panicked long by now if any of this was obviously broken. Here though, we can now start to do things like, assert that the input fd is final depth, id is input depth, and we wanna assert equals the version is equal to the final version specifier. So here, we need to actually map these fields and for the input depth, I thought it was version. So why is it not version? Oh, this should be input depth dot two. And the final depth here is of type index and the index dependencies. So registry dependency and a registry dependency has a name and has a requirements. And I think requirements is the one we're after. See if that works. And as you can see, we could go on with this for ages, just adding lots and lots of checks. Now I'm just adding them mostly for completeness, right? So there are a bunch of, we wanna check that optional matches, we wanna check that default features made it all through. We wanna check that package made it all through. So optional in the index, optional is not optional and that's because it defaults to false. And then we wanna check that here. Default features, defaults to true. A package here as D ref, I'm gonna de-reference both of them and then they should be matchable. D ref twice. We wanna, this is a cal string. This one is a cal string. So here we're gonna do as ref so that they're actually comparable.map s. There we go. Just needed more stars. And then what else? Is there anything else that we actually write out here? Version, optional default features and package. Now, one thing that's interesting is this will actually break if it generates two dependencies by the same name, which we haven't told that it shouldn't do. It's also kind of interesting. Nice. So this now could run basically forever and generate every possible dependency list. But at least now we have something that does the round trip check with a bunch of different configurations of package options. We could add more things here that make it into the cargo tumble that are not just in the dependency list, but at least now we know that that dependency transformation does something reasonable. So now that we have this as prop test, I think the other thing we wanna do here is actually, I think we wanna ignore this. The reason I think we wanna ignore this is because this particular test is actually pretty slow. And so what I wanna do here is now go into test and say that not minimal, not required, actually required is probably fine. So we're gonna go down here and say prop test is gonna run only on stable. I don't think this is one where we need to run it on multiple ones, tool chain stable. We don't need to generate the lock file separately. And here we wanna test prop test ignored. This is not all features in all targets. It is test prop test ignored. So this will actually run the test. So the idea here is that the, we're not actually gonna run the prop test in the normal test step. That's gonna be ignored because otherwise we would end up running it on Windows, on macOS, like in all the different configurations where we run test. So we want a separate job that's actually gonna run prop test. There's even an argument here for saying, this job should only run if the basic test suite succeeds. And I think there's a way to do that. So if we go to GitHub actions, workflow syntax, jobs needs is the one I'm asking. So we can say here that this needs required. So the idea here is that we're not even gonna run the prop test job, which is gonna run for a long time. We're not even going to run it unless the basic test suite succeeds. And now if we go back to prop test, one of the things we'll see is there's a configuration down here, test timeouts and forking. That's not actually what I want. Configuring a prop test. So because we know that our prop tests are pretty slow, okay, so currently 256 executions have to execute for a test as a whole to pass. Now 256 for us is actually gonna take a while, like you saw how slowly they executed, but I think it's probably okay. I don't worry too much about this. I think we can also skip this print line. I don't think it's that important. But 256 might be a little high. Let's see how long the CI job takes to run. Prop test. The other thing you'll see is we now have a new test slash prop test regressions thing. Prop test regressions. And this has the seed for failures. So that it replicates over time. They recommend that we check it in. We're not gonna check in the initial one we had, but over time it's a file that we're probably gonna wanna check in. And we could even here do, do, do, do, do, do. So this is gonna run for a little bit. And now you see it didn't generate the file because it only generates it on failure. Okay, so git add dot commit add protests. And then git push you prop test origin prop test. Right, so now we open a pull request and let's see how this turns out in our CI. So it should now be the case that it kicks off a bunch of different CI rules, just as it did previously. If we go to details here and we go not to check, but to test and go in here. It's just, yeah, here we go. So we see here that there's this required and then there's this one, which hasn't started executing yet. And this is prop test. So it won't run this one until that one's finished, which is gonna make the total CI time longer, but it also means that we're not gonna start running prop testing until we know that the basic tests actually work correctly. So while that's going, what we'll do is we'll go configure the branch protection. So for main add branch protection, what do we want? We want to require a pull request. We want to, these are nice, but they matter less for a repo that only I work on. I do require status checks to pass. And in particular, I want stable formatting to pass. I want Ubuntu stable, which is just the regular test suite to pass. What's annoying here is these are the the work, it's like the workflows, but not the individual jobs. So it can be really annoying to dig through. And I want features. So these are gonna be required. I don't wanna require minimum supported rust version, for example. I don't wanna require minimal versions. Like those can be pretty annoying. I don't wanna require nightly. I don't wanna require beta, but these ones are like clearly ones that should succeed. I don't require linear history. Don't require deployments. Create. Let's go ahead and do that over here. Okay, so now we have a branch protection rule that's gonna start to kick in. Windows test failed. Why did Windows test fail? Hate Windows so much sometimes. I think this is just an actual random failure. Ignore this. But let's go ahead and see whether our tests actually... Oh yeah, you're right. And ignore equals would be useful here. I forget whether that's stabilized yet. So there's this new thing where you can say ignore equals and then give a reason. And in this case, we're ignoring it because prop tests are slow and should be run explicitly. And so now if I run this, hopefully it should print that ignore message. Yeah, that's a good call out. I'll put that into the check. So if we now go back to test, add prop tests. So these are still running. Why are these still running? Why are they so slow? Oh, it's because building our dependencies takes forever. Yeah. I think there's a given the size of our dependency closure, especially via cargo, there's a decent argument for caching the cargo home directory, which I just haven't set up at the moment. But there is like an... No, not there. In here, I forget exactly what the command is, but there's like a step you can add here that's like cache home slash.cargo or something. Or in our case, I think what we actually care about is the target directory. And by caching the target directory, it's slower if you don't have very many dependencies because it means you have to... It means the GitHub CI actually has to copy over the cache directory from previously, from previous builds. And that has a lot of binary artifacts, which are slow to copy over and as fast as build them from source. But in our case, we actually have a decently chunky dependency tree that takes a long time to build cargo in particular. And so doing the copy of those binary bits might actually be faster. Okay, so that kicked off. So now we should see this one run. We'll look at the summary in a second. Let's see if I can find job. And yeah, it's tricky whether you use caching because now also if your dependency closure updates, then it's annoying. This is one of the places where checking in the lock file also really helps. Oh, interesting. They don't have a specialized version for a bunch of languages, but not including Rust. So we can say here, yeah, so what we want here is probably just use this for required. One of the annoying things, right, is that when we, for example, build with MSRV, then that has to be a different cache. And so this is why you have to define a cache key. So it's a little bit annoying to set this bit up. And so we would need to do something like target cache restore, target. And key here is going to be matrix.toolchain, restore. And then there's a separate like save step, which we run after cargo test. It's going to be save cache target here. Save steps.target cache restore output. Cache primary key. And then in theory, at least, we should be able to use that same cache down here. And we're going to restore, but we're not going to save afterwards. And in fact, we could restore the cache here for things like minimal versions too, at least in theory. But I don't actually think I want to do that. And for prop test, the restore here is just going to be stable target. So that's the thing we can try. We'll see whether it makes a meaningful difference here. So now the prop tests are running and they're taking forever to run because of course they are, it's prop tests. The reason I want to do this in a separate commit is because I want to see to what extent it makes a difference. And that means I need to know how long it takes to run when we're not doing this. Huh? It finished compiling in four minutes and then it's going to run the test. Explain, ignore. And then I'll go ahead and push now because I don't think it's going to cancel this one while it's running. And we'll see, it'll be interesting. It's not going to speed up the first time because it has to populate the cache. Although it might speed up the prop test from the very beginning. Huh? It completed. I want to see the output of this step. So prop test, that's not too bad, right? It finished compiling after what, four minutes. And then it finished the whole job in 458. So the actual prop test only took, let's see. Yeah, the actual prop test only took 51 seconds. So that's running 256 different combinations of this input space. Now we are exploring a pretty large input space because it's all the feature combinations. We're also not correctly taking advantage or making use of the dependency kinds. So there's some more work that has to happen here. Arguably, I should note that down in test prop test of to do, split these by their dependency kind, def.one. But the prop test passed, that's nice. And now we'll go back here and see for tests. Add prop tests and I want to look at these jobs. And in particular, I want to look at stable. So there was store here, there was no cash. So it didn't make use of anything. This is going to take about four minutes, right? That's how long the bill takes. And then it's going to save. And then hopefully in the prop test job, we should see that speed up. Fantastic. Okay, so we have, oh, and this should also be improving our coverage. Oh, but our coverage doesn't run the prop test. So that's another good question, whether test coverage should this also run prop tests? And I guess let's do market to do. The trade-off there, of course, is that when you add this instrumentation for coverage, it also executes more slowly. And so having prop tests run more slowly to get coverage is unclear whether it's worth the trade-off. So let's see. So we have gone through all the bits of CI. We've done a bunch of talk about prop testing. We've implemented some prop tests. We've looked at configuring CI for prop tests. We didn't really talk about documentation, although arguably that's maybe a topic for another day anyway. I think that's probably where I want to end this off. I still want to see just that this does roughly what we want. But beyond that, I think I'm happy with today. While we wait for this to finish, are there any sort of follow-up questions from what we've done today? Check that you access the restore cache key correctly or you even allow it to reference that key from different steps. Yeah, I'm pretty sure you are because one of the points for the cache is that it is cross job and I think even cross workflow. So if we go back to here, you see there are caches and the caches are I think entirely global or they're per branch maybe. But that's very much intentional because that's sort of how the cache has to be to be useful. But are there other questions about how we configured CI or about any of the stuff that we talked about today so far? Just while we wait for the final job here. No, everything makes perfect sense. I'm glad to hear it. Let's see, I initially claimed this would be two hours. It's almost three hours, but I said two to three. So I think I'm okay. Come on, come on. It's interesting too because I think in theory, coverage here could reuse the same cache. Where it gets tricky is that for coverage, you need to instrument your code with coverage. And I forget whether this also means you instrument your dependencies. I think the challenge is you end up modifying rust flags to pass dash C instrument coverage. And even if that doesn't affect the build of your dependencies, cargo takes it into account for the fingerprint for whether it's allowed to reuse the cache files. Can you cache target or rebuild only when cargo lock changes? That's effectively what we end up doing here. Okay, so the build finished in five minutes. And now we're gonna see here what saving the target file looks like. So saving it took, what, 17 seconds? So that now finished. So now let's go look at this prop test job, which hopefully now shouldn't need to do a five minute build. That's the main thing we're going for here. Install stable, hopefully this should be able to restore the cache target directory from the step that just ran. Yep, it did that, downloaded very fast. Cache restored, shouldn't need to update the git repository given that we pin the revision in cargo Tommel. These warnings from the patch are a little unfortunate. I really hope this can now avoid doing the build. Oh, I can't wait for sparse registries to be stable. Arguably I should just inject that configuration right here to make the CI faster, but it's about to land anyway. I think it's gonna be the default in 169. Come on, git clone. So many rockets and hoorays. Come on, the clone of the index is so slow. I'm so excited that we have sparse registries. Downloading crates, that's fine. So it has to download all the crates because we're not caching home slash .cargo, right? And as a result, it has to redownload all the sources, but it shouldn't recompile the O, but it recompiles some of them. That's almost certainly because of build scripts. So cargo has this property where build scripts will, build scripts can say rerun me and therefore recompile if any of these files have changed. But the way that you indicate that is, or the way that cargo detects that is solely based on the timestamp of the file and not the contents. And so that's why when we redownload, the timestamp is newer, so it rebuilds, which is why it ends up rebuilding cargo because one of cargo's dependencies has such a build script. Still two minutes though is way faster, but that means we can tune this too by having it cache the home cargo directory as well. So we can go here and in save, we can have it also save that and then restore. We can have it also restore that. Can't you pass a flag to save those sources to target and not use home slash cargo? No, so the stuff that's in home slash .cargo are the source tar balls. It's not the target, it's not the target. The target directory doesn't store those things. We could override cargo home to make that be a sub directory of target, but it's a little dicey. So this is now also cache cargo home for build script timestamps. But this did now run a lot faster. And in fact, in theory, if we run a lot faster, and in fact, in theory, if we do get push of this, should run even faster than that. I think this is a good place to stop though. We're getting towards the end. I don't think there's too much more to cover here. You can follow this PR if you wanna see me tweak the final bits, but I think this is a good place to end. Thank you all for coming out. Hopefully this was useful. My guesses will probably do, I might do another stream on this just for our right documentation because I think there's enough to cover in that separately. But for now, I think I'm happy. All right, thanks folks. See you later.