 Hi folks, welcome back to yet another crust of rust. In this one, we're gonna look at something called the drop check. And the drop check is another sort of weird topic, sort of like subtyping invariance like we covered in the previous video. That's like kind of niche, kind of weird, but you also run into it in more situations that you might think. It usually only comes up in the context of unsafe code. And so we're gonna sort of fiddle a little bit with that and try to figure out both why it's there, what it's for, when it comes back to bite you and how you work around that. The picture of my cat awkwardly trying to figure out how to lie on the lap is only anecdotally relevant. All right, let me leave up the drop check here. So the Nomicon has a page on the drop check, which is pretty good. I recommend you give it a read. And it goes through sort of why the drop check is there, what some of the sort of awkwardness around it is and how to work around that awkwardness. And that's basically what we're gonna be working through today. I think it's helpful to sort of walk through an example and sort of real time code and look at the compiler errors, which this article also does a decent amount. So if you prefer reading to sort of watching the code go live, feel free to go read that article and it will probably cover most of what we do here. Although it's a little bit different. All right, so what we're gonna do is we're gonna start as has become standard on the streams. We're gonna start a new Rust library. In this one we're gonna call box, spelled the Norwegian way. So this is a Norwegian wrapper type for box. And we're gonna CD into box and we're gonna edit this and remove the test. I don't need a new version of Rust analyzer. Thank you very much. And we're gonna do here, oops, whoa, whoa, whoa, whoa, whoa. My vim binding's got weird. So we're gonna create a new box type, box T and it's gonna hold a pointer, which is gonna be a star mute T. So this internally really just is the same as a box is the idea, just with a Norwegian name to make it easier for Norwegian programmers to understand what's going on. And it's gonna be pretty, we're not gonna sort of do all the fancy stuff that you would want for a type like this. But what we are gonna need is a constructor, right? So we're gonna have a constructor for box and normally you would call this new, but because we're in Norway land, we're gonna call it knee, which is how you say new in Norwegian. And this is really just gonna return a box where the P is a box new of T and then we're gonna use box into raw to turn that into a raw pointer. So really we're just wrapping boxes, all we're doing should be a capital T. We will get to variance and non-null, yes, settle down. We will get there. And of course we had a warning the P's never used, that's fine. In this case, what I want to do is actually set up a little test here just to see that actually, I kind of want this to just be main. Let's make this a binary instead. Let's move source lib to source main, source main. No, I don't need a new Rust analyzer, thank you very much. All right, so what do you do with a box? Well, you have, let's say we just do, let X is 42, let B, which is gonna be our new box, and we're gonna do box knee of X. And that's all we're gonna do. And let's just see that this runs, just sort of the very basics, okay? There's some unused variables and stuff, that's fine. One thing you might immediately notice here is this box is never freed. When a Norwegian box goes out of scope, it doesn't bother calling the destructor of the inner type, it just leaks the memory, which of course is not what we want. So we're also gonna need a drop implementation. I'm gonna put that up here so that it's near the type it's implemented for. So we're gonna implement drop, not dorp, drop for box. Drop for box, and it's going to take a, oh, my typing today is awful. And all this is really going to do is it's going to do a, that there are a couple of ways we could do this. We could either do like a box from raw of self.p, which is unsafe. So this constructs a box from the inner type. This is probably the easiest way to do it because it both calls the destructor of the T and it deallocates the box. We could also just drop the T if we wanted to, so we could use steady pointer drop in place of self.p, which would drop the T, but it wouldn't free the box. So this is why we do the first one rather than calling the destructor directly. And now indeed, if we run it, you won't really notice any difference because you can't really tell the memory is leaking, but it gets us close enough. So now we have a working box type and everything is happy. I guess the one thing we are gonna need here is we're gonna need to implement dref for box because otherwise you can't get at the inner type. It's target is gonna be a T. It dereffs the self into a self target and it does that simply by doing a dereference of self p. This is unsafe because we're dereferencing a raw pointer and the safety here is valid is valid since it was constructed from a T, from a valid T, and turned into a pointer through box which creates aligned pointers and hasn't been freed since self is alive. And a similar argument is gonna apply for dereffmute, right? So dereffmute is the method that lets you mutably access the thing that's inside. Dereffmute at mute self. And it's gonna return a mute to self target. The reason we can refer to self target here just as sort of an anecdote is because dereffmute is a sub trait of dereff. So anything that is dereffmute is also dereff and therefore self target, the compiler understands that self target is the associated type dereff of the parent trait of dereffmute. And here the argument is the same as here. Also, since we have at mute self, no other mutable reference has been given out to p. Also a misspelled reference, great. So now in theory, if we do something like a print line of star B. So the star here goes through the dereff to get there into the value of this word below. And indeed, when we run that, we get out 42. Great, so we have a working box. So far, so good. All right, so I had two questions from chat. Why unsafe? The unsafe here is just because we're calling box from raw, drop itself is not unsafe to implement. And indeed, this should have a safety comment saying, p was constructed from a box in the first place and has not been freed otherwise since self, since still exists, otherwise drop could not be called. I figured I consider writing these safety comments in Norwegian, but I figured that would be less helpful to everyone involved, except the like three people in chat who know Norwegian, Danish or Swedish. Okay, so we have a working box and we can just ship it, right? We're done. It turns out not quite the same as before, not quite, this maybe will not surprise you, but it turns out there's more to this. So let's look at something that you can do with a normal box that actually doesn't work with the box that we wrote. So here I'm gonna create a new variable y. I'm gonna say b equals box knee. And what I'm gonna give it is a mutable reference to y. And now I'm gonna print out the value of y. So this is gonna read a little weirdly to you. So what we're doing here is we're constructing a box that contains a mutable reference to a variable that's on the stack. And then here I'm gonna use that variable. So notice I'm not using it through b. If I used it through b, we'd be fine. This would run just fine. Instead I'm gonna try to access y here. And if you think about it, this really should be okay because even though b holds a mutable reference to y, it doesn't do anything with that mutable reference to y. Like even though conceptually down here there's a drop of b, right? When b goes out of scope. And dropping the b is going to sort of get rid of this mutable reference but it doesn't actually use the reference for anything. So there's no problem in sort of the borrow ending early and allowing us to access the thing that has been mutably referenced over here. And yet the compiler doesn't let us do it. You see it says cannot borrow y as immutable because it is also borrowed as mutable. The mutable borrow occurs here which we assign it to b and the immutable borrow occurs when we try to use y. And as it rightly says, the mutable borrow might be used here when b is dropped and runs the drop code for box, right? And if you think about it, the reason it does this is actually fairly sane. It sees that box has a drop implementation and there's nothing that really stops box from doing something like this. Is that a bad example? Yes. Okay, so this is obviously weird, right? Like no drop implementation would do this but there's nothing that stops the drop implementation from doing this. From like trying to read what's behind the pointer in the drop implementation. And it doesn't even need unsafe. In this particular case, it needs unsafe because we're storing a raw pointer but you can imagine that there are cases where you store something in the box and you touch it in the drop implementation and you like read out of it or something. And if drop did this, then suddenly this code would not be okay. We wouldn't expect this to be okay, right? Because when b goes out of scope at the end, when b is called here, this is gonna read from the mutable reference to y, which means that this read of y is not okay. And this might be more obvious if here we used y mutably. If we set y to 43, then now we have a mutable reference being used here but also a mutable reference being used here and things get weird. Okay, so what the compiler here is trying to guard us from is sort of a drop implementation that might use the value. And this is what's referred to as the drop check that not quite true, but bear with me. But basically the compiler wants to make sure that you don't have a drop implementation or when the value gets dropped, it needs to know whether or not it has to check any thing that the thing that gets dropped might contain. This is, that's a little convoluted of a way of saying it, but it's hopefully gonna become clear as we go. So if box doesn't do this, right? If it doesn't have a line like this, then we expect this to be fine. And in fact, if we tried to create a B using box new, then this code compiles. So our box is clearly somehow different from the box that is provided by the standard library, right? For some reason with the standard library box which does implement drop, right? It has to because it needs to free the underlying memory but somehow for the standard library box the compiler knows that that box won't sort of read the thing that it contains. It'll just drop it. It won't access it like with the unsafe bit that we wrote above and therefore it allows this code. But for our type, it doesn't know it. Nothing tells the compiler that this is okay. Specifically, nothing tells the compiler that the drop implementation for our box doesn't try to access the inner value. And to understand why that is, we need to get into a decent amount of like weird oddities in the language and the compiler. And specifically it relates to this thing called the drop check which is that when some variable goes out of scope or otherwise gets dropped, the compiler needs to know whether to consider that drop a use of anything that's contained inside the type. This is not related to NLL. So in particular, the compiler has a rule that says that if you have a type that's generic over some type parameters, in this case is generic over T, then the compiler will assume that dropping the thing will access a T. So dropping a box T, the compiler will assume that that will use a T if box T implements drop. So regardless of whether the drop implementation for that type actually accesses the T, the compiler will assume that it accesses a T. And in this case, that means that it will assume that the drop implementation accesses this mutable reference and therefore any intervening use of the target of the mutable reference is a conflicting use. And the drop implementation here is important because if we comment out the entire drop implementation, then now the compiler knows that there can't, the dropping a box T can't possibly access a T because there's no drop implementation for this type. So there's nowhere where it could do that use. And if we comment out the drop implementation, this code now compiles, right? Because the compiler goes, oh, down here you drop a B, but the box doesn't implement drop. Therefore dropping a box doesn't access the T. Therefore this mutable borrow, I can shorten to be just straight after you create the box, sort of after the last use rather than at the end of the scope and drop. And therefore this use at Y is valid. This does mean that implementing drop for a type is a breaking change as one example. If it is a generic type. If it's not generic, I don't think it matters. So implementing drop is actually backwards incompatible for another reason, which is you're not allowed to move out of the fields of a type that implements drop. So if a type has public fields and you try to move out of that field, then you're not allowed to do that if the type implements drop because the drop implementation needs to have a mutable reference to self, which is complete and has nothing moved out of it. It can't be partially moved. All right, so how do we fix this, right? Because we need the drop implementation for box. This has to be here because otherwise the memory would leak. But we also know that we're not accessing the inner value here. We can look at the implementation given that this is compiled, that this isn't here. We can look at this implementation and go, we know that we're not accessing the inner type T here. So we should be able to communicate that to the compiler. And there's actually no way to do this in the stable compiler today. And part of the reason for this is because we don't have a, we haven't figured out what the right mechanism is. But there is a temporary workaround. This is a feature that is sort of going to be permanently unstable because we're waiting for figuring out what the right solution should be. And that is something called a feature, the drop check I patch. And I guess I'm gonna have to override this instead of nightly. And do this, so Rust Analyzer picks it up. So the drop check I patch is a permanently unstable feature that as the name implies, lets you apply an I patch to drop check. So remember how I said that if you have a generic type, the drop check is gonna assume that if this type implements drop, then the drop will access a T. And what the drop check I patch does is it lets us sort of opt out of that part of the drop check. It lets us mask a given type parameter from the drop check. In particular, with this feature enabled, you can say unsafe imple, and then you can say may dangle, as sort of a precedent to any number of the type parameters. And what this tells the compiler is that even though box holds a T, and then those generic over T, I promise, hence the unsafe keyword here, I promise that the code inside of drop does not access the T. It might drop the T, we don't make a promise that it doesn't drop it, but it will not access the T. And once we add this, then now this code down here compiles. Right, so now just like it used to be possible with a box to have this code compile, now our code compiles as well, right? Because now the compiler understands that when B gets dropped down here, there's not actually a call to drop, right? It's just dropped at the end of the scope. When B gets dropped down here, it understands the promise we made, that we won't touch the inner type, we won't touch the mutable reference, and therefore it can shorten the borrow of the Y to start just up here, and therefore allow the use of Y here, the shared borrow, the immutable borrow here. So I'm gonna stop there for questions, and then I'm gonna go into why this code is still wrong. Access here includes creating any kind of reference to the T, right? Um, access here is use the T, which I guess you could say is create a reference to it, but I don't know that creating a reference is a problem. Actually creating immutable reference would be a problem, because mutable references are never allowed to alias, so even creating a mutable reference would be illegal, even if you don't use it. So with this feature, a trait can sometimes be safe and other times unsafe to implement, or is that a general thing? No, so this is one of the reasons why this is not stable, is because we don't have another instance of this, of having, implementing a trait sometimes be safe and sometimes be unsafe. You can sort of think of this as a, this unsafe keyword isn't for the drop trait, is for this attribute. You could sort of think like, maybe it should say unsafe here. So I don't, so certainly the drop check IPatch is not gonna land in its current form, and I think the idea is that we want some other more fundamental rearranging to make this work correctly. But no, it's not a general purpose mechanism. This is a different problem than what phantom data solves, yes. But you will see shortly why we still need phantom data. What if your container holds two Ts and only accesses one of them in drop? It seems strange to me that we're talking about access of T without being more specific about which inner T. So it doesn't actually matter. If you contain two Ts, they are the same type. You're right that they could both be mutable references and you'd then be tied to both of them. But at the same time, if you only have one T, then they must also have the same lifetime. And so therefore it seems kind of the right thing to say that if you access one, it's equivalent to accessing the other. If you really wanted to distinguish them, you would have two generic parameters. This is sort of similar to the argument around, you might want multiple lifetime parameters to allow them to have different variants. And it's sort of the same property here that if you want two fields to have to have different drop check properties, you might actually want to have different type parameters. You are creating a box T to the T which like mutable references may not alias. Yes, we are creating a box T which means that we're not alias, we're not allowed to alias the T but we're not aliasing the T, right? Nothing here ends up aliasing the T, mutably aliasing that is. The box from raw here, if that's what you're referring to, the box from raw here does create a, it creates a box to the T which means that you're not allowed to alias the T. Like at this point that the T is now sort of owned and then gets dropped and that's fine because this is a raw pointer and not a reference. So even though there is another pointer to it, it's not another reference to it. Is it possible to make T keep track of all the references and then drops them when accessed? I don't know what you mean. Although keep in mind here that the T here doesn't have to be a mutable reference, right? As I showed in the first part of main, the T can just be like an I32, like it doesn't need to be a mutable reference the way it is down here. And then the question here is that came up is also don't you still need to access the T in order to drop it? And the answer to that is no, you don't or rather it matters whether you access the T or whether you just drop the T. I think of something like a mutable reference to T. Dropping a mutable reference to T does nothing. It's a no op, right? Like in the compiler borrow checker, it is an operation but it doesn't issue any instructions. It doesn't actually dereference the mutable reference but accessing the mutable reference does. Okay, so now the question becomes what's broken about this? Because this is not quite sufficient in its current state. In particular, let me write a sort of malicious type. It's not a malicious type, it's just a type. Leave me alone. So I'm gonna make a new struct and I'm gonna call it, okay, I'm gonna call it top all. I wonder if this works, I think this will work. Okay, so top all, I think Unicode works in types, right? Top all means to touch in a region. And a top all is just gonna hold the T. Is Unicode types unstable now? Really? Oh man. All right, then we're gonna call it. Oh, that makes me so sad. Ah. Okay, then we're gonna call it Oisam, Oisam. Which sort of means like oopsie daisy. And we're gonna implement a drop for Oisam. Oh, I guess I could have just enabled the feature. But Oisam is fine. And what we're gonna do here is that this is gonna print line. It's gonna debug print the inner value on drop. So Oisam is going to, Oisam is going to access the inner T when it gets dropped. So nothing, there's nothing unsafe about this. It's just a normal type that happens to access the inner value. And notice, this is one case where the drop check sort of does the right thing, right? The drop check assumes that any generic parameter that or any type that you are generic over is one that you may access in drop. And therefore that it should assume as accessed on drop. And indeed, that assumption is correct here. So it's always safe to make that assumption. It just might be constraining like we saw with box. So why this is type causes a problem? Well, now down here, let me do a let mute Z. All right, it doesn't even have to be Z. It can just be Oisam. And it's going to hold a, actually no, here's what we're gonna do. We're gonna have a let mute Z as 42. We're gonna create a new box and it's gonna hold an Oisam of a mutable reference to Z. And then down here, we're gonna print out Z. So this compiles, but shouldn't. So here's an example where when we drop the box, we're gonna drop the Oisam. We're not gonna access the Oisam ourselves. Like the implementation of drop for box does not access the inner T, but it is drop it. And the drop implementation for Oisam does touch the inner T. So when we drop B here, that drop of B is gonna access this mutable reference. So it's sort of, this drop down here is sort of equivalent to doing this. But now there's a borrow of Z up here that is used down here. And there's an immutable access in between which it should be illegal by the borrower checker. And indeed, if you try this with the standard box, it will not compile. If we look at the compiler error, it says immutable borrow occurs here, immutable borrow occurs here, and the mutable borrow might be used here when B is dropped and runs the destructor for the type of box Oisam. So box clearly understands that this isn't okay, but our box does not because we use the I patch to say the T might dangle. And indeed, what's missing here is that the compiler, what we've told the compiler with this is that we won't access the T. But we haven't said anything about whether we're gonna drop T or not. And by adding this, the compiler just thinks that we don't do anything with T on drop, but that's not true. We do drop a T, but the compiler doesn't know that. And the reason the compiler doesn't know that is because if a type is generic over T and implements drop, it assumes that that drop will access the T, but it doesn't make any assumptions about dropping the T, but an access is sort of stronger than a drop. Like more things need to be true in order for it to be valid to access something as opposed to drop it. But there's nothing in here that says that we're gonna drop a T because we don't, this type doesn't hold a T. It doesn't, like if this said like, if it actually held a T, then the compiler could look at it and go, oh yeah, obviously this is gonna drop a T when it drops. But it just holds a raw pointer to T, which the compiler can't assume anything about. It doesn't know whether or not we drop a T. The only reason we drop a T is because of this unsafe bit here that the compiler doesn't necessarily know about. Or it doesn't know the semantics of this operation. So what we need to do is we need to tell the compiler that while we're not gonna access the T, we will drop a T. And this is where phantom data comes in. So phantom data is a type that can be generic over some other type, but doesn't contain that type. And it's the only type that really works that way. So if you have a phantom data i32, its size is zero. It doesn't hold an i32. It doesn't hold anything, in fact. But as far as the compiler thinks and the type system thinks, it holds an i32. And that's exactly what we need here. So here we're gonna have a T that's a phantom data of T. And why do we do that? I guess I'm gonna have to also have this be a phantom data. Oops, phantom data. And the reason we do this is because the borrow checker will again look at this. And when looking at any type that implements drop, it sort of has to ask itself two questions. It has to ask, does it act for each type parameter? Does it access that type parameter and does it drop that type parameter? The main angle tells it, it does not access the type parameter. The phantom data tells it, it does drop the type parameter. And in fact, now this code will not compile with our box either. And the reason it because we've now taught the compiler that when the box drops, we're not gonna access the reference. We're not gonna access the inner T, so this code is fine. But we are going to drop the inner type. And so you need to think of it as it tells the compiler, you need to go check its drop implementation to see that it does the right thing. All right, this is like for fairly deep down like a rabbit hole, but does this make sense? Does it make sense why we need this extra attribute, what it's for and why this is sufficient and necessary to make the compiler understand our type correctly? Without the phantom data T, the only uses of T will be moves or drops. No, that's not true. Without the phantom data T, the phantom data T is what tells the compiler that we may drop a T. In fact, I believe, and this is something that I don't know whether it's documented in the Nomecon. I saw there was some discussion in a PR actually, or an issue on the Nomecon, is that I don't think the phantom data is necessary unless you use made angle because a use is stronger than a drop. So the normal assumption about generic parameters is sufficient. What if Oisan didn't touch the data and drop? Would it compile? So if Oisan did not touch it, if we just did this, then no, it still will not compile. And the reason is because the compiler still makes the assumption that because Oisan is generic over T and implements drop, it has to assume that it might access the T. But we could do the same thing here and say unsaved simple made angle to say that we promise that this drop implementation also does not access the inner value. And then the code indeed compiles. Or if this type didn't implement drop, then the code also compiles just fine because dropping Oisan will drop the T, which is fine, but it does not access the inner T. Phantom data is a sort of special type. I don't think you can write it yourself. And the reason for that is if you try to write, I mean, we could try it ourselves. Like if you try to write a phantom like this, like a unit type, the compiler will complain that the type parameter is never used. So you can't actually write this type yourself. So I think phantom data, phantom data required some special compiler magic to allow that just for that type. Okay, so the question now is, are we done? Is this all we really needed? And sort of, this does get us everywhere we wanted to with the drop check itself, but there is one more bit here that's a little awkward. And that is, and this gets back to the previous video on subtyping invariance, our box is currently invariant in T. So what does this mean? Well, there's a lot to cover from the previous video, but here's what I'm gonna do. Let's say that I have a string new, and then I do, what's a good example of this? Oh, I don't know how to write a good example of this off the top of my head. But if I do, let B is box new, Z. And then I do B equals box new, hello world. If I now, sorry, I'm just writing this, and then I will explain it once it doesn't compile if I did that right. Yeah, okay, great. Maybe not. Damn it. This gets to work for silly other reasons. Let X, let, I'm running out of letters. ZZZ is gonna be a static stir. It's gonna be hello world. And then we're gonna have this take a ZCC. Great. Ooh, integer, that's not right. This compiles, check static. Sorry, let me, I'm gonna explain what I'm doing in a second. I just need to do some setup. Ooh. Interesting, why did that not work? Oh, actually, here's a better one. If I write a, ah, I think it's gonna be too hard to come up with a good example for this. So let me try to talk through it instead. Yeah, it's, no, the type can't be box static stir either because then it shouldn't compile even with box. Okay, so let me try to talk through it instead. As we talked about in the previous video on variance, anything that you have a mutable reference to is invariant in the inner type. And the reasons for that, we're not gonna go over again here because they're covered in that previous video in a lot of detail, but this means that in general, if you have, say, a box static T and you wanna pass it to a function that expects to receive like a box of a reference with a shorter lifetime than static, like a tick A, except it can't be generic over tick A. This is part of why it's hard to come up with a good example for it. Then you basically, the invariance means that you cannot treat this, oh, that's not at all what I meant to do, cannot treat this as this or some shorter lifetime. Even though normally, right, I guess this should say box as box, even though you can always treat a static stir as some shorter lifetime stir. And you can treat a box of this, like this is a box with an X as a box of this. So our box is sort of broken in this way. It doesn't behave the way that you would expect box to behave. And that's because of the invariance that gets introduced here. All right, I have someone give a code example in chat. Let's see if that demonstrates it. Let S string from high, or let's make it high because it should be Norwegian. Let box is box S, let box two is box static stir. Let box two is box static stir is box knee. High sum also Norwegian box equals box two. Oops, two. So what does this complain again? Ah, yeah, okay. This is a good example. You're right. Thanks, Ellis. So the problem we run into here is exactly this, that this box has a lifetime that's not static. It's a lifetime that's shorter than static. This variable has a lifetime that is static. So it's longer than box. Let's call them box one, box two, I guess. So box two is like strictly more useful. It has a longer lifetime. And so it should be fine to assign box two into box one. And indeed, if we made this box, box, box, new, new, then this compiles just fine. But if we use our box, I'm just making this easier to talk through. So you see that the example with box compiles just fine because box is covariant in its contained type T, but our type is not. So even though you have a box here with a short lifetime, you cannot overwrite it with a box with a longer lifetime because it's invariant in its inner type. And that invariance is because of the mutable reference over here. So the easiest way to fix this is we can make this const that does, and then we need to deal with the stuff down here. Realistically, what we'd probably use is instead say, bring in the non null type. So the non null type is sort of like a mutable, sort of like a mutable reference or mutable pointer rather, except that it is covariant. And because it's not null, it supports the niche optimization. So we can replace this with non null over here. This now has to be box from raw as mute. This has to be non null from this. I mean new and in fact, new unchecked because we know that box never creates a null pointer. We know that box never creates a null pointer. Safety box never creates a null pointer. And this has ref and this has mute. And new is new. Er, no, that's right. So now you see that this actually compiles. Now the example when using our box, our Norwegian box, also supports, is also covariant over the T. So it allows us to take this box with a contained type that's a subtype of the one we already have and assign it. Basically you can think of it as the compiler realizes that it's allowed to shorten the lifetime of this borrower of this type parameter when assigning over here. Nice. So now we've made it invariant. Sorry, covariant, which is nice. You might also have seen stuff like this. This also makes a type covariant, but this means that we are no longer subject to the drop check. So with this, you would now be allowed, it would open up this bug again. So you might wonder, well, why then, given that it opens up a bug, because this now compiles when it shouldn't, right? Why then would you ever see this? And you see this in types that do need to be covariant over the T, but they don't drop a T because they don't contain a T. The best example I know of of this is if you have something like a deserializer. So a deserializer, phantom, oops, phantom data T. If you have a deserializer, like a sturdy deserializer type or something, like a custom deserializer, then it doesn't contain a T. It's just, it's generic over the type that it's going to deserialize into. Like imagine a VEC deserializer, that deserializes into a VEC of T's. Then it doesn't contain a T, but the way that we've written it, if we just do the sort of the naive thing with phantom data, this is now gonna mean that the drop check gets applied to the deserializer, right? It's gonna make it so that dropping a deserializer T, the compiler is gonna check the, is gonna do the drop check for whatever that T is. So if someone constructed a deserializer of Oisan, of a mutable reference to say an I32, where remember Oisan will touch the inner type when you drop, this would not compile because the drop of the deserializer, the compiler's gonna assume is gonna touch the mutable reference that this is to, even though the deserializer doesn't actually contain an Oisan, it isn't gonna drop one. And so this is one example where you would use this, because now this is one way in which you can, you can be covariant over T, right? So this is the covariant over T, but you don't contain the T's, you don't trigger the drop check for your generic type. And the reason you don't do this, oops, the reason you don't do this is because this is contra variant, not covariant. So in general, this is what you want. Or if you wanted it to be invariant, you would do this. Whew. The empty iterator is another example. Yeah, if you have a, that's a good example actually. The empty iterator is generic over T and it implements iterator for empty. So we're gonna do this. And the type item is gonna be T. And it returns an option self item, but it always returns none, right? So this iterator never yields a T, it never contains a T, but it doesn't need to be generic over T. So we need to stick T inside of phantom data somewhere. And we want it to be covariant. And if we just did this, it would be covariant, but now we would trigger the drop check of T, which is not what we want. So we use this construct instead. Oh, interesting. The empty iterator in the standard library just uses T. So that means, okay, let's try to play with that a little. So if you have a iter empty, then I think this is going to, um, so if I say, let A is 42, then I say, I want to create an, ooh, let O is equal to sum of, it gets annoying to construct something that has the type that I want it to have. Oisan of mute of A, and then say O is empty, empty, does it have a new probably? Dot next. Or I guess I need to create empty, is it default maybe? Probably default. I T dot next and then print line A. Let's see if I can make this trigger. I can just say, sign twice. That's fine. Interesting. Oh yeah, you're right. So normally if empty implemented drop, then this code would not compile even though it should because the iterator, and so this is because in the standard library, empty is defined as struct empty, struct empty phantom data T. So this means that dropping an empty T is considered as dropping a T. So the empty type that we have here is going to be considered as holding, you can sort of think of it as going to be an oisan of the at mute of A. That's sort of the type that it's going to end up having because of the, because of this, right, because O has been given that type. And so when the empty gets dropped at the end of the scope here, right, that's going to consider it as dropping an oisan, which because it's a phantom data T and not a phantom data this, that's going to be considered as dropping an oisan, which in turn is going to touch the inner value, which is a mutable borrow of A. But in fact, it should be even without an implementation of drop. But because there's no implementation of drop for empty, the drop check never triggers. Although it should still be considered dropping, I think it should still be considered as dropping one of these. So something here is not right. Like if I drop IT here, interesting. I wonder why that works. I don't think that's right. I think that this should fail. So if this is actually the definition of empty, this should fail because dropping the iterator down here should drop the empty T, which should drop this, which the compiler should assume drops a T, even if empty doesn't implement drop, this should drop a T, which means that you consider the drop implementation for oisan, which does touch the inner type. So interesting. Yeah, I don't know why this doesn't compile or why this does compile rather. Given this implementation of empty, let's go look at empty. Itter empty. So if I look at the nightly version, interesting, hum, hum, hum, hum, hum. Yeah, so that is the implementation. So why then is dropping an empty not considered dropping a T? Hmm, hum, hum, hum, hum, hum. So let's see, this will, we're gonna drop IT down here. This is going to be just a double check. This is gonna be an oisan of a mutable reference to 932. If I drop the O here, interesting. I honestly don't know why this compiles given this definition. I mean, I think you're right. I think it's because there's no implementation of drop for empty, but I don't see why that should matter because the drop check should still consider this as dropping a T because it does. According to this definition, this type drops a T. I think what probably happens here is, oh, does empty implement copy? Good catch, it probably does. No, no, it implements clone, but not copy. The only thing I can think of is that the lifetime that gets assigned here gets shortened. Yeah, that's almost certainly what happens. That I think what happens here is actually that the lifetime, so this gets a lifetime of, let's call it tick A. And I think what happens is that it just considers tick A ends here. I think that's why. Like there's nothing stopping the compiler from saying that tick A is just the duration from here to here. Yeah, but then dropping the iterator afterwards should still be considered illegal. Oh, I'm stupid. No, no, no, no, no, no. I mean, we could try it.next, but I don't, I don't think that's right either. I don't think that should make a difference. Because the call to drop it here is equivalent. Can we see the drop handler run? Yes, you can, let me second here. So let me just copy, let me get rid of these. 4242, so why does that get printed twice? It get printed once when this gets dropped, and then it gets printed again when this prints. Which is right, because this, it next returns none. It doesn't actually return a noise on. Are you accidentally using your own empty? No, I call that empty iterator. So I'm not using that one. Huh, the drop of, so the explicit drop of O is necessary because otherwise it thinks that O is dropped down here, and then the O, just O itself conflicts with the A. Oh yeah, I guess we could do, I guess we could do this. Default for empty iterator, just to like see whether there's some magic going on. I feel like it's weird for there to be magic here. Like looking at the source, there's no magic, but who knows? So if we do this with, instead of empty, we do empty iterator, and we do this. So it compiles with mine as well. I'm just like, I'm not sure why. This presumably definitely compiles, should be strictly more general. Yeah, I don't have a good answer for you. I don't have a good answer for you. I feel like this code should not compile, and I can't quite explain why it does. Could you try wrapping all this into function? I supposed to use the same lifetime for it and O. We could try that. I don't think it's gonna make a difference. Let's say we generic over tick a, and it's gonna take an iterator, which is gonna be an empty of oisan of tick a mute. It might only need to take one argument, and it does, next. Oh, what did I do? I did something wrong. Oh, this should be fm. And what, am I blind? Empty oisan. Expected type found, oh yeah, you're right. Okay, so actually what happens here, yeah, yeah, yeah, yeah. It's because, oh yeah, right. I needed a type after mute. Okay, so the reason is because a tick a mute t is invariant in t, but covariant in tick a. So what happens is that even though this has the lifetime, so the lifetime tick a gets shortened to this lifetime. No, but even then dropping the, so it's true that empty is allowed to shorten the lifetime of this into one that ends here. But then when we use this here, this should be illegal because the iterator contains a value with a borrow that's already ended. And if the borrow is extended to the end of the scope, then this use should be illegal. The it may return a static. No, it has to return a t. Oh, you're saying this gets extended to be a tick static? Oh, I see. Because it never yields one, it's allowed to claim that it does. Yeah, that's awkward. Yeah, you're right. I wonder if we can, yeah, and if I create a function it's gonna bind the, okay. So, ah, all right. So what actually happens here is the, as you can see from this annotation, like the empty gets created as a thing that has a static mutable reference because it never, yeah. Because it never holds a t, like it can produce a, wait, that's not why? Why not? No, this is definitely it. It is allowed to produce a mutable reference of any length because it doesn't actually produce a reference. So it's able to produce the reference of any length. Right, so the O we get here, I bet you this is a option oisan tick static. Is that right? No, this won't be right. Yeah, that's not a static. So here O has a lifetime that's associated with the A here, but then when we reassign O here, its lifetime gets shortened because it can be. Like we're overwriting O anyway. Yeah, so the lifetime of O at this point has been shortened to be here, but empty is still sort of allowed to claim that it can produce mutable references of lifetime static because it never actually has to produce them. And therefore when we call it next down here, it's fine because the empty is not actually tied to the lifetime of this because the static mutable reference that it returns here gets shortened to be the lifetime of the mutable reference here. So think of this as like this returns a blah, blah, blah, tick static mute, yeah, tick static mute of I32, right? But the O here is of type tick A, mute I32 and the compiler because mutable references are covariant in their lifetimes, what actually happens is that the tick static that the iterator returns here gets shortened to the tick A over here. So everything still works out as long as empty has a static lifetime. And therefore when we drop it over here, we're dropping an Oisan with a static lifetime, which is not a use of A because it's not tied to the lifetime of A, it's just tied to the static lifetime. That's why this works out. So I still think that the phantom data of empty should be this, although someone mentioned it's because of const, which makes a lot of sense. This probably, you can't probably constify this is my guess, but it's just hard to come up with an example where this is a problem where it triggers the drop check. Yeah, yeah. Okay, so at least now we understand why this compiles, even though it is somewhat convoluted, but the reason it doesn't trigger the, so it does trigger the drop check, like this still considers itself as dropping a T, so it considers itself a drop of this and dropping Oisan is gonna use one of these, but because this is not tied to the lifetime of the borrow of A, using this mutable reference, using, it's not actually gonna use anything because it's none, but using this mutable reference is not a use of A and therefore does not conflict with the use of A down here. And so the drop check does get triggered, just it doesn't consider it a use of the thing that's borrowed. I'm sure you could construct one that does where this would be a problem, but it's probably weird enough that you don't run into it in practice. Yeah, I wonder if this could be, this could probably be this, although if you made it this, it wouldn't be send and sync, so it's a little awkward. All right, great. So even though this was more about variance than the drop check, it was related to the drop check, so I don't feel too bad about going through it. And I feel I understand it better now too. I'm glad we managed to work it out. I recommend you go back and read this again and see if it makes sense the second time when I upload the video. All right, so now we have, now we have our own box type, we have Norwegian box type. And hopefully, let me get rid of the stuff in between here. Okay, so just to sort of do a recap, we have a box T, it contains a non-null T because well, we know it's non-null because it comes from box and we want the niche optimization, but more importantly, because we want it to be covariant in T, which a mutable reference, a mutable pointer to T would not be. We include a phantom data T because we do drop a T when box T gets dropped. And we want to signal that to the compiler so that it will check the drop implementations of T when box is dropped. We have an implementation of drop for box, which we need to because we need to drop the box and call the destructor of the inner T, but we annotate the T as may dangle because we sort of were promising by virtue of the unsafe keyword here that the implementation of drop that we have will not access the T, it won't use the T or dereference the T, if you will. Not necessarily dereference, but use the T inside of drop. And therefore the compiler doesn't need to check the, it doesn't need to assume the T is used unless the drop of T implies the T is used. So hopefully that now explains all of the weird, all of the weird bits of syntax that we're using in here. And keep in mind too, that this is probably not gonna stay in the language long-term. This is like a temporary fix for this problem that is nightly only and will remain so basically forever, I think. Whew, all right. Hopefully that was interesting. I think we're gonna end it there. There's not really that much more to say about the drop check. It's more just sort of talking through why it matters and why the different bits and pieces are needed and how they fit together. So I think, I think we're in a good place. I just wanna push back. There was a comment in chat saying this is complicated for no reason. I don't think that's true. As we walked through here, right? When we started out with sort of the naive definition, then the reason why we needed to add this complexity was because without it, it's totally safe to not have this made angle, but it restricts the use of the type and it restricts it for a good reason. It restricts it because the compiler doesn't know what you do in your drop implementation. And so you need to tell it that you don't do certain things and by telling it that, you allow your type to be used in more ways. And then we sort of overcompensated to make it so that this makes the type too flexible, more flexible than is safe and so we need to rein it in with this. It might be that there's some other way to like expose this pattern and that's part of the reason why this is not stable is because clearly this is sort of a sort of roundabout way of getting at it, but there are some fundamental problems involved for why we need to tell the compiler this in some way. And this is the only way we currently have of telling the compiler, but if you can think of a better one, then I think that the Rust standard library maintainers would love to hear one because it's not entirely clear what it should be and there have been multiple sort of draft designs. All right, I think that's the end of what I wanted to talk about. I don't know quite when the next stream will be. I'm guessing in like three weeks or something and I really want the next one to be like a live coding one. I just still haven't quite decided what we're gonna code. I'm still working through that and trying to figure it out. I've had some people reach out to me to sort of try to give some ideas and that's great. I love just getting the input. I can't promise I will do any of them. I sort of need to find something that just speaks to me because if we're gonna do like three, six hour sessions of coding, it needs to be something I really care about and so that's part of the challenge, but I think it's gonna be fun. All right, thank you for joining everyone. Hope this was interesting and not too confusing and I will see you all next time. So long for well. Auf Wiedersehen, goodbye. Ah, no, I don't wanna excite it. I want to stop recording.