 Hello everyone, welcome back to I guess or welcome for the first time I suppose to the first episode of the unsafe chronicles. I feel like maybe I should be wearing some like scary outfit or something, but but the reality is that unsafe code has this this reputation in the rust world of being like scary and hard and somewhat unknowable and To an extent like that's a that's an earned reputation, right? Like the number of ways you can shoot yourself in the foot with unsafe There are a lot of them and sometimes it is really hard to get right But but I think being scared of it is not the right approach The right approach is to sort of look at what can go wrong and try to learn for it from it So that you don't make the same mistakes in the future to me unsafe code has always sort of been like If you're in unsafe world, you're just writing C So if most of your code is not unsafe, you're still better off than what you would have been otherwise Regardless though, this this series is is focusing on a subset of that problem, which is just As the name implies talking about stories with unsafe like cases where I guess Starting off like cases where I run into oddities with unsafe where I didn't know that something was okay And then I learned and then I fixed the problem And I'm hoping that just by sort of surfacing a lot of these stories about where where you might go wrong with unsafe or Where there might be tricky? Tricky scenarios to get right just talking about those cases and what the sort of underlying principles are might be Helpful for other people who are trying to write on safe code and do so safely And in particular for this for this first first Chronicle What we're gonna be looking at is our crate called left right and left right is a crate that I built in the my previous stream, which is sort of a longer live coding stream I'll link it in the video somewhere around here. I guess and It at its core It's a concurrency primitive that allows you to have really fast concurrent read access to some backing data structure And it does this with very little synchronization between the readers Which means that it scales really well as you as you add more threads that end up reading from the data structure Where left right gets a little complicated or one of the places it gets complicated is that it it does this by keeping two copies of the the backing data structure if it's a map for example you would have a left map and a right map and The data structure sort of keeps these in sync and then make sure that the Readers only read from one and the writers only write to the other And this is how it ensures that the reads can be fast because they don't have to do any synchronization They can just read from the the read half if you will and Left right sort of realizes that duplicating all of your data is probably not okay So it wants to pull this trick of aliasing where you have Think of it as instead of having all your values be stored in both the left and right data structure You have both of them store a sort of shallow copy if you will or an alias of the true value that's stored somewhere else So the the trivial example here right as if you have something that's on the heap instead of having one heap allocation for the left map and one or for the left copy and One heap allocation for the right copy You just have one heap allocation. You have the left and the right both point to that same heap allocation and sort of conceptually this works because Left right only ever gives out Read access to the values that are stored in the copies like the moment after you've inserted it You can no longer get a mutable reference to it anymore Except when the value is actually dropped And and so this should be fine like aliasing should be fine because you only have readers and it should be fine to have shared references To to a value like multiple shared references. We know from the Rust Baro checker is okay and What we're gonna be looking at is is specifically that Mechanism for aliasing because there there's sort of necessarily has to be some unsafe code if you think of this as something like If you have a box what you're now going to be doing is you're gonna construct two boxes that point to the same underlying allocation Which means you need to use the actually let me pull this up here box So the box straight has a from raw method Or function I guess is a freestanding function that takes a raw pointer and gives you a box T And similarly there's a to raw or into raw that takes a box T and gives you a mutable pointer to that T And going from a box to a pointer is a safe operation because it gives you a raw pointer and dereferencing a raw pointer And rust is unsafe anyway, but going from a raw pointer into a box is unsafe There there are many reasons for this The primary one being that you need to guarantee that this pointer actually is the box that it's like a pointer to a heap allocated thing Because box when you drop it is gonna free the underlying object, right? So So far so good, right there is a mechanism for for if you have one box to sort of construct a second box The points to the same underlying heap allocation and this was the trick that That left right and the and its predecessor EV map Used for the longest time it would just create two boxes to the same heap allocation Or if you had a VEC it would create two VEC elements that both pointed to the same underlying data structure on the heap and that seemed Fine enough and I I thought that this was entirely safe because I was Remember the data structure guarantees that it only ever gives out shared references to the backing memory So what's the problem? well, it turns out that there are two problems and Both of these actually came up in the previous stream when we were working through how to factor out the concurrency primitive in EV map and Some of this is described in this PR that I started working on shortly after that stream But I'm gonna try to summarize it here With some actual code. So let's do a cargo new lib Aliasing Okay, so the basic problem we have is that we have a let's say Struct aliased of T It stores internally a box T and Then what we're gonna be giving out is aliased T, right? We're gonna have like We're gonna have I guess how we're gonna describe this like We're gonna have a Here may actually write out the code that's gonna help Let's imagine that we have a left which is an aliased where the real is a box new Let's just use a number for now Aliased this and then we're gonna construct an alias just by doing Real box from raw box into raw Left dot Real This won't actually compile and I'll talk about that in a second to but sort of conceptually. This is what's going on Right, we have a left and right. They both contain a box, but the box is the same underlying pointer And as long as we only ever use left dot real and Oops and Right dot real like if we only ever use those behind shared references Then why is this not okay? And The the first issue we're gonna talk about here is about aliasing rules. So box I don't know. Let me pull up the documentation for box Why is my scroll bar look weird it's weird Where is this so this might actually not be Jocumented So the document change for box talks about a bunch of the sort of invariance about the boxes Right, so it says that a boxy value will always be fully aligned a box will always be a non-null pointer and If we look at From raw Which is the unsafe function we're using It says that for safety this function is unsafe because improper use mainly to memory problems, okay That's fine, but it doesn't really say very much about what isn't isn't okay It just sort of assumes that you know what the the correct thing is It does say for this to be safe the memory must have been allocated in accordance with the memory layout used by box Which is sort of trivially guaranteed for us because we know that we constructed this from a box in the first place But it turns out that In general in rust when you have a box T What this compiles into in like LVM like in the underlying code generation pipeline As it turns into or it uses an attribute called no alias or no alias Forget whether I think it's no alias and What this means is that rust is telling the underlying code generation tool that The pointer that the box contains is not aliased anywhere else in the code and Normally, that's true, right if you have a box T Then you own that box in you know that no one else has the actual box So if you're operating on this box, you know that that pointer itself is not aliased It's it doesn't exist anywhere else in the system And you might wonder well, why does this matter? It matters because LVM gets to optimize your code under that assumption the the silliest example of this might be Let's say that you Let's let's say that we didn't have right. We didn't do anything funky with this. We just have like a a normal like box somewhere Because the compiler knows that box is Is not aliased it doesn't exist anywhere else and It sees that five fits inside a U size What's to stop the compiler from just not doing the heap allocation, right? Just have Five be stored in the the field where the pointer normally would have been stored I don't think LVM does this currently, but if we know that there are no aliases of this pointer We know that no one else has the actual pointer Then why do we have to have the actual pointer if the value fits in the pointer itself anytime? We move the pointer somewhere else. We can just copy the value that's stored in the pointer field instead If we give out pointer it like references to the box Those are just references to the actual U size. So everything sort of just works, right? And the compiler might not do this optimization today But we have no guarantee that it won't do in the near future Because we are telling the compiler when we construct a box that this pointer is not aliased But if we write code like this where we are aliasing the box Rust doesn't know that we are aliasing this box So it will continue under the assumption that box the box is not aliased because the rules for box is that it's not aliased and Therefore let's imagine that it did the same optimization for the box here, right? So it just didn't do the heap allocation here and instead just stuck the value five into the pointer field of box and Then when we did this aliasing it would basically just copy the five into this other pointer But now let's say that we modified this value Right, we modified the value that's pointed to by left In theory and we do this sort of Guaranteeing that no one is using right yet like right is right someone has an alias pointer But they're not using it for anything. So we're allowed to mutate it. Let's assume that that's the case Well, now let's say we set that value to 10 if this optimization has been applied It's just gonna change the value inside of the box pointer to be 10 because it's using this optimization but anyone who reads from the other alias is not gonna see that change because The optimization has removed the pointer. There's no longer a shared thing They're pointing to that the the the change to left actually mutated They just mutate their own local copies. You can think of this as box sort of becoming copy under this optimization right and so now this is just gonna read lead to completely weird behavior where Changing one side doesn't change the other. It's true that we only ever access them through These these shared references, maybe you think everything was okay because we can't mutate anything anyway but then you have types like unsafe cell or mutex or I Guess ref cell technically right or or cell for that matter That give you interior mutability. They let you mutate a type with a shared reference Well, what happens if left or right contains one of these like let's say that this contained a unsafe Or a cell of u-size the optimization can still happen, but now Changing the left is not going to reveal that to things in the right. So basically by aliasing box We've we've invalidated an optimization that the compiler still thinks is legal and again this particular Optimization isn't really the point. I don't know that this is an optimization that even makes sense or one that LVM does it probably does not but it's more that because Because of the aliasing rules of box, which is you're not allowed to alias a box The compiler tells LVM the doing any kind of optimization like this is okay It's telling LVM that this pointer is Not aliased and therefore you can make optimizations accordingly And so it's just not okay to alias a box Now this is a problem for us, right because the whole point of left right was to use aliasing to share the underlying values and and so I was trying for a while to try to figure out what like how to fix this problem because Ultimately if we can't alias a box, we just like can't do this trick We just can't share values between left and right because you can imagine that you have some data structure that internally contains a box somewhere And then this optimization would become invalid again Isn't no alias still fine as long as they're all The no alias pointers are only used for reading Which is how reference T can be no alias knows That's the point that even if we only use the box through shared references on both sides This is still not okay because this optimization again If if what the box contained was a cell of u-size the optimization can still happen But now if you mutate through one, right? You're doing a completely safe operation Which is to mutate a cell through a shared reference, but it doesn't but that change doesn't actually propagate to the other side Which that in itself is not unsigned behavior Well, and I don't know what unsigned behavior this might trigger But that's sort of not the point and this is something that comes up a lot in discussions of unsafe, which is You don't know what optimizations might happen in the future, right? So you need to code against what the what the guarantees are and what the requirements are of the underlying types So if box says it can't be aliased even if there's no current problem with aliasing a box you should still avoid it because There might be an optimization in the future that then randomly breaks your code Now as we saw from the documentation on box there It's not actually documented anywhere that you're not allowed to alias one. It's just sort of kind of implied And so after the discussion on the previous stream someone filed an issue or how Someone followed an issue against the repository that that I was working on for the stream saying that like basically Ralph Jung who's one of the one of the big people working on like safety and rust and how to do things like formal verification of the safety of rust programs and just figuring out what the spec for unsafety even is Pointed out that like you're not allowed to alias box at least that was that was his opinion And I think he's probably right about that that the rust currently assumes that that is the case And this ultimately ended up with an open issue in the unsafe code guidelines working group So this working group is working on basically trying to codify What the what the actual requirements of unsafe rust code is what you need to guarantee in order for your code to be sound and really what came out of this discussion was Box probably should have the requirement that the value that the pointer is not aliased But it doesn't currently specify this anywhere Like this is not a current part of the requirements of box The question is whether it should be and so I recommend you read through this discussion It's gotten pretty long if if you're curious about sort of the trade-offs that are involved here and and the arguments for Why should aliasing for aliasing a box be okay? Why should aliasing a box not be okay? and where that discussion sort of ended up was that We probably want box T to not be aliasable, but we probably also want some way to override that behavior Like if you do want to aliase a box There has to be some way to do it that doesn't violate the the sort of soundness requirements of rust code and what we ended up with there is Rust has this type called Maybe uninit So maybe uninit of T Is is a wrapper type this wrapper type is This wrapper type is wrapper transparent So that is it has the same memory layout as whatever the underlying type T is and maybe on in it is interesting because you know it Doesn't require that the sort of bits inside the bits that make up the T actually make a valid T the the most simple example of this is in rust if you have a boolean this the the sort of validity of Boolean says that you can it the bit pattern for a boolean can only be zero or one It's not allowed to be anything else if you have some bite whose value is like 255 that is not a valid boolean And there are a bunch of other types like this like if you have a if you have a box The bit pattern of all zeros is not a valid bit part pattern for a box because the pointer is not allowed to be null right so In general rust always requires its values to be valid and maybe uninit is a way to tell the compiler that The bit pattern for this type might not be valid yet And this basically forces rust the disable a bunch of optimizations because it can't assume the same things that normally assumes about the inner type and then on maybe uninit On maybe uninit There is a so so for example if you if you try to create a T by just zeroing a bunch of bits using mem zeroed Like that's not a valid reference for example because references have to be non-null But if you stick them inside a mem uninit, then you're fine because rust doesn't assume assume that the that this is actually a valid T yet And then maybe uninit has a method This documentation is great. By the way, you should read it if you're curious It has a method called are you Assume in it And assume in it consumes the maybe uninit and gives you the inner T And this is unsafe because when you call this function You are telling the compiler now the bit pattern of the inner type is valid And so now you can start to assume have all the normal assumptions you have about that type Like for example for a bull you can now assume that it's zero one if it's a pointer if it's a reference You can assume that it's a valid reference. It's a live reference. It hasn't been dropped. It's aligned all of those good things The reason this comes up as as being relevant for a box case is that if we stick a box Inside a maybe uninit. We're disabling the optimization of passing things like no alias, right? and And this means that now we can alias the box We can have a maybe uninit box T and then an alias of that a maybe uninit with an alias of that same box And the compiler won't emit no alias because it's not allowed to assume anything about the inner bit pattern about the inner type And then what we can use is we can use the the The as mute pointer method on the maybe uninit or the as pointer method to get a Reference to the inner T which we can then sort of dereference and go the whole way because we know that while it while Rust assumes that you're not allowed to alias a box If we cast it into a reference into the inner T We know that that is a valid reference right so To try to rephrase that if we have a actually maybe I should write it down. So if we have a maybe uninit box of T Right rust assumes nothing about the validity of this box But if we now call dot as pointer, right? So that gives us this gives us a a star const box T right and then we do a This on that ah Right that's gonna give us a reference to T and This reference to T matches all the validity requirements of a reference to T Right specifically it has to be a valid pointer, which it is because we know that the the value still lives It's aligned. We know that that is true because a box creates an aligned value It hasn't been dropped because we haven't dropped the target value and Crucially a reference to T does not have any requirements about it not being aliased because shared reference or references are always aliased And so by doing this transformation We're now able to alias the box T and still get out shared references Okay, that was a lot of technical detail So so I'm gonna take some questions on how this works before we talk about a related problem around aliasing boxes But There's a lot of complexity down pack here, so please like ask questions about What why this is or is not okay? Any part that was unclear and I'll try to go into more detail. This is from pretty hairy stuff And it's hard to talk through too because there's so many like subtle interactions and a lot of new types probably for many of you And and it's also hard to It's hard to explain the problem concisely because we're just working with this sort of very basic example here But the the basic summary of the the problem and the solution is that you're not allowed to have two box T's That point to the same T And this is because box T requires that it is not aliased that is a validity requirement of box T But we can use a maybe on in it because maybe on in it Relaxes the in the the requirements on the inner type it for example removes the requirement that the type is valid And that makes things okay What are the aliasing rules for manually drop T that's a great question that we're gonna get to next Does this fix the issue with nested boxes being alias because we will sometimes reveal both shared references That is a good question so so the question if I understand you correctly is what if we have a type like box Box box box T right or we have some like wrapper type That has a box T or something right then what we're doing is we're constructing a my maybe on in it of that type Which we're then going to turn into a reference to this type and the question is is this okay because now we're this box is still aliased right because we were aliasing this type And the answer is that yes, this is okay And the reason this is okay is because I'm allowed to write this code Right. This is totally valid rust code. So Rust doesn't assume it. It's not that rust Doesn't allow you to have multiple pointers to a box multiple references to a box It's that it doesn't allow you to have multiple owned boxes that are aliased So this is fine, which is also why this is fine Because here that the shared reference part is what means is is what sets the The sort of aliasing rules for this value if you tried to access this box through the shared reference Rust knows that it can't give no alias because you're already going through a shared reference Where this is not the case if you have like we had above right if you had left and right that are actual aliases of the Box, then if you do a left dot alias dot Like Something that accesses the inner value, right? Let's say you did these concurrently as far as rust is concerned This is not going through a shared reference and neither is this It's just accessing a box that you own and therefore it does give the no alias attribute That makes sense Um Can you unpack the types of ampersand star star more dear of the pointer and getting a reference? Yeah, so the the transformation here is from maybe on in it un in knit box T Goes to a star const of box T goes to a star const of T, so Then we do so just to give like the methods we call this is the first so The as pointer gives you this the star gives you this the second star gives you this and the reference gives you reference T Is the progression and We can't just do a reference here because if we did a reference here You would end up with this which is not what you wanted So you need the star and then the reference which ends up being a no op right this ends up just being a cast And you could do it as a cast instead if you wanted to Why is it called maybe on in it it's called maybe on in it because Maybe on in it is primarily used for cases where the inner type is like just zeroed bits For example, so the value is not initialized. It just so happens that It's sort of poorly named right because really it's like maybe valid would maybe be a better word for it because It's not only that it hasn't been initialized. It can also be there's some other reason why it isn't valid yet Or at the moment Why is box no alias in the first place box is no alias because of optimizations I'm sure there are other cases where no alias allows you to do optimizations But I'm not sure I think this is Part of the reason why this is still an open issue in the unsafe code guidelines group is because it's not entirely clear whether Box itself should be no alias. So it makes a lot of sense for muti to be no alias right muti is no alias because What's the the canonical example here is the one that's given in in the Nomecon See if I can find this alias saying Yeah, so this is a let me zoom in a little maybe so the Nomecon is great for anything that has to do with unsafe And this function is kind of neat. So it takes an input and an output and The output is a mutable reference in rust because and this is specifically because of the no alias optimization Lvm the code generation tool knows that Input and output cannot point to the same value and it knows this because output is no alias So it knows that Even though input is a reference it cannot be a reference to whatever output is point to because the output pointer Or the output reference is not alias and that lets it do a Neat optimization here where it only needs to read the input once instead of twice Which in turn means that it can do some other optimization So no alias does enable some cool optimizations for Mutable references to T whether it also enables cool optimization for things like box T is a little unclear so far But that's what that discussion is about Why would you need to alias a box so the reason you didn't need to alias a box is so that the two the left and right copy Both share the same underlying data What's the advantage of using maybe on it box T Over-turning the box T into muti and just aliasing that one The biggest advantage is actually one of Generality so it's true that any time where you have a box T that you want to alias What you can do is you can turn it into a star muti and then you just alias the star muti But the more general case is you have some type X And you want to alias X. This is the case we run into with the left right map where X is a user defined type That may be a box, but it might also be a vector, right? And if if someone gives you a vecti then the correct way to aliases is like U size U size star muti right That's a real way to alias a vector so What we want is some way to do aliasing in general Rather than how to alias a particular type because left right is generic over whatever the backing type is And maybe on it, you know, let's just do that because we can alias box T by doing maybe on in it box T and We can alias vecti by doing maybe on in it vecti Right, so it's a general purpose mechanism Russ doesn't give no alias to muti. I think it's currently that optimization is disabled at the moment But it is an optimization that they would like to have enabled When can you assume a maybe on in it is safe It is valid you mean so The maybe on in it assume in it method, which is the one that goes from a maybe on in a T to an actual T Which is unsafe the safety requirement is there is basically you need to know that the bit pattern that stored inside the maybe on in it is Now a valid representation for T Sorry, so so for example if you have a maybe on in it box T in order to get the box T You would have to ensure that it now meets all the validity requirement of box T Which would be things like the box is not aliased And also things like the pointer is not null the pointer is aligned the pointer points to a heap allocation That was originally allocated by box basically every every requirement for the backing type Is there a way to disable those optimizations and in the example of the double fetch actually fetch the value two times So the way you disable the optimizations is to Well In general you should not assume that you can turn these optimizations off Even with optimization level zero Some of this might leak through because these are These are optimizations about the compiled code that may or may not Result in optimizations. Remember the code generation tool is always Converting your code into assembly and it might choose to do so in a different way if it knows that something is not aliased In some sense maybe on in it here is one way in which we disable that optimization And and if you're thinking of disable these optimizations, you should think really carefully through why that is Keep in mind that generally rusts Rules around borrow checking are for a good reason And if you're choosing to override those What's the reason if you're trying to do things like mutate through a shared reference then use the unsafe cell type Because that's what it's for And one of the reasons you will specifically want to use the unsafe cell type rather than doing pointer Like one thing that's just never okay in rust is to if you have a ref tea to cast that into a mutable reference There are all sorts of ways for you to do this like with a transmute you can make this happen But you should never do it because even if you manage to do it It is not safe and one of the reasons is things like no alias The only safe way to go from a shared reference to a mutable reference is using the unsafe cell wrapper type And the reason for that is the unsafe cell wrapper type basically The compiler knows about this particular type and knows not to emit things like no alias for it Think of it as this is the way that you disable those optimizations Is there a difference when you say shared reference and alias? Yes, so This is a shared reference to a t Um Whereas if I have a box t whose value is a pointer like this and I have a a second box t I have a second box t that has the same pointer value. Then this is this is alias So a shared reference to t is always an Always alias is t right because you can have multiple shared references And those are aliased pointers to the t but you can have aliases that are not shared references So they're not quite the same Does maybe in it do more than basically just transmuting No, maybe on in it is really just a transmute, but it's a type that Rust knows about and knows to for example, not assume aliasing about It it mostly just changes the flags that it passes to lvm um Why not use raw pointers instead of an alias box? The reason is the the one that I talked about before which is we want a mechanism that works for any user defined type and not Just specifically for box What does maybe on in it do for pointers if there's no null maybe on in it doesn't Deal with pointers maybe on in it is just you can think of it as whatever type is the inner type So if it holds a box t Then the maybe on in it is a box t but with all the but but that you can't directly use without unsafe code Because it doesn't have the same validity requirements So maybe on in it is just a transparent type that makes it unsafe to access the underlying type Okay, I think we've we've now talked through most of the stuff around Around aliasing but there is one point that comes up and which did also come up in in chat a little bit which is One trick that left right was trying to pull was it didn't actually star a box Because what would happen is if you have if you have this case, right? You have a left and a right to both point to the same heap allocation Then if we drop left And then we drop right This is a double free Right the drop of left is going to drop the box and that's going to do free the heap allocation And then we drop the right it's going to drop its box We're just going to try to free that heap allocation a second time Clearly not okay So what left right did was it used a type called manually drop in the standard library Um and manually drop let me pull it up here Uh, so manually drop is also a transparent wrapper type Sort of similar to maybe on in it, but not really the same um manually drop is a Rapper that just disables dropping. That's all it really does. Um, it doesn't require any unsafe to access the inner t Maybe arguably it should but it it does not Manually drop just is a t but will never drop that t And there are a bunch of cases you might think of that where where this is useful And this is one of them right where now if you drop left, it's not going to drop the t because manually drop prevents the drop But now if you drop the right that's also not going to drop the box t And so now you never end up dropping but at least that's sound It's going to leak a bunch of memory, but it is sound behavior Um, but if you look at manually drop you'll see I wish this was the default expansion um That manually drop implements deref and deref mute So if you have a manually drop t you can easily just get a shared reference to that t or an exclusive reference to that t Just by using like ampersand mute or or stars to get at the inner or dot for that matter to get at the inner type There's also an into inner which just removes the manually drop wrapper and gives you the inner t and you'll notice that that function is not unsafe right that function just gives you the t and That's all there is to it So in other words manually drop just disables drop. It doesn't do anything else It doesn't prevent it doesn't enable you to do aliasing or anything like that So so far it's so good that looks nice Um, but we do obviously sometimes need to drop the box t No, that this would the the code I've written so far like if you have manually drop would never drop the box t Like when you drop left nothing gets dropped or the box doesn't get dropped when you're up right The box doesn't get dropped and when the scope ends the box doesn't get dropped the box is just leaked Uh, and this is obviously not okay The way that this the way that I did this in Or the way you're supposed to do this to start with that part is that you're supposed to like Drop Something like left right and then when you know that you actually want to drop the inner type In our case because we know that this is the last copy Then what you do is you do right into inner Uh, and remember into inner gives you the the t that's inside a manually drop And then that you drop Or just let go out of scope like you could just do this as well and it would work just fine Um, but basically you sort of tail manually drop. Okay. Now. I do want to drop the t Uh, and at that point you would only end up with a single free Okay, so far so good. Um, where we run into issues is that imagine that, um I don't actually have a manually drop box t What I have or right, I do have a manual drop box t but what I have is actually like, uh Let hash map equals Hash map new Insert I don't know zero Right, uh, I don't know. I'm not necessarily writing valid code, but that's fine. Um, let me comment this out So actually what we have is we have a Left map and a right map, uh, and they each one contains an aliased copy Right, and then imagine that the user wants to execute an operation like they want to call retain right so, uh, you want to call like left Hash map dot retain, uh retain if you don't know of it is a Method on most collection types that just, um Walks the it walks all the elements of the collection And if the retain closure returns true for that element, it remains in the collection if retained returns false for that element It gets removed from the collection It's a really easy way to do just like remove a bunch of things that are related or keep only the things that have a certain property So let's say like e I don't know in this case e not equal to five Or yeah So this is going to remove any element in the map, uh, who's I guess Technically this is key value. So let's do this So it's gonna remove any value in the map whose value is not five and that includes left right because left's value is five Um, so Here that's going to get removed Which means that the map is going to drop the value now. That's fine because in this case Let me add these annotations here to manually drop new And here as well manually drop new Uh, this All right, let me just get rid of some warnings here manually drop and collections Just to make this a little less noisy Uh, yeah, yeah, yeah, yeah, that's fine Uh box five box into raw I'm just cleaning this up so that, um, the code will actually compile because that way they're fewer distracting error messages from raw v And it also shows you more what the code actually used to look like Uh Hash map I can't can't spell Um Those are expected to take two distinct arguments fine and V dot real V dot real Ah Now what? This Great. Okay. So now that compiles and now you can believe me that it's possible to write this code Okay, so the retain here is going to remove left right because the left value is indeed five So it doesn't match retain so that map when you call retain the inner code of retain is going to end up dropping, um The left now left it has a manually dropped for the box. So it's not going to drop the box great But now we need to do the same operation to the other map, right So we call the same thing on the right hash map great, but this Also ends up dropping right But it drops right with the manually drop wrapper So this means that the underlying box the box that's alias does not actually get dropped anywhere And we don't really here have a way to use into inner Remember how before we would call like right into inner when we actually wanted to drop it Well, how do we do that here because we are not the ones writing the drop code? Of course, we could re implement retain ourselves And have it like walk the thing and anything that we decide to remove we call like into inner on we could do that But it's really annoying. It would be nice if the code could just look the same So this is where the the second trick uh that ended up biting me in the ass earlier Came up, which is I pulled this trick where I did the following I'm going to write it out and then we're going to talk about why it's wrong Sorry, let me just I think I now have it right Uh do user op just to Signify that step specifically This is actually what that used to be And then the manually drop was on the alias Sorry, I promise I will explain what this does. I just need to write it out first So I just move the manually drop from being on real to being around the alias And then this Okay, great So what I decided was to do this cast And uh in order to understand this cast, which is looking really ugly We need to look really carefully at the types here so The type of write hash map Is a hash map From in this case, I guess the key is an i32 And the value is a manually drop aliased U size Right, that's the type of the right hash map And this should make it pretty clear why Removing something from the right hash map doesn't drop it It's because the values are manually drop and so dropping the manually dropped does nothing So what we're doing here is we're casting the reference that we have to this thing the mutable reference we have to this thing Into a mutable reference to this thing And if you look really carefully You'll see so we have this and we're trying to turn it into This thing So essentially what we're doing is we're doing a type cast that casts away the manually drop Right, and you might think this should be okay, right manually drop has the annotation refer transparent so It has the the the refer transparent Which means the casting from this to this should be fine because this should be laid out in memory Exactly the same as this is Right, and why does this solve the problem? Well, if we have one of these And then we drop This thing There's no manually drop here So it is actually going to drop the inner value which means that it's going to drop the box So now we can keep using retain and it will do the right thing We'll end up actually calling drop when we do this operation the second time on the on the right map on the second alias, which is the last alias This is before I talk about why this doesn't actually work. Uh, does this make sense like why why this seems like it should work Uh, refer transparent. Does refer transparent work across generics? Yes, so So refer transparent all refer transparent does is it says it when you put refer transparent on a type That type has to have only a single field. That's uh, it has to have a only a single non zero sized field Which is to say it basically just has to have one field And the layout of the outer type has to be exactly the same as the layout of the type of that one field Uh, so that includes things like generics It's just the outer type is exactly the same as the inner type for any value of the outer type For any, um, instance of the outer type, sorry So in that sense, yes, it does work across generics Uh, oh op here is operation Operation Right, so we do some operation on the map here I've chosen retain because it's sort of an obvious case of where we don't choose how the dropping happens so the The idea behind this cast right is that it lets us just not Worry about what this operation does internally and instead just Cast the way the manually drops so that whatever it does when it drops things it drops them correctly So this trick seems nice enough. Um, unfortunately This is not sound. This is a problem And to understand why we need to dig into the trait system So let's do start some code up here So we assume the standard library hash map, but let's just make our own hash map So our hash map is going to take a k and a v And it's going to hold like values. There's going to be a veck of tuples of k and v Is not actually how the hash map works, but we're going to go ahead and do that Seems fine. And then we're going to have a bunch of methods on hash map But now this particular hash map that I have is wonky It's going to require that the v is wonky Uh, wonky is going to be a trait that we define And wonky, uh, has an associated type Foo Um, it doesn't have any requirements on that type. It just has an associated type And we're actually going to store a foo right here. We're going to have a prefix which is going to be a v foo Let's do this Well, actually, let's do hash map. Let's do my hash map Uh All right, so it's a little weird like who would write hash map this way, but there's nothing stopping anyone from writing code that looks this way Um, right, like this could be any wrapper type that is in some other crate, uh, or even in the center library Who knows right and now what I'm going to do is I'm going to implement wonky for alias Of t Oops, uh, and here foo is going to be a u8 And then I'm going to implement wonky for manually drop alias t And foo is going to be u32 Do you see the problem? The layout of my hash map depends on what this type is And this type is u8 if you have an alias t and u32 If you have a manually drop alias t So this means that the layout of a my hash map Uh i32 alias t Right is a u8 followed by a vector of i32 and alias t But the layout of a my hash map manually dropped alias t is Also a vector of these but it's prefixed by u32 instead because that's what the associated type foo is for the wonky trait So these are not the same. They're not laid out the same at all And this is a trivial example where this is like another field But you can imagine that like it sticks some of those in the vector itself Like the whole structure could be laid out differently depending on whether you use this or this Now this seems a little malicious, but there's no unsafe code here right And now imagine that We don't like here either imagine that the user could choose what hash map implementation to use here But but crucially imagine that they just the the value type the t that we're wrapping in our box has Well, not the type we wrap in our box, but But the collection type we use If the user if if there's anything at all in that wrapper type Right inside of like hash map contains a vector and maybe the vector contains some other structure If any of those are dependent on any associated types depending on the value Then now this transmute does not sound because going from a this to a this Is like going from casting from a mute u8 to a mute U32 it's just not okay. They're not the same you can't do the same things with both of these So this transmute even though it seems like it should just be Identical it isn't they can be different if you have a particularly wonky outer structure All right, so how do we fix this this sounds like a huge pain and it is Finding a way to work out this work around this was really hard and I have a solution that is Probably sound But it depends on another thing that has still not been decided in the type system Which is okay. We have this observation that the reason why this isn't sound is because someone could depend on Some implementation the uses an associated type and really what that comes down to is Some someone somewhere else someone outside of the current crate Can tell the difference between these two types and therefore can choose that the layout of their type depends on it So the solution has to be to construct the types that we cast between In such a way that no other crate can distinguish between them And the way we're going to do this is as follows instead of having manually draw What we're going to have is this We're going to cast we're going to add an associated We're going to add a sorry not an associated type We're going to add a another generic type parameter to alias That dictates whether or not it should drop the inner type And so so you can imagine that we implement drop for alias, right? And then depending on what the type the second type is We choose whether to actually drop the inner t and sort of call manually drop into inner Or we're going to not do that And casting between these two Should be sound as long as these two types are both private And the reason for that is If no other crate knows about this type if they can't name this type and they can't name this type They have no way of implementing a trait for this differently than how they implemented for this This also means that these types no drop and do drop have to not cause any differences in what alias looks like So for example, if alias implemented Debug only if it was no drop, but not if it wasn't do drop Sorry, if alias implements debug If it has no drop, but it doesn't implement debug when it's do drop Then now we have the same problem. So we can implement the wonky trait for any alias Sort of tu where u is debug and then implement Uh the wonky trait differently for any alias tu where u does not implement debug So they have these two have to be indistinguishable outside of the current crate This is complicated But it does work the way we're going to do this is we're going to have a trait drop behavior That just has like a do drop method Uh and do drop doesn't doesn't need to know about the current value. It's just a Um a method on the trait if you will returns a bool And we implement drop behavior We have a struct no drop and we have a struct do drop And they must not be public And we implement drop behavior for no drop Uh, which it says false and we implement drop behavior for do drop which returns true And then we implement, uh, we changed alias so that it takes a t and a u It holds a manually drop box t again So it no longer holds we don't put the manually drop outside the alias. We put it inside the alias again Um, and it holds a u It has to be rep or transparent Uh, and I'll talk about the u here and rep or transparent in a second um And u is going to have to implement drop behavior And then we do this Impole drop for alias tu Where you implement drop behavior and When you're asked to drop one of these If you do drop Then self dot real dot into inner drop that The code can't actually look quite like this But it's close enough to to sort of explain what's going on and so now There's no way for sort of an attacker if you will or just for an Oculus library That doesn't know that you have this requirement that they're castable to implement Like they have no way to distinguish between No drop And do drop It can't because no drop and do drop are not public. They're not available Outside of the crate the defined alias. So if we don't export them all it can do is this But these are overlapping implementation. So the compiler will reject them It doesn't have a way to change the layout depending on what this uh, this other type is Which means that these the associated type here must always be the same Which means that the layout should always be the same Which in turn means that this cast from alias no drop to alias do drop should be valid Because there's no way that this type can vary in its layout depending on what this inner type is So it's a fairly sophisticated argument and it's not actually guaranteed that this is sound So this gets back to another discussion that came up, which is talking about deterministic but undefined layout This is also an issue from the unsafe code guidelines issue number 35 And here I recommend you read through the discussion. It's fascinating stuff And what we got to here was basically I proposed is this a valid cast? Let me see if I can dig into this some more so further down here Um Yeah, so here is where I give this proposal a transmute from some wrapper my type t And then some private second type to the same type with a private type b Uh, where their unit structure implemented no traits, uh, and where the the sort of my type is wrapper transparent And the response was Uh That sounds like a very reasonable request Right for there to be some way to have this kind of type Permetricity, uh cast be safe Now it's not currently guaranteed that this is valid But it is much more likely to be sound than the previous version, which we could clearly demonstrate was not sound This one I haven't been able to demonstrate is unsound Uh, and this is why this issue is still open because it's not clear that this is under the compiler currently guarantees or something it wants to guarantee Um, and this is why the unsafe code guidelines work group is so important because they're trying to figure out like What are the actual rules and what are the rules that we're willing to commit to right because if you commit to Some kind of behavior being sound you're not allowed to add an optimization That requires different behavior in the future because people have written code that assumes that a given behavior is sound Um, so yeah, this is the the solution that I've come up with and that I think to sound and hopefully it will remain sound Um, I didn't actually modify this cast but the cast remains basically the same Right, it just turns into this And Yeah, there's one more thing here, uh to get back to which is Repertransparent requires that there's only a single non zero sized field So we're not really allowed to store you here because Even though we only ever use it with no drop and do drop which are zero sized Um, there's nothing stopping someone from creating like an aliased U size U size Right and now this couldn't be rep or transparent because it's not just one field And so we use this is where the marker trait phantom data comes from Which is We want to retain information about U at compile time, but we're not actually going to store a U in this type Uh, and this means that this field is always zero sized all it this this field basically disappears at runtime It's only used to use that compile time for for sort of carrying around the generics information Um And uh, the reason why it's okay for us to store a phantom data here is because the Drop behavior trait does not take a reference to self It doesn't require that you actually have an instance of the trait In order to answer whether or not to drop because the drop behavior is determined by the type overall Like do drop always drops no drop never drops and there's no value for us to care about Okay So this now gets at how we ended up getting Hopefully at least sound aliasing and left right We use um, so we had to combine both these approaches, which I haven't showed you in the code so far Which is aliased has to contains a maybe on in it. It can't be a manually drop. It has to be a maybe on in it because um, we have to um Remove things like the aliasing requirement from box So the real aliased in left right if you go look at it actually holds a maybe on in it not a manually drop Uh, and then we need to have this associate or this not associated type the second generic type parameter That is entirely private that we cast between in order to make sure that the um, the cast is actually valid Huh, all right. That's a lot of complexity that I've thrown at you. Um, and So let's do some more questions around this. Um, I don't What's a little weird about this particular type of stream is that There's like not that much coding. It's really just talking through the complexities and and how to resolve them So I don't actually have anything more. I want to talk about today, but we have talked about a lot of really complex stuff and interactions And so I'm sure there are a lot of just related questions and and things that I can go into more detail on and Help you understand why things work or don't work So please like fire away with questions and that will probably take some time to Um, if I understand correctly the user of this library is only given an abstract aliased all the fields are private Yes, that is correct. So, um, if we go back here and look at Left right So left right has this, uh aliasing module But that actually talks a bunch about this um And it has an alias type and the alias type has no fields But the alias type Ah, the alias type does implement D ref Into the inner type. So if you have an alias t, uh, then you can always get you can safely get at the, um at the inner t The reason why you can do this is because it's sort of like the same argument as manually drop that It's unsafe to construct the alias in the first time and it's Well, uh, it's unsafe to cause the alias to be dropped to drop the inner t And therefore as long as that part is unsafe and you have to manually guarantee that it's safe Then it should always be safe to derefin alias because of the safety invariant of actually dropping Uh, these arguments are all understandable But any unsafe seems so difficult to get right accidentally being unsound is so easy. I feel unsafe somewhat deserves its reputation You're not wrong. Um It is true that unsafety is complicated, um And it sort of is complicated for a good reason right like the the reason why this code is so hairy or this Why the reasoning here is so hairy is because There's a very sophisticated property that we're trying to guarantee Like think of if you if you were to do this in in c code, right? All the same conditions would apply Um aliasing not so much because in c you generally don't pass no alias tele VMs. You don't get the same optimizations Um, but if you try to cast between two different types and they have a different layout You would have the same problem It is true that in c it's sort of hard for them to not have the same layout because you don't have things like Generics in the same way. So the the same problems can't arise but the Properties you're trying to maintain are the same. I think the reason why it becomes complicated in rust is because There are so many interactions with the rest of the rust type system to to think about um And it's hard like ev map did this the wrong way for years But it worked right like it was being used in Sort of a real research system for for lots of benchmarks and testing and everything worked just fine. So like it's tricky right because Unsound and unsafe behavior is um or unsound behavior specifically is like It is a problem because it could randomly break at any time Right the next compiler release might suddenly cause everything to go wrong and it might go wrong in really bad ways So that's why you should avoid unsound behavior or undefined behavior But if it currently works, then that means as long as you don't upgrade anything, you're probably fine That's not a good safety requirement to rely on but but oftentimes it's like good enough and I have a lot of understanding for People who write unsafe code and go This might technically be unsound in some way, but it generally works the way that I use it. So therefore, I won't fix it I understand the argument. I think it really depends on what the Um What does it have blast radius of being wrong is right if you if you're just like writing your own code base for some hobby project It probably doesn't matter But if you're writing like code for the space shuttle This probably really matters so you get all the unsafety right and that does require you to do a lot of pretty sophisticated reasoning I think in general you can get pretty far by just being extremely skeptical of Anything that's as unsafe in general the the best rule of thumb I've heard for unsafe is that unsafety is a crate property If you do anything unsafe in your crate, it's not just about like whether the surrounding lines or right or anything It's that you need to guarantee that all of the other code in your crate Is still sound now that this unsafe thing is happening And that requires really hard reasoning and that's why for basically any piece of unsafe code you write You should write a safety comment above explaining why this doesn't invalidate any of the other safe code in your In your system But it's hard. Um, the the other way to get at this is to run everything through miri So miri if you don't know about it as a really handy tool that basically it sort of runs your Runs is a fudgy word when it comes to miri But it basically runs your rust code in a in a mode where it can detect unsafe behavior Now miri is a little limited in that it only Detects unsound behavior if it occurs So it can't detect if your code like sometimes does something unsound if your tests don't trigger that particular case But but it is really good at if you do something that is unsound it will tell you And this is why for for unsafe code like testing is just absolutely key You want to run it through miri. You want to run it through things like Like address sanitizer Or through val grind You probably want to run it through something like loom, which is a Sort of concurrency testing tool that's really handy in rust You just need to be really thorough about vetting unsafe code How'd you figure out that the cast without do drop no drop trait was incorrect in the first place So this came up in the previous stream actually where someone was like Are you guaranteed that this cast is safe? And I was like, I think this cast is safe, but let me ask so if we go back to Here Let me see if I can find my particular comment Yeah, I recently came across a case where I specifically wanted to cast a some type t into a some type manually drop t And in trying to figure out whether that was safe. I found this discussion I think that case works correctly in the compiler today, but I suspect that it's still an open question Whether it should or will be guaranteed by the language to be okay subject to the discussion in this issue so That's really where the stem from that I was like I think this transmute is okay, but it might not be okay because this is super subtle stuff I googled around for a bit. I found this open issue. I commented on it and then a lot of really interesting discussion happened Um, can you explain it like at five like I'm five that unsafe cast line? Okay. Yeah, so this is a sort of fun aside Uh, this cast line. So let's dig into what this is doing Uh, this is like worthwhile to go through because casts are are nasty. So write hash map here Is really a hash map right of uh alias use high 32 alias use size no drop Right, um, we can't cast the actual own type. We want to cast a mutable reference to it So that's why this takes a mutable reference out of that. So that gives us a Mu dash map Of that same inner type Um Then we you're not allowed to cast directly between reference types. So if you have a like a You you can't just cast directly from this to this You can only do that kind of type casting either by using mem transmute, which you should generally avoid because it lets you do Really unsound things Um, but you can cast between raw pointer types to things like this So for example, if you have a One thing that comes up a lot is if you have a mute u 32 And you want to cast it to a mute u 8 4 This is sort of a this is safe because they're the same layout, right? Um, you can't cast directly between them without using a mem transmute, which you want to avoid So what you do is you turn this into a star mute u 32 um, and a star mute u 32 You can cast into a star mute u 8 4 And then you use a star mute star to turn it into a ref mute So so that's the same song in dense it goes through here this um This turns it into a this turns the mutable reference to the hash map into a mutable pointer to the hash map This is what actually does the cast To do drop The reason this is a cast to do drop is because I've specified that the type should be do drop here So the compiler can sort of reason backwards to what type What type I wanted this cast to turn into And this star mute star is the way that we go from a mutable pointer to a mutable reference Have you found that most unsafe usages can be replaced with safe coded no performance penalty? Yes unsafe code generally You should only generally need it if you're writing like Core primitives like I write a lot of concurrency primitives things like left right or Like concurrent hash maps and stuff where you're you're really in the low level and there you sort of need unsafe because You're relying on invariance that the compiler doesn't know about and that aren't common And when I say common what I mean is like If you look back at I did a stream a while back about interior mutability and things like cell and The interface to sell is completely safe And it's common enough that someone actually wrote the cell type and put it in the standard library Usually if you're writing like your own concurrency primitive the invariance you're relying on are so specific to your particular Problem domain that no one else has written the thing that you need and so you need to implement it But outside of writing like Really low level code. You probably don't need unsafe There's probably already a safe wrapper that you can use and in general Those do the unsafe internally to mitigate the performance cost. So it's probably just as fast as the safe code You should not think of safety as requiring a performance cost because um If you can do it with unsafe code and have it be safe That means that there's some way to write a safe wrapper around the code That is exactly the same thing you did and therefore has no performance cost Isn't the compiler going to drop the manually drop in case of memory optimized programs since the code won't require the variables again No, so the whole point of manually drop Is that when you drop it it never drops the inner type? How could an implementation of the left right look like that does not use unsafe aliasing at all the safe implementation of left right Well, there's other unsafety and left right. That's not related to this particular problem But if we look just this particular problem, the version of left right that doesn't use this unsafe aliasing is that you use ARC So use an arc of u-size or an arc of whatever the inner type is And then you just keep one clone for each map And that's totally safe Left right could do this There are a couple of reasons why I didn't want to do it. One is that this means that you have to The copies are well You end up having to do reference counting every time you want to Create an alias and any time you drop the aliases and that reference counting isn't really needed Because it just sort of does the right thing already um Like it the left right Primitive already knows when to when you should when you're dropping an alias So you shouldn't drop and when you're dropping the last copy in which case you should drop Um, so the reference counting is sort of just overhead The other reason why I didn't want to do an arc t and this is sort of the primary one Is that imagine that the user specifies a t of like Box foo Then now what the stator structure is going to store is arc of box of foo Which is sort of wasteful right like you're having a an extra heap allocation just to store a box which is a heap allocation Um, even worse yet imagine that the user gives a type of arc foo Then now we're going to store an arc of arc of foo So every access is going to be sort of penalized because it has to go through an extra level of indirection now It's this is not impossible. Um, it's not Like you could totally do this The cost would be fairly small for many applications But left right is specifically for where you have really really really high throughput reads where you care a lot about the latency of each individual read if you didn't You just use like a reader writer lock or something like that And so I'm assuming that the people who use left right will care about things like an extra pointer dereference And therefore I didn't want to force the additional use of like a reference count update and the extra pointer dereference Which the aliasing avoids That said like this would fix another outstanding on safety problem in ev map, which is um So Actually, this is worth touching on briefly, which is imagine that The retain here right retain here is doing a comparison Right of the inner value now. Let's say that instead of being five. This was some user defined type like foo And I imagine the implementation of ek for foo is non deterministic It just like randomly returns true or false Then when you do the operation on the left map, you might choose not to remove the value because the randomized implementation of ek returns false Um, and then in the right hash map when you do the retain it randomly turns true and so it does get removed Then now you didn't remove the value from the left map But you did remove it from the right map, but because you thought it was removed from both You end up actually dropping the value even though an alias still exists Uh and guarding against this is really really difficult. There's a there's an open issue already on ev map for this problem And with arc this wouldn't be a problem because the reference counting would notice that it's not safe to drop the second alias because one still exists So, uh, I don't have a good solution for this Arc would solve the problem, but it comes with its own costs. It might be the solution is that Creating an ev map has to be unsafe and that you need to guarantee the that every operation like hash ek partial ek is deterministic An analogy I was thinking of here is that rest is a lot of powerful tools And the more powerful the tool the worst things can go wrong when you have to take off the safety guard It's a good way to think about it. Um the alternative way to think about it is that You have a really complicated machine and you're fiddling with one little bit on it, but Everything else is connected. So if fiddling with this one bit might cause a problem over here Uh, and so this is like the interconnectedness of there are lots of features that interact in interesting ways And you need to reason about how all of them interact with the change that you're making It's sort of tight coupling if you will Uh, and I think um this point came up in chat too that one challenge, especially with generics and unsafety is that You have to ensure that your unsafety holds for any value for the generic type, uh, which might be tricky Why not transmute to maybe un-init instead? That should always be fine Um Here You have the same problem if if this was like casting between Not not maybe un-init and maybe un-init or the other way you still have the the wonky problem um Now of course internally alias does have to do have a maybe in it. So if you look at the code for left right You'll see that that's there Uh, don't u32 and u84 have different alignments Uh, yes, they do so You're not allowed to cast this into this But you are allowed to transmute this into this So you can go one way but not the other Um, doesn't alias already introduced a layer of indirection with the box Uh, to me that looks like an arc without the reference counting. No, not quite. So Maybe un-init is not a heap allocation. Maybe un-init. It's not a pointer Uh, same with manually drop. It's not a pointer. Um, the definition of Uh, manually in it you can uh, maybe in it You can sort of think of as this It doesn't store like if this means that if you have a maybe, uh, maybe Un-init If you have a maybe un-init box t that really is just a box t. There's no there's no additional pointer or anything um It's just that the compiler knows that you can't actually assume that it that it's a valid box t But there's no extreme direction Uh All right, let's see This test seems uh, kind of simple at this point might as well just make it wrong Um, okay. So hopefully that was useful. Hopefully you feel like you learned a little bit about, um Learned a little bit more about how this stuff works um under the hood um, and and There are many issues with unsafe like they're this journey is not over. Um, and The point of the stream was not necessarily to like Teach you how to write safe code and that's not that's never going to be my goal with this particular series it's just to expose you to The kind of things you have to think about when writing unsafe code I don't know when the next stream will be it'll be whenever I have another interesting thing to talk about Um, but hopefully just sort of talking through the kind of problems that that can arise is is useful and interesting um There is and hopefully it was hopefully it was possible to follow like this stuff is super involved and I highly recommend that you look at the um, the Issues both on the unsafe coding guidelines and the ones on ev map And maybe and look at the code of left right. I'll link all of those in the video description eventually um Because it's some really fascinating discussion around it as well That I think is is useful if you're going to write code like this yourself just getting into the right mindset helps a lot um, someone asked about uh structural eek for the eek bug um structural eek doesn't actually solve the problem because First of all structural eek is really limited. Um, it it's so the advantage of structural eek is that it's Derived by the compiler and so you're guaranteed that it's deterministic Um, but it doesn't actually but but it means that if someone has their own type that is not structural eek Then they can't use your data structure, which seems really sad Uh, the other is that structural eek is not the only problem. Uh, imagine that We do this. Um, so here's an example of something that is not does not rely on eek Uh, so imagine that we have this instead And of course we do the same thing for um, do the same thing down here for the right map Right now this depends on the iteration order of the map Right, whichever thing you iterate to first, that's the thing you're not going to drop You're going to drop all those all the subsequent elements Well, what if so the iteration order of a hash map depends on the hashes of the values that are in there So if the user controls the key type Right here, let's imagine this is like foo some user type And the user implementation of hash for foo is non deterministic You have the same problem Because the iteration order will be non deterministic. So which thing is first it also varies Uh, so we actually need to require both that. Sorry. Let me leave the code up Uh, we actually need to require both that the keys and values are deterministic In the improvisation of eek and of hash And maybe even of or depending on what the value type is I think maybe the takeaway from from this is like Do not write on safe code unless you have to There are some cases where you have to where there's some Invariant that you know is maintained but the compiler can't know But for most of the code you write that shouldn't be true, right for most of the code you write There someone probably ran into the same sort of data structure problem You had and wrote a data structure that is safe that internally does the unsafe stuff Like don't do unsafe stuff yourself unless there really is no safe implementation of it that you can use Because getting unsafe stuff right is hard. It requires a lot of thinking And not just a lot of thinking but a lot of understanding of what the actual rules that the compiler enforces are and that the typesism enforces are Um All right, I think I think that's everything for today. Uh, hopefully that was useful I'll upload this to youtube and Then I will see you next time. I don't know when the next stream will be I don't even know what topic it'll be on Whether it'll be sort of a crust of rust or a long live coding or another unsafe chronicles I guess we'll see But hopefully this was interesting if you think that this particular series is like valuable then please like Please at me, which is the opposite of what people usually say or just like mentioned in the comments It's really useful for me to get a sense for What people find interesting and useful and educational and what they don't if this kind of stuff just sort of goes over Everyone's head and you feel like the only takeaway what everything is hard then this is not a valuable stream But so so any input you can give me is always valuable All right, everyone. Thank you for coming and I'll see you next time So long for well. Bye