 Hello, everyone. Welcome back. It's been several weeks now of, or several months, actually, where we've been working on implementing TCP. And that's been a lot of fun, but I figured we want to sort of keep iterating through new topics as people get interested in them. And so if you recall, there's this website I use that I set up where, in fact, I live-coded this website, too. You can find it in earlier stream, where people can vote for what things they would like to see me stream. And then, particularly, since procedural macros was the thing that most people were excited about, it was a pretty neck-and-neck race between that and a second take of the open source contribution stream. So my guesses will end up doing that next. But for now, we're going to dive into procedural macros. So for those of you who are not aware what procedural macros are, let's start just with what macros are. And to do that, I'm going to actually, let's not do that. Let's start a new empty project here. Actually, oh, how do I want to do this? Let's just start a VimTerminal. I can't type today. It's going to be a chunk. What? I'll file a format. Fine. Let's do this instead. So while writing Rust code, you may have used macros before, something like Vec. So if I do this, so the exclamation mark here means that it's a macro. And what a macro does is essentially it is given the piece of code that you give it, and it returns some other code to run instead. In the case of Vector, you could imagine the Vec macro being something like the Vec macro up here really being something like let mute V is Vec new, V.push1, V.push2, V.push3V. So you could imagine that this is what the Vec macro expands to. In reality, it's actually more optimized than this. You don't want to keep allocating, for example, as you grow it. So this is not actually what it expands to, but you can sort of imagine that this code ends up being turned into this code. And that is essentially what a macro is. Now, normally, you can write macros using macro rules. So let's say that we wanted to define the Vec macro. We could do something like, in this case, we probably want, like, eexperstar, comma. So this is an example of a normal Vecro, normal Vecrush. This is an example of how you would write a normal macro in Rust. So if you want to write it inline in the file. So you use this macro rules macro. That is a macro that lets you write other macros. And you basically write pattern matching rules for the arguments to the macro. I think this might have to be parentheses. And then you write the body that that should expand to. In this case, the pattern is saying anything that's an expression separated by comma, 0 or more times. It expands into the following code. It's a block that generates this Vec. And then this syntax is saying, write out the statement in between the parentheses as many times as they were in the input. So this is saying, write out this line many times. And this Vec macro we could use instead of the one from the standard library. You wouldn't want to, but this is sort of the way macros work in general. Now, these are all very well unhandy, but they're pretty restricted in what they can do. So you have to be able to write this kind of pattern. And sometimes you want fancier things. Like, if you've ever used the certi crate that lets you do serialization and deserialization, it allows you to do things like struct foo. It has a bar type U size. And you can do derive, serialize, deserialize. And what derive is, if we think about it, is really a way of taking some kind of code, annotating it in such a way that some program is going to run to generate the code needed for serialize and deserialize. And in fact, this turns out to be a macro. This is what we call a drive macro. So a drive macro is going to take, it's going to be given as input, this business, and then it's going to generate something like impulse, serialize, for foo, and then whatever stuff is necessary in here. There are also other type of macros you may have seen. So a macro-like test. So this is a macro that transforms this function into the appropriate format for a test. This is what we call an attribute macro. Usually, you could also use these with parameters. So for example, if you've ever used Rocket to the sort of web framework, you could do stuff like route get slash. And now this is going to be generated as one of the routable parts of your application, where if you get a get request for slash, then this function would be evaluated. And so this is an attribute macro. All of these are the class of macros that we call procedural macros. So the idea of a procedural macro is that you write a program that's going to run at compile time, and it's going to be given essentially a stream of tokens. So tokens, in this sense, are a bunch of the building blocks of the Rust language. You're going to get a stream of them, and you're going to produce a stream that's going to replace or add to the original stream. So someone asked here, how can I see the macro expansion? And there are actually many ways. What we're going to be using today is this a crate called cargo expand. I'll get back to this a little bit later. But basically, there's this command that you can run on your own. So if you run cargo, Rust C, and then with these arguments, which are nightly only, then the Rust compiler is going to run and it's going to basically print out with all the macros expanded. So here's one example where there's a derived debug, and if you run cargo expand, you get the fully expanded, or you get one level of expansion for that code. Actually, you might get a fully expanded. These are sort of special. So that is one way to do it. And probably the one we'll end up using today. And this is going to be really handy for debugging procedural macros later. So procedural macros are, if we go back a little bit, they're effectively, so this is from the Rust reference, which actually might be less helpful than we want. But essentially, a procedural macro are mostly a conversion from token stream to token stream, right? There are some variations. So if you write a function-like macro, so that would be something like VEC that we saw earlier, then it's just take a token stream and give the replacement stream. If you write a derived macro, then you're given the token stream of the thing to transform, and the output token stream is added to the same place as was, right? So if you're going to add an impulse, for example, you don't want to replace the original struct, right? Because that struct still needs to be there, and the user has defined it, so you're not really going to be changing it, but you're going to be adding stuff. And then finally, attribute macros are things like route or test that we saw before, and they're given the token stream of the attribute, so this would be the stuff that comes between the parentheses and the token stream of the thing that has been annotated. And they're supposed to generate a token stream that is, let's say here, the return token stream replaces an item with an arbitrary number of items. We're going to be talking about items a lot here. Items in Rust, in fact, this is the reference probably, so items are basically anything. They're like modules, functions, type aliases, different types that are defined, implementations, traits, external blocks. Think of them as basically anything with curly brackets. It's not quite that simple, and it can be simpler things too, like use statements or also items. And so procedural macros are basically a way of taking a stream of items and doing something with them and then producing a stream of items. We won't actually read through this document. I don't think that would be helpful for us. Instead, what we're going to do is we are going to follow the proc macro workshop. So this was essentially a workshop from the Rust Latem conference. And this is written by David, who is a really prolific Rust developer and has built many of the tools that we'll end up using today. So he's written in particular the syn crate, which is what we'll use to parse a token stream into something that we can modify. And the quote crate, which is the inverse, it takes essentially an abstract syntax tree, like something that we're going to be modifying and turns it into Rust code again, basically turns it back into a token stream. And David wrote this as sort of a workshop guide with a bunch of exercises for this conference. And so what I thought would do is basically try to work through some of the suggested projects here because they're basically built for people learning procedural macros. And I have not done them before. I haven't even written that many procedural macros, so this will be an interesting learning experience where I have less experience than what I usually do for these streams. Do input items have to be valid? That's a good question. I believe the only requirement is that it lexes, but not necessarily that it parses. So all of the individual parts need to, basically you need to be able to generate a valid token stream from them. So you couldn't have like an unterminated open double quotes, for example. But as long as you can parse it meaningfully, then you're given that stream of tokens. But what you produce, the token stream you produce, is what the compiler is going to be running on. And so that obviously has to be valid for the resulting token to compile. Yeah, but something like just an identifier open curly brackets and some comma-separated identifiers, which would not normally be valid, Rust is totally fine. In fact, the thing you give there is a valid pattern, I think. But that's sort of separate. OK, so we're going to be working through these. And in particular, we're going to start with essentially what this suggests we should start with, which is write derived builder. So for those of you who don't know what builders are, the builder pattern is essentially a way of constructing something that you're later going to build. So an example of this would be, and the example that's given here indeed, is imagine that you want to run some kind of command line utility from Rust. Well, when you do that, you're going to have to specify the name of the program. You're going to have to specify arguments to that program. You might want to specify where the program should run. You might want to specify whether the program should replace the current one. It'll be started in a separate process. You might want to say where the input should come from, where the output should go. And all of these are arguments you may or may not want to set for any given execution. And so rather than have a constructor that takes a million fields, you have a builder, which has defaults for basically most of the options. And then it has methods for overriding particular parts of that construct. So here, for example, you create a new builder. You dot executable returns a builder whose executable has been set to cargo. And so you can keep sort of layering on here, continuing to configure the command builder. So basically the command you haven't run yet. And then eventually you call build. And that's going to actually produce the final type that you want to run or indeed run it, depending a little bit on the setup. In this case, it's sort of trivial. But here the idea is that what this is going to produce is a command. So our derived builder is going to produce a command builder that has these methods. And then build is going to give us one of these back with all the fields set. And so what we're going to start out with doing is we are going to just clone this. That's where we're going to start. So we get clone this. Oops. Apparently I just cannot type today. Seems less than ideal. All right. Great. So let's see what we have here. So for this, the skeleton is located under the builder directory. So we're going to cd into builder. So this is essentially its own self-contained crate that we're going to be solving this in. The cargo.toml has already been set up basically correctly. So you'll notice that for lib it's set to proc macro equals true. So this is what we want. We're writing a library that's going to be a procedural macro. And you need to communicate this to cargo so that it knows to compile it in a special mode where it has access to those things. For example, you don't require a main in a procedural macro. What is also neat about this is this also comes with a bunch of tests that we can use. So if we look at the test directory, let's look at progress first. So this uses a crate called try build, which is handy for writing procedural macros. It basically tries to build the given source code and then checks that the compiler output is what you expect. So if you expect it to compile, that's all well and fine. Try build will do that. But you might expect it to fail with a particular error message. So you might want to check that, for example, the compiler highlights the right part of the code if something doesn't work out. And we'll see some examples of that later. In particular, the first thing we want to pass is the parse test. So it looks for a derived macro with the right name. So we're going to have derived builder. It doesn't require any specific code to be generated. So really, all that matters is to see that we can parse it correctly. Why did the chat go away? Weird. So the only goal here really is that we can parse the input, but we don't have to produce anything. In particular, what it's telling us to do here is use the Syncrate to parse. Well, let's take a look at the Syncrate. So the Syncrate, as it says, is a parser for Rust source code. And essentially what it does is it gives you a parse macro input, which lets you parse, well, a token stream into a syntax tree. And particularly, it generates this kind of derived input business. So let's take a look at, no, that's not what I want. Let's take a look at the docs. So down here, where is parse macro input? Great. So that should give us a derived input. So in particular, what we want here is we want to parse the token stream we're given as a derived input. So the derived input here is the input to a derived macro. And you'll see that what we get back is something that has atters. So atters are any other attributes specified on the struct that we're deriving for or the item we're deriving for. The visibility of that struct, the struct might be public. It might be create only whatever ident, which is the name of the struct. Generics, which are any generics on the struct itself. And data, which are the contents of whatever data type that we're deriving over. And so in our case, the test here is basically just a test that derived builder on command produces code that compiles, which would include adding no code, which is indeed what we're going to do. So here, what we would expect is, given this, we would expect atters to be empty because there are no other attributes on the struct. We would expect visibility to be pub. We would expect ident to be command. We would expect generics to be empty. And we would expect data to contain these fields. So if we look at what data is, so data can either be a struct, an enum, or a union. In our case, we're expecting it to be a struct. And so it's going to have a struct token, which is a struct. It's going to have a bunch of fields, which are going to be these. And it's going to have a semicolon token. It may or may not have a semicolon token. In this case, it does not. So the fields are going to be named. So unnamed fields would be like a tuple struct. Then the fields do not have names. They're just numbered. So we're expecting to get fields named. And so here, we're going to have a brace token, which is going to be this. And then we're going to have the named fields. So punctuated here just means that it's comma separated values. So this is going to be an iterator that gives us comma separated fields. Every field in here, again, has all of these. In particular, executable, for example, it has no attributes. It doesn't have any pound square bracket above it. Its visibility should be private. Its ident should be sum executable. So it's an option, because it might not have a name. Colon token is, well, a colon. And tie is going to be the type, which in this case is going to be string. So you can see that this, going all the way back up, this derive input basically gives us access to the entire thing that's been defined. So let's try passing the first test. So we're going to start out by uncommenting this and then run cargo test. Let's see what that gives us. Apparently I'm lying. What's the, no, we're here. Oh, that's probably because I'm in the wrong directory. Here, cargo test. The other thing we're going to need here is for the builder, we're going to have to depend on the syn crate, which is currently version zero 15. So let's see what that gives us. Okay, so great. So it ran zero one parse. It failed because the proc macro derive panicked. In particular, because we haven't defined any derived macro called builder. So we're going to have to define one. Great. So going back to the source for this crate, right? Here, what we really hit here was we hit this not yet implemented, right? So this has already been set up for us so that it sort of defines the basic setup that we need for proc macros, which is we need external crate proc macros. So this is a special crate that the compiler exposes to us that gives us access to this token stream type, which is what the compiler produces to macros and expect macros to produce. And then the proc macro, when you write a proc macro, you also have access to three attributes for functions. It's proc macro, proc macro derive and then proc macro atter, which is you define the three different types of procedural macros that we talked about earlier. In this case, we want a derive macro. So we're going to write proc macro derive and we're going to give it a name, which is the name that the user is going to put inside of the derive statement on their struct or enum or whatever. This is going to be a public function because it's going to be essentially called externally and it has the signature that we expect, right? So the input is a token stream, so this is going to be our struct and the output is a token stream that we're going to produce. And in our case, what we want to do here is we want to parse, let's call this an AST, I don't actually need this anymore, I think, because 2018 edition, maybe? Great, I guess we'll find out. So what we're going to do is we're going to run something along the lines of what was given in the example down here. This, this obviously means that we're going to have to use parse macro input. So let AST is going to be equal to this. Notice that there's no result return type here. The reason for this is procedural macros are expected to always produce a token stream and a panic is considered a failure to compile. And so you really just want to panic if something goes wrong and what the compiler is going to do if you panic is, and we saw this back here, it's going to highlight a span that's contained in the error you produce in the panic. Basically the span there was processing at the time. And then the message it's going to give you is the panic message from the procedural macros. Okay, so this is going to give us the AST and let's just print out the AST. And then what we want to return is an empty token stream. I think there's a empty on there. Let's see what that gives us. Okay, so we can't do that. Do I actually need to, that's a little weird, but okay. And we need to use derive input. Okay, so we're going to have to look at the proc macro here. All right, so proc macro token stream. Oh, it's just new, which gives you empty. Okay, so instead of empty, it's going to be new. And I think this implements display maybe. Isn't there a nice way to debugging? Feel like you're supposed to, oh, you're able to print token streams. Okay, that's fine. I'm surprised though that this doesn't implement debug. It does. What are you on about? Why can't I do that? Derive, derive input does not implement debug. Pretty sure it's supposed to implement debug. Send drive input cannot be formatted. Huh, how weird. Oh, are we not building syn with derive? So syn has a bunch of features. I guess the real question is, what features are on by default? Derive is enabled by default. Extra traits. Okay, so back in Cargo Toml, we're going to do version is this, and we're going to say features, add this on, and then in theory, that should give us debug for these types. Oh, it might even say in the debug tips, prog macro workshop. See what it says. Yeah, so Cargo expand I already know about. Yeah, see, we found it ourselves. We could add pretty printing, but, you know. Great, okay, so here we now pass the first test because we're basically because we removed the unimplemented, right? That's the real reason. Previously it was crashing, now it's no longer crashing, but that also means that we're parsing the input. So let's make this a little bit nicer for us. Let's use pound question mark. Right, so here we get to see the full parsed syntax tree for what we got. So let's see if it matches what we expected. So there are no attributes. Remember, this is zero and parse. So it's this struct that we're comparing against. So there are no attributes. It's public. The identities command. There are no generics. Data is a data struct. It's struct token of struct. It's fields are named and the fields are executable, so types are specified with paths. In this case, string is just the final part of the path. There's no proceeding path, which is why the path has no leading colon and it only has one segment and that is string. The next field is a vector of strings. So we're gonna see, yeah. So comma field is called args. This is a, so this also only has one path segment. A vector is the identity of the type and the type also takes arguments. So if we look up here, the arguments for string were none. The arguments for the vector is deep down their string. So clearly our type is being parsed correctly. We don't use any of the fields. Like we don't actually do anything with any of this, but at least now we have the type. All right, so let's look at the second test. So we want to have the macro produce a struct for the builder state and a builder function that creates an empty instance of the builder. As a quick start, I try generating the following code but make sure the type name matches what's in the caller's input. Right, so here we want a builder function on command that returns the builder and we're gonna have to define a new struct called builder, right? So we wanted to generate this business. Great, yeah, so if you look at what the test actually does, it expects there to be a builder method on command, which of course there isn't by default, and then it just sees that that returns the thing it can represent. The resources down here mentions the quote crate. So we talked about this a little bit early on. So quote is basically a crate that lets you go the other direction. So sin lets you go from a token stream into the sort of tree, and then quote lets you take a tree that you can specify in a convenient way and turn it into a token stream again, roughly. And so that's what we're gonna use to go the other way. In particular, if we go back to our cargo tumble, here we're gonna have quote version zero six. I don't need this to do anymore. And so now we're also gonna use quote, quote, right? So what we want to do here is the quote macro lets us use this, it lets us write, it's sort of like a templating language almost. It lets us basically write Rust code and then interpolate things in there. So for example, the name we wanted to use here was it just an indent? So we want to generate an import for command, right? But we need to know the name command. So where does that come from? Well, we know the derive input has an indent that is the command, right? So we can just do this, which is gonna be the name. So we're gonna import, so pound is the way you signify replacement of some variable. This is gonna generate a builder function. It's gonna return some type that we don't really know yet. And who knows what's gonna go here, right? So we're gonna need some kind of struct here that's gonna be the builder, but we don't just wanna call it builder. Cause if we just called it builder, then you couldn't have derive builder for two different structs in the same file cause they would generate the same builder type. So we actually need to generate a unique name for this. So how do we do that? Well, there are a couple of different ways. The easiest way is to generate a new indent. So if you look over here, let me look at... So an indent is sort of a name in Rust. Like a variable would have an indent, a type would be an indent. In our case, we wanna create a new indent that's the concatenation of two existing ones or of some strings that we wanna combine. In particular, what we want is, let's call the builder name. So it's gonna be, we can just, we're just writing a normal Rust program, right? So we can just do this. And give in the name. Can I just do that? Or do I have to do something fancy? This implements display. Great. So that's gonna be the name. And the be identifier is gonna be syn ident new. We're gonna give in the b name. But ident takes a second argument. And the second argument is a span. So if you look at new, span here. So spans are basically parts of the original input file that we want to be able to refer to. So in this case, if something goes wrong with creating this name, we want the compiler to highlight the original name that costs us to build this string, right? So for example, imagine that the user wrote something like struct this, which is not a valid name for a struct. Then our builder that we're gonna generate, let's make this simpler. Our builder would generate this, right? Which is also not a valid struct name. And what we want is for the compiler when it tries to compile this to point at this being the error. And what that means is this identifier here, we want its span to be this. So that the compiler knows to point at that when it talks about this. Does that roughly make sense? So here, we're gonna take the span of the original. So that we're gonna take the span of the original name and use that as the ident span for this. So now this can generate by that. So currently it generates no fields, but that's fine. We're not, this test does not require a stat in a field. We might add some later though. So let's see what this gives us. So it doesn't compile because, do I need to do this in two steps probably? Expanded equals this, expanded. So what quote gives you back is not quite a token stream. It returns some type that we can turn into a token stream, hence the int to expected stir, that's fine. Oh, I wish this took int to stir, oh well. All right, so let's see what this just gives us. Okay, so that compiles, but let's, oh, I guess we need to actually turn on the test as well. Great, so now we pass both tests, but let's take a look at what this actually generated. So if I run cargo expand, it's gonna give me the expanded source code. Let's do it just for test 02. 02, what's the syntax for this? Oh, I see, sure. So I guess we're looking at main and we're looking at builder 02. So let's do cargo expand, see what that generates for us. Oh, right, not there, here. All right, so this is what gets generated well. This looks pretty sensible, right? So this is the original struct, the one that had the derive on it and we generate struct command builder and we generate input command. So this actually generates what we expect. Let's try to see what happens if we put something weird in here. Like if in our main we try to do what we talked about earlier and make this like 777. What now happens, expected identifier. So the way I can make this not work might not be. I wanted to see whether the span's aligned correctly but I don't think there's a nice way for us to test this just sort of quickly. Okay, so if you remember back to the test it asks us to, once we generate all of this to actually generate this struct, so that's what we'll do. We'll go back to our source and instead of generating an empty one we're gonna generate this one. This, they want to generate, please do the, great. Okay, so now let's just see that that actually works correctly. Okay, so we still pass the test, right? Test zero passes, test two passes. Time to move on to test three, call setters. So now we wanna generate methods on the builder for setting the different values. So this is what we talked about before, right? When you have a builder, you want to be able to do dot arg and give one argument and have that be added to the list of arguments that are currently there or like dot executable to set the executable and so forth. So here you'll see that it still has this derived builder and now with the builder it gets back it's expecting to be able to call these and have them do the appropriate thing, right? So they give an example here of the way this method should look like is fn and then the name, this should be the signature and the method should return the builder itself so you can keep building with it. This is basically so that you get to chain calls for a builder which is sometimes handy and all it's really gonna do is it's gonna change the appropriate attribute and then return self. And of course now in the very beginning what we can do is just generate each of these manually. I don't know what the later tests do but it could be that eventually we want to generate these dynamically based on the fields because this macro is kind of useless if it requires these to be the four fields. We want it to be able to handle any set of fields. In fact, let's just sort of peak at what optional field repeating field. Let's look at like seven. Okay, so these are sort of different. It might be that we just wanna do this the straightforward a first and then expand to other fields later. Okay, so for the third test we wanna generate a bunch of methods. In particular, we want to input bydent and then we want methods like this, right? And we want, I guess we want them to be pub and we want args. I guess what does the test test for? Executable args and then current error. Right, so let's see what that does. Great, so we still pass all the tests. Now we're seeing this is becoming really verbose because we're repeating the fields from the original struct multiple times over and this is really what we want our macro to do for us. We don't want to have to write these out because then the derive isn't particularly useful. And so we basically need at some point to write a thing that's gonna walk all the fields of the struct. So let's look at four, call build. Okay, so we now wanna generate the final build method that's gonna go from a command builder to a command and this should, let's see, this method should require that every one of the fields has been explicitly set. It should return an error if a field is missing. The precise error type is not important. Consider using boxed in error which you can construct using the impulse from string for a boxed in error. Okay, sure. So now we're gonna generate the build method as well and this is of course not gonna return a command. It's gonna return name, right? Because in theory at least we're generic over or not generic over but we're abstracting over what the type that we're derived from is. And so here it's gonna be something like if let if self.executable.isNone then return, oh great. Okay, so we can just do executable isNoneSet, right? And here again we see this repetition that we don't really want that we'll eventually fix by instead just looping over the fields which you remember we have in the parsed AST. Just checking is the chat still working because it froze for a second there. And of course otherwise here if all the fields are, ooh in fact we can do this even nicer but let's ignore that for now. So this is gonna be name, this executable is gonna be self, executable, ooh in fact here's a better way to do this. Okay, or isNoneSet, I think that should work which is much nicer and right there instead of having all these ifs. See what that gives us, recursion limit while expanding the macro stringify. That's interesting. Does it say anything about recursion limits? No, huh, also what is happening with the chat? Just checking that this is, huh weird. Oh, oh I guess we get out of that, that seems fine. Oh, we need to actually use that. Yeah, that's fine as long as the stream is still working. I didn't see anything either, thanks though. Yeah, so here this error is actually kind of interesting. So if you look up here you see that it says error was not in scope back when we had this, right? And the reason for this is in generated macro code you can't really rely on something being in scope. Like you can't rely on the user having written like use error somewhere as you really need to use full paths for things or explicitly use them yourself. Otherwise it just wouldn't work. It's still failing though. Cannot move out of borrowed content. Oh, I see this is gonna have to do like a, right? Because we only have a mutable reference to self here and so we can't move out of these fields. We need to clone out the string. Great, okay so that passes. Let's see whether the next one, oh we did four. So five should be a freebie, great. So five should just work, great. Can't you use the trait at the input level? Use the trait at the input level. Yeah, so I mean you could in theory use the trait here. My point is more that you can't rely on the user having imported the thing that you're naming. So whenever you're naming a type that's sort of external to you, you need to explicitly give out this path or explicitly use it. Okay, so six, optional fields. Okay, here's what I want to do. Before we do this one, I want us to fix up this one. So instead of doing this one yet, what we're first gonna do is have this no longer be specific to command. I want us to actually iterate over the fields, right? This is, it doesn't explicitly say this in instructions but we can do whatever we want. We're learning. Okay, so let's think about this. Well, so quote, if you take a look at it, it has, what is it, right? So it has the ability to do repetition. This is similar to the dollar parentheses star that we saw for the macro rules in the very beginning. And this lets us basically evaluate an expression multiple times as many times as the variable that is used inside appears. In particular, as long as the variable is something that is iterable, it can do the appropriate thing. So what do we want to do here? Well, what we want to do is we want to iterate over all the fields of the original thing and wrap the type and option, right? So how's that gonna work? Well, remember how we got the syntax tree way back up here, right? So really what we want to do is we want to walk the fields of the data struct and then we really just want to replicate them in our own, but we want to wrap the type in an option. So that's gonna be something like, I guess, let fields is gonna be AST.data if let send data struct is this else. Unimplemented. So keep in mind that the user could in theory put like a derive builder on an enum, for example, instead. And that is something that we currently just do not support. So we're gonna assume that the derive is on a struct. And here you can imagine that we should give better error messages. I don't know whether that's a later exercise, but you could imagine if the user puts like derive builder on an enum, we really want to highlight the enum part and say only works for structs. All right, so the fields are gonna be fields. And we really want to do, if let, actually it can be even better here. Um, so the struct contains a data struct, which contains a fields. We don't care about the other things fields. We're expecting to be a sin going back here. So where's the data should be under D data struct. So a data struct has a fields. And so this is gonna be a sin fields and we're expecting it to be named, right? And that contains a fields named, fields named contains a bunch of other fields. And the one we care about is ref named. So that's no joke, right? And then let's take a look at what happens if here we put fields. Don't actually know what this will do. Actually, that's probably gonna not work in all sorts of ways. But if we go back to here, let's see what this expands to now. Oh, I guess, if let, what did I do? Oh, here. This should just say named. Let's do our cargo expand. Ooh, data struct does not, this should say fields. Okay, so this now expands to command builder and notice that it now auto filled out all of the fields. All the fields are here. They're twice because we wrote them out manually as well, right? So we both have this hash fields and these. But what we really want here is when we're printing them out down here, we want to print them out essentially the same as they were, but with an option around the type that they contain, right? So named here is in fact an iterator. If we look back at here, so named is punctuated. And a punctuated, if I remember correctly, implements into iterator. Does it also implement? Great, yeah, iter. So really what we want to do here is down here, we're going to do expansion of fields, right? So let's see what this will do. In theory, I think this should generate basically the same thing, this way. Great. Yeah, so this now generates the same thing. And the reason for this is this pound open bracket star, what that means is call into iterator on the variable that's used inside, iterate over it, and for every element, run the thing inside with the variable set to the item you've got. And in this case, fields is the iterator over all of the fields. And so for each field in fields, we're printing out that field followed by a comma, which is the same thing that you would normally generate. Of course, this is not actually what we're gonna generate, optionized, is gonna be fields.iter.map. And then this is gonna be optionized, right? Because really what we want to do with these, as I mentioned, is replace the type with the same type but wrapped in option. So going back up to where we got the AST. So we wanna leave all the other things, but we might wanna set the visibility manually. In fact, maybe we just wanna create our own field. Come to think of it. That's a good question. We might wanna do that. Because we don't wanna, we don't want to inherit things like attributes on that field, the visibility of that field, colon type. We don't wanna inherit any other things. We just want the ident in the type. So let's manually construct a field instead. So we're gonna map the fields that we get and we're gonna map them to sin field. So let's look at what sin field gives us. So a sin field, field. Oh, we can just construct one. Great. So we wanna generate a field whose address is an empty vector. Whose visibility is a sin visibility public. Actually, no, just inherited this line. The ident should be the same as the one in the original field. The colon token is gonna be same as the original one, I guess. Seems fine. And the type, this is where we don't actually want this to be the original. But let's just see that it expands to the thing we expect. You need to clone this probably. Great. So this still generates the same thing, but now we have a starting place for actually modifying these fields as we desire. In particular, down here, we want to create sort of an option wrapped type. So f.tie.clone. So this is the original type. And we're gonna be modifying that type to wrap in an option. So if we look at something like vec, right? What that really does is it's, so vec contains a t, right? And we wanna generate option that contains a t where the t is the original field type. And so we basically want the same kind of wrapping as is applied to vec. So looking at this, so what we really need to generate is the same structure here, right? It's gonna be a path. It's gonna have a single path segment. The ident here is gonna be option. And the arguments are gonna be basically the same as for vec. But the inner args here is gonna be a vector of the original type. So you'll notice here that here it starts with path, right? This. And that is the same as this path. Everything below sort of nests. So we're really just gonna nest that inside the option. So let's see it how this is gonna look. Let's get part of the stream. This is based on David Tolne's proc macro workshop. So we're gonna have to generate, this is like original type. And then our type is gonna be a sin path, sin path, which contains, ooh, leading code. Oh, I see, sure. It does not have a leading colon. So it's gonna be this. Leading colon is gonna be none. Segments is gonna be punctuated path segments. Okay, so how do I construct a punctuated? New, great. So let segments is gonna be sin punctuated. So punctuated here just means an iterator that's separated by some punctuating character. In this case, a comma colon for a path. Punctuated, punctuated, new. All right. And then here we're going to segments.push. Right, so there's only gonna be a single segment here and that's gonna be option, right? That's the only type we actually want in here. So remembering back to Vec, right? The original thing is a, what's the difference between this path and this path, though? I wanna, oh, it's a type, isn't it? Yeah, interesting. So we actually need to generate something slightly higher, which is, it's gonna be a type path, which is gonna include a type path. So this is gonna be a sin type path, which contains a Q self. I don't know what Q self is here, but it is none for the Vec, so it'll be none for us. And then it's gonna contain a path and that's gonna be this path. So for the punctuated one that we had, oh man. Right, so we're generating, remember we're generating the segment to the option type. So this is the, let's see if I can explain this better. So look at a type like this one, right? What is the type of args? Well, the type of args is a segmented path to a type, right? Logically, it's something like A, B, C, that. So the segments of this path are A, B, C, and option, including its parameters after. When we, given that we just wanna generate option, we want a path with a single segment and that segment is gonna be option of the type that the user gave us, right? So hence, we create a new segment, we're gonna push a value, not, so remember a punctuated stream has delimiters as well. We don't wanna push in delimiters, we only wanna push the value and in particular the value we're gonna push is a path segment, all right? So what is a path segment? Oh, here, great, fine. So we're gonna push a sin path segment. Path segment has an identifier, now we're getting somewhere, and the identifier is gonna be the identifier of the original type, original type dot indent. Dot indent, actually is that even true? Dot indent, yeah, right. Should force it to standard option, option? Yeah, arguably, we could do that. I mean, it's pretty easy to do that, so we can do that. But I'm gonna not do it right now. So let's do a to do, use standard option, option, not just option. So the reason for this is you could imagine that the user defines their own type named option, and our code is currently written would end up using that option type, right? Which is not really what we want here. We want to use the one from standard option, and this is why really the macro should generate this path. And then the path segment also contains arguments, okay? So the arguments is gonna be, wait, I'm lying here. This should be option, this should not be that. This should be sin, ident, new, option. And what span should this have? I don't actually know how to give this a span. Oh, you're right, we can just use quote directly. That's a good point. That is far better. I totally agree with you. That is far better, I did not think of that. So this is just gonna be, quote, good thinking. So this is gonna be, quote, name is gonna be f.tie.ident, tie is gonna be f.tie.tie. Actually, I'm lying, this should be ident. This should be tie. You're right, that is far easier because we can get rid of all of that and do this instead and say name colon. Now we can even do this pretty easily. I love quote, I forgot how much I like quote. Like so, let's see how that works. Great, that was far easier, you're totally right, thanks. Right, so now we see it actually generates all the fields for us, right? And now we don't need these manually added fields, right? They are now just noise. And you can imagine that we can do the same thing down here, right? So for the methods is gonna be the same kind of thing. It's gonna be, quote, pubfn name of the field. It's gonna take mute self, name colon tie mute self. Self.name is some name self. And now this can just be methods. Let's see if that works. What does this expand to? Oh, did I lie? Probably lied. Yeah, we need to extract. So this now expands to, great. So we get all the same methods. Right, beautiful. Almost as though it was planned. And what else do we want? Well, in build, we also wanna generate these. So let's generate them. Clone to take in build. So okay, I'll address that when we get to it. It's a good question. Okay, so we're gonna generate this build method and it's gonna have all these lines. And in particular, it's gonna do something like name self.name. in this case clone, okay, or. We're gonna have to figure out what should go here. I think what we want is stringify. But can I, oh, that's a good question. Is there a way for me to join these strings? I wonder, isn't there a concat, forget this, concatenates literals, great. So I want concat this with is not set. And now this is gonna be same thing again. Build fields, comma separated. Okay, so one question was raised here in chat. Here, cargo expand. What did I do? Two token is not, this is to be a star. This doesn't need the type. Okay, so this now generates the right build. And there's a question from chat, which is instead of this being clone, should it be take, right? So option as a way where you can take the value that's in the option and leave a none, that way you don't have to clone it. The reason we want this to be clone not take is because if it's take, you couldn't reuse the builder without resetting the fields. And that would be, it would be fine, you could still use the builder, but it would be unintuitive. Arguably here, build should just take self. In fact, that's something we can fix. Build should just take self to sort of document the fact that it doesn't change the builder. And now hopefully all the same tests will pass. Ooh, not so much. Why did type string found results? Interesting, missing question mark in build. Oh yeah, you're right. This should be have a question mark. You are totally right. Good eye. Beautiful. Okay, so now we pass all the tests, but we no longer have any of this repetition. I guess actually we need empty fields as well. So this is gonna be build empty comma separated with start. Okay, so at this point, what did I do? What did I do? Specter identifiers, remove this comma. Right, so now we pass all the tests without specializing to the particular fields that were being given. In theory, it now works across any fields that are given. Beautiful. All right, so now we can try our hands at the next test. So optional fields, let's look at this. Some fields may not always need to be specified. So currently remember, we require that all of the fields are set, right? We do this like unwrap or business. And specifically something that's an option T in the original struct shouldn't need to be set later on. Have your macro identify fields in the macro input and use type as option and make the corresponding builder method optional for the caller. In the test case below, currentDir is optional and not passed to one of the builders. Right, so currentDir is option and so building one where you don't call currentDir should not be wrong. It should allow you to have that be set to none. Let's see, be aware that the REST compiler performs name resolution only after macro expansion is finished completely. That means during the evaluation of a procedural macros types do not exist yet, only tokens. In general, many different token representations may end up referring to the same type. For example, option and standard option option and VEC option T as into iterator item are all different items for the names for the same type. Conversely, a single token representation may end up referring to many different types in different places. For example, the meaning of error will depend on whether the surrounding scope is imported standard error or standard IO error. As a consequence, it isn't possible in general for a macro to compare two token representations and tell whether they refer to the same type. In the context of the current test case, all of this means that there isn't some compiler representation of option that our macro can compare fields against to find out whether they refer to the eventual option type after name resolution. Instead, all we get to look at are the tokens for how the users describe the type in their code. By necessity, the macro will look for fields whose type is written literally as option and will not realize when the same type has been written in some different way. The syntax tree for types, parts from tokens is somewhat complicated. So this is what we've already dug into this, right? Which we're gonna get into in this question. So here's the nested, this is the same thing that we saw, and in particular, what we're really looking for is this business, right? We want to look for somewhere where the identifier for the type is option. All right. So the only place where this really comes up is for build. So in build fields down here, we don't want this to fail for fields that are optional, right? So we want to do something like, so the type is gonna be, and we're gonna have to sort of basically dig down into the type hierarchy for this field. So this is gonna be f.tie, which is a type. We're gonna assume that it's a type, if let, sum type path is f.tie, if, right. If, right. So p now is a path, and a path, we remember have the fields q self, and so a path contains a type path, which has a field path, which has segments, which has to have at least one segment, right? So here we could do something like if segments it are last unwrap, because we know there's at least one element there. So that will give us a path segment. Actually, let's do fp.path.segments.len is equal to one, and the last one's ident is equal to option. We probably won't be able to do the equality this way, but we'll see. So if that's the case, then we want this. Otherwise we're gonna do the standard thing. Then we just want this, right? So I think we always know that this is a path. No, that's not true. They could inline a type. But if it's a path, and that path is, does not have any colons in it, basically, and I guess at this point, this can then be zero. And if the identifier of that one path segment is option, then we know that they're just option types, is we don't need to do this unwrapping. There's a separate observation here, which is if it is an option, we don't need to store it internally as option, option. So that's the other thing we might wanna do, which is up here, we could do, let, in fact, let's just do a convenience tie is option, tie is option, which takes a tie. Actually, we can do this even, right. It takes a field, and it's gonna do this. It's gonna tidy up our code a bunch. Excellent. Now you could imagine us optimizing this for like, if they wrote standard option, option, it would also work, but we're gonna ignore that for now. And now down here, we can say if tie is option for field, then this, else, this. The reason I wanted to do that is because we wanna do this for the fields as well, so that we don't end up storing an option, option, right, which is just unnecessary, because currently, we're just unconditionally wrapping an option. So we wanna do if tie is option, then leave off the option part. And similarly, for the methods here, we want if tie is option, otherwise this, then just set the field, right. So this will still be option because that was the underlying type, but now because the inner type does not contain an option, we're still good. Let's see what this generates. Do I really? So this is gonna be a sin field. So if we look now at currentDare, this example does not actually have that. So I guess let's take 06, which had an optional field. I think so. And then change main to have this instead. And now do a cargo expend. So now currentDare is now stored just as an option. Notice it doesn't double capture the option. It just uses the option type from the original code. And currentDare just takes an option, sets self to be that, and currentDare does not unwrap. And so let's see if we now pass six. Ooh, we fail six. They're expecting currentDare to not even take an option. That's interesting. Oh, that's very interesting. So specifically, if we look at 06, notice that when they do give currentDare, they're expecting this to just take a string argument, even though the underlying type is option string, which would mean that there is no way to unset currentDare, but I guess it makes sense for ergonomics. So this is gonna require us to do a little bit of trickery because we basically need to remove the option, the wrapping option type, which is a little more annoying. So this is gonna be let unwrap option T, which is gonna take a type, send type, and it's gonna give you a send type. Assert this, shouldn't be reachable because this should only be called if this is true. I guess we can instead just do assert tie is option. And I guess this should take a type, shouldn't it? So this is gonna be type, this is gonna be type, and this is gonna be, so we're gonna assert that tie is indeed an option. And then in here, if you recall back to, do we have a convenient way to get at the expansion down here? So the actual type that's contained in an option is like hidden down in this arguments under args, right? So we really want to extract the type that's in here. So how do we wanna do that? Inner tie is gonna be P. So P is the path segments zero arguments. So what is the type of arguments? We're gonna assume that that is this, f arguments, this, and otherwise here we can actually give a helpful error message, which is like, ooh, which is something along the lines of unimplement. It's really a panic. It's a panic saying that option did not, option type was not option T, right? Because this means that it's taking some other arguments or it has no arguments. So arguably this should be a part of this assertion. So I guess, and if let, we really want this, don't we? Actually, how about this just returns a reference to the inner type? That is much nicer. That way we don't need to write this multiple times. Yeah, let's do that instead, that's much nicer. Is some, is some, and is some. All right, so if it's angle bracket, then this should give us this business. So let's look back here. That gives us one of these, which is, so this contains the full syntax tree, right? Which is why it needs to have all these surrounding tokens and stuff as well. All we really care about is args. We don't care about the, the surroundings. So we're going to do inner type. Dot args. So those are going to be the generic arguments to that type. And then we're going to, that's going to be another, another iterator over generic arguments. And we expect that to only be a single type. If I, this punctuated, does that give me a length? Does, great. So if length is equal to one, we could also invert some of these. So if this return none, makes the code a little less indented. Okay, so here we know that we have exactly one argument within there. If let, there's a first or something for this as well, right? For punctuated, first. Great. So if let, actually no, it's not even that, is it? Punctuated, first gives me an option pair. Okay, so inner tie.args.first.unwrap, right? So we can unwrap because we've already checked that the length is one. And that gives us one of these pairs. And in particular, what do I want from the pair? I want value if let, right. So what that is going to give us is one of these, same generic argument type, t is equal to this inner type, then return sum of that t. Great, so this should give us the inner option type, which should mean that, well, so here in theory, we could rewrap it an option. I don't think that's important, but it does mean that down here, if let sum inner tie is this, then this should take inner tie and set it to sum of that 75. Who did not like that? Oh, this, we can just make this a normal function. It takes one of these and it returns one of those without a semicolon. Why is that not okay? Fine, borrow checker, value, that's fine. What does it expand to? That's what I'm more curious about. So now it still generates option string, that's fine. Current error now takes a string, so that is what we wanted. And this seems to generate the right thing. So why is it complaining? It is complaining that it expected a vec. Oh, are we expecting the vec as well? Yeah, so for args, notice that we're turning args into string as well, even though that is a vec, not an option. So really what we want here is if the segment is not one or if it's not option, that's what it really should be. Yeah, so now the only thing that is kept option is current dear because it already was an option. All the other things retain their inner type, although this is showing that our type extraction is working for vec as well. And current dear is a string. Beautiful, now let's see if we pass the tests. The real question. Hey, look at that, excellent. Okay, so we now pass the first six tests. Time for number seven. Yeah, so command here is based on, in the standard library, there's also like a command builder basically. But when you pass many arguments, you sometimes just want to add one argument. You don't want to replace all of them, right? So in particular, you want to do things like this. You want to give this is an arg and this is an arg. Currently we can't really do that. In particular, what this is asking is, let's add an attribute to fields that have fields that are iterable basically, or that implement extend and generate an appropriate method if they have this field attribute tag. So that this is really saying add a builder method arg that when called extends this thing with the given argument. The generated code may assume the fields with this attribute have the type vec and should use the word given in the string literal as the name for the corresponding builder method, right? So this would be called arg, this would be called n. In order for the compiler to know that these builder attributes are associated with your macro, they must be declared at the entry point of the derived macro, otherwise the compiler will report them as unrecognized attributes and refuse to compile the callers code. Okay, so this means that up here, we're gonna have to say that this also takes attributes builder, attributes builder. These are called inert attributes. They do not correspond to a macro invocation on their own. They're simply looked at by the other macro invocations. Great. Okay, so this means that for methods we also want a different one. We want extend methods, which is gonna be basically if, so looking back at what we get back from a data struct fields, fields named, point-oriented field. So remember that fields have this atters, which are other attributes on that field and it's a VEC of attributes. Okay, so if field dot atters, if not field dot atters dot is empty, then we're just gonna sort of print out F atters and just see what it is. I don't know if this will work. I just want to quote an empty thing to still retain the type. And then I want O7, so that's a good word cargo expand gives us. Interesting. But how do I, oh right, I don't actually invoke it. Do I extend methods go here? Right, so it did find some attributes, right? So there are two things in our original input. That have this builder attribute on them. So we would expect to see two attributes that are non-empty and they're indeed both here. And what we wanna look for is we want to look for ones where the path, which is the thing before the, so if you look at it here, the attribute has some like symbols and stuff and then it has a path, which is the name of the attribute and then it has a token stream, which is the tokens contained within that attribute. So in this case, the path is builder, right? So recall, this is the pattern we're looking at. So the path is builder and then the token stream inside is gonna be each equals n. So if we look at here, we get a group delimited by parentheses, right? So those are the arguments. And the stream inside is, there's an ident each. It's punctuated by an equal, then there's a punctuation equals and then there's a literal, which is nv, the string nv. So we have all the information that we need here. And so let's use it. So what do we wanna do here? Well, first of all, we wanna, we probably want to do, this has to be a filter map, right? Because we don't wanna emit any methods. Oops, that's not the right place, is it? Here. Let's base these out a little. So if it's empty, then this should return nothing. Otherwise, we wanna deal with the attributes. Well, first we wanna check that it's actually a relevant attribute, right? So we're gonna do like something along the lines for, actually, then we might not need this for after in F dot adders. If we get back here, we're gonna say none. And for each attribute, we wanna look at, if we wanna look at the path, adder dot path. And the path has segments like before, right? Dot lan, not equal to one. Actually, if it's equal to one, and adder path segments zero, which is a path segment, dot ident is equal to builder. Then we wanna do something with it. Well, what do we wanna do with it? Well, then we wanna look at the token stream and look for, we expect it to say each, right? We expect it to have an equal sign, and then we expect the string that's gonna be the name of the method. So here, we're gonna do something like assert adder dot, let's see if let, so the token stream is gonna be adder dot TTS, right? And we wanna walk the token stream to get to check that these tokens are actually the right ones. So we wanna assert that TTS dot next, assert equal, we want that to be a group. So what is a group? Well, I guess this probably re-exports token stream would be my guess or would be my hope. Apparently not, oh, maybe it's a re-export. No, interesting, okay, so we'll go over here. So a token stream, right? So a token stream implements into iterator and it gives us token trees and a particular group, which is what we want. So if let rock macro, oh, we can probably even parse this. What does SIN give me here? Maybe SIN has a convenient parser here. Parse, parse, they simplified too full struct. Oh, it's just gonna be a parse macro input and then we're just gonna have to figure out what to parse it as. In particular, we wanna parse it as attr attribute, okay, let's see, drive input. We're gonna go some syntax, I'm like fairly sure there's gonna be a, or I really want like inert, but it doesn't seem like that's there. Could be attribute, yeah, but the real question is how do I parse, oh, I see, it doesn't, it wouldn't know what to parse it as is the real problem. Ooh, examples, attribute parse outer. The TTS fields can do the rest of the other body token. Try parsing the tokens of an attribute into the structured. All right, so it's already parsed basically as much as it can. So we're expecting this then to be token tree group G is attr.tts.next. Let's see that that actually gives us what we expect. This probably has to do like, is there a convenient way to clone this? If I have a token stream, sure, that seems fine. Proc macro two, ah, so now we get to, so notice that it's referring to proc macro two. So proc macro two is a crate that's just a wrapper around the compiler provided proc macro crate that basically gives you backwards compatibility. It gives you features from Knightly that aren't stable yet and it gives you some, basically the ability to use this outside of just a procedural macro so that you can write tests or like basically do things when you're not in the context of running within the compiler. So these shouldn't really matter. Like it's fine for us to do this. And I don't think this has anything more useful on token stream. I think it's basically the same. Yeah, we probably need to depend on it, don't we? Macro two is zero to four. This probably has to do something like some quotes just so that it can infer the type. All right, so what does this give us? All right, so it gives me the group. That's great. And now we wanna look at the stream. So this is proc macro two group. And we want the stream from that. And then we want, what do we wanna do with that stream? Tokens is gonna be stream.intuitor. We wanna assert equal tokens next. It's gonna be a, I guess we wanna do next. I don't know if this will even work, but we can try. That's really what we wanna say, right? We expect that there's an ek. We expect that there's an equal. This won't actually type check, but, and then arg is tokens.next.unlap. What rust plugin are you using? Ale, it's pretty nice. I'm a big fan of it. What do you mean group does not have stream? Oh, fine, fine, fine. G is gonna be g.stream. Token tree, you say. Can I do, ooh. So I can probably then do quote each into. Is that something that would fly? No. See, I just don't wanna fully parse this. And I don't think it should be necessary, but I guess unwrap, unwrap. So this really should be, how about now? Will that let me do this? I mean, it's not terribly surprising. I want this to be a proc macro to token tree. Actually, I just wanna use proc macro to token tree, just cause I'm gonna be repeating it a bunch. I want this to be an ident. Probably I'm not allowed to say that, am I? Expected ident find reference. Let's see, what can I do with a, do I really need to like walk it? Cause that sounds really unfortunate. I guess this would be panic unexpected this. It might be happier about this. What was the thing before? Thing before is a punked, which we want to be equal to this. Expected each, expected equals found that. See what this gives us. I'm not allowed to compare punked to that. Okay, what can I compare punked to? Apparently nothing really, but as car, I can compare. This does need to be mutable. That's true, beautiful. All right, so now we get just the literals, which is what we wanted. Now we just wanna extract the literal from the literal here and that gives us, but how do I get out the value? Oh, it implements display, I suppose. But that might actually not matter. We don't actually need to get out the value. All we really need is now we have that. Let's see what this gives us. So if we now do FN arg, like if I do this, what does that give us? What does that expand to? Expected identifier found arg. Yeah, so now it's, basically this turns into each, right? Because the way that this is written is this is a string and I want just the inner part. So the question is how do we do that? Is there a way for me to turn a string into an identifier? Yes, there is. It's gonna be ident, oops, arg is gonna be ident new. Using arg and arg.span, this is syn ident. Right, because I want to create an identifier for this method and I want it to be tied to the span of the original each part so that it's spelled wrong, you still get something reasonable. So this I want to be this and it needs a string. Proc macro derived panicked, did it now. What, arg is not a valid identifier. I really, I can't print this? How do I get, how do I turn a string into an identifier? Let's see if there's something in 07 here. Okay, that's fine, inert attributes, the word inert indicates that these attributes do not correspond to macro implications. That's fine. So this token tree, when I have a literal, how can I extract the contents of that? Because display is just gonna display it as a string. So how do I parse that out? Interesting, interesting, interesting. Syn, can I parse an identifier or a literal lit stir? Yeah, specifically, can I turn my literal into one of these? Is there a parse literal maybe? Parse macro input, is there a literal type? All right, so how do I make one of these? Aha, great. So I can do synlit new from this. So that gives me a little, that gives me a lit, and then I should be able to do if let synlit stir, is this, right? Actually, I can just do the same here, right? So if I get a synlit stir s, then that's great. Otherwise, panic, expected identifier found, it didn't really expect an identifier, but it expected a string found whatever that is. And now, right, but if I got a stir, then I'll presumably from a stir, I can actually get the string, value, great. So now I can do s.value and s.spn. Let's see how that works out. Expected literal found token tree. Oh, right, we need to let arg is match, chokens.next.unwrap, and we have to do the same thing here, except this should be literal, l, l. Expected string found that, and then this is gonna match on the arg. So panics are bad practice, but when you're writing procedural macros, you are specifically supposed to panic. That is the way you signal errors to the compiler. Now, I think that the next exercise is basically how do you give good compiler messages from procedural macros, but it is still all about panicking. We'll see that in a bit in the next exercise. Be formatted with the default formatter. How am I supposed to use it? Great, so here you see, we did in fact get a function with the name arg and the function with the name end, which are the things that were in here. Great, so we did actually get something semi-reasonable. Now, of course, what we really want this to be is we want this to contain the inner type. So I think here's something we probably want. We want this to take wrapper to be a string. So this we want to extract from option. This we want to extract from option. This we want to extract from option. However, here, we want inner type from VEC. The specification said we were allowed to assume that this was a VEC. So this is gonna take arg, I guess arg of type inner type. It's gonna also take, I guess, mute self and it's gonna return mute self and the body here is gonna be if self. So I guess there's still name here, right? F.ident. If self.name.isNone, then self.name is some VEC arg else. I guess we can do it the other way around. If let some refmute args or values is self.name, then we just do values.push arg else this and then return self. See how that expands too. Expected lifetime parameter, really fine. This string is not a part of the return type so it's just tied this way. Now what does this expand to? You can't do anything without using unsafe. That's not true. There is no unsafety in this code. There's in fact no unsafety in most of the code I write. So that's just not true. Okay, so what does this generate to? Arg is if self.args is some, then we push otherwise. Yeah, so this is what the VEC macro actually de-sugars to. Great, does that pass though? That's the real question. This theme is called groovbox. I recently switched to it. It's pretty neat, pretty happy with it. Duplicate definitions with name nv. Oh, that's sneaky. So this one, the name of the builder method and the name of the field are the same. So we actually don't want to generate a, for anything that has a builder, we specifically don't want to generate the standard one. It's a little awkward. Like the question is basically for args, should it be possible to specify all the arguments at once? It sort of has to be, right? Because the previous test assumed that you're able to just call args. So it's specifically if the name is the same as the field, then don't generate the normal one. How is it possible for the past test to even work here? Like this is spec, this test expects nv to take a vector. This one, this one expects, well, this one doesn't actually expect anything. Oh, that's super weird. I don't actually know what this wants me to generate. I think nv still needs to take vextrain. Because otherwise, the other code samples would no longer work, right? So let's look at five here, for example. This still calls us args and still calls nv and it calls them both with vectors, right? So here, we still need to generate args and we still need to generate nv. Actually, that's not true. I guess using this could imply that you don't want the default one. I don't actually know what the exercise wants us to do here. It sort of makes sense to generate both when you can. The question is when you can't. I think when you can't, you don't generate the vector version because the user has explicitly said they want this one. So that's gonna be a little awkward for us because it means that if you generate an extended method and the name of the extended method is the same as the name of the field, then don't generate the normal method. Also, I don't think this interacts correctly with if the user had to type those option back, this wouldn't work correctly. Although that doesn't actually come up here. Hmm, we want to do this. It's a very good question because we could do these same checks when generating the normal builder, but I think really what we would want is just a single one, I guess, avoid conflict. And then down here, we want to say, if arg is equal to f.ident, then avoid conflict is true and then we'll just generate this. Yeah, I think we're gonna have to do something like that. We're gonna have to tidy it up more than this, though. Extended method takes an f, which is a field, and generates an option token stream. Combination, I guess, of a bool and an option stream. So this would be instead of this, this would say method equals this and then return some, and if arg is equal to f.ident, then it conflicts because now methods, this can also be, this doesn't rely on anything outside, so this can go down here, tidy up our file a little bit. So really what we want to do here is conflicts are conflicts and extend method is, so let's see, so if it conflicts, we don't want to generate this. Although, I mean, we can generate it regardless, right? So method is this, I guess, set method, and then we can do here, match on that. If what we get back is none, then we just return the set method. If it's some of true and extend method, then we give extend method, right? Because we can't also include the set method because the set method would have a conflicting name, and if we have false extend method, then we produce quote of set method, set method and extend method. See what that gives us? It gives us a whole bunch of errors, what it does. And then now this should just be methods. Oh, I see, into expert is that, and then it's a bunch of errors. It from that is not implemented. Some things very funky here, extend, I guess this should really just say extend method. Why is this getting weird? Token stream implements two tokens is not satisfied. I guess one question is, you don't really need to do the into down here. That's mostly just causing us pain. In fact, this isn't even returning a token stream currently because I see, oh, I see what's going on. This is a proc macro two token stream, but are here, so I didn't, shouldn't, I didn't knew just gives me a new right, do I need, I don't need to unwrap it or anything. Yeah. And f.ident, if I have a field, oh, I see, could the fields ident be, okay, I see. So this really should be, really, I can't compare two items. Am I just confused about what? Oh, yeah, this no longer needs that. Great. All right, so let's see what it now generates. Puff and args, puff of an arg, puff of an nv, which is the single extend. And it does not generate the conflicting, the conflicting nv that sets all of the method. Great. Okay. See what that does. Ooh, failed something. Seven failed. nv is not set. Oh, they want us to not even make those, when they have each set, it shouldn't be turned into an option either. Interesting. So the example to consider here is, if no nv is ever given, should it fail or should it give you an empty vector? In all the previous examples, it gave you an empty vector. So it seems like seven is a little bit under specified. Assume the fields of the type veck. If we can probably turn no. Okay, so I think this is assuming that if something has a builder tag, then it should no longer be an option. It should be an empty veck instead. Which I think means that what we want here is, built, I guess. And that's gonna be something like F dot, that's all sorts of weird. Enter any F, builder of, which returns you an option, I guess, proc macro to group. So this is gonna be these, then return some g. Otherwise give me none. So now, this should be let g is builder of F. And this no longer needs a return. So that is a little nicer. And what this means is also that here, else if builder of F is some, then we also just wanna do that. And here, what we wanna generate is different as well, because it's no longer gonna be an option. So instead, this is just gonna take name, right? So if a vector is given in, it should just set the vector equal because the vector does not have a type wrapper. And I guess down here, if that, oops, and I guess these are the same. And down here as well, or builder of F. And here, we wanna generate, when we generate the initial builder, anything that is a builder, we wanna generate as a back, an empty back. Actually, let's do, can you, otherwise we're gonna generate a none. All right, let's see what that expands into. So I enter type. So now in the builder, args and n var both just vectors, they're not options anymore. When we create a new one, they're just created as vectors. When you call build, oh, ale does, when you call build, they're just cloned, they're not required to be sum. That's weird too, I guess, sure. And for args, nope, args should take a vex string. So that is wrong. This should take the type. So args takes a vex string and just sets self.args to that. Arg takes a string and pushes it. And n, because they would conflict, is only the builder version, so it only has a version that takes string. Let's see what that gives us. We still fail the last one. Spect its direct vex, found option. Oh, right. The code we generate for the extended method here should now always just do self.name.push. Cause it's now no longer an option. So notice how previously the code for adding just one of these things for an extendable attribute would like do a sum match on self.thefield. But in reality, because there are now no longer options, it should really just do the push. Let's see, hey, okay, we passed seven. Now we're getting somewhere. So progress, compile fail. Okay, so this is an interesting attribute of try build. So try build also lets you write tests that are not supposed to compile, basically so that you can check the output of your compiler. So you want it to provide helpful error messages. And so this is probably gonna just fail, but let's try to run it and see what happens. In fact, let's just speed this up a little by commenting these out for now and see what it gives us. So here, let's look at what this, so this is someone asked earlier about about the fact that we're writing unwrap in so many places and isn't that bad practice. And it is bad practice in Russian in general. In the case of PROC macros, panicking is the way you communicate to the compiler that something went wrong. Preferred way to report an error from procedural macro is by including an invocation of the standard libraries compile error macro and the code emitted by the procedural macros. Interesting, interesting indeed. So here, right? So here the problem is this is missing an H. So let's look at sin error. So here, oh interesting, what does this even do? Interesting. Okay, so we probably want to generate a new one of these. It's a difference between new spend, some other spend with a parse stream. Yeah, so that's a good question. Yeah, we don't actually have a parse stream, although it could totally be that we could. But at this point, remember what we get from the token stream is like ident equals literal. As I don't think there's any point in parsing that directly with sin. So instead, let's just generate the message manually, which would be by doing, so this is up for the each, right? So here instead of asserting that it's equal to each, it's like if I not equal to each, then well certainly sin error and the span is gonna be the span of the I and the message is gonna be, the real question is what do we do with this error? See that's what I'm trying to figure out. Because once you have one of these errors, we do render the error. Like why does this return a token stream? Oh, I thought I mentioned that. So I use the ale, which is a Vim plugin that gives you highlighting up compiler errors inline. Ale, how do we make it? Like what does this method do? Oh, I see it generates, it generates a compile error statement. Oh, I see what's going on. So really, it's not really that we wanna panic, it's that we wanna generate the syntax tree that has, so there's a macro in Rust, that's compiler error that lets you emit a compiler error at a particular point. Okay, so in theory, this should be able to return some false this to compile, maybe. So, oh, let's see what this expands to. Yeah, so here we now get expect, yeah, so now we get what we wanted in the highlighting because we take the span from the right place. Gives us, in theory, the right error message. So really what happened here is the, what we produced was, basically when we tried to generate the method for this line, like when we're trying to generate the output, what we instead generated was an invocation of the compile error macro, which caused it to produce this error up here. I don't know if that's what I wanted, but in theory it might be. So what do we see here? It wants us to highlight the entire thing, so we actually specifically highlighted the keyword that was wrong, and it's wanting a wider highlight, which of course is relatively easy for us to provide. So instead of this being the ispan, we're gonna make it the dot span. Our builder off is too helpful. Oh, that's fine. We can just have this be F dot adders zero dots span. That's me just sort of bullshit guessing, but, so what does, what do we have on field? Field adders attribute, huh. But how do I get the span of it? Why can't I get the span of this attribute span? The span of this attribute span. Oh, really? All right, take a look at that. What does it now highlight? Oh, what that doesn't seem right. That should not be the span of this attribute. So I mean, what it wants us to highlight is everything between the square brackets, and it seems like we can only really get the spans for this, or the span for the attribute is just this for some reason. So maybe what we need to do is generate, can I merge spans? Probably, join, what does join actually do? That's why I can't die. Really? Of a group, oh wait, just send. So on a span, if I join to, what does that do? Join, inner, what is inner? Oh, DocsRs proc, where is the real proc macro here? Does this have a span? Great, what does span join do? Join, okay, what does it do? Self-dodged zero, thanks. Span contains a bridge client span. That's very unhelpful. I basically want to know, I want to know if I join two spans, what do I produce? Is it like disjoint parts, or do they get combined? Because I think really what I want is, I want to sort of, what happens if I join this with the i.span, for example. Like this won't produce anything particularly useful, but oh, you don't, that's annoying. The real oddity here is that the span of this attribute is not the span of the attribute, right? Like this is not a sensible span for that attribute. It might be that the easier part here is actually to, so basically to do what builder of does. No, see, it really just should be of the adder, the whole adder. So if we look at what a syn attribute is, right? The syn attribute, I guess it includes the pound, and we really want the path and the, like does this implement span? No, oh, it does, okay. So what happens if I do like adders zero path span? So that highlights just the path. Right, I wanted to, and if I join it with f adders zero TTS span, what was the requirement for me to get join from proc macro to join? December accept and not exposed by default. Can I create a new span though? Like basically how do I extend a span? This really seems like span is just wrong for attribute. What's the style though? It doesn't implement span, so it doesn't matter. Hmm, let's look at this. Join spans, okay, join spans. Is that a method that I get? Token macro instead. No, it doesn't really help me either. How do I make my own span? Hmm, I mean we could just leave it the way it was, but it seems a little bit sort of dissatisfying. Oh, actually, maybe this is a bug here because the token stream, like the span demomitation for attribute is that it joins the spans of the attribute's token stream. Oh, okay, it needs this for that to work, right? See here? Like it doesn't actually join anything unless it has this. So that's why it's broken, which suggests that we actually, I think the span in the test case is the one from attribute parse meta. Sorry, from sin. So attribute parse meta. Oh, I see what you've done. Okay, so what we did here previously is we sort of manually parse the stream inside of the attributes, I see. And it just so happens that there's a, attribute has a convenience method for parsing things that have a particular format. And so when we instead use parse meta, that gives us basically what we want here is name value, right? So name equals value pairs. I see, I see. So really what we can do here is something along the lines of meta is parse macro input. I guess that would be, can we do G as sin meta? That well defined, did not like that. Although into, is that something I can use here? Oh, that's not an expression, is it? Oh, sorry, this can just be, this is just the attribute itself. So rather than do all of this business, this is going to return an optional, I guess, sin attribute and then return adder. Yup, and then that will give us this. And then we'll not do the join, whether it likes this expected token stream, right into token stream. All right, I guess the alternative would be that I could just call this. I guess the question is what is this result? Oh, I see. So if I get one of these, no, that's not what I meant. G parse meta. So there's gonna be this. If it's inner then I, if it's an error then it's already determined what the parse error is. And so at that point I can do this business except the error is already constructed for me. Although can I give it a new message? I see that's as if it fails to parse. So this can still just be E and then we might need this for later. And this is not gonna be used. And now the question is what is this business? So this gives me attribute parse meta, gives me a meta and if sin meta imvalue. Actually let's just do then we're gonna do some stuff. Actually then we're just gonna return nb. If we get some other meta then we're gonna have to figure out what to do. Haven't decided that quite yet. Now down here, I guess if meta.ident not equal to each, then we're gonna pull this trick again. Right, then the ident is wrong. So we're gonna return false and then basically the span of the meta with expected each, great. E token we can ignore and then this is just gonna be meta.lit which is now much easier to get out. So that saves us a bunch of stuff. Let's see this is gonna be meta.lit. Excellent. And now the question is what do we do if we get something, if someone does like builder foo and doesn't say foo equals and in that case, we can also just construct an appropriate message something along the lines of this is still meta span. And I guess that's probably fine. We also don't need this and it doesn't like, right. This is to error. Expected proc marker two literal found. Oh, great. We even get the literal even better. So now, okay, so it's still not quite right. The highlighting now highlights builder which is still not what we wanted. And now it's using the YouTube stream is down for you. Well, that's kind of damning. Is anyone on YouTube and able to confirm whether or not you can see the stream? I guess I can check here too. YouTube as a whole is having a massive outage. Great. Well, I guess go to Twitch. I guess now there may be more people on Twitch. We'll just continue. I mean, people can always just catch up to the recorded video that's uploaded after. Okay. So we're still not getting quite the right span for, I'm assuming we're hitting this message. Oh, it's cause we're already icy. No, that's still not right. So I mean, we're presumably hitting this just to double check that I'm not going insane. I see. So it means at least you found one from the other. Expected builder. Oh, really we're hitting this one? Why are we hitting that one? What kind of meta am I getting here? I'm getting a meta, I'm getting a meta list. Oh, I see. Okay, fine, fine, fine, fine, fine. So this should be a list of FNVs.len not equal to one. Then we have another error, which is specifically a, oops, I expected only this. And then, okay, so what's the type of the inner thing here? Nested meta. What else could be the nested? Like the thing that I get back from this, if I get a meta list, nested is a punctuated nested meta. Okay, so we're going to do, we're going to match on NVs first. And if it is a soon nested meta, meta, name, value, then we're going to return that. If it's a, if it's any other kind of meta, same thing here, we're going to return this error in a lot of places. So arguably we can make this nicer. What does this give me? Meta list, you say, oh, not nested. Actually, that's not even true. Why does a meta list, I see what's going on. The list is, okay, so the list that it's naming here is if you look at this, this, right? If we look at this in isolation, so this is a list that contains, it's, it's a list, right? And that list contains builder, whatever, right? So that's the list. And then I guess builder, up here. Yeah, so it's a list and the, I didn't have the list as builder. It contains, nested inside of it, it contains a name value pairing. So that's really what the structure of this thing is, which means that we need to parse it as such. So we expect to get this back. We expect the list to have only one item, I think, right? In fact, meta list is not a list at all. So a list, what is it? Okay, a list contains a meta list. And a meta list is not a list. Oh, I see nested is a list. I see, oh, I see. So really the reason this is coming across is you might actually have someone write the following. If I write a equals b comma c. So this is a list whose identifier is builder, where the first thing is a name value and the second thing is a, just a, is a literal. Well, it might be a meta word, actually. I see, so that's why that can be that way. So if nv.ident not equal to builder, in fact, what does builder even do then? So our builder of, so this should just be true, because we call builder of and builder of checks that it is, in fact, the builder. So this is fine. And then if the nested length is not equal to one, then we write this. And then if the first thing is not a name value, then we also show this. That span should have the meta, which is not including builder. This span should have including builder. And this should have if nv., so that's a meta name value. If nv.ident not equal to each, then also return this error. So we're gonna create this error a lot of times. I would like this to be nicer. It's gonna take a, and it's gonna be this business. Great. So now this can be error nv.span. That is so much nicer. This can be the same. This is error meta span. This is error also meta span. And this no longer needs to happen, because that's, oops, because that's already happening outside. Whoo. All right, let's see how that expands. No method name first. That is true, because this is, we're looking at the nested arguments to builder. That gives me a pair and I want the value. In fact, in fact, this, we need to unwrap this. And then we're gonna match args value. This is yelling at me because, what exactly? nv.nested, it's because the punctuated iterator, when I call first just gives me access to the first element, which I don't own. Which means that, so for the this, is there a way for me to just take the first? Sure. nv.nested, pop, unwrap, because we know the length is one. That gives me a pair. And then probably something like into value. Great, into value. So now I own it. And now this needs to be mutable. All right, how about now? Great, and this gives me an error. And it's just builder. So we've gone like full round trip on getting the span to be back to what it was. Now that said, I think the code is much nicer now because we're using this like meta parsing. It's probably also more robust. So I guess, no, it's not that one. It's this one. So this we're saying should really have the span of, should have the span of the, oh, is it because we pop it? So if I do nv.sspan, and then down here do this, like does that help at all? No, so it's not the pop. Good old fashioned debugging. Yeah, okay, so it is failing there. It's just that this span, so this span contains just the ident. And I'm pretty sure that's because we're not, we're not including the feature that lets you join spans. And therefore, sin is also not able to join spans. So I think what we really wanna do is enable, actually, let's take a look at the cargo Tommel for this crate, see how we might get at, yeah, because specifically I think what we want is we want the, from the proc macro to crate, I think we want this feature. So the question is, how do we do that? We wanna, I can currently only underline one token. Oh, I see. So I can't use this, I need to use new spanned. And what does new span do? Sorry, for those not watching the chat, David who wrote these assignments is actually in chat and is pointing at all the things I'm doing wrong. Oh, I see. Interesting, well, I guess we'll do that. Because I feel like the other option would be to just join the spans manually, but I agree with you, it seems better to just not, to just do this instead. So in that case, I guess T, this would be T. This would give in NBS, this would give in NBS. Although now it might be a problem that we pop it, come to think of it. This should give in Meta. So I guess in these cases, we really just wanna give in the full token tree. That's gonna complain somewhere. Oh, I see. And so now this, I guess has to be one of these. So this has to be generic over T, which is Syn Tokens. T, T returns me, what exactly? A, one of these. And now this should be possible. Right, yeah, so we can't use error new because you can't join spans as a span. Unless I guess if they're consecutive, but then you still need the nightly feature. And two tokens is somewhere else, it's in. Now what does it highlight? Hey, it highlights the right thing. Great, all right, now let's get rid of this because no longer needed. Now do we pass the test? Okay, great, good job team. All right, let's check that all the other ones also still pass. Beautiful, now the final one, redefined pre-loop types. So your macros still work if some of the standard library pre-loot items mean something different. Yes, we talked a little bit about this earlier. And in fact here, we get pretty close, right? We already use standard option option. We would still fail on this assumption, I guess. And these would fail. So some is also a part of the option type, right? So this needs to say option, colon, colon, sum. Just like this does. Now man, option, option, sum. What else do we have down here? Those are all fine. This would have to be, where does Vec live? Standard Vec Vec. This would be standard option, option, none. This would be standard, where does result live? Result result, where does box live? Standard box to box. You'd say these are all fine. What else does it try to override? Result box, option, sum, none, okay. This needs to be standard, result, result, okay. There's another question of like, what happens if here, like if they had a module called standard? And of course, this was very nice of you, David, to remove the other things that were option here. But in theory, you could totally imagine that like they then use this option type on this struct. And remember how we have this assumption that the option type is just called option, which would totally break if that weren't the case. Is that the right one? Progress. How was the right one? Excellent. Do we pass all? We pass all the tests. Nice. Of course, there's still more things we could do with this. First of all, the code is a bit of a mess because it's all written in one, but like we did the thing. Take that proc macro workshop. Why does my output give so many warnings? I wanna get rid of these warnings. See, it gives me warnings from the test files. Can I just suppress that somehow? Is there a, I want this output. I wanted to look literally like this, but I'm getting all these warnings. Are these the generated fields? Oh, there's something the implementation doesn't test, but that it we very clearly need to do, which is, oh no, we already marked it as pub. This should probably also be pub. Maybe override rust flags. Oh, I probably do. I see. So you're saying, well, I don't really wanna remove that. I guess arguably it's unclear whether target CPU native makes a difference for me. I guess. Maybe I should just remove that at some point. Well, all right, but we do pass all the tests. So yeah, we pass all the tests there. Right, so things we might wanna fix about this. Well, first of all, the code is a bit of a mess, right? There's a lot of this, like, things that look almost the same, but not quite. For example, like the first and last case here probably we combined maybe even with this one, if you made this optional somehow, like you could just construct, you could set name to be quote this of name, for example, in this particular instance. So in theory, all these three could collapse into one. I'm also a little bit sad about this VEC business for the methods, right? So for the extended method, the fact that we have to do, where's the check that we have? Yeah, here, right? Like don't include the set method if the extended method ends up having the same name. But I feel like these are things that you're gonna run into when you write real world macros regardless. In fact, how about we do some of that tuning? Just I don't think I wanna start another one of the projects. Like one thing we might do is actually do us. If there's enough interest in this, we might do a second stream on proc macros on some of the more advanced ones, right? So the suggestion here is basically do this one first and then do the other ones because the other ones have other things you need to deal with that are potentially harder or different. And so I kinda wanna do the other ones too, so we might do those in a subsequent stream if there's sufficient interest to cover more of proc macros. I think this is a good intro to like, here are all the bits and pieces you need to at least get started and get things to fit together. Which is why I think what I want to do for the remainder of today is tidy up this code at least to some extent. So the first is what we do about this, generating these methods that are like almost the same. So what we really wanna do here is, let's see, how are these different? So this is the set method. So the set method will, the only thing that differs is what we choose to use as the argument type and whether or not we use option, right? So let's try to structure this a little bit differently, which is we're gonna use arg type and value. So in this case, we want the arg type to be inner type and we want the wrapper to be quote this. In this case, we want the type to be type and we want the assignment to be name. And in the last case, we want type to be type and name to be this. And then set method can now be equal to, without this, to be equal to arg type value. Ah, they're different, aren't they? All right, this is our ident versus token stream. Oh, I see, sure. Just to make them have the same type. Right, so that already makes me feel a lot better and then let's also leave circumference for ourselves. If the field is an option, don't require, wait, why does this need to take, setting is an option, T. Setting should take just a T, but we then need to store it within a sum. If the field is a builder, it is a vecti. It is a vecti. So we take the regular, and the value in the builder is not wrapped in an option. So we shouldn't wrap the value in sum. And finally, otherwise, we take the type used by the target and we store it in an option in the builder in case it was never set. Yeah, I'm of two minds of what I want REST format to do with my comments. It would be nice if it broke comments, but at the same time, and not in this particular case, but in some cases, I really want comments to be, I don't want it to wrap my comments if I've intentionally wrapped them, which is hard to detect. I agree with you though that it would be nice if it broke comments at a max width of 80 columns. So currently it allows a line to run for as long as you have space for within your maximum column, but I never want text to run that wide. I want it to force an earlier break. So here, this is not a builder, so just use the regular set. Actually, let's not even do it that way. Let's say we need to take care not to include a builder method with the same name as the set method. For example, consider this struct. Derived builder, I want my comments to be later. Comments should be more important in code. Struct command, nv vex builder. It would not be okay to generate both nv vex string and n string for the field and n string for the builder. I don't need to be parenthesized. Here, I think here we're decently happy with where we're at. I mean, we could simplify this, but I'm okay with these being separate. I do sort of want some observations about what these are. So these are the builder fields. These are the methods. These are build fields. So this is for when you call builder build. Build empty. And then I really wish this, I didn't have to assign this to a variable, but I guess it's okay. So the other thing that would be neat here is if the proc macro also generated comments, it's unclear that it can generate particularly useful comments here. But the one thing it can do is refer back to the original builder using the new fancy auto-linking that Rustdoc supports. In particular, this could say implement the API guidelines, flexibility checklist builder. Is there more than one builder now? No, great. So this implements the builder pattern for, and then this is kind of cool. I don't know whether this will actually work, but in theory it might for, like we'll quote extrapolate things inside of doc comments. I guess we're about to find out a name. Like if I run cargo expand now, what do I get? No, it does not. Does not generate that. That's kind of unfortunate. Think you meant to use doc equals. What do you mean for what? Oh, I see, I see. I mean, I guess we can do that. That just seems less, seems a little sad, but I guess, what was the concat? But am I allowed to use macros here? I'm not actually sure whether I'm allowed to use a macro here, right? Like this definitely seems weird. Am I the only one who thinks this seems weird? I mean, I guess we could try it, but produced unparsable tokens. Yeah, see, I don't think you can use macros in here. So I don't think that helps me at all. Let's look at quote and see what it says. Entifiers, method calls, huh? I mean, we could always generate it manually, but that seems unfortunate too, right? Like one way for us to do this is to just create a new token stream manually and then just extend it with the appropriate token trees. I guess one answer would be if this didn't need to do that. If like, is this okay? And that could very well be. Well, that produced something kind of like what we wanted, although that doesn't seem right. Like is this, yeah, it also didn't do expansion. It could be that we just need to generate this manually. So I guess one question would be, how do you even generate comments? Like, I don't actually know what a comment would parse to. Although, just bring this back to what it was for a second. I think what we can do here is cheat a little by going up here and then doing give me the whole token stream and then go to main and then do like hello and have nothing in there. And now let's see what this expands to. So now, oh, that's interesting. So triple equals really does expand to pound doc. You're right. Well, that's useful. Okay, so in theory then, what we should be able to do is, here's what I'm thinking. Like, I guess let doc is something along the lines of quote doc equals empty string. And then let's print out what exactly doc is. And then we just like stick doc here, right? Does that make sense? And then we're just going to modify it in place. Oh, doc, please. All right, so that is great. And now we just modify this string to be the appropriate, I don't think doc compounds that way, but we can just generate it manually, right? So we do this and then we do, actually, can I even mutate a token stream? Actually, I probably can. In fact, what I can probably do instead is now that we have that say, token stream new dot append. And we're going to append a, I guess, that's not even one, append a token tree, a token tree punked pound spacing, you say. What is a spacing? Sure, alone. All right, and then we want to generate a group. Okay, and what is a group? A group is a new delimiter is bracket. Limiter, bracket, and a token stream, which is this. Okay, so what's an inner? It's an ident and specifically an ident doc. Okay, so how do I make an ident? Because now we're no longer using the dint types. Can I just create like a new span? Is that a thing? Sure, that seems fine, call site. Okay, so then we have that. And then what we want to append next is a punked equals, it's going to be alone. And then a literal, there's got to be a nicer way to do this because this is a pain literal string. And that string is going to be this new line, new line. Actually, let's do this, formats a little nicer. This, this, with name. And this is going to produce that, going to produce that, and this is going to expand to TS. I have no idea what this is going to give us, but we're about to find out. So this can now go away. And now what will this expand to? Well, it's a syntax error somewhere. I mess up here maybe, let's see what this gives us. Oh, does it not take append? What do I have to do for a token stream? It implements extend. Really, so I need to do, it's going to be like, extend back of these. And this too, extend back of this. So, I mean, I mostly just want to see whether this could even work. 34, why doesn't this do something useful? Okay, okay. Theory, that should be all valid. Quote to, I can't do that, can I? This has to be a proc macro two token stream. Oh, it's going to be annoying. This has to be a proc macro two. This has to be a two, this has to be a two. These have to be two, this has to be two. Well, I mean, I don't know if this produces the right thing. I'm not actually sure. It compiles. So this is just a car we expand, it's actually doing the wrong thing here. Or, not necessarily, but it doesn't realize that the new lines technically would need to be done the same thing with. But that's fine. So if I run cargo doc here, what would this give me? So I think this actually generates the right documentation. I mean, it's terrible. Yeah, command builder, implement the builder pattern for command. Now, this short-tanned URL will not actually work unless you're on nightly and stuff. But this is such pain for generating that documentation though. It seems like quote just doesn't have a good way to deal with generating comments, specifically with interpolation of things. I wonder whether proc macro, what about issues? Yeah, it doesn't really say. Maybe this is worth raising as a thing. But hey, I mean, it works. I don't know whether we wanna keep it there, but you know. This ugly business generates a doc comment for builder that points to the original type. And this experience tells me that, oh, I see, oh, I see, you just format the message. I see, okay, sure. So here's a suggested alternative method of doing it. Doc is format is this. I think this is what you mean, right? So this and then none of this and then doc is equal to doc. And you're saying that should work. Yeah, I agree, that's nicer. It's a little weird, the interaction is a little weird, but yeah, that is much nicer. Great. I don't think for the functions, it doesn't really make sense to add any doc comments. I know that some people like to have comments to just say what the method do, but these methods, it would not be useful to have comments. Builder maybe, but unclear. This is now a decent amount nicer. What we could do here just for our own sanity is, list here is builder each equals of elements inside the last element, and then I guess here NBS nested 0. Ah, no, sorry. That's not true. It's this list here is dot, dot in and then this is hope. Hopefully each equals, yes, nested 0. This is just sort of, this is the thought process that we went through when writing this code and it's useful to document what each point is. So the next time we get to this code, we don't have to do it all again. So this shouldn't be necessary. This is two was either just a literal or I guess it was not key equals values, really what that comes down to. This is, what would that even be? Like what other metas are there? Let's save ourselves some future trouble. No, inside of there was either just an identifier or so that would be something like this. That would be if it's a word or a key value mapping which would be builder equals neither of which are okay. Of course, this kind of syntax might actually be something we want to let the user use. But this is more to document what each of these cases are. This can go in here as a nested one because it's not supposed to be used outside. Builder of we use other places. We do even, yeah. Okay, I think I'm decently happy there implement builder. Okay, I think that's where we're gonna end today then. I think it's a good place to stop. What I'll do is I think what I'll do is I'll do sort of an informal poll probably on Twitter although also message me if you have particular thoughts of whether people wanna see more procedural macros or whether we should consider that topic now covered and then move on to whatever the next thing would be which currently looks like this one although there are always like a bunch of shakeups in the votes after we move on from one stream. Also keep in mind I've added a bunch of new topics here so feel free to vote for more things that you think are interesting. So okay, I will upload the video, start a poll for whether people wanna see more proc macros. If so, what we'll do is basically work through one or, I mean in theory these will be faster now that we know what we're doing maybe but work through one or two of the other projects from here from the same workshop and otherwise if people don't wanna see any more of it then we'll do whatever the next voted topic is. Great, thanks for coming out. I hope this was interesting. I certainly learned a bunch and I'll see you next stream which will probably be in three weeks from now maybe little unclear I'll announce it as usual. So thanks for watching. Have a great day.