 Alright. Can everyone hear me? I'm supposed to start, so can everyone hear alright? Yeah. Alright, okay. Hi. My name's Tom Tromey. I work at Mozilla. I recently started a project to improve the debugging story for Rust, and so I'm here to talk about that a little bit today. And first of all, before I start, I wanted to thank Mozilla for letting me come to this conference. I don't know. It's pretty nice. Let's see. So first, I like to give an overview. We're going to talk a little bit about Rust, not a whole lot, but a little bit. We're going to talk about some of the concrete problems with like expressing Rust's constructs in debug info. I'm going to tell you a little story about like a recent problem I've worked on. And then I'm going to talk about, well, it says, hear recommendations about what we should do, but really what it is is complaining about other people's work. So Rust, who knows Rust? Alright. Yes, Rust is a system language from Mozilla with an emphasis on safety. I personally feel like it's what the programming world needs. But this emphasis on safety doesn't matter at all to debuggers. It's not important, like from the debugger view. What does matter to debuggers? Well, what matters is the type system, the ABI, things like macros, you know, ways that the source gets transformed that developers want to understand while they're debugging. And all of these have like problems when you get to like dwarf and teaching debuggers how to debug Rust. Now, I wanted to talk a little bit about like similarities. There's a lot of constructs in Rust that come from other programming languages. The Rust world, I think, sometimes says that nothing in Rust is revolutionary. It's all, you know, it's the combination that's revolutionary. They take ideas from other things, and it has a lot of influences. So for many things in dwarf, that's very easy. You know, like you see an integer, dwarf has good representation of integers. But it turns out that there's things that Rust has that dwarf doesn't really understand. So first we're going to start with the type system. So the type system in Rust, it has like a full complement of scalar types, you know, integers and floats and stuff. It has an algebraic type system. So that's a fancy word for saying there's structs and there's unions, except unions are like safe, not unsafe. It has pointers and references. It has this thing called slices, which are basically a view into an array, like a C++ array view or something. It has traits, which are sort of like a Java interface a little bit. It has trait objects, which are also sort of like a Java interface a little bit. And then one thing that you should know about Rust is Rust, it has this emphasis on safety, but it has an unsafe mode. Because it's a system language, sometimes you need to be able to do things that you wouldn't, like, that are unsafe, right? You need to drop below the hood and do some crazy stuff to implement the language or, in Rust's case, implement some sorts of data structures. Unsafe is basically C. And, you know, conceptually, it doesn't look like C, but it's basically the same sorts of rules as C. And so Rust's type system includes sort of a super set of all the things that you would find in the C type system. So, let's talk a little bit about some of the things in Rust that make it a little tricky for Dwarf. Here's one. Rust has three kinds of structure types. It has structures. And a structure, it says name and then, like, you know, there's, it uses curly braces and every member of the structure has a name and a type, just like C or whatever. Well, Rust also has tuples. Who says tuple? Who says tuple? No. No? I don't know. I'm a tuple person. But, you know, I don't judge. So, a tuple is basically, it's a heterogeneous type where, like, a structure, right, where the fields don't have names. They're just referred to as integers. So, like, with a structure, you would write, you know, blah, blah, blah dot name. And then, a tuple, you would write blah, blah, blah dot zero. Now, what's convenient about them is they're anonymous. You don't have to declare them. So, you can just write them in your code at different places. And that's very handy in Rust, like, if you do pattern matching and stuff. You know, it reduces the amount of, like, you know, typing you have to do. Well, there's also tuple structs because maybe you want your tuple to have a name and not be anonymous. And so, yeah. Anyway, what this means, though, is like in Dwarf, there's only one kind of structure, you know, DW tag struct. Well, technically, there's also class, but Rust doesn't have classes. So, and you want to be able to differentiate these in the debugger. So, I think, you know, like here, the solution's obvious. Let's add some new tags, right? Currently, like in GDB, what we do is we try to guess which one it is by looking at the names of fields. But that sort of thing falls down if you have a structure or a tuple with no members. There are no fields to examine. But it still might matter which one it actually is. The other thing that can happen with Rust is the Rust compiler actually does some structure optimizations. It rearranges the fields of a structure to pack the structure more efficiently. I forget if it goes into sending or descending order of size, but you know, now that turns out to be easy to represent in Dwarf. It just comes out looking a little strange where you have all the members in one order and then the offsets all over the place, you know? Okay. So, but I want you to remember this, like here, new tags. We'll come back later to what is involved in adding a new tag. Now, Rust has enums. An enum in Rust is actually two things. There's C-style enums, which is basically more or less a list of constants. But there's also enums where each enum variant can carry some data. And those are basically discriminated unions. Unlike C, where you can construct a union and look at different views of it and change, you know, like get yourself into trouble. In Rust, that's impossible. You can only refer to the variant that is, you know, the correct variant at any given moment, whichever one it actually holds. So, now, Rust does some optimizations with unions as well, with enums. Like, for instance, this enum, no, and then yes, holds a string. Well, no doesn't carry any data, right? And it's just a con, you know, it's a, I forget what the word for that is. And yes, carries a string with it. Well, this enum actually takes the same amount of space as a string takes. Because the Rust compiler realizes, well, this thing can never be zero. So, we'll have like one word. And if this, you know, there's a slot somewhere in the, in the payload. And if that slot is zero, it means no. And if that slot's non-zero, it means yes, and that is the string. All right? And historically in Rust, the Rust compiler emitted this in like this crazy way, which is also sort of horrifyingly traditional where it would emit a, yeah, I know, it would emit a union with a special, a special field name that the debugger has to decode. But late last year, they landed more optimizations like this. The Rust compiler can optimize your enums, you know, more cases. It can handle different kinds. And that old approach is completely broke. So, something better has to be done. Talk about that in a second. Then there's also univariant enums. You can have an enum with just a single, you know, variant in it. Why you want to do that? I don't know. Sometimes there's times. But univariant enums, like these enums, this one, the tag is kind of hidden away, right? It's tucked inside of this. Now, if you have an enum with a lot of variants, each of which has a payload, there's a tag, like at the beginning or somewhere, you know, that describes, it's basically an integer saying which branch is valid. And univariant enums don't have a tag at all. They're just a structure, you know? Now, so it turns out that WUF has a way to represent, represent tagged or, you know, discriminated unions. This is a thing called DW tag variant. And there's a thing called DW tag variant part. I'm going to tell you a story about that later. That's the story I said I'm going to tell you. It's like, I don't know. Anyway, so this is one of these things where, okay, so while you're working on this, you know, you're working on improving debugging, you find something in Dwarf and you're like, hooray, I don't have to add a new tag for this. You know, it's worse. All right, we'll get there in a little bit. We're still covering the type system. So a trait, like I said earlier, a trait is like an interface. A trait has a bunch of, a trait is a description that has a bunch of methods. It's abstract, you know? So for instance, there's a trait which is used to implement operator overloading called add, I forget what it's called, but basically it has one method, which is how to add two things. And you can implement a trait for any type. So for instance, all of the scalar types implement this add trait. And they have this, you know, they have a concrete implementation of this function. And so you can add like two integers, all right? Well, you could define a point type and implement the adding trait for your point type. And then you define that function. When you implement the trait, you define that function, and now you can add points, all right? So Dwarf has something for this. We're going to get to that. There's also something called a trait object. Now, the way to think about a trait object is it's a trait, like if you have a concrete type that implements some traits, you can add those things together, all right? But that's just, you're just working with members of that type. But suppose you want something abstract that can work on anything that implements the add trait. Well, one way to do it is you can, there's this thing called a trait object. And what a trait object is, is it's a pointer that has two slots. One slot points to the payload, right? Some piece of data, your integer, your point. And the other slot is the virtual table. And the virtual table is, it's kind of like C++, where, you know, you have one slot in the table for each method implemented by the trait. And so you can write these functions that take any sort of object and know how to operate on it. When you say it has a virtual table slot, you mean it has a pointer to a virtual table, right? It has a pointer to a virtual table. Yeah, it's a fat pointer with, it's actually just two pointers, one to a virtual table and one to the payload, right? Now, suppose you're in the debugger and you try to print a trait object. This is a bug I fixed last year. If you try to print a trait object, you get a gobbledygook. There's nothing there. In fact, previously, all that Russ would tell you is that it's like 16 bytes of, I don't know what, you know? So what we did is we added code to describe the V table in Dwarf, which by the way, this is something that C++ doesn't do. What C++ does is it just differs to the ABI here. So in C++, virtual tables aren't in the Dwarf at all, and the debugger does unbelievably terrible things to try to find out what's going on. Our solution was a little different. We actually just emit something that describes what these trait objects look like. We emit something that describes the virtual table at least a little bit like, and what we did is we made a Dwarf extension where we have, the V table has this thing, we add a containing type attribute so that this V table points to the concrete type for which it was implemented. What that means is when you get a trait object in GDB and you go to print it, GDB knows how to, it knows it's a trait object through hackery. This is another area where we should add a new tag, but it knows it's a trait object. It extracts the virtual table, it finds the virtual table somewhere in the debug info, and then it has this concrete virtual table that has a link to the containing type. So now GDB knows all this other slot in the trait object points to that type. So you can say, print my trait object and get the correct answer. Does that all make sense? What is the correct answer? The correct answer? It's whatever the payload is, right? The trait object points to some other object. A trait object is like a fat pointer. I don't like that term, but that's what they call it. But the payload doesn't have some kind of display trait. Well, you wouldn't know at run time. It's like in C++. You get a pointer to some base class, right? What does it actually point to? At run time, right? You don't, you know, how do you find out? Now, I think C++ should adopt something like this, but it's not in Dwarf at all. So, you know, there's just, I don't know of any compiler that does it, GDB uses hacks. Yeah, I'm not going to do that. All right, so those are kind of some of the corners of the type system that are troublesome. And remember, like, all these things like this is, you know, this little extension. It turns out there's actually another extension that uses containing type to mean something else that's been in GCC for 20 years or whatever, which is how I got LLVM to accept this extension because there's already a comment saying this is an extension and I just added one more saying now it's a double extension. But that is the extent of the documentation of this, you know, a comment in LLVM. Yeah, I know. We'll get to some of that in a minute. All right, so Rust has this other thing. Rust has a hygienic macro system. They use exclamation points to tell you it's a macro. So, like, if you read some kinds of Rust code, it just looks like a lot of shouting, you know? Yeah, as an aside, I feel like programming languages of the future should be written in uppercase and and just kind of back to the Fortran days. And then I think all keywords should be emojis from now on. But people, I keep trying to propose that and no one in the Rust world takes me seriously. So, hygienic macros, unlike C macros, they're not like just textual substitution. They can do sort of a pattern matching on their arguments. And so they're not, they can be more complicated. So you can kind of think of them as there's two different kinds. Like, there's some like printing, print line or debug, or there's a few like that. They look just like function calls and they act like function calls. So when you're stepping in the debugger, you sort of want them to step on or through them or over them like you do with function calls. But there's other ones like you can write complicated matchers that take a block of code as the argument, as it were, and destructure it and stuff things in there and stuff like that. And the question is like, when you see a macro, when you're stepping through code and you hit a macro, what should step do? Where should it go? Or what should next do? Well, next is usually easier. But what should step do? Should it step into the implementation, you know, like print line might be a significant chunk of code in some library over there. Do you want to step into that? Or for this matcher, do you want to magically step over whatever like prologue it might introduce and go into your code or? Now, I don't know if there's a really good answer for this. There's a proposal to like let macros pick one or the other, sort of. And then C actually has this problem. It just doesn't admit that it has this problem. If you've ever looked at GCC, nowadays it's not so common, but back in the day, GCC used to have some functions or like some macros that would expand to like 200 lines of C code. So, you know, that's a case where maybe what you wanted in GCC was to step and see the body of that macro and step through the macro. But then you have like problems, right? The macro has arguments. How do you see those? So anyway, this is just an unknown area. Dwarf has no solution for this. We're probably gonna do some work around and, you know, revisit it in a couple years and see if it worked or something like that. I don't know. If you have an idea, you know, I often think like Lisp, I think you can stuff like a debug form in your function and the debugger could know about that. Maybe like we need something like that, you know? Yeah, I don't know. All right. So now I'm gonna tell you a story about variants. I feel like I'm burning through this talk like really fast. So remember enums and how, you know, Rust used to emit special field names that sort of describe the layout of one of these compressed enums, an optimized enum. Well, that's not any good, especially as more optimizations are done. It can't. It just doesn't make sense to do that. Like if, you know, this compressions, this naming scheme relied on, you know, you'd have this weird field name that told you what slot to look in to find the discriminant and then a special name which was the name of whatever field got, you know, just dropped that doesn't exist anymore. Like the no field in that, the no variant in that, that example. But nowadays it can sometimes drop more than one data list variant. And so schemes like that just start to get ugly or none there. So I thought, let's, you know, let's fix that. You know, I've been actually focused on something else and I haven't been working on like just sort of some of these plain bugs so much. I'm gonna talk about that more tomorrow at a different talk. But I thought this one's really timely, you know, and how hard can it be? So I found out there's this thing, DW tag variant. And it's, it looks perfect. It's like it was introduced into dwarf for Pascal, according to dwarf. And it represents the variant part of a record. Now it's a little funny. It can only appear in the member, in his structure. So for Rust, we'll have to emit like a dummy wrapper structure that's empty because, you know, there is no wrapper, it's just that you know. And then it'll have a variant part that will, you know, the variant can point to the discriminator and then it can have a sequence of variant parts and it can have a default variant part which would fit that, yes, no example perfectly, right? No would be a variant part with no payload, no members, and a discriminant of zero. And then yes would have like this string in it and it would be the default. Perfect, right? So step one was add this to LLVM. So I did that. And I did it in what I thought was the obvious way. So I wrote this patch and I, I, you know, I added it to LLVM and I added support for that to the Rust compiler. And then I found out, well, GDB doesn't actually read variant parts. It just ignores that field entirely. So if it's in there, it's just completely skipped. In fact, LLVM also doesn't read variant parts. Nothing reads variant parts. And then I thought, well, does anything actually like make variant parts? So I went to look to see what makes variant parts. I thought, well, maybe there's a Pascal compiler. I don't know. You can barely even find a Pascal compiler anymore. And I looked through GCC. GCC generates variant parts. I'm like, yes, you know. It does something. I can see what they look like. Well, except it doesn't really. What it does is they're in there for Ada. Who knows Ada? Oh, man. Where were you guys like two months ago? No. What I did was, what I did was I wrote to someone I know who works at AdaCorp to say, how do I even make this work? And he said, well, you know, because the Ada compiler doesn't generally use Dwarf for all of its constructs, which is who even knew that. Remember that thing about like crazy field names? The Ada compiler weaponized that. That's all it does. Anyway, he gave me an Ada program to show me how it works. So I run the Ada compiler on it. And I get this nice Dwarf output. And I'm looking at it. And I'm like, this is perfect because it's exactly what I wrote for LLVM. It works the same way. So I'm very happy. And I wrote like some code in GDB to make it work because, you know, when I do this, I like to test it end to end before I start sending stuff, right? So I make it all work. I send my LLVM patch. And the very first review is, well, you know, this looks pretty good except the Dwarf standard says this all works a different way. Like you're completely wrong. And I went and re-read the Dwarf standard. And that's exactly what happened. I thought, you know, you have this variant part. And somewhere in the variant is a discriminator, right? At some field that has some bits in it that tells you which other fields are active. Well, I thought, well, surely that goes in, that's a fixed part. It goes in the record. You know, it goes outside. And then the variant points to it. Well, no. In Dwarf, the discriminator's part of the variant somehow, which to me like didn't make any logical sense. But I don't care. But it turns out that, you know, this is as part of Dwarf where no debugger reads it and all the known implementations do the wrong thing. And I started searching for it. And I found a discussion on the Dwarf standard list from like 2005, which arrived at the same conclusion that we should do it the wrong way. So, yeah, yeah. Oops, you know. So, I'm sorry. Fix the standard. Yeah, well, you know, that's, yeah. Well, we'll get there in a second. So, I agree. And I mean, this one, actually, I don't care. You know, wherever it is, it's fine. Because this is one of these things where it's like rust works the way it works. The expression of it in Dwarf isn't super important. You know, like, is that member here or is it here? The debugger's going to cope either way. You know, because the debuggers have to know a certain amount about the language they're reading anyway. But what happened was, I, you know, I got a review from LLVM pointing this out. And I'm like, well, you know, you're completely right. It's depressing. It's not the first mistake I've made. You know, Andreas' talk was basically a list of like bugs I wrote. So, yeah, no, it's totally true. But so, I thought, well, you know, maybe the Ada guys know something I don't. So, you know, I send email to my Ada friend who sends it to someone, the guy who actually wrote that code in GCC. And I filed a bug in GCC to track it. And then he comments on the bug. And he, he agreed, let's try to fix the standard or at least get some clarification. Because it doesn't quite make sense to us, you know. So, he files a Dwarf bug. And, you know, three weeks later it shows up on the Dwarf web page. You know, say what? That's fast. Yeah, that's fast. Exactly. So, and this kind of all, this, you know, leads me into the next thing I wanted to talk about, which is coordination, you know. Like, this process is terrible. Like, it shouldn't rely on me sending email to someone and having like three side discussions while we agree how two compilers and 2D buggers should be able to interoperate. Right? That's what Dwarf is for. However, in that discussion, you know, everyone involved said, hey, maybe someone else should bring this up on the Dwarf discuss list. You know? Yeah. Well, why did they say that? You know, I mean, the reason is, the reason is, if you send a message to Dwarf discuss, usually what you get is a message saying, well, you should file a bug and please don't discuss it here. It's like Dwarf, Dwarf, please don't discuss. And, but then you file a bug and it's, you know, like that bug, it was weeks before it's even visible. And, you know, filing a bug like that is what you're saying is, I think I found a problem with my, with this project I'm working on. You know, like, I'm deep into writing this patch and I'm trying to fix the debug information for my compiler and I would like to now completely not participate in the resolution of this and please get back to me in a few months so that I can continue on. Like, that's that. That's how the process actually works. That is not a good way for it to work and it can work better, you know? So, what I would like to propose, you know, is since Dwarf is the logical clearing house for these kinds of things, we should, like, resurrect Dwarf discuss, like, not, not let things get pushed into the bug tracker right away. And ideally, you know, I, what I would like to see is Dwarf development be more open. Like, you know, plenty of projects are on GitHub. I know some people don't like GitHub, but there's other options. But, you know, if you file a bug, there's no reason for it not to show up right away, for it to be easy to comment on, to track, you know? Dwarf could accept pull requests. Why not? You know, it's not like that's gonna poison it somehow. Yeah. So, but Dwarf doesn't want to work that way. But it should work that way. Who is Dwarf? Who is Dwarf? What I'm talking about is, like, right here, you know, like, in this particular variant problem, right, it's like people who work on, you know, the flagship free software compilers and debuggers. And we have to coordinate by privately sending email and commenting on some GCC bug or writing a comment in the LLVM source. That's not a good way to work. Dwarf may not want to work that way, but, you know. Maybe LLVM comments are really bad, but how I would do this, because I know Dwarf doesn't want to work. Right. This way is, okay, we have something which is different. We add something to the glue vendor space. We work it out. We exchange it for GDB. Right. To the GCC, we document it. Yeah. And once everything works, send it up to the Dwarf committee and... Yeah, well, let me, so this last one, constant assignment. Remember way back, I was talking about like tuples and tuple structs, and how I'd like, you know, some kind of new tag, like DW tag tuple type or something like that. Well, you know, Dwarf, and Mark mentioned this in the last talk. Dwarf has a vendor space. I could add a vendor tag, right. But what that means is I add a vendor tag to LLVM, and I add it to the Rust compiler, and I add it to GDB, and I add it to LLDB. Then we ship all that stuff. Right. And then there's executables out in the wild, and different versions of the Rust compiler for years. Then in a few years, like three years from now, the next version of Dwarf comes out, and it blesses DW tag tuple type. So now I go back, and I add support for both tags, you know, like the compilers can just flip the switch. Although sometimes compilers don't like to flip the switch. They're like, gosh, I don't really want a debugger upgrade for this tiny thing. Can you add it now with a flag? And then in a year, you're allowed to flip the switch, you know, and then the debuggers have to read both versions for all of time. And what that means is you take something very simple, because DW tag tuple type, you know, some of the things that go into Dwarf are complicated, and it's good to have a proof of concept and an RFC and all that stuff. But some of them, this is, we're talking about a constant, which is like necessary and simple, and we're talking about doubling the amount of work and spreading it over the course of a few years, instead of saying, you know what, you can have the constant 173 today. That's what we're talking about. So I think it's absurd, you know, yeah. So anyway, this is what I want to see. I don't actually know how to get here, you know, like I'm not on the Dwarf committee. I could be, if that was possible, is it possible? I don't know. And if it could lead to a better outcome, you know, like, I don't want to be on a committee or, you know, like I don't know about you, but I don't think it's good to join a committee just to be on the inside track to get things done. You know, that's part of the problem, right? So, yeah. Anyway. Mark, do you know why the Dwarf committee doesn't want to work that way? I thought you made the same suggestion once, right? Yeah, yeah, yeah. But I learned that it doesn't work that way and I accepted it. You know, yeah, I mean, you know, I accept it. It is how it, that is actually how it is, right? But I don't think it's how it should continue to be, or, you know, yeah. Actually, the standard is in a git repo. We could just fork it. We could fork it, yeah. I was thinking about something. At least have, if they don't want to change now, maybe they can be, they can get, they can get some, okay, how can I turn that positively? Get some incentive to do so, like compilers and debuggers, communities get together. Rather than having their own vendor space, they have a common vendor space, like, you know, I don't know. That's an interesting idea, yeah. And not GNU, but, you know, NERA and 2GDB, sorry, GCC, LVM will have the same one, and there are other. And so, in practice, that will be a shadow, you know, like Dwarf. Yeah, and in the end, yeah. Either. That still has this duplicate problem, you know, and it's not like these patches are huge or something, you know, I don't want to oversell it, but it's. Yes, but with time, you know. I mean, we all went through this vendor. Yeah, right, you know, yeah, exactly. If they don't want to change, it's okay. Right. And we can only try to give them incentive. Yeah, you know, anyway, so I am planning to add some extensions for Rust, and I would like to document like this Vtable one, you know, I think it's something C++ should do as well. But, you know, and I mean, I will file the bugs or whatever. I just wish it was less like, you know, that it was more streamlined, that I could get DW tag tuple tomorrow instead of in 2020, you know, yeah. So, all right. Any other questions? I think I'm like pretty much out of time. Well, just going back to the macros, do you see them as functions or as statements? Right, you don't know. They can be anything. Okay. What I'm asking is, have you seen Alex work on the statement for tiers? I read his paper, but I haven't seen like his dwarf proposal or anything like that. Yeah, I don't know if you see, because I think it depends on if you see macros as functions which get optimized differently for a statement which sometimes can. Well, they can be either one depending on the macro. Yeah, right, yeah. I would suggest look at the statement from tiers and see if they make sense as macro from here. All right. I think they look more like templates. Like templates. They can instantiate either functions or other stuff. I mean, in the case of pattern matching stuff, to me it looks like when you resolve Right. Yeah, I see what you mean. All right. All right, thank you very much.