 Hello everybody, thank you So as was mentioned this talk is going to be the state of Delve Kind of piggybacking on state of go I thought it'd be an interesting interesting concept since I've I Spoken about Delve the past few years at Fosdham So it's always nice to come and kind of give an update on what's going on with the project which changed One thing that I feel the need to address right off the bat is if you look at the schedule You might read some nonsense about hands-on debugging session and this this and that So that was the original plan for this talk But trying to trying to get basically a workshop into a micro workshop Format was kind of difficult and so I decided to go with a more traditional talk instead And after watching some of the talks this morning It actually works out quite nicely because making Francesc act as a microphone stand for 25 minutes I just I just can't I couldn't do it to him. So I'll save him some of the trouble so Regardless, we'll have a good talk and I'd like to go over some of the some of the things that have changed Since in the last year some of the things that we'll be looking forward to in the next year as far as new features and developments And I also would like to take a deep dive into a fun a feature that was Implemented within the last year that I think is really interesting And so we'll we'll get into some more technical things there Which is nice because I kind of like to use this effort as an opportunity as well to to encourage contribution you know, this is a very open-source conference and a lot of a lot of really good Smart people here. So always looking for new contributors and deep dives into this kind of stuff. I feel like encourages it So just to introduce myself after that long-winded introduction My name is Derek Parker. I'm a senior software engineer at Red Hat I Maintained the go package for rel internally at Red Hat support all of our internal customers and external customers and Also do some upstream contributions and contributions to the runtime and some other things Aside from that. I'm also the author or is a creator of DELV and Maintainer I also have a another co-maintenor Alessandro who I just like to give a shout out to because He's been a huge help with the project especially over the last couple years So state of DELV Like a like I said, I I'd like to go over what's changed in the last year. What new features have we added? how what have we done to keep up with changes and go in the go runtime and Additionally after that what what can you expect? What new features are on the roadmap and and What would be if you would like to contribute some of the highest impact areas where you might be able to contribute in the future? so just to do kind of like a little recap and And give a little bit more back back story to the project The project is actually about four and a half years old now Which which for me is is kind of surprising looking back on it But it's grown it's grown a lot and tons of people are using it now, which is great We've gotten a lot of support from the community and a lot of support from a lot of the core team members and things like that Which is which has been great and building these maintain these these relationships with the community And especially the core team is is very beneficial and also necessary for Implementing some of the features that we need to implement in delve that might involve coordination with the runtime and things like that And I'll explain that a little bit more later the first the first commit of this project was actually in May 3rd of 2014 which was a little bit after the first gopher con which was There's a panel there that was actually the the inspiration for the project as a whole Total we've had 88 contributors Which is pretty good for kind of a weird esoteric project not a lot of people hack on debuggers So having this many contributors has been great Again always looking for for more people to come and help hack on the project So without further ado, I'd like to dive into some of the the interesting changes that we've had since last year and some of the Give a shout out to some of the bigger features that we've implemented recently So one of the one of the biggest and most exciting things for us just as maintainers Was the release of the 1.0 So that happened actually I believe pretty shortly After Fosdham last year so before that we were considering the project pre 1.0 A lot of people a lot of folks were still using it still using the project still using it very successfully We we we actually considered it pretty stable and ready for use Even though for a while we had like a scary message on the repo, you know saying it was pre 1.0 and all that stuff but Earlier this this year we or last year I guess We we finally released the the 1.0 version and subsequently we've released 1.1 So the version scheming kind of follows the upstream versions so One 1.0 was around the time of go 110 1.1 is around the time of go 111 so subsequently as we're we're gearing up for 112 113 so on and so forth our versions our release cadence will for for the minor versions will will map to go versions so 1.2 for for 112 and and so on and so forth This this release marked a bunch of new features and More importantly just kind of a feeling of stability API stability compatibility for Anybody who does ID and IDE integration or uses the the API to interact with the debugger so Over that over that time period We we ended up adding a bunch of new features So I'd like to kind of take some time and go through Shout out some of the some of the bigger features that we've added certainly. There's been tons of new other features improvements performance improvements fixes things like that that have gone in and This doesn't also cover a lot of the work that we do upstream with helping helping Improve debug information that's generated in go binaries and things like that But just as far as the the delft side Over over the last year. We've we've added support for 110 111 and also 112 So with that with every new go release what we try to do is improve the quality of Debug information that's found in the go binaries so Myself Alessandro other folks who do IDE integrations we we we tried to to communicate with the runtime team the compiler team and Suggest fixes and and contribute code where necessary to ensure that when you type go build It produces a binary that is that has as much information as as we can possibly get to Create the best debugging experience for you and When delft builds the binaries it builds them without optimizations and things like that We we also want to make sure that You know you have that you have the best chance to debug your binaries and have all of the information that you need So as new versions come out we support new new missing features that that are in like the dwarf information and things like that that are present in the in the the binary So with that one of the one of the improvements that we've been able to make is when you're debugging optimized binary So when delft builds a binary for you if you just type DLV debug It it builds the binary in such a way that it disables all the optimizations that are enabled by default So function inlining Variable registration things like that And this this just makes it a lot easier to be able to debug your binary But What we what we also ultimately want to do is is let it Ensure that you can debug any kind of go binary and you'll have success so even production ready more optimized binaries so Part of what we've done is added added more support for debugging inline functions so that you can debug Optimized binaries a lot better Another another feature that we've added is support for DWZ compressed dwarf info This is useful for So dwarf information for those who are unaware. This is the information that's stored in the binary that that kind of has a bunch of information about about the process and in the Program that you're that you're debugging which kind of tells you Information about what the stack looks like where variables are in memory and things like that How to how to how to access all of the things and and do well get all the information that the debugger is able to then ultimately present to you so Allowing for supporting compressed compressed dwarf info it is kind of allows for for Folks to still build binaries with the dwarf information enabled But also get smaller binary sizes because dwarf information can can take up a lot of space So as we as we improve debugging production optimized binaries Adding support for Compressed debug information is important as well Additionally we've we've added support for being able to Specify separate debug information. So with a lot of a lot of binaries that are distributed by package managers a Lot of times there's a separate there's a separate Package for the debug information From the binary package and so if you install something like this You can you can use delve to specify an alternate Debug information file so that you can still debug these programs that have the debug information stripped out of it But still available elsewhere on the system Um We've also we've also made improvements to the debug server Being able to run headless without a connection This is more of an implementation detail, but it's nice to have maybe like a headless server running that you can connect to and disconnect to Say if you want to run delve within a container or something like that and run the client outside of it It it makes it a lot. It makes it a lot easier to do kind of ad hoc debugging If you want to connect and disconnect and and not have a session running constantly Another another big improvement is the ability to call functions in the target program now This is something that I was very excited about and and this is something that I I'd like to take a deep dive into in a little bit But this was a long time coming. It was a very complicated feature to implement and As you'll see later it involves a lot of coordination with the go runtime just because of goes memory model garbage collector There's a lot of really complicated things when it comes to Calling a function in the in a target go program that we had to overcome. So This was a big collaboration between Us on the delve team and a couple folks from the go runtime team. So I really I'm really excited about it And it's something that we've been looking to add for a really long time Some other some other niceties We have added just an edit command so similar to if you're used to any other debugger Wherever you're stopped in your program you can type edit it'll open it up and you know buck editor and you can You can start, you know editing your program or viewing your program that way from Wherever you're stopped at in the debugger We've added support for a position independent executables. This is just another nicety if you if you're building pie binaries Through your build system or for production reasons security reasons anything like that you can now debug those binaries with delve Another feature is We finally got the information that we needed from the the debug in from information in the binary to be able to show Return values when you're doing CLI tracing. So similar to Doing like S trace or something like that you can trace your go program from delve without having to start up like a full debug session And with this feature you don't need to do print line debugging So you can just trace your program from the outside print out any information that you need about variables Or you can even generate a stack trace and do all kinds of interesting stuff without having to Do a full debug session and litter your code with a bunch of print lines and all kinds of other stuff Additionally, there's been just performance improvements. This is mostly for like IDE integration and stuff like that but we've done a lot of memory caching so we're not reading from the process memory so much and Some improvements regarding Being able to list like go routines and stuff like that Another small thing is the the repo has been moved from my personal github account to the go delve organization Nothing's really changed just the location of the repo, but something worth mentioning I suppose So another thing that I want to talk about is Upcoming features so things that we'd like to do within the next year that Hopefully maybe I can tell you all of these things have been done next time next year at Fosdum So some of the big things support for 113 of course which should be coming out next year Like just improving the the debug information upstream improving how we how we Parse and use that information And so on and so forth So if you're ever interested in some of the upstream debugging changes that are happening or are currently in flight If you go to github and look at the go issue tracker, there's a debugging label So you can always take a look at that if you're ever interested in the state of upstream debug support We will we want to continue adding support for debugging optimized binaries A big one is adding support for more architectures right now our support matrix is x8664 and So we want to add you know arm 32-bit architectures things like that and this is a place where community contributions can really shine Another huge feature that we're looking to add is supporting a scripting language. So Instead of just interacting with Dell through the API We're we're looking at adding a integration with a scripting language right now. We're looking at Starlark, which is The programming language used for I forget the name of the build system Yeah, basil basil it's the it's it's used for that. So we've been talking with Alan Donovan And trying to trying to see what we can do to integrate that But you'll be able to define your own functions and it'll really make the experience seem a lot more interactive We want to improve go routine inspection. So Just show you more information about go routine more information about like the go routine stack Where like where it's located the size of the go routine stack things like that Improve process inspection. So just making like slash proc information available showing page mappings and things like that Improve stack frame inspection Improve and improve function calling. So there's some limitations to function calling which I'll get into but I'm getting short on time. So let me just jump into that so we can get that the deep dive deep dive into the function calling so As I said, this was a really interesting feature and it took a lot of coordination, so I kind of wanted to Just dig into it and show everybody how it works and and just share something that I thought was pretty exciting and a really cool implementation So The syntax for calling Function is just this so you call Whatever your function is you can pass it arguments. There's some limitations on which arguments you can pass But I'll get into that a little bit a little bit later You can also call methods. There's limitations there just because in reality what's happening is The method receiver is being passed as an argument. So same argument limitations apply, but again, I'll get I'll get into that later Um So in order to in order for the for delve to to make a function call in the target program There's a procedure that must be followed. So I want to talk through that right now So when you type call my funk what actually happens underneath the hood is Delve does a bunch of stuff. So it checks go routine state and make sure that the go routine is in is is actually running So that's a limitation that we have right now The go routine it has to be running it pushes the current PC on the stack it's it writes the argument frame size and and It writes this on the stack as well and this is to communicate with the go runtime So the go runtime needs to know how much space we need for a stack to to call whatever function that we're trying to call We save machine registers This is just because if you call a function it's going to do a bunch of stuff It could clobber registers things like that So once the function call is over and we return back to wherever we were we want to make sure that we haven't Like permanently change the state of the program We set the the PC to runtime debug call so when I say PC that's the program counter register So we we tell the CPU when we we when we tell you to continue again start executing the code at this address And the runtime debug call v1 That's the integration with the go runtime that we that we use to make this happen And then following that we just continue execution Then what happens is we switch back to the runtime. So since we set we told the the processor to execute this To execute this runtime function it starts doing all of its things. So it copies register contents to stack This is for the GC to be able to If there's any pointers in the registers it allows the garbage collector to still have To still see those it performs some some safety checking and it allocates a stack frame for the function call Setting the stack pointer register Appropriately and then it communicates to delve through the AX register So it sets the AX register to zero and then what it does is it calls And three which is a which is a it's a like breakpoint interrupt that That is that debuggers listen to and it will switch It will essentially transfer control from the target process executing runtime code to the debugger To do what it needs to do next So state of execution transfers back to delve delve will on the new stack frame that was allocated It will write the argument frame so When arguments are passed to go functions they're passed on the stack So it writes all the arguments on the stack. It sets the trapped PC as return So the trapped PC in this case is where the runtime code left off So what this does is when the function that we're calling is done executing It's actually going to return back into the go runtime instead of returning wherever else it might have returned And then it sets the the program counter to the called function and it resumes execution Basically doing like a jump and allowing the the function that you wanted to call to execute once that happens the the The the function executes and once it's done executing since we set the return value it returns into the runtime And the runtime sets the ax register appropriately to communicate what happened back to delve so It sets the register to one if the function call was successful if the function panics for any reason the runtime actually wraps That and will set the register to two and allow delve to read the panic reason and report that back to the user And then it does the same breakpoint instruction to switch execution context back to delve and Then delve reads the return values on success from the stack and then or reads the panic message on failure and That's pretty much the whole procedure. I thought that was really interesting So I want to take a deep dive into it. So there's there's some limitations. I'll go through those pretty quickly So only pointers to stack-allocated objects can be passed as an argument. This is because we don't have like escape information present in debug info right now, so if you if you pass a stack argument and The function that you pass it to makes that makes that escape to the heap for whatever reason We're not able to really track that and warn you of that so that could cause weird memory and corruption issues We do some automatic type conversion for you when you when you supply arguments, but only some are are supported They can only be called our functions can only be called on running go routines And the current go routine that you're calling from needs to have at least 256 bytes of free space on the stack Functions can only be called when go routine is at the stop at a save point and Calling a function will resume execution on all go routines A feature that we want to do is being able to just continue one go routine while this function is being executed to minimize side effects But that that that'll require more Cooperation with the runtime team and so hopefully we'll get that done in the next year And it's only supported on Linux and OS X right now So we're we're gonna add continue Adding support for that as well And that's everything so thank you so much