 Welcome to Better Solidity Support and VS Code by Hartat. We're going to be covering, this is going to be like a jaunt, quite an easy jaunt, through the different visual studio code extensions for Solidity. There will be shilling of the extension I work on, but then more interestingly we'll move on to a couple of the technical blockers that are holding us back from giving you the Solidity experience you all deserve. Then a plan for getting around those blockers. My name is John Cain. I'm from the Nomec Foundation. He probably knows us as the Hartat people. Nomec is a not-for-profit foundation paid for by donations with a remit to improve the Ethereum developer experience. Hartat's a big part of that, but the remit's general. We've identified a gap in editor tooling and that's why we have built and released a VS Code extension called Solidity by Nomec Foundation, of which I'm the team lead. So you can find me on GitHub or hanging around our Discord server, particularly the Hartat VS Code channel. So if you want to come and chat or suggestive features or you've got a really detailed bug report and a matching PR, come chat, come chat. Edit or tooling. Let's kick off by addressing the Microsoft-branded elephant in the room. Why are we focused on VS Code? The reason is we have good reasons to believe that VS Code is the main way that Solidity is getting developed today. So if we take a look at the most recent Solidity developer survey, you can see that VS Code is dominant. In fact, the 60 visual studio people, we suspect, actually meant visual studio code rather than the venerable.net IDE. Similarly, the remix, that's an online Solidity editor, but pulls in key components from VS Code. So you're actually down at them before you're not talking about VS Code or VS Code adjacent. And we suspect that the Solidity developer survey skews towards those who are deep in the ecosystem, those who are on Twitter and click on the survey link when it goes out. Put another way, it's overrepresenting this long tail. It's overrepresenting the shadowy supercoders who are doing their development in Vim in a teamwork session over a VPN to a military satellite that they repurpose for the day. Actually, let's just test, right? Hands up anyone who, when they do do Solidity development, they do it in VS Code. And hands up if you're a shadowy supercoder who's doing it in Vim. See, that's a trick. No shadowy supercoder would admit, right? And actually, hands up if anyone, any of the two Emacs users are in the room because I'll buy them a drink. I'm having my doubts. I've created the wrong incentives here. And I'm immediately going to renaig on that promise, right? I'll be a lesson on soft promises and the value of escrow. Right. So by a preto analysis, VS Code's where we should put in the effort. I'm in VS Code. I want to just love the development. I'm looking in the marketplace, what should I do? And I would say that there are three options. If you have a different setup, get in touch, I want to hear it. But those would go down as Solidity by one Blanco, Solidity by the Narmic Foundation, and Truffle for VS Code, right? And we're going to cycle through all three of them quickly. And if we were doing it by installs, we would start with Solidity by one Blanco. But as I work on this one and it's my talk, we'll start with mine. And as I'm a coward, I'm not going to do live coding. So here is earlier, John, to run you through the feature set. So if we jump into our Solidity contract, as you can see, we have table stakes. So syntax highlighting. We also have diagnostics, or otherwise known as inline warnings and errors. You can hover over variables or functions to get useful information. Note that currently we're not showing documentation. Though I will come back to that. We have code completion on local variables. Or more sophisticated examples where type information is required. Completions work inside of import statements as well. Both at file system level, but also direct imports. So in Hartat's case, that's looking inside of NPM modules. In terms of navigation, we can find usages. Similarly, we can jump to definition. And this is working across the different packages of the monorail. Refactoring includes rename, which obviously you shouldn't use to do any evil, though we have implemented undo. We have a set of quick fixes which appear in response to salty warnings and errors. You can find them under the bulb icon. So let's bring back public visibility and similarly making sure that we set the override specifier. These are small annoyances with function signatures, but we have more complicated quick fixes. For instance, if we have a contract that needs to implement an interface or multiple interfaces, we can leverage the admising functions from interfaces quick fix. And this will do some of the heavy lifting for us. It is scanned through the inherited hierarchy of interfaces and base contracts and figured out what the smallest set of functions that needs to be stopped for salty to stop shouting at you. Oop. So once that's thrown now, if I push play, this is going to play again. Ah, good. I want to add three caveats to that video. Caveat number one. Like any sort of advanced technology, it's indistinguishable from a rigged tech demo. What you just saw was a rigged tech demo. You didn't see all of the places where jump to definition doesn't work or you're aiming for completions, we didn't resolve the type of set of completions. Caveat number two, I want to just re-emphasise that the documentation against function wasn't appearing on hovers. And there's a technical block to that that I want to come back to. And three is the biggest caveat, which is there are key features in there, actually diagnostics and quick fixes that only work inside of a hard tap project. And that's a problem that I want to come back to again. But let's take a look at the other extensions quickly. Solidity Y1 Blanco. This is the OG. It's been the work of Solidity Development for years with over 800,000 downloads relative to ours. Let's just say we know how that feels. It provides syntax highlighting, integrated formatting, has navigation and diagnostics. But the default structure assumes you're organising your contracts under source and lib. So setting it up for a hard tap is difficult, but it's going to work better if you're doing boundary or dap tools or brownie. Next, we have truffle for VS codes. This leverages Solidity Y1 Blanco for the syntax highlighting, diagnostics, et cetera, but adds a layer of UI integrations into the truffle tools suite so that you can run compile through the UI, through the task subsystem in VS code, for example. And that's great for onboarding if you're new to truffle or you prefer the UI to the CLI. One other really cool feature is integration with the truffle debugger. I just wanted to mention as well, this isn't from general development, but the diligence team at consensus has a set or suite of extensions that are security centric, so they're going to give you analysis and visualisations when you're doing a smart audit. Plenty of options for Solidity then. Lots of great programmers pushing the ecosystem forwards, but I don't think any of us would claim that we're yet in the same league of editor support as other language ecosystems like .NET or Rust or TypeScript, right? So how do we move from where we are now to that best-in-class experience? One of the answers is just time and resources, right? But there are a few technical blockers. I'm going to spend the rest of that stock digging into those. First off, feature fragmentation. So we have features which are siloed by editor and also siloed by development framework. The other is feature brutalness, right? Can you rely on a feature to always be there and to do the right thing? For fragmentation, clearly extensions, we have built-in VS codes. Maybe the lack of support in them is why there isn't as many people using them, right? If they had those advanced features in them, they would be using them, right? So that's one form of fragmentation. The other form of fragmentation though is in developer framework. Not all of the features that were listed there work for all development frameworks, right? If you're working on a truffle code base, you're probably using truffle for VS code. And if you're in a hardtack code base, hopefully you'll use Denomic extension. And if you're in Foundry, you're likely using one Blanco's extension, right? And that makes sense for some development-specific features. But there are core editor features which should just work across all of them, right? You're going to want rename in Foundry just as much as you're going to want it in hardtack. But currently, it's siloed in hardtack, right? How are we going to get around this? Sorry, there are multiple reasons for that. One of the reasons is just Conway's law. So if you've got four teams working on a compiler, then you're going to end up with a four-phase compiler. The code structure matches the communication structure of the teams, right? And the development teams that are writing editor support, it's easier for them to add it for their own development framework. It's easier for me to harang the other hardtack devs or cajole them or convince them or go crying to the CTO to try and get it changed. Then it is for me to convince the entire ecosystem to just do what I say, right? But there's another aspect that's part of Solidity, right? Which is that Solidity doesn't specify how import loading works, or doesn't specify that part. It leaves it up to the development framework, right? So if we've got an import statement here pulling in other Solidity codes, SolC requires a custom loader to actually resolve that, and it has to be provided by the development framework. And if it's a relative path, the development framework is probably going to do that as relative on the file system, right, from the current file. But if it's a direct import, then there's multiple different interpretations which make sense, right? In hardtack, we would interpret this as looking inside of load modules and then find open sample and resolve the rest of the path, the IRC-20 Sol file. And that makes sense, right? Hardtack leverages JS, including load modules, but that doesn't make sense for Foundry, right? It's going to resolve this based on remappings in either the Foundry.tomo or a remappings.txt, right? And that has profound implications for editor tooling, right? So say you're on IRC-20, this token here, and you want to do jump to definition. The editor can only resolve that if it understands this at open zeppelin import line, right? That implies the editor has to understand the import loading logic of the development framework that this file has been dealt with under. Okay, so how are we going to work around these fragmentations in different editors and fragmentation across the development framework? I'm going to do it in two steps. First, we're going to use the language server, which is language servers, a technology that came out of VS Code, but tries to deal with the fragmentation you get across editors, right? So instead of each editor implementing language features for each language, we gather up the language features under a daemon, and we have the editor put that daemon up, ask which features it provides, turn those features on in its UI, and whenever those are requested by the user, it delegates them off to the daemon, and the communication between the editor and the daemon is covered by a clearly specified language server protocol. And that's what we're doing in the NOMIC extension today, right? There is a language server, which is encoding all of the completions, diagnostics, hovers that we provide today. And that is embedded in VS Code, but there's nothing stopping us from exposing that out so that other editors, say them, can take advantage of it, right? But what about the important or more difficult form of fragmentation between developer frameworks? If you have a language server in the NOMIC extension already, that I don't have advanced completions in Foundry, right? And the reason because that is because currently the language server is really a heart hat language server rather than a Solidity language server. We are all through the code base encoding heart hat assumptions to deal with things like the import loading that we saw earlier, right? And the reason that we've done that is we use bits of the heart hat code base, right? Which allowed us to get up and running quickly. We can make heart hat dance, right? And subtraction is hard, right? But we are looking now to refactor the language server to isolate out these heart hat particular points, the development framework particular parts behind an adapter interface. We'll have a heart hat adapter, but we also can then have the implementations for Foundry, Truffle, Brownie, right? And in that way we can add the feature once and have it available across the ecosystem. Well, our point I just wanted to mention recent versions of SolC have a language server built in, right? So why don't we leverage that? And we're big fans of language servers to the northern area but the SolC language server is embedded in a particular version of SolC, right? And heart hat code bases can have versions that are older than the ones that provide the language server. And certainly we have many complex code bases and heart hat code bases which have multiple SolC versions, right? And we need the language features to just work across those versions. So that's why, right? But just to summarise, the NOMIC extension has an in language server that we are making develop our framework agnostic and that we're going to make stand alone so it can be used in other editors. And that's how we can build a feature once and leverage it across the ecosystem. Feature britonus, right? This is about quality, the editor just doing the right thing. So, when you do jump to definition you can do jump to definition of the places you expect and actually goes to the right place. And the NOMIC extension has blind spots, right? There are lots of places where features don't quite work as we would like. And there are several areas that we need to work on this but I just wanted to focus down on one in particular today, right? And that is that we are not giving the best experience when people are annoyingly trying to use the editor to make edits. Walk through, so you're bopping about inside of the you're bopping about inside of the editor you're doing navigations, you're exploring the code base, you jump back again and then you start making an edit and you remove a semicolon, right? Shame on you, shame on you. But suddenly jump to definition doesn't work, right? And why is that? So the video just showed you a change in the document and that change dot gets passed to the language server, right? And the language server does a parse and it builds an abstract syntax stream, right? And that's the data structure which represents the syntactic elements in the code. The code might represent a function and then subnodes would represent substructures like the function name and the parameter list, the function body. And notice well that the parse produces an abstract syntax tree, right? So some trivia is thrown away, stuff that isn't strictly necessary, like white space, like comments, right? Because they are not typically used in later phases by for instance the parser, right? And that's why when we do hovers we don't show the documentation. The abstract syntax tree is abstract because it's thrown those elements away and the ASD doesn't have it, we don't have it so we can't provide it in a hover. The language server takes the ASD and it combines that ASD with the ASDs of all the other code files, right? And then we do a further analysis stage. We overlay it with type information, we also scan through the nodes and we find definition nodes and usage nodes, right? So a definition node might be a function declaration and a usage node might be a function declaration, right? And we layer on top of the ASD that extra information, these extra links so that you can see which node is connected to which other node. And so I change so that when the user requests a jump to definition the language server receives that request including the cursor position it looks up the nodes underneath that particular, underneath the cursor and finds say a function invocation, right? To find the definition nodes and we return that location, right? In terms of its file line column, actually both the start and the end. And VS codes then jumps to that location. Next the user makes an edit and introduces a syntax together. The change.coms through, we do a parse and it fails, right? And why did the parse fail? Well, we like other projects in the ecosystem leverage Solidity parser, right? For parsing. But Solidity parser it's a GIS library and it's designed to give binary answers to the question of is this syntactically valid? If it's syntactically valid we produce an AST otherwise here's a list of errors and that makes sense, right? Most use cases of a parser stop if there's a syntax error. You don't proceed on to compilation or generation steps if there is a syntax error, right? Editors have other needs, right? In the editor's case we want both the list of errors but also the syntax tree, right? Or as much of the syntax tree as possible at the same time, right? And that's not what a standard parser is designed for. The way that we get around this in other languages is by building specialist parsers that are more tolerant, right? And they're designed specifically for that editor use case. And that's what we are building now at NOMIC through our slang project. The slang parser, one of the components of that project, looks to leverage best practices from other languages, right? From Roslyn, from Rust, right? Things like red-green trees. And the goal is to always produce and put always to produce a syntax tree, right? Even in the presence of errors it's just that sometimes the syntax tree will be valid and other times it will not. But hopefully those error recovery will mean that a missing semicolon in one function doesn't mean we don't get the syntactic structures in all of the other functions so that we can take advantage of those in editor features, right? One of the other advantages of building a parser specifically for the editor is that you can include that trivia that's normally missed out the white space, right? The comments so that we can show the comments in hovers, for instance, right? Or we can do advanced formatting, sorry, refactorings, right? Right now we're blocked in refactorings because developers probably won't be too happy if on doing a refactoring we stripped out all the comments in the file, right? That's probably unacceptable. But with a parser designed for the purpose we have that information and we can make sure it's there. Okay, so what we're doing here is swapping out an excellent but general purpose component with a component specifically designed for the editor. And that's how we drive up feature quality to summarise. If you're using HRTAP and you're using VS codes we think you should give Solidity a binomic goal and we think it represents an excellent set of features. You've just heard how we're driving up the quality of those features by building editor specific components and swapping out the general components that we have. And we are looking to make those features available across other development frameworks, right? We want a credibly neutral language server and we're doing the refactoring and putting in the work to or putting in the work for subtraction to support that. We also want it to be a standalone language server. So we want to pull it out for being embedded in VS codes it's going to need to do a small amount of cleanup. This is actually one of the easier things we need to do. But hopefully, sometimes soon a shadowy super coder will be able to hack away in Solidity so that's content. So that's how we're going to improve the Solidity developer experience for everyone. Thank you very much. Are there any questions for John before you go? Sure. Are there any analysis tools that can benefit from these improved more robust abstract syntax history? Sorry. So the questions are there any other tools that can take advantage of this? And we would hope definitely, yes, the slang parser we're building a component that could be used in dev tooling across the ecosystem, right? And it's going to give more detailed information and work across different Solidity versions, right? So it should be giving you rich information across versions. Other parts of the Solidity project also intend to add other components to write even more information. So usage and definition would be the next one that we're looking to tackle. So yes, hopefully lots of analysis tools will be able to leverage these components. Hey, thank you very much. This is awesome. So two questions, real quick. Go for it. It wasn't clear to me at least if this slang parser is being used right now by the extension and the second question is is there any intention from you guys to support .enbeam which is like a yep. So right now we do not have the slang parser. We're using the Solidity parser and we're looking to dogfood the slang parser within the VS code extension. I am not giving out estimates, mainly because it's not up to me, but yes, we're hoping to pull in the slang parser in the near future. In terms of VIM COC and that's part of making the language available and VIM COC would be the first target for making it work with that. And that's something that is within just the VS code teams remit, so hopefully that should be in the near future. When the slang parser comes out how do we do we need to install it or is it already built in? How can we get access to it when it does come out? It will be available as a separate library. I believe it's in Rust but we'll make it available in Node and JavaScript as well.