 Okay, yeah, hi. I'm Daniel. I'm in the compiler team for yeah over four years now and Yeah, originally the plan was for this talk to be held by Chris who couldn't make it at Bogota So I have to improvise a bit. I brought a lot of code snippets Which will explain what we have currently in the compiler currently the state of the language and where we are headed with this Yeah, I guess you if you are familiar with Solidity, which I hope most of you are you probably know it mutables But now it's been around form for a while and it's a very highly appreciated feature from our impression Which yeah, I still will quickly explain how they work. That's the owner variable here It's looks like a state variable, but it's not a really proper state variable in storage But a mutable variable which can only be assigned to once in the constructor And then can use this many times as one wants read only in the runtime code as a contract But there it's the access is very cheap. It doesn't require an S load, but it's basically inlined in the Code as if it was a literal So, yeah, you probably know this and people liked it a lot and an apparent and very obvious extension to ask for is Why do we only support that for value types? That's the restriction you had so far that it's only integers only addresses, but not a raise of things for example So this was for example, it's taken from and get up issue that I think open several in open with us to Yeah asked for an array of immutables, which would then yeah the initialized by index accessing and otherwise work Just as As one is used to with immutables And we're working towards that end, but there's some issues with that. So, I mean we have these assignments here where a randomly assigned Into elements of this array and in general I cannot check that this is really assigned only once I mean the question is is this still an immutable thing in that sense Also, if I start having a race as immutables, I will want to have local references to them Which I could then reassign. So if I keep the name immutable I just changed what this points to so this is really it doesn't look immutable immutable doesn't fit this concept anymore properly So let's yeah, let me explain a bit further how immutables actually work in the constructor Owner is actually a position in memory. So we store Whatever you write to this variable in memory and then when the actual runtime code of the of the contract you're part of deploy It's copied in memory to be returned by the constructor. We fill in From this memory location into the bytecode the value that is there which in the end Results in the runtime code to actually have it as a literal in the bytecode So this variable in the constructor actually is a memory variable in runtime It's actually something that lives in code But filling in literal values in some so basically the push argument into the bytecode won't work anymore for Dynamic types if we have steadily sized arrays We could still do that and we probably will for efficiency reasons But at least for dynamic types to be put go full way We cannot do that anymore because we don't know the length of the thing so we can't reserve space in bytecode for that So instead we need to rely on code copy and Yeah, I already mentioned we will probably want to pass the immutables around by reference. We will want to slice them Which if you think about all that together Makes immutables not an annotation for a safe arrival anymore, but it will become a proper whole data location But this will probably look like this in the end As I said the right in the end will live in code So code and the opcode that will access it is code copy. So code is a natural name for these things So here I have a data variable abides and dynamic array In code which I can then in the constructor just treat like any old memory variable Assigned to it modify it freely You can drop the requirement that it's only written to once in the constructor because that was always an artificial requirement so if we actually be Yeah, call it what it is a code variable that will actually be inserted into or use as in the code in the end We can freely modify it and Then in the runtime code It basically behaves like a call data reference a read-only reference only that it doesn't come from call data But from code so I can slice it Have local references to it and pass it around to functions all with very little cost This will be a bit tricky to write to type check in the end because yeah in the constructor It is a memory variable in the runtime code Be as differently. So if the constructor calls functions, we need to type check everything twice But we have different calls for refactor in the type checker to actually do that as well So we will do that and there's still some considerations of gas It would be very handy if they had a code load upcode similar to a call data upcode to actually write read single words from code, but We don't so we will whenever possible probably still use the same mechanism as immutables use now to actually fill it into a push argument in white code if not, we will probably yeah code copy either in an entire memory region or if you just need a snippet of a Dynamic array code copy to scratch space and em load from there But it will still be significantly cheaper than anything in storage Okay, yeah, that was the first topic I wanted to talk about the second thing that you may have noticed in the recent versions is our move to Allow user defined value types go to test case for this concept was always fixed point types Which we decided to not implement as a first citizen type in the in the latest type system But to build the infrastructure to define it as a user defined type. Yeah, I'm assuming that you're familiar with those so I won't go into that much more detail in the basic concept which What we recently added or more recently added is this global keyword you see in the first using statement which means that currently a using statement is only locally effective whereas a Using statement with a global at the end will only be allowed in the Very same source file in which you define the type But we'll have that the effect that wherever the type is available the functions you bound to the type will also be available So this is what you can do then in the end practically to write type libraries But you're in that library define the type Equipped with some functions and then anyone importing the type only can still use all the functions on that Still of course one major inconvenience with that for defining for example fixed point types that are actually nice and usable Is that we have only functions in? that can be defined and Equipped to the type of using statements, which is what we will soon change So this is open PRs currently that are under review, which will probably yeah in finite amount of time. We actually Merge and come through which will allow you to define operators and literals in In the first versions in a very limited setting So the addition for example can only be between the two between two Expressions of the tie that you define such that is not getting messy and you know no longer know what Functions will actually be executed if you have an Operator expression, but it will still already be yeah very useful. I hope In making these user-defined types more yeah convenient to use literal similarly we will Introduce a way to define functions that can Take up as argument a Mantissa in that sense and an exponent which is basically yeah here for example for the example below it would be 100 115 and the exponent is 10 to the power of minus 2 Which will allow to yeah, then deal with for example a fixed point setting rather gracefully and have literals without Yeah, especially code constructs Eventually we will have a proper rational Rational literal type instead of splitting it up in a value mantissa and exponent here I think at least but for now this is the fastest path to get this work But yeah so far these user-defined types we have only our value types This might be equal cost to want to encapsulate and abstract dynamic types arrays structs and so on and there's also of course Yeah, the desire to have proper algebraic algebraic data types in general so the more user-defined data types We have the easier it is to write a usable code and Yeah, there's questions about that for example where data locations would go currently the data locations are associated with variables and not types which makes that a hassle Which we will also probably change for independent reasons but yeah, I mean You may See where I'm going with this if we have all these kinds of types and maybe I would also use a different containers and all that It will no longer be feasible to To write functions to cover all these types and that's separate functions for all these types So all this Increases the need for having a construct of generics in the language Which has been long an idea to introduce to the language and which yeah, we're finally now making moves towards But yeah this if we have that The natural question is if even the built-in ties we have now can be user-defined instead Which leads me to another topic that we already started Just a standard library The idea here is to move whatever manually hard-coded constructs we have in the compiler now or as many of them as possible to actual implementations in Solidity written user code and then to ship that as an Standard library that's integrated into the compiler So yeah, this is what this is in the first instance going to look like that you can enable The standard library version of the compiler which in the end we will move towards and then import Built-in constructs from the standard library and it will be possible to export the standard library as a set of files Out of the compiler and they can then inspect what this function actually does for example Atma as a simple example, which can then actually be implemented in Inland assembly with additional checks that works To reduce the footprint of the compiler and to yeah make it easier for people to inspect what's actually happened and to extend it even But the problem there again is this is of course very restricted in the scope in which we Have what we couldn't move into the standard library as long as all functions is a little bit of monomorphic As long as we don't have generic functions that can handle multiple types So the full potential of the standard library will only be unleashed if we have generics Yeah, and the question is also whether we can not only move built-in functions But also built in types which would end then as you had earlier as user defined types in the standard library So the end go over the standard library is reduce the language to a small simple core language and have most of the current Features that we have in the language Defined in a standard library that is written in Solidity can be quickly iterated on can be extended by the community and can be audited and so on and so forth But to get to that point in full in each generics So the idea is to yeah I get inspired by a nice system of generics, which is based on a logically grounded type system We have products I have some types function types Which if your mathematician or category theorist the type system will probably form an petition closed category Which is how these things work usually in probably design generic languages like Haskell or like the trade system in rust or the generic system in rust And yeah, we will have polymorphic functions that can have it take arbitrary types and Yeah, ad hoc polymorphism using type classes or that's in rust. Maybe if people are more familiar with that trades And yeah, we will also for this to work out You kind of need to go all all in on it and I'll go to go for the web code the whole way that will also have Agri-variate data types and Yeah, all of this will also be only really useful if you have compile time constant expression evaluation because you will In generic functions have yeah constructed parameters that should be evaluated compile time. So Yeah, that's also something We will have which is as I said earlier for the code data location We will need to refactor the type checker to have be more flexible for this. We will need that as well And yeah, maybe we will also have linear types, which is how rust sporo checker has constructed But yeah, I will now have some examples with some made-up syntax and some made-up constructs But I'm going to say this is all early research stage. So we have nothing fixed here yet I'm just telling you where we had it and what we'd want to do in the next However long it will take So yeah, the go-to down-to-earth example of a use of generics is something like a resizable erase some container which Yeah, and this case is just an array that if You want to append something to it and it see the array is already full You just relocate was twice the size copy things over and yeah, otherwise you can just add the element to the end of the array and Yeah, then we can also Define an index index access as a user defined operator, which will then have this thing actually work nicely Yeah, one would need an allocation function and so on equipped with that but one can then build a library to Actually build these things without having special casing for each type each kind of base type But I have ears a bit cheated though because in reality things are more complicated, of course We have data locations again Maybe I need to restrict the base type as by some trade mechanism to something that can live in memory Which may be a value type or another memory array something like that So in the end it will get complicated, but it will be worthwhile and yeah if one wants to take this Up a notch and all the way one can actually then think about even defining the most basic types In Solidity itself This would be the definition of There are current representation of memory arrays in Solidity Written as user defined type with the customly defined index access to it So in the end for any day based on a memory array is just one stack slot We refer to memory arrays just as one stack slot. We index access by it by fetching. I mean the stack slot Points to some memory area where there is a size of the array and follow up by the data. It's a current layout at least And yeah, in index access we can just fetch the size Do some bounce checking and fetch whatever result we have at the offset that the index pointed to And again, we could define then that index accesses work as expected and yeah a length field could be added to that all defined in language Which as the advantage we had often the request to allow slicing for memory types Which we can't because the representation I show here doesn't allow it Since we expect the size to be at the first Memory offset where the memory pointer that is the representation type of this points to is the size We can't just slice away from the first element for that to work Memory arrays would have to work different similarly to call data types. There you have Offset and size on stack It would be a huge effort for us currently to change the entire compiler to come to change the representation of memory types If we had things defined like this It's minimal changes. I can just say now the memory array is defined as a tuple of stack slots one of them is pointed to data area is the size and Yeah, have a similar definition as before slight changes, but a few source changes in our standard library would then Be the same as changing the entire layout of Of memory types, which would be yeah month of month of work Why we maintained everything hard-coded in the compiler Of course, it's all the disadvantages if we actually keep this extremely generic like that We will lose semantic information. This will actually make memory optimization harder because yeah But the compiler sees it's just a bunch of stack slots There's no idea that that's actually memory areas you're talking about that are allocated may only be allocated temporarily And stuff like that so I'm not sure whether we will actually go this far and if so we would maybe probably do this in a Yeah compiler internal manner, but we still assume certain semantic properties about these functions without supporting Similar optimizations as we can do if we know what these things do in Yeah, random user code But yeah, why did I say stack slot all the time? That's yeah maybe obvious but Just to mention the stack slot would be the one primitive type maybe apart from From product types and even the basic Indigit types size integer types we have right now can be defined generically just Yeah, like the other cases we had so I mean this will really reduce the footprint you can Here could also then distinguish between types that are checked arithmetic and uncheck arithmetic by having very few functions that are generally written and Yeah, so yeah, but not to give to Make you expect this to happen too soon. We are still in early design phase for generics There's a lot of questions. This is a very complex thing to do and a very dangerous thing to do because Yeah, all of this needs to be logically well defined to not bite you in the back in the end so I mean we will take some time to design this properly and Syntax is also a question. There's very much Deferring opinions on how the syntax for these kinds of things will end up being I'm not that concerned by that now I would first want to get the semantics, right? But eventually we also need a good syntax for it And yeah, what I just said we need to decide what to do with this trade-off between Making this the language really self-defined in the in the very deepest sense or to have some fixed functions Which are fixed in the compiler, which means we can assume there's a mandates Or there are compromises between that, but we'll see So yeah to summarize What I was talking about and what I wasn't talking about and we will still do hopefully We will in the future Try to allow more pre-competition either in the constructor by the code data location or in compile time by compile time constant expression Evaluation which is something yeah a lot of people have asked for and which obviously makes it easier to Write things in that you don't need for example magic constants embedded in a contract or whatever because you can Compute them on the fly without it costing And yeah, the huge The huge topic for the future will be to make the language extensible and self-defining by means of improving user-defined data types pushing the standard library and making a move for generics But we also of course can't just ignore is that we're still wasting a lot of memory. I mean Whoever has used memory in Solidity will know that yeah, we basically don't free memory which for a long time Wasn't the main concern, but in the meantime is very huge pain point in Yeah for cost of contracts there were several Approaches we discussed so far for improving the situation there we a long time we Wanted to deal with this on the you level it turns out that may not be as simple So maybe we will move actually to analyze and Solidity which has the properties right there We shied away of doing that for Solidity being the more complex thing to analyze But maybe it's fine We will see and yeah We will also of course try to move completely towards via our code generation that we have some heard burdens there to overcome Still like them performance of the optimizer Better tooling support You still need to define but the background data for the tooling tooling to consume to actually make the experience as Nice as with the legacy code generation The details there would be that the tooling expects certain patterns to remain in the in the bytecode in the end Whereas the new optimization pipeline will mess them up by optimizing better But tooling needs to understand that we need to output data for it being able to understand that But yeah, that's hopefully where we had it and Yeah, I'll close with that if you want to give us any feedback help us with designing generics or Criticize what we are planning and saying we are crazy to do that. You know that reach out to us That's the channel see where each other and Yeah, thank you Yeah, it used to be the case that the compiler is too back-end path at the moment So I mean it used to be the case that Solidity was directly translated to EVM bytecode And then only up the only optimizations that took place were On the bytecode level For the past years we have Moved away from that and have a different new code generation pipeline that translates Solidity first into Yule and an intermediate language Which preserves some structure and which allows for more complex optimizations for inlining more analysis and then only to Translate Yule to EVM bytecode as a second step which can reduce gas costs significantly in some cases in some cases It's the same as before but yeah, and Yeah, the new pipeline the Viya IR is via the intermediate representation for compilation via you Thank you for the talk you mentioned generics I'm wondering if you could speak to how you're planning on implementing that whether you're going through or monomorphization Because I worry that the code contract size will balloon if you start, you know Doing the C++ style duplication of implementations or if there are some you know uniform representation You can do a la okamal or you know Java. I think there's not much you can do actually I mean Generics of C++ are different in the sense that they're analyzed differently, and you only get errors and On instantiations but we will still need to instantiate and generate code for each specific case But that's not worth A worse than what you get now what you get now is writing by hand for different types different functions They would end up separately in bytecode so Yeah It's not nothing is worse than that that's Duplication in code and in bytecode if you have generics at least only have it in source in only in the bytecode So my question is related to that question regarding generics, but from a different aspect so Type system understand but once we get generics into solidity Wouldn't the developer have to focus on ten more things instead of focusing on writing business logic? Good question. I would I would think that the Yeah, down to earth go to smart contract writer will not bother with this It's mainly something for us for defining a standard library and for people writing libraries to support smart contract developer so I mean The language supporting generics and having generic types doesn't force you to use them And it doesn't mean that anybody has to use them But it will make a language in the the evolution of language much Faster and more streamlined we can in the future ask People if they propose a feature to just implement it in a standard library way and then standardize it in the end If it works out which will yeah All the advantages one can think about that, but for user code for the end smart or contract code the difference is not that large probably I go into implement lambda functions just with this equation Definitely eventually I would say I'm not sure whether I mean getting the basic type system Going and all that will already take some time But yeah, eventually this is of course something that will make things easier to read easier to write and are beautiful So and maybe not I mean as long as you don't want these things to capture variables. It's easy Capturing even maybe something at some point we can also consider but yeah Not it's not the first thing we will do but eventually Have you been thinking about? Integrating on China computation and off-chain computation in the one source file. This is of course very problematic But in some moment in future we might need it not necessarily in solidity, but maybe you should try to think about that Yeah, I mean on the solidity level we don't yet Interact that much with yeah off-chain computations. They are two stuff or whatever But yeah, we are aware that we need to interact with that and support that where we can I think you mentioned something about the performance of via AR Did you mean how long it takes to actually use it? Yeah, I mean the compilation time Yeah, so relation time is kind of a ten times or even worse in some Exactly anyone that's used via AR today has experienced that so how much of an improvement do you think we are going to see I? mean So far the via our compilation pipeline has not been written with any performance considerations in mind at all they've written it for correctness first and Only now are starting to realize how bad that got and That we need to do something about performance there. So I couldn't mention that we can get quite away But yeah, it's hard to tell before actually doing it Regarding generics, how much thought is put into the audit ability for external code auditors? Will it improve the story make it worse more training? Can they forget stuff? I? Think it will actually improve things I mean We will be able to be happy standard library definitions of all the built-in functionality Which can be exported which can be analyzed I think at the point where we have generics going the standard library going we will actually at some point not promising that happening soon either but at some point be able to Define a form of semantics for the core language that remains which can actually help formula verification a lot and Things like that So I think reducing the language core that is built in and hidden in the compiler is actually a good thing for a form of vacation and audit Yeah, you were you mentioned that the data location is going to be coming part of the type instead of being associated with the Variable is that mean that we are going to be able to start writing things like an in-memory array of storage pointers or a In storage array of code data pointers because those are all well formed But something like a storage array of call data pointers makes no sense, right? Yeah, so what is the well-formedness look like? How would that how does that sort of you see that restricting that I mean? First part of the question. Yes, this is what this means. This is what will be allowed Second part of course there are invalid combinations there combinations that are don't make sense which then the types of stability check Okay, then thank you again