 Well, hi and welcome everyone to the session. What's new in Gradualizer? Gradually typing in Elixir and Erlang with Radek Shintishan. We're really glad you're here with us, Radek, and without any further ado, over to you, Radek. Thank you, thanks for having me. So I will talk about what's new in Gradualizer and about gradually typing Erlang and Elixir. Okay, so what's the agenda going to look like? First, I will introduce myself. Then I will talk a little bit about what Gradual typing is and how it fits in the picture of strong typing, weak typing, dynamic, and static. Then I will talk about the approaches to gradually typing Erlang and Elixir. About what's new and actually since when it's new because the tools I'm going to talk about are still in development, so they change quite frequently. But the meat of the presentation is going to be the demo or two demos, one with Elixir and Gradient and the other one with Gradualizer and the Erlang language server. And then I will finish with some practical tips. And I'm happy to answer some questions throughout the session and we can have a bit longer discussion at the end. So let me introduce myself. I'm Radek Shanshushen. You can find me under the Irish handle or nickname on GitHub, Medium, or Twitter. I'm a tech leader at Erlang Solutions, specifically the crack of Office for Erlang Solutions. I have been working at the company for almost 11 years now. I'm a static type system enthesiast, and I'm saying enthesiast because I don't have professional background developing type systems or academic schooling in that. But in general I have a master of science, computer science background, so I think it's a good foundation to deal with these topics. And I'm contributing to Gradualizer as a core team member and I'm also co-creator of Gradient, which is an Elixir frontend to Gradualizer. I also believe that sleek and good quality docs can make a programming language community and ecosystem easier to approach, easier to start learning and start contributing. And that's why I co-authored Erlang Enhanced Proposal 48 and also implemented that as part of E-Doc, which is part of Erlang LDP24. And this was the groundwork for using the Elixir X-Doc sleek and modern docs for Erlang projects. And I'm also an instant messaging expert, an XMPP engineer, I'm an XMPP server core team member, who I'm recently not very active on that, on that front. And in general I'm an Erlang and Elixir programmer professionally. So more on the topic of this talk, what is Gradual typing? Let's start with this simple diagram. And obviously it might be a bit biased, but the point here is not to classify and argue about which language should be in each, in which part of the diagram, but to think about how we can classify or what are the angles or the axis on which we can classify programming languages. So let's think of two axes, weak typing and strong typing being one of them and dynamic versus static in the other one. So for some examples, how can a strong type system look like? A perfect example of that is OCaml, which has a very strong type system, which is very explicit but sometimes clunky. So examples of statically type programming languages are Haskell, OCaml, Fsharp, Scala, to some extent C++. And what's on the other end of this axis? So if we have static, then we also have dynamic. So how do these programs differ? Programming languages differ. So in this case, a program does not have type annotations, but if it's a strongly typed language, then data has well-defined types. So it's impossible to do invalid operations. But if it's impossible, there has to be a check done at some point. And in these dynamically typed languages, this check is done at runtime. So just before an operation is carried out, it's checked, whether it's a legal operation or not a legal one. So for example, in languages like this, it's not possible to add an integer in a string because this results in a runtime exception being thrown. And this might happen ideally when we test the program, but sometimes it might happen as flight as production, which is pretty bad. So whatever pros of these languages, they are usually more flexible because there are no type checks, there are no compile time checks. So we can just pass that around where we need it. And this leads to a flatter learning curve. But there are also some disadvantages to this. For example, it may be harder to navigate code because we sometimes deep in some code, we are not really sure what code we, sorry, what data we work with. So are these numbers, strings, some hash maps, or more complex data structures? So the APIs are less defined. We also have to test such programs way more and achieve very high coverage to have confidence in the logic of the program. And there's also performance penalty because the checks are done at runtime, they slow down the program a little bit. And examples of such languages are Erlang, Elixir, Python, JavaScript, or PHP in Lua and more. So let's see an example of how a dynamically checked program language looks like. Here we have a snippet of Python where we define a lambda function or a closure which accepts a parameter. And we immediately apply number three to that closure. And because this closure just adds the parameter to a constant two, this evaluates the five. But if we try to apply a string value, so the string a, we get an error thrown at runtime and this is how the error is reported. That type error type must be string not int. In the case of a statically typed program language like Haskell, when we define something similar, so a lambda function which accepts the parameter a and adds this parameter a to a string with a number two. If we try to apply an integer to that function, this program simply will not compile and we will get a very specific error. So it's not possible to run it. There is no space for runtime exceptions. We'll get this feedback at compile time so way, way earlier. There's no risk of production issues. So we see some benefits of both approaches, dynamic and static. We also see some deficiencies of both approaches. So why not try merging both or taking the best of both of these? And this actually has been tried multiple times in programming language history for different programming languages and also using different theoretical methods. So gradual typing is one of these approaches but we can also find the notion of optional typing in literature which was not as strictly defined formally but the practical implementations of gradual typing defined by Jeremy Seek and Waleed Taha are for example TypeScript and Flow for JavaScript. MyPy or Python or TypeTracket are probably the best or the airlist example of gradual typing in practice. Hack from Facebook for PHP and also a gradualizer for along and gradient for now gradient for Elixir. And when we talk about optional typing it also requires underlining that there is a well-known and quite widely practiced use of type specs which were popularized by a dialyzer developed by Tobias Lindel and Kostas Saganas. And thanks to that our programs which used to look like on this slide now quite often look more like on this slide. So we have the type information, we have the better defined APIs but could we use these types and function specifications more effectively? So the advantages of gradual typing are that we can move between a completely dynamic program so like this one to a statically type check program with like this one but not do it all in one go so we don't have to specify the entire program to reap the benefits of using the type checker. So we can add the types and specs as needed for example as a team app skills and learns new techniques or as project grows more people are added to the team or it matures and there are more and more APIs and we want to make them easier to navigate and also when quality becomes more and more important it's not a prototype or not a prototype anymore it has to actually work under a lot of different circumstances in production. So another aspect of gradual typing is that some parts of the program are checked statically the ones that have type specs the rest is still checked dynamically at runtime and it can benefit from the let it crash philosophy or the supervision trees restarting parts of the program etc etc so we go we do away with the all or nothing approach that it's either all type checked and annotated or it's none and it might all crash if we don't test extensively okay so now that we know a little bit about the approach in theory we can have a look at the practical implementation for Erlang and this is Gradualizer this is a Gradual type checker created by EOS of Spanningson and the first introduced at Codebeam Stockholm and the assumptions are that we can start with writing dynamic Erlang as usual and that message passing will always be dynamically typed so Gradualizer does not use any mechanisms to type check this part of our programs static typing is opt-in so it only happens when we add a type spec to a particular function without a type spec there is no check happening so we can add the specs only where we feel it's a prep right and then we just run the type checker and profit from the box code and from the feedback we get from it all that before runtime and in general the tool is still experimental but it's constantly improving and on the right hand side we have the list of top contributors to the project a little bit about the theoretical underpinnings so there is no global type inference in Gradualizer but there is a mechanism called bidirectional typing used which relies or incorporates a limited local type inference and the amount of type inference can be controlled with the infer flag passed on the command line there is also pattern type inference and there is a limited type refinement based on pattern matches or actually mismatches and guards and here is an example of type checking a very simple Erlang program at Gradualizer so we define a simple function taking two integer parameters that just adds them we also define a print function which doesn't have a spec which is fine because it's gradual typing we don't have to add a spec everywhere and we define a test function that misapplies the add function to two different arguments because one of them is not an integer and then we run the type checker on this module and we immediately get feedback that the string on line 8 is expected to have type integer but it has type string and the error is highlighted or underlined so it's very similar to the Haskell example and I would like to introduce Gradient which aims to do something very similar for Elixir basically it's a frontend to Gradualizer which allows to use the tool for Elixir programs and kind of it's the glue between the Elixir code and the Erlang syntax tree that's actually produced by the Elixir compiler and also is used by Gradualizer to do its type checking but Gradient is not just that it's also a little bit of macro magic to provide more type checked or a bit better type checked frameworks all the assumptions are very similar to Gradualizers and the tool is created mostly by me and Tromek Vojtasik a colleague from the Krakow office for solutions and it's also experimental but it's constantly improving and here is an example of using Gradient with some Elixir code the code is exactly the same as with Erlang so we have an add function taking 10 teachers we run a mixed task gradient and we get the feedback that's the bit expression line 7 is expected to have a integer and the error is underlined in red so that's it for the introduction of the tools now I would like to very quickly go over the features that were added since the last approximately half a year so it's exhaustiveness checking of non-trivial types it's type checking of maps and therefore elixirs tracks it's exhaustiveness checking of map variants so this is connected with the first point but it was quite a different chunk of development it's property based testing of the type checker to well squash as many bags as possible and make it robust which also led to the discovery of some infinite loops and fixing them there is also a gradualizer diagnostic for the Erlang language server which makes a gradualizer feedback immediately available in the editor of your choice and many many many bag fixes so I won't go over all of this in detail at this point because I really wanted to focus more on the live demo which I will switch to now are there any questions at this point I guess there is nothing in the Q&A box okay thank you so I hope you can see the screen clearly and specifically if the font is big enough if not please let me know so on the left hand side we have an example elixir module and let's try type checking it okay so I ran gradient type check file on this module and it returned okay so it means that the code type checks properly so this code describes a gen server which is an implementation of simple server handing two kinds of messages one of them is an eco request so the server should receive a message and then just echo it back to the sender and the other request is a hello request so the server should just receive a request of this type and then print the name or well the payload of this request but it doesn't send anything back and the idea here is to make this message passing interface not only the function interface but also the message passing interface explicit so how can we do it how can we leverage the type check or type system to do it we define the message type which is a union or some type of these two aforementioned requests and let's see if the type checker and this definition can guide us somehow on working on this code so for example we know that the type checks at this point but what if we change this definition to something different let's say I remove the hello request and let's type check again right now and we get the error that the clause on line 63 cannot be reached let's check line 63 and we see that this is the implementation of handlefall the gen server behavior callback for a tag tuple with the hello tag and a name so okay we declared that the type of our handled messages is just the echo request but here we have code for a different request so okay if that code is never going to be used so let's comment it out and type check now okay it's good we fixed the bug so let's try again with the expanded version of this type so this is exactly the same as the first definition but here we just expand the types to their definitions instead of using references to ring those types and let's type check at this point and we get the warning that not exhaustive patterns on line 58 and the example value is hello so let's jump to line 58 and indeed here is the implementation of the handlefall function for the echo request but we are missing the implementation for the hello request which is now part of the declared messaging API so okay let's fix that and bring this code back and type check now and back to normal we declared we handle to different requests and we have to handle them because otherwise we get an error reported so how else do we use this mechanism like let's try what happens when I do a silly mistake like I do echo rex instead of correct time and let's run the type checker now so we get the warning that pattern echo rex on line 58 doesn't have the type message and indeed because type message defines an echo rex type so let's fix that type check one more time so what do we get we get dead code detection if we declare in the message less types of requests that the implementation does we get warnings about missing code and we get warnings about silly typos and issues when we implement a handler for a message that doesn't exist so far we get it seems pretty useful so let's go to the top of this module and look at the echo API function this function is exported to the users of the module and of the server so so far we looked at how type checker can help us when we implement the logic of the server but can it also help us on the side of the server process so before we actually make the call to the server and yes it can in some way let's see how so we know that to call the server we use gen server call let's check functions documentation but we know that this function returns a value of type term the most general value that exists it can be anything there is no information about that so for example if we wanted to pattern match on this value because we know that an echo response will be something different than a hello response or that there might be different echo responses based on the state of the server we might want to pattern match on that and do something with the response afterwards but since the gen server call returns a type sorry a term of type of this type of a general one the type checker cannot help us so we have to we have to help the type checker help us in this pattern and we do it by using the annotate type macro where we just do the call as usual and then we also pass information about the return type of this call and thanks to this we reap the benefits of the type checker so for example let's try what happens when I do a silly mistake like this we get the feedback similar to the previous one that's the pattern echo that on line 30 doesn't have the expected type so that's nice let's fix that check again back to normal so this syntax might look a bit heavy so we can also do it like this to make it more similar to ordinary elixir so the annotation can come last and it's probably being piped into it or we can complete to get rid of the explicit annotation and just use an auxiliary function like this for the same effect so let's see what happens when we do it like this and let's actually introduce a bot okay this worked so what's the difference and why do we even need it so another type doesn't do anything at runtime it's a no-op it just takes the value of that we passed to it and it returns it back but it embeds this type information into the abstract syntax tree of the compile module so the type checker can use it for its reasoning about this base expression and if we don't want to use the annotation but we are more happy about the auxiliary function this information is simply passed in the spec of this auxiliary function so we need to pass this to the type checker because otherwise it just won't know the type of this value of this response so sorry to interrupt just letting you know there's about 10 minutes remaining thank you do we have any questions at this point yes we do have two questions in the library books okay so I'll read the first one so we get a question from sorry if I pronounce it wrong but some of the gradual type checkers a little smoother gradual typing so just having an option such as minus minus strict what do you think about this approach so I'm not sure what the meaning of this option would be but I think that one option that's pretty similar in Gradualizer is the infer option it's a bit more strict in the default mode the infer option it will try to tell the type of more more terms more things in our program so I think that it might be a little bit related but maybe not exactly the same thing okay thank you for the question so before answering more questions I would like to go to the next part of the demo to stage 2 and here I would like to ask the question like how can we use the type checker to do a little bit more how else can it help us and one of the ideas is that in the airline virtual machine or in the electric runtime we have a lot of concurrent processes we have multiple servers running but there are different kinds of servers like one of them accepts different messages than another one and one of the possible mistakes is that we send a message of a certain type to a server which just or a process which doesn't handle it and it will lead to a bug in production it will lead to a bug in runtime or even a crash at runtime so let's try to run an example and see how it works okay sorry I was to run the example and I run the type checker so I run some code the code of a test and now I will comment a line that I think will lead to an error and then I'll run it again and then we'll go over the code of this test and yes something else is happening like it seems that this got blocked and yeah after a while we got a runtime error that gen server call timed out so well this is definitely not something we like to happen in production so let's look at the code to see why it's happened so we create two types of processes one is of our particular server type so we created with server starting and the other is just a random process that does something completely different than our particular server and when we use the server API to call our server process it responds with with the expected things so it just returns the payload but if we use the server API to call a random process like just in some arbitrary process well we see what happened on the right hand side the server sorry the request timed out because the server cannot handle it and we would like to avoid it so let's see what happens when we run the type checker on this code right now and we get a warming the variable on line 103 is expected to have type T not just the arbitrary or the general type PID and so we get feedback sorry we cannot use the server API with any process with any PID we have to pass a PID of this particular type and how is it how does the type checker know that so when we jump to the top of the server definition we define a type T which is actually equivalent to a PID but it exists as its own type a type of this particular server and then all the API functions accept only this type T or return this type T instead of a PID so when we start link we return okay T or already started T the eco function only accepts the type T the auxiliary function called eco also only does that all the rest of this example is the same as the previous one but we see that thanks to explicitly defining the type for our process we can avoid the API of this of the server being misused with PIDs of a different process so another win we know this we type check the code and we know upfront before running it that we misuse the APIs and I will quickly jump back to the next example without asking for questions at this time because one of the things that Robert Burding one of the creators of Erlang pointed out when I was showing this demo to him that we could actually generate these auxiliary functions and it was obvious immediately obvious to him but it took me a while to figure out how to do it but indeed it's possible so we don't have to implement this function ourselves we can use some macro magic that's provided by gradient to generate them this function is actually generated to be exactly the same as this example here so we don't have to uncomment this code for our server to run and to type check see if it type checks and indeed we got an okay so how is it possible that this code is generated and is there anything else that we can benefit from so it can be generated because instead of using a gen server reply we use type server reply and as in the previous example we used the type of the response in the annotation or in the auxiliary functions spec here we use type server reply and we pass the type of the response so we have more locality because this type is type of this expression so it's less possible that we make a mistake even the value and the type are on the same line and moreover thanks to using the same macro or these two things being arguments of the same macro this is it's not possible to actually do a typo or provide an invalid value a value that's not of this type here because if we do it we will get a warning that the variable line 71 is expected to have a type contract echo response but it has type echo sz and we get it highlighted so not only we can generate the auxiliary function but we also get type checking on the response so let's fix it and that's it for this example I have one more thing that I would like to show and this time we will jump from the elixir world to the Erlang world and here I would like to show the integration between the gradualizer running the background and Erlang language server which gives us feedback and this is a different code I won't go into the details about what this code implements but I will run the type checker on it and see if I can get okay so there is definitely some errors in this code so let's just browse this and see how the integration with Erlang language server works okay so for example here we get a warning that the pattern of R doesn't have the type abs something something so let's scroll more that's the operator plus as expected to have a different type okay so that's how the feedback is displayed in the editor and again we see the information the pattern of R does not have the type something something so okay there seems to be a repetitive warning about the pattern of R so let's search for this pattern in this code like I okay we found the definition of type term which has quite a lot of variants like probably 15 or maybe in 20 of them and one of them is commented out so okay let's go back to one of these reported errors and comment this out and yeah we fixed the error at least we think so because it's satisfying but now when we change the type definition and go back to that place we get a different error that in the place where we first tried to fix the issue we got feedback that there are non-exhaustive patterns that pattern of R is never matched so we see how we see live feedback about how our types of representation mismatches if it's the case and it's quite easy for it to mismatch if we operate on types that have like 20 different variants so live feedback in Gradualizer also seems to be from Gradualizer and Editor also seems to be pretty useful and I like to stop demoing now and go back to the last slide with a few closing remarks like I think it's worth to leverage the tools as much as possible so use the Erlang language server for live feedback in the editor use the compiler options like one missing spec and one missing spec all because the more specs we have the bigger confidence in the code we can get there is no elixir language server integration available yet for gradient but fingers crossed we're going to get there and we could see how using these tools leveraging these tools we could do a bit more type driven development by describing or domain with data types writing them a limitation letting the type check refine bugs and not necessarily test everything even the trivial parts of the program and it also helps us with aggressive refactoring when once we have the domain described with types but we have to extend this description and we have to modify the type definitions or a function spec we just run the type checker and let it figure out what else has to change in the program to match these changes in the domain model however the tools are still a bit experimental so if there is suspicious code or suspicious warnings or some warning is not clear it's worth writing a test and if you happen to find any issues like this please do report them on the respective project sites on github so gradualizer or gradient and that's it thanks for joining thank you so much for joining us everyone and thank you Radhik for sharing your experience with us today