 Start for the next talk. We're gonna have Martin present. He's been a regular at various elixir comps With as both an attendee and as a speaker He has a very extensive collection of libraries that you can all Look into and use over at his GitHub account, which would be I guess github.com slash QQWI and Yeah, he's also an elixir forum moderator and without further ado, let's give it up for Martin for his talk going meta little So welcome everyone. I'll be talking about going meta with elixir So what does it mean to go meta? Well within the context of Programming meta programming it means that rather than just why writing a program that takes some data's input and transforms this data And perform some tasks as output. It actually treats the program itself as data that it can create and modify while it is running So in this talk, I'm going to Explain to you in what ways elixir helps you with doing meta programming and why this is useful and give you a couple of very practical Real-life examples, of course some of these examples have been trimmed down somewhat. So my code fits on the slides and This talk is geared towards an audience of people who may be new to elixir or may already know a little bit of elixir But are new to for instance programming with macros It's not a problem if you cannot follow along with all the code examples that I show you It will all be very quick because we only have 20 minutes But I will link to a lot of resources and Show you many names of where you can find the stuff that I'm talking about so you can read about it in more detail So in elixir there are two main techniques that you can use to do meta programming We can both influence how code is actually being compiled while you're creating your program and once your program is running We can also change what code the program itself actually consists of and this might still sound a little bit vague But I will explain to you why these two techniques are very useful. So let us first start with running code at compile time The simplest and most general technique to use to do this is dynamic compilation If we have an elixir module like this simple example here that has a single function to Compute the Fibonacci the end Fibonacci number Then all the part that is inside the function clauses is is called at runtime when the program is actually running and someone calls this function However, all the parts outside of the function are evaluated at compile time And you can use any elixir code including of modules you wrote yourself that have already been compiled and just call That stuff over here and it will run at compile time and do all kinds of interesting stuff as a very simple example we can just use an if statement to conditionally compile one implementation or another or use one implementation and Raise an error at compile time to explain. Hey, would you can't use this feature? You cannot use this library because we're missing some very important thing that's underlying runtime should provide So then that way we can propagate errors to show you errors early during the development cycle So you get very quick feedback, which is a lot nicer than finding out Only once your program is running or once your program is deployed So let's go back to our Fibonacci example that we used before Now this code is this is very naive implementation of the Fibonacci function And there are some very clever ways to make it faster, but this is a talk about meta programming So we're going to solve this problem with meta programming one thing we could do is to Pre-calculate a couple of known results to the Fibonacci function and then use a for loop to iterate over these known results and Define function clauses for each of these known results. So now The Fibonacci function does not need to recurse for these simple Function calls and that makes it a lot faster So writing code like this Does exactly the same thing as if we had written this code by hand But of course once we're adding more and more stuff to this this becomes less and less maintainable As a real-life example, this is how elixir uses This is how elixir is able to have proper unicode support They take the unicode data dot txt file that is released every couple of months by the unicode consortium And they parse this file to find out properties about the various characters and there are thousands and thousands of them that make up the unicode standard and Then we iterate over all of these ranges of characters to define in this case whether this is a uppercase character or not and This is a very clear example where you could not maintain this code by writing all these function clauses by hand Now if plain dynamic compilation is not good enough We can also use a more powerful tool and therefore Back then and in some cases still is to power phone switches, which Really need to be have have a very very good uptime be available all the time and they have this great advertisement video of now thirty four four years old where they explain How they're able to call each other and use this software that that does this phone that that manages this phone switch And when one of the cold break calls breaks, they're able to fix the bug coordinate over the other calls and Deploy a new version of their software without any of the other calls having to be brought down and this is not only nice For this particular example, but also really nice when working with certain types of web applications where it's really Important to be able to still access your current daytime memory or have these connections to other services open all the time it also really helps while Writing and debugging your software in in the console because if something goes different than you expect You're able to make quick changes to your code Recompile that particular module and all your data is still there You don't have to refresh anything any part of your data It also really helps here that elixir is a functional language with immutable data types But now for some other examples where hot code reloading really makes sense Besides being able to deploy new versions of your software without stopping a program One example which is by some people called a little bit of a heck is known as fast global and nowadays There's also a built-in version of this persistent term because for most cases it is fine if you store your data in a single elixir airline process and When another process needs to read this to communicate coordinate with this process However, all memory in elixir airline processes is completely Separate from each other. They're not able to immediately access it and this means that in some cases They might become congestion on certain processes that have certain data And there are some other ways to resolve this for instance using ETS or long-term storage But even that has at some point its scalability issues and one thing you can do to fix this problem is to actually Recompile data into a piece of code that is then shared between all the processes again Of course, there are many cases in which this doesn't take sense So benchmark before you use it, but if we wanted to build something like this by hand It would work like this We have a simple module with a get and a put function and in the get function We see if we check if any value was already stored and if so we return it if not we return a default value and When putting a value we evaluate some ASD that we're building here at runtime and here We define a module that has the given key as module name with a single function that contains only the value we're interested in Of course, don't write this code at home nowadays starting in OTP 21.4 or whatever the version exactly was there's a built-in way to do this using the persistent term library that now ships with Erlang itself and Again, this is somewhat of a heck. You only really need this if you have efficiency problems But let's look at another example TC data is a very Right widely used library for keeping track of time zones because what how time zones and daily Summertime winter time is handled daily savings time is handled changes between countries But also sometimes it how this within a country is handled changes based on political decisions may sometimes we only have like a weak Notice before it happens and so it really doesn't make sense to pre-compile your Program only to have the time zone information change and then you have to do it all again So it makes sense if this happens outside of your normal program release schedule So what tc data does is at runtime once a day? It performs a request to remote location where the time zone database file is stored and if there are any changes it Recompiles that information into your application currently It's actually still using ETS, but there is work being done right now to replace this with persistent term exactly for scalability issues that they're facing another very interesting application of this is Using introspection there are many languages and environments in which when you want to find out about your application has a memory Leak or is behaving in an unexpected way in the wild You have to stop the program add some tracing statements to your source code recompile it started again, and then hope and pray that the program will trigger this same weird edge case however in elixin Erlang you're able to actually patch this tracing information into the calls where you wanted and The rest of your program is completely unaffected. So in this case we start with the same Fibonacci examples before but now When starting introspection deburring statement where we say we're only interested in calls to the Fibonacci function with one as an argument Then that call to the tracing functionality is only used Over there and the rest of our application still works exactly as it was before Now when we call it we immediately receive all these tracing messages So we can use that to debug and once we stop the tracing information is removed again And now our program works identical as it was before we started tracing So this really helps for instance when working with race conditions, etc Because all other parts of a program are completely unaffected and this works because we're running on a virtual machine with bytecode So there you have it There's our two tools you can use and the compilation at runtime is maybe the slightly more archaic one and has more niche Applications where you could really use it but both running at compile time and recompiling code at runtime are very useful And of course also share a little similarity between them as a final warning I would like to say that these are great tools and you can build some really amazing stuff with it however, if you're going too far then your code might become very hard to read or maintain because if someone writes Certain code and the macro completely transforms it into something else Then it becomes difficult to know what is going on But of course also very important is to have fun and to enjoy what we're working with All the things I talked about have some information on the internet which you can look at my slides We'll of course also be available online and With that I want to thank you very much for your time listening to me today Thank you very much And now there's some time for questions Or The question was have you run any benchmarks to compare putting stuff into ETS versus Recompiling a module and reading from that module. I have not done myself There are some benchmarks available of other people who did this for various different projects and essentially Recompiling something of course takes a lot of time and also the more processes you have the longer it takes because all the processes will need their code store refreshed However, once it is stored it is a lot faster and a lot more scalable than ETS But only at very large scales. So for most applications starting out It really makes more sense to first start with an agent and then at some point with ETS And only when you're really reaching the limits of that start looking into persistent term storage Do I have any advice on how to test code that uses macros well, I have written some libraries in the past that used macros and Something that really helped me while debugging is that after creating my AST in the macro To just put an IO inspect statement there And there are some helper methods as well in the code module that elixir ships with that allow you to introspect your macro or your your AST as readable source code again and that really helps to find out what exactly is going on there and besides that try to come up with a lot of Examples of what you want it to look like when someone uses the DSL that your macro provides Also try to come up with a counter examples of things that people might try so you can test for Yeah, common errors and of course I'm a big fan of using property testing wherever it's possible And that might also really help to catch some edge cases that you would otherwise not have considered great question. Thank you Yes Yeah Great, I have not read it myself, but it's probably a great resource because Chris record really writes very well Thank you for mentioning it. I will add it to my slides. Thank you very much