 So we are here to show Mantis, our ScalaClient, Ethereum Classic ScalaClient, we have been working on it for more than a year and a half and so we are going to show all the cool stuff that we have made and also I'm going to do a host a small workshop where I'm going to hack it and to show how can you turn it into a statistical note so we will be watching cool graphs about the blockchain and and and the transactions history first we're going to go through the code and the architecture just to engage people and then we are going to tweak the source code to change the database that we are using which now is RocksDB we're going to change it and use PostgresDB so we then we will be able to run some queries and we are you going to use a cool library that is going to display these nice charts on screen so it's going to be fun so for example we will be we will be showing some queries like which is the top miner at a certain point of time and that kind of stuff so it's pretty cool. Hey guys thanks for coming this talk or this workshop is named Hacking Mantis so hope you like it I want this to be like a more relaxed session so if you have any questions just raise your hand and interrupt me it's fine. I'm this guy I'm Alan I'm a Scala developer we work for HK we are the Attics team which means we are a small subset of the growth and the team that it's located in Buenos Aires Argentina you might have noticed my my weird son my English let's say so let's start so I told you that we are going to hack Mantis so before hacking Mantis we need to learn what it is and what the tools we have used to create it right and then we will do this hacking session so what's Mantis? Mantis is a Scala based Ethereum classic it has been written from ground up in Scala and here you can see the release history so in July 2017 we released our alpha version which includes transaction execution and block synchronization. Transaction execution is basically the the EVM so we wrote it in Scala from scratch again and August we released our beta version which included the Shasion RPC API the mining and pruning. Pruning is the mechanism that allows you to remove some state from the state try that you don't need to keep your database small and in January we released our first version the production one which includes the Dallas integration the Dallas is a wallet developed by IHK and so you can interact with Mantis using an ICI and it has some stabilization and the monetary policy ECIP and the next release the 2.0 is going to include some performance improvements as well as the new EIPs the ones for Misantium and Constantinople. Those requests are ready for review so they should be merged soon and one thing that I really like is that Mantis is being used for other projects and not just for IHK ones so we have seen some other people using our code base for their cryptocurrencies or their blockchains so that's quite nice and here is the repo it's public open source you can use it tweak it and there is the documentation site so you can get in there and start using it so functionalities again it has regular and fast sync transaction execution the JSON RPC API peer discovery pruning so it's full ETC compatible so you can just launch this client and download the chain and start processing transactions so you can run your full node or your light node using this client and it's only 15k lines of code so it's quite small and that's because of Scala and I will tell you about that later and also it has a lot of tests so I'm not including within this 15k all the tests that we have we have unit tests integration tests and we are also running the Ethereum test suite which is a centralized repository with common tests that are language agnostic that you can run to check that for example your virtual machine runs okay or in general your node is compatible with other nodes so that's pretty cool so wait Scala do we have any yeah yeah yeah yeah no yeah this this is an open a different repo have you ever heard of it the Ethereum test suite so this is the Ethereum test suite and we are compatible so we have a sub module and you can run all of this test but we can check the repo if you want to so do we have any Scala developers see so I don't want to get into a language wars or anything like that I love all languages besides C so so no I this is just I'm going to speak about Scala and why I like it it doesn't mean I don't like other languages like Haskell or like Java or JavaScript so why Scala and first of all because this is the first client written in Scala so that's brand new and but also because it's JVM base JVM is the Java virtual machine and that virtual machine is battle testing production some people might say it's slow but it has been running for years in enterprise grade software and it's really really good and it has a lot of libraries and it doesn't have as much libraries as you might find in Node.js for example that when you have a package for turning a string in uppercase you might find 50 packages to do so and in Java related languages or JVM related languages you have like two different packages that do the same but they are really good and they are heavily used so I think that's a strength and also you have a lot of tools like the devalera is awesome the IDE is the I mean it's been around like for 20 years so tools are great it also supports two different paradigms you can code in object oriented paradigm so as we know domain objects are represented as objects that you can send messages between them and they hold state but also the thing that I like is that you can do some functional programming and actually our code base is mostly functional programming style so before jumping to the functional programming definition the type system is awesome in Scala I mean the compiler is super great actually the the people that created Scala they are especially think rating compilers so Scala is like it's super good at compiling and actually you don't need to define some types and the compiler will look them for you so it's quite awesome and we feel protected using this type so and functional programming so have you ever heard about functional programming no raise your hands yes so we're using functional programming which is a different kind of modeling software it's it allows you to model software as functional as functions like in math they don't have state that they don't change the state they are like just pure functions that you can use and compose to create your software and that's quite cool because first of all it's closer to math so you can if you in a small program I'm not talking about this big project but you can start thinking about a functional application and you can start replacing values and understand what's going on because it's like doing math at high school so it's quite awesome and also it's less verbose and that's why we have 15 can lines of code instead of 50 so and it makes the code easier to read right so that's why we like functional programming because I just love it and so I'm going to show you some snippets in Scala that is not to teach you Scala but so but to let you know just a couple of things that we found that are really really cool for example you are seeing here a function that it's called from bytes which receives a type which is an integer a payload which is a array of bytes and the protocol version which is another object right another type and it returns a message so just looking at the types you can understand what's going on there right so that's pretty cool and and we have here I would show you two cool stuff that you can do in Scala the first one is that we have these stuff that it's called pattern matching it's sort of the structuring in JavaScript somehow but you can match it it's like the child between the structuring and switch and ifs and all that stuff so let's see an example so here I'm creating a tuple with the protocol version and the type right and here I'm matching this tuple against different values and cool things that I can do within this match I can just ignore the first value for example I just want to match for this one because in order to parse the hello message I just need the type I don't need the protocol version because it's the same message in all the versions or for example I can match this type against both the two protocol versions at the same time with this with this little guy here and what I can also do is after matching I can use this value in a function call so basically I'm matching the something and I'm processing it and if you wanted to do this in JavaScript it would be like a bunch of if statements with returns or a switch statement with another switch statement inside or something weird so in Scala just this and also one thing that you might not have noticed is that for example look at this this sentence so we are saying payload dot to hello but I told you that payload it's an array of bytes which is a standard type it's an array so how can you do that one in Scala you have this thing name implicit I'm not going to speak about it because they are a little bit confusing but basically you can attach methods to existing classes so I'm able to attach this to hello which converts this array of bytes to a message just to the array so that's pretty cool I guess and the code is quite readable yes yes actually it's wrapping it so you can wrap it and extend it it's quite cool actually in this example you are not able to see how am I wrapping it wrapping it because but you just need to import something on your file it looks for the implicit and then when you look for this sentence the compiler knows which wrapper to use and magic sorry exactly and this is a really cool way to add methods for example if you are importing a library and you want to extend it you just can do this but implicit can be tricky so you need to be careful and this is a functional programming concept so I'm not going to get too deep into this kind of type but we are going to talk about options so first let's see the function signature so this function is named get block transaction count by hash so this is a JSON RPC API method that allows you to return the transaction count for a given block hash right and it returns a service response blah blah blah which is the thing that we are going to return from our server so let's think about javascript so how can you do this so you need to look for the block in your database by hash and the block might or might not be there right and if it's not there what do you need to do in javascript your dv layer should return null or I don't know empty string I don't know whatever you want to that's weird and then you need to place an if statement saying if this is null return zero or null but if it's not null just let's count the transaction list right size so that's that's like a little bit weird I mean you have an if statement you need your I mean the code is not as fancy as this one so what I'm doing here we have this type which is named option and option it's like a type that might or might not have a value so it has two instances it might be none which means no value or might be some and can hold the value right so and that's actually what we are trying to represent with the with the database in the database the block might be there and hold the value that's some and the value or might not be there so it's none it's not null why not yeah exactly yes and this is embed in the language and and look what we are doing here so there is no if statement so you might ask me okay what happens is the block is not there right so basically here you will see that we are calling blockchain let's get block body by hash so we are retrieving the body and if it's none this if it's a none this map function won't be doing anything it will just return the value it what it will return the then all sorry then none it will it won't try to access the value because there is no value but if this function return a block body which means a sum block body we can just take the value apply this transaction list dot size which means let's count the the transactions and it puts back again into the sum container so basically this is a nice container that you can take the value process it or if it's not there it won't do anything no null pointer exceptions no if statements nothing so it's quite nice right and then you can represent this none with a null value or whatever you want to for the end users but within your code you don't you don't need to face with those kind of things and this is a similar example it's not the same I'm presenting you this either type which is similar to option basically either might have two values left or right and so let's look at this function definition so this function is named validate and it receives a block header and a parent header and returns either a block header error or a block header valid message so basically what this function is doing is to is validating the the block right so first of all if you want to do this in another language what how could you do it I mean you can throw an exception if it's invalid so why would you throw an exception right and or you can just create this hierarchy of objects that are the parent is block validation and it could be error or not and then you can switch somehow in your code or check a message or something in Scala it's way much easier and actually I'm really proud of this piece of code because it's really easy to read so this is a construction that in the end it does something like the map but it's not map it's flat map it's it's similar and basically you can see that we are calling different validation so we have validate extra data validate timestamp validate difficulty and so on so forth until we return a block header which is valid but if we find an error here for example here the timestamp is wrong the validation or the the difficulty is wrong it will return a left right because we have the errors on the left and the valid on the right so in Scala we have this map is right bias so if we return a left the map will stop right so that's pretty cool because if we want to short circuit the execution of this piece of code if we return a left it will just end and return the error so look at this validation so if we found an error in gaseous we are not going to execute neither this line or this line or this line we are just going to return the left with the error so this is quite cool because if you wanted to implement this in JavaScript it will be a bunch of if statements with returns and or not or maybe no yeah so i mean i i don't have anything against javascript i really like it but in scala this piece of code is quite readable right yeah this yes this this gets translated to a bunch of flat maps chain this is like sugar syntax and this is called for comprehension it it's it's quite good and you can add filters and like if statements with conditions and stuff you can do assignments and it's similar to haskell do notation so i'm going to speak about a little bit at mantis architecture so because this is required so you can understand the hack that i'm going to do afterwards so first of all one of the goals that we had when we were coding mantis is try to avoid uh advanced libraries and you might ask me why and we tried not to make too hard for newcomers to approach the code so we are not using scala c cats shapeless on it or any other library and but don't get me wrong cats is a super awesome library but it it requires you to understand a little bit more about a type system and scala and more advanced topics so if you want to introduce a new haskell a new developer it could be more difficult and actually scala c it's a port of haskell libraries into scala so for concurrency we're using aca so aca it's like a wow it's it's an incredible library so in mantis we have a bunch of stuff going on at the same time we are downloading blocks headers bodies we are receiving messages from the shason rpc api everything is going on at the same time so you need to a way to handle this kind of concurrency right and you have several options in at least in shava virtual machine you can use a mute exist and synchronize blocks which they are fine and you can spawn threads and do all that stuff which is fine but it's more difficult or aca it's an implementation of the actor model and the easiest way i can explain you about this model is i can imagine the actor model as a bunch of little guys that have an inbox and every time you want them to do something you just leave them an envelope and basically when they receive this message they process them one by one and if there is no message they just sit without doing anything so your cpu is fine so every time a message comes they process them and they can call other staff and then they shut down they don't shut down they just sit so that's the way we process everything so if we are handling messages that are coming like a new transaction we post that to the transaction to the actor that receives the transaction it processes it place it in the mempool and then it's it and it's quite cool because it allows you to model very complex concurrency based programs in that way which is really really easy to understand but aca is much more than that you can do clusters you can spawn these actors in different clusters in different machines and they can coordinate actually you can you have this supervisor mode when you can if an actor dies you can respawn it and stuff so it's it's really cool and it's similar to the erlang actor model for crypto we are using bouncy castle so i placed this one because i wanted to show that you can use any jvm library in scala and it works out of the box it just works and for the db we were using level db but we have found that it doesn't work quite well so we changed it to rocks db and i placed this one because rocks db it's running it's written in c right i hate c but there is a wrapper around c that it's running in java and you can call native c functions from that wrapper so it's quite cool you can also call native functions obviously this is a uh this is a little bit tricky because these libraries might have memory leaks so you should try to avoid them but if this is the only option you can do it as well and while we are using sears as a jason parser which is a scala library and we are we are using scala test for test and this is this one is really really amazing so there is a library in haskell which is named quick check and basically it's a property based uh testing framework so let's suppose this case so you want to check that this function receives a natural number and another number and it adds them right so what kind of tests you can do that there is a property that we might say which is if you add two natural numbers plus zero let's suppose the result should be equal or be greater than the first number because that's how you some numbers right so this framework allows you to create tests based on properties and you let the and you need to tell them tell this framework how do you create these these test cases so you say okay i want to test all the the natural numbers and the way you generate a natural number is like this and then this framework start starts running a bunch of tests with different values looking for corner cases and believe me you can find a bunch of a bunch of corner cases using this library so it's quite cool it's it's really impressive and this is an architecture diagram it's simplified i mean we have a bunch more stuff around but i want to make a point about this diagram so first of all we have the ledger which is the one that receives the data and process and the informer that receives the blocks and process the transactions it runs the validations and then it calls the java virtual machine to the ethereum virtual machine to run the the the transactions we have on the bottom layer we have the network which receives the messages and sends them to the sync package which is the one that runs these actors and receives the messages and asks for more data and all that stuff and we have the db which is the package where all the data access is being hold we have the domain where all our domain classes are are stored and we have this json rpc which connects to the ledger to for example if you want to estimate gas we need to execute the transaction so you can then you call to the virtual machine or you can interact with a key store if you want to unlock your personal account so why i'm placing this diagram because i want you to pay attention to this little guy because i'm going to hack it so now you might have a better understanding of what mantis is do you have any doubts about it it's quite straightforward so this is what i'm going to do i'm going to turn mantis into a statistical node so i'm going to replace the db package and i'm going to inject a postgres database a postgres database instead of being a key value storage it's a relational database and you can run some cool queries and i'm going to use another tool to display some nice charts so let's get into it this is the main method of mantis this this might remind you of the java static main whatever and here we're creating our node right and when when we create our node basically we are extending in runtime this is quite cool you can extend interfaces in runtime and you can change the behavior in runtime in runtime and what i'm going to do is to change i'm going to change here the you can see here i have a the blockchain so the blockchain is the thing that accesses the data it's like the interface to work with the data so let's see blockchain first so trait is like an interface it's the the same concept so what can we do with a blockchain we can get a block header by hash or we can get a block body by hash or we can interact with the state or we can get the best block number or we can save information so we can save the block with the receipts and and stuff so we had we used to have a default implementation which receives a bunch of storages these are all level dv or rocks dv storages key value ones but what i'm going to do is to replace this one with a postgres dv one i'm going to use a library named quill but you can use any library you want to it's fine so what i'm doing here is instead of receiving a bunch of storages i'm going to receive a postgres storage and i'm not storing the state or any other things into this database because i just wanted to do a a short demo but you could do it as well so if you look at the postgres storage what we are doing here is to define a bunch of methods that interact with the blockchain for example we are saving a block and while we are saving a block we are creating a transaction a dv transaction and we are saving the block header and the block body and saving the block header basically is inserting it into the database and but to do so we need to transform the block header which for example holds a byte array and something more interesting for us so we can run some queries and here i'm turning it for example the byte arrays into strings and so what i'm doing here as you can see is create a new sorry here so instead of creating this all blockchain implementation which is the key value one i'm using a postgres one and what i'm trying to do is to insert data instead of a key value storage i'm going to insert it into a postgres database and you might ask me why would you want to do that are you stupid or something so no the trick is that what i can do is something like this so hope this works because wi-fi it's a little bit uh it's not working well for me but i have data reloaded just in case so here we have an sql id so i'm connected to the database so i'm select selecting the max number from the block headers table and you might see that they have the 56 something right so what i'm going to do is to start synchronizing from the network and inserting data into this database so with a bunch of like 50 lines of code i was able to change the database layer so it's running it's now it's compiling compiler it's a little bit slow because it does a lot of stuff so the database started the connection and now it's trying to connect to different peers while it's looking for peers so this might not work let's see if not i will show you some graphs don't worry well there are no peers let's see so and you can see here a discovery message so it's trying to look for peers uh it's trying to do handshake well it's not my wi-fi it's bad but but if you believe me this works we can try it later on a starbucks or anything like that what you might get is this so this tool is named pg charts it's like a really cool tool you can connect this tool to a database a postgres database and you can run some cool queries so let me show you the one that i like most the number of blocks mined by address right so i processed the data and if you look at this query i'm not a sql hacker but you can see i have a query here and what we can do is to get something like this so you can see that we are getting this nice pie chart with based on a on different addresses and you can see the top five miners doesn't mean it's the same miner it's the same address it's the same beneficiary in the block here but you can see for example this one got 10 percent of the of the of the mine blocks it's others others so this is the sum of all the miners other queries that you can run if you tweak hand if you hack mantis to the way i did so for example you can see the top transaction senders uh it's just a stupid query and you can run your own and you can create indexes even you can insert this data into a in two tables so you don't need to run on the fly queries but you can see for example this account has sent 969 transactions no no i'm not inserting that the information but you can do that i mean it's a it's a nice challenge i guess so for example this one uh displays the block difficulty evolution for a certain block so you can see growing and what else i have a month uh transaction evolution i don't remember what's this one so based on a date you can see the amount of transactions giving a month uh what else you can see uh the the number of omars evolution by month so more omars were created here which means more miners probably so and this quite cool and you can run your own queries well no blocks so i was expecting that so basically with 50 lines of code and actually there are other projects that have coupled the virtual machine and plugged in a different virtual machine and they have also changed the the consensus layer so they are using a different consensus algorithm for example raft so that's why i think mantis is a good piece of code and that's it