 So I guess we can start. Thank you everybody for being here. Sorry. And coffee. So this talk will be, this seminar will be given by runtime verification and microsoft. You should benefit from microsoft. And we try to give you a broad overview of what we can do with formal verification and then we'll dive a bit into our own techniques and approaches. But to set a mood for the seminar let me start the famous quote by a famous person Saithra who said that program testing can be used to show the presence of bugs but never to show their absence. So what that is telling us is that while testing is very useful we cannot rely entirely on just testing is what we want is to prove correctness of our program. To prove correctness we need formal methods which are mathematically grounded techniques to model and reason about systems. And before we dive deeply into formal methods I'd like to also see another quote by another famous person Tony Ward. Both Tony Ward and Saithra are currently in the world and he said that the job of formal methods is to elucidate the functions of which formal correctness depends. So if somebody tells you that hey, I formally verify or I formally model my system what that really means what that should mean is that they clarify all the assumptions of which the property they claim is important. So this is an overview of our presentation today. We are going to give an overview of formal verification and then we'll dive deeper into our approach based on the case framework. Then we'll show you some examples of how we formally verify smart contracts and we'll use three different approaches that we want to put together under one umbrella. And then we'll come to it. So what is formal verification? At a very high level formal verification can be seen as taking as input a program or a system or a protocol, something that computes, something that evolves from one thing to another and that's something that's called the program P and a property or some properties or specifications which are properties of this program P and it's called the S and what we'd like to know as output is whether the program P indeed satisfies the specification S meaning that no matter how the program is executed or whatever inputs or whatever environment and whatever non-determinism that you might have in that program the program will satisfy S, no matter what or if you cannot prove that then you'd like to have a so called counter example you'd like to have some round or input, some environment in which P violates S and that's very useful because then you can use that to improve the program or the specification. The typical that we as a community know that formal education has other following so the obviously what you really want for formal education to be able to implement the program is correct but that depends a lot on many factors on what property, what you mean by correctness and in general what really can play is that your program doesn't have certain vulnerabilities for deployment and those vulnerabilities are whatever you specify in your specification if you forgot to specify a certain vulnerability then the program may still have that problem in the extreme case you can prove that your program satisfies the specification through for the empty specification and yes it's satisfying yet but it doesn't need much also formal education helps you find very subtle bugs bugs that you haven't thought about when you wrote your program initially because formal education will enforce to go through all the execution, all the behaviors of the program and because of that you will know when you don't tell you when a certain case or a certain case that you forgot about still learns into your thoughts also it helps you it helps you focus testing and auditing effort because if you have a function in your code that you formally verify 19 but you couldn't verify one of them you know that you should look at that and also if you use it routinely, if you use program verification routinely, you have tools and there are tools that you can use on a regular basis then that can help you in your everyday development of your code you start thinking actually differently about your code you think of the code through abstract languages of the properties satisfied and because of that you write metal code in here that's probably one of the benefits of formal verification what I like to do next is to go through the entire spectrum of formal verification or formal analysis not even formal analysis, tools and technologies, techniques that you can method for your program and I would like to use a metaphor, this dial metaphor to present them and I'm already upfront that it's going to be very rough approximation of the reality an entire universe of research and work we cannot minimize all that in a couple of slides but I think it would give you an idea that you can go as deep as you want with formal analysis and formal verification but at the same time you can stay very light and you have to find your right balance where you think you should stay so at the minimum so if you turn the dial to the left you get very cheap methods but at the same time very little assurance as you go to the right you get more expensive methods but also you get more and more assurance so the minimum you can do and you should do is testing and you should have lots of tests for your program, ideally giving you for smart contracts even complete coverage in reality, in real life in real software it is very rarely achievable but in smart contracts there is not too hard because they are smaller but keep in mind what Daistra said no matter how heavy the test you are still not going to get any formal guarantees any assurance of correctness you have more and more confidence that your point is correct but you don't know if it is correct so that's why you say testing is the minimum once you have lots of tests then you can move further and do what I call here runtime monitoring that comes up with lots of names some people call it runtime verification others call it runtime assurance there is lots of different names so what runtime monitoring is really what I mean by runtime monitoring for this slide is that you can run the program but now observe its execution don't look just at the speed without relationship look at the sequence or the behavior of the program and you can monitor the program as it runs and if you see any violations then you can report errors an example could be division by 0 right so on EVM division by 0 is fine but you may not want to do but then you can monitor for division by 0 or other properties could be that you want to revert before the end of a function if you have an overflow during the function so you have to look at the sequence of steps and you can monitor them because it's pretty easy to achieve once you have a running system of course you have to instrument your program somehow to observe what's going on but that's much lighter than the other methods later on however like testing runtime monitoring does not ensure correctness unless you combine it with more complicated backups further you can do static analysis static analysis there is a large variety of static analysis techniques out there and I strongly encourage you to use any static analysis that you can put your hands on because they give you immediate feedback about your code that you can use at the point of your code so static analyzers typically they scan your code and they look for known patterns code or bad coding practices not right and the advantage of static analysis is that they are typically very fast you get immediate feedback they show you where, they color your code they exactly where problems by being your code the problem with static analysis could be that they make certain errors that they are not trained for they don't implement or they may give you false because they have to make tradeoffs you can never prove a program correctly complete automatically so you have to make some tradeoffs and one of the tradeoffs that static analysis tools make is to give you potential errors and then when you investigate and deeper realize the truth it's actually not a problem because I know why static analysis tool can analyze the code scan the code but it don't go deep into the semantics of the programming language if you want to go deeper actually into the semantics of the programming language you do what we call symbolic execution and what symbolic execution is, you're still running the program but you run the program with inputs which are not known if your function has two inputs a and b you run the program with inputs a and b without knowing exactly what inputs are you look at the body part of this a and b and as you execute you track of all the logical constraints and what's going on in the running program and the advantage of symbolic execution is that you cover a larger state space or space of behaviors of the program essentially in theory all possible behaviors but the problem now starts being that it doesn't scale at some moment it will be very slow or at some moment you have to make compromises whether there are certain conditions or not so if you want to make it automatic then you have to again set some trade-offs and one of the trade-offs that's happening in the combination of symbolic execution is what we call bounding model checking where you explore all the program behaviors up to a certain bound let's say up to two iterations of each loop up to two recursive poles for each function as most once you put such limits then you can explore explicitly the state space but the problem is that you enforce yourself a finite smaller state space which means that you may lose errors if you want to be absolutely sure that you don't lose any behaviors then you have to go into formal verification and here I'm calling it state-of-the-art verification because this is what most of the people who do formal verification understand by verification and probably most of the people in this room which is the fact that you have a programming language and that programming language provides some specification language, another language on top of the existing language that allows you to specify properties of interest and then your tool, your formal verification tool analyzes all the behaviors of the program against those specified properties and with some help eventually you may be able to prove maybe some additional hints, additional lemma you may be able to prove the properties correct so this is the conventional program verification and I say conventional because what you're going to see next beyond that so the limitation actually for the traditional verification of programs is that you need to be stuck with one particular language or version of your programming language and more important is one particular version of your specification language you think that some language constructs are really important to specify or some specification construct like always eventually are really important so you add them to your language for properties but then later on if you need something else you're out of luck the tool doesn't support that if you want you have to start going into deeper reasoning and deeper analysis of your system then you go to formal modeling and validation of your model meaning that you take your program and you model it in some other system a theorem group or interactive verification environment where you can now reason completely mathematically about the program forgetting any programming language you don't care about the programming language anymore you don't care about the condition system in your more abstract environment and then you can reason about detail important properties there without taking the code into consideration and that's really useful because it helps you find high level errors potentially in your code that may have been hidden in the actual code so when you are more abstract you can see more abstract problems for example you can see design issues of your business logic for many customers to work with we take them to the various steps and at some point we get to this point and they say oh actually we really want that so once you understand the value of modeling then you really like to go that path but now you have a gap between that model and the actual code that needs to be filled and this is what takes us to the final stage which is verification of proving your code correct using frameworks alright so you take a programming language you have a formal mathematical model of what you meant to implement your code and I have to prove that these two are that the programming needs to implement the formal model and to do that you need another language which can set the code of this and do all the rules and you may have heard the system like Koch or Isabel and the system that will escape falls into the same category right so as we go from left to right cheaper less assurance expensive but more assurance and ideally we would like to do all this automatically but unfortunately it is not possible there is only that much that we can automate and traditionally we can automate everything up to symbolic execution and even some part of symbolic execution but when you try to go beyond that a human expert tends to be needed unless you stick yourself to very simple properties so there is a general picture for program analysis we cannot have anything free for automatic but nevertheless you should use everything that is automatable in our contracts if you are practicing in high assurance it should be that you should use all these tools so what we intend to do today is to show you how we approach this spectrum of formal analysis tools from angle and our angle is the k-frame one the k-frame one is great in watercolor expressiveness so with putting the right hands you can do almost everything you want to do but on the other hand many people find it heavy to use because of the heavy notation and the lack of automation so our challenge here is how can we automate as much as possible we can do and here we look at two types of automation automation that we know about things that we have done with k and we know that those can be done better and faster and that's what we try to do with the firefly tool that we have ever talked about so basically firefly tries to automate everything we know we can automate with k for smart contracts and then there is a lot of automation going on in the programification community developed by decades of research by amazing researchers I should end here and Microsoft research actually is a leader in that field and what we like to do is to take advantage of all this automation incorporated and we want to do that by connecting the k-frame one with very sort of programification tool for solidity contracts developed by Microsoft and I will go into more detail before we dive into our own approach I would like to tell you that there is a suite of tools with a high degree of automation for some analysis of smart contracts that you should definitely look at and try which are not developed by us and many of them have talks actually in this depth work like me, sex and me too and I think we are not here to talk about 10 up and out and we are trying to identify and correct by change security what is amazing and amazing tools that we should definitely try to use them especially when they were out of the box and now I am going to go into our approach based on the k-frame work so why do we like k-frame work it mainly because it supports all languages it is designed to work with all programming languages and in particular that means with all versions of a given particular language we don't have to touch anything with k-frame work to switch from one version of a language to another or from one language to another completely and this is the philosophy or division underlying the k-frame work that you should define a programming language once and for all that includes syntax, semantics, everything and everything else all the tools that you need for that language should be derived automatically or semi-automatically from that one definition of a language that includes not only parsers and integrators but also symbolic and efficient engines module structures, deductive program verifiers and everything and all of these black boxes are separate tools but they are all completely language agnostic or put it differently when you implement a tool a formal analysis tool or analysis tool for a language you typically pick the language and you pick the tool which is hardware to go out to that language and to that particular type of analysis so you have languages and then you have tools and many tools pick one language and one with a Java or a Java for example so what the k-frame work proposes is actually to not do that it proposes to encapsulate all the tools and to make them all parametric in a programming language and the programming language then can be formalized and passed as input and when you take a programming language you don't need a translation of the programming language in some other intermediate language so just exactly that programming language so that approach we started working on the k-frame work more than 20 years ago and only 2 years ago we switched to a blockchain and it was really easy to do because all we had to do was to plug and play blockchain language everything else takes exactly the same and for the Ethereum blockchain we actually looked at the Ethereum virtual machine we didn't want to verify program to the security level at the beginning we wanted to start with a bytecode to eliminate potential errors in the compilers from the trust base and we had never actually ever and other students I know if they should so your students taking classes and they said oh actually we can go from a semantics to a k-frame and that passed project into a research project and then into an actual product so there is a complete formal semantics of the EVM in game open source everything we do is open source and it is complete in the sense that we can run any EVM program with it like a client and actually we generate from the semantics number one of the blue box it was an interpreter with that interpreter we generate an actual client, EVM client and it is pretty efficient or efficient enough that you can use it for actual real work in particular it is a bit faster than Ethereum JS which is the javascript emulator of EVM in Netrafo and it is only a few times slower than the C++ implementation but that really makes it usable as an implementation but remember just a mathematical model the exact same mathematical model to use for your verification without any gap between the implementation and the verification also several people in the community use the EVM for the EVM semantics as the canonical specification of EVM and we have also a gelopaper.org which was completely automatically generated from the EVM semantics and we generate it, we update it each time and modify the semantics and that is human readable you can go and read it in the canonical specification of the EVM alright and the Firefly tool that everyone will talk about is as I mentioned it tries to automate what we already do in K but in a way that other people can also use for this automation not only for experts in K or for all methods so it is an instance of a K-frame with EVM semantics and with lots of automation in various boxes and all under the hood so that for new users a Firefly is just a push button thingy just a push button and you don't even need to see what it comes to be so Firefly tries to automate whatever is there in the K-frame already while the combination with Verisold the dimension here is to build upon the legacy of successful tools developed by Microsoft Research and many of you may have already C3, BlueT, Coral so all these amazing technologies that are being incorporated in Verisold the problem is that there is a gap between what Verisold does analysis of the solidity level and what we do with K-frame which analysis of the bipolar level using an actual form of semantics so what we like to do now is to connect the two and to regard Verisold as a user friendly lightweight front end that generates all the verification artifacts that you need down there and to reconstruct the proof of the bipolar level and this way you achieve everything you want to achieve your formation a very small traspace because now we only get the formal semantics of the EDM and at the same time and with that I'm going to to learn the actual more clean than the tools and we'll start with Ushabendu give us a little bit of Verisold part of the second I was quite honored I work at Microsoft and I see some familiar faces thanks to all the shout outs to the research from Microsoft I don't claim credit for any of them but I've been in the engine room while these technologies like Z3, Boogie and Koran can you hear me? I've been developed so I've been fortunate enough to sort of be part of some of these technologies or be in the engine room while these have been developed so the goal was so we had a team in Microsoft called Azure Blockchain and they started taking dependence on smart contracts so it was naturally quite interesting to us to see if we could help them so that's how the journey got started and then we'll be talking with Grigori about how to combine these two techniques and what's the landscape of tools and so on so what is Verisold so the way I think of it is a formal specification for Solidity Smart Contracts and it's a research project in my mind so there's the GitHub page there's lots of collaborators from Microsoft from Austin and a whole set of collaborators from other places in Argentina and so on so what was the motivation for Verisold so to me the motivation was that we need to empower the developers who are writing smart contracts for a living because as Grigori said it's a very crucial important set of applications that you build with lots of security goals and you would like to reason about them not just for audit but also while you're developing so you have some idea and this initially started with the focus on the Azure Blockchain smart contracts that they were building for the infrastructure and all the samples so we had some that was the starting point and also as Grigori said one of the goals is to expose a lot of research that is sort of matured in the research community bring it up to developers with fairly well defined expectations so not all of them are the most automated tools in the market but they have sort of an expectation that exactly what is expected of them so these three many of the tools use C3 for symbolic reasoning Boogie is a state-of-the-art intermediate verification language instead of K it's a little different flavor like it's a verification language that you can map different languages and then Koran is a push-button inter-procedural checker so it's completely push-button scalable driven by the assertions that you want to prove not by the number of parts in the program and there's a lightweight but very predictable inference of annotations when you don't have all that and also a kind of push-down a lot on the recent and as I said these are not the most optimized for the purpose but they have a sort of well defined expectations I've been fortunate to use some of these tools in production in Windows IE and found lots of bugs people used it in the static driver verifier the ship with the driver verifier so these tools have a history of being tested in production people are taking dependencies on these tools so I will go ahead and give a few examples as I go along we recently released a .NET global tool so you can try to install it at some point there could be bugs and I have a set of examples that I'm going to walk through today just to give a fever of what the tool looks like so we'll start with the DAO example it's most famous and notorious so so this is the DAO so this is the notorious DAO example and we all know about the re-enhancing bug but let's say you're just rewriting the DAO example I mean nothing about re-enhancing so what is it that you were looking for let's say a language that doesn't have re-enhancing what is it that you should be thinking about so here is one example that says that there is a withdrawal method that is responsible for withdrawing funds from the balances of the DAO but I guess one of the ingredients that you want to preserve is that no matter what the cost is because what you want to say is that if I call withdrawal then the sender should not withdraw more than they have been trying to that seems like a natural specification it should be equal ideally but I'll show you examples that actually can equality is too strong because an attacker can actually go and donate something but this says that roughly that anybody should not get more money than they want so you write the specification so it uses some old contracts libraries that is support so it uses mostly what it is and that ensures that all are the new things in there and so I have the bug in there I'm 27 the command line tool right now and it tries a couple of things it tries to get a proof first and if it doesn't find a proof then it tries to look for counter examples so in this case it sort of gives you a couple of examples that say you have to call the constructor with some arguments then you donate because without donating you cannot withdraw so you donate to withdraw all the withdraw methods and there is an assertion that we wrote down case and for unfortunately only for windows at this point we have a viewer that can help you debug to the tape trace I'm not going to show it but you can actually step to the trace and see values so you can see values printed along the trace just to get to the better sense of what's going on so that's the experience and so I'll go ahead and do the fix which is you update so basically many people know we basically update the credit of the sender before we before we do the so in this case it actually found a proof meaning that this DAW will be able to satisfy the property that we wrote down and the interesting thing is how do you model the adversary we have a motion called combat where the adversary can call it to any of your functions so with that model we have a proof so this is sort of the experience so I'll show you the CRC-20 so this is sort of I guess many people are familiar with the CRC-20 program this is the one from open zeppelin with sort of modified hardware and the thing that we're trying to reason about here but again the question is what do you do with the token once you design the token what do you look for so one nice specification is that the total supply that's in the token should be the same with some of all the balances in the of all the addresses and that will sort of get a good mental model of what an internal ingredient is and if you show this to a customer they might say okay just go through the process if you can show this maybe on the trust report then so the thing I'm going to show is up here sorry so this is the method called transfer and interestingly open zeppelin doesn't check for when you send an amount the sender has to have that much amount and the reason it does it because it relies on the under flow semantics of the safe match so the safe match sort of makes it safe and you won't have enough money the safe match will fail and then you'll abort the execution so how do you find that without knowing the internals of safe match so the invariant that I wrote out if you change the safe match to a regular subtraction then so again it refuses to find a proof and it calls you to show the counter example where you're trying to drain an amount one from somebody who did not have any French syndrome and that and this is interesting because it requires you to reason about underline semantics if you're not careful and don't use modular arithmetic you might see a proof where it does not exist so this shows basically the various levels of abstraction you can reason in the kind of categories you can get the modular arithmetic is not a very scalable solution so if you try to use it for other line programs there's lots of different values so you have to choose but you sort of set up an expectation of where you want to invest okay I just want to show one more example which is an example from this tool called azure blockchain that was one of the products that micro rock shipped about a year back and it had a very nice way to decompose a workflow policy they realized many smart some kind of workflows they're getting very popular in the enterprise scenario so they provided a nice language a JSON based language for clear semantics of what a workflow looks like and then the question is does the solidity program implement the workflow you make it full screen so people can see right so here is the specification language with a bunch of state machines and actions and access control and there is the solidity smart contract and the question is whether the smart contract defines the state machine so in this case it turns out there was a discrepancy it turns out it was a specification but nevertheless there was something that was different in the two versions and so this requires sort of a very deep analysis of the seven steps to go through this table so shallow execution will not be able to get to this part so this is, I'm going to show this example so in this case it did not find the but it takes a little while to actually find the counter example but still it's reasonable it's a few seconds but it just shows the complexity of the search space so since this is assertion guided it can stay a little better but of course it hasn't cleared you can see you have an issue 1, 2, 3, 4, 5, 6, 7 transactions to reach this so it's interesting that I don't miss this okay so getting back so it shows the flavor of verification so for this one for example if you just want to do a push button verification you have access to this corral if you want to do proofs you can write ingredients and interact with the tool but the nice thing we have observed is that some kind of advice in that the open environment of adversity makes it easier for the verification to prove that is actually a nice so that in perspective very some looks like this so we have a solitude program with specifications for specifications it includes this cold contract library and I call it cold contract because it's sort of the syntax you get the resolution from solidity you can get syntax errors and the specification directly from the compiler and then we're working on translating the subset of solidity to boogie that is very recently high level so we're not going to try to do deep verification about how data is lay out by the bytecode compiler that's something that it does very well that's something very high very hard to automate and it also sort of tries to prove what the compiler is doing so three of the reasoning is much more abstract level which I think is good for business projects that we see in enterprises and then the host of proofs that we rely on which are being extended by us, by the community as the feasible and so the outcome has three outcomes that's the proof or the trace or in the middle or something like that not be able to verify it fully or find a trace but I have this guarantee that any transaction of depth k any set of transactions if you issue up to k transactions then I'm going to give you some assurance so this gives you a reasonable guarantee to maybe stop testing and focus on verification at this point so the way it works is it takes a contract to construct a bunch of arguments and then creates this driver or the harness that sort of in an infinite loop calls any of the veterans in any other arbitrary argument so you can imagine if you can verify something of this program you're verified it for all executions so the capability of this point is the old contract library and we're consciously taking the decision to start with a known language so developers can understand them without needing a whole bunch of sophisticated semantics of languages our initial customer with the folks in Azure Blockchain who are systems developers but they sort of understood the specifications at least to some extent and so we have a scalable push button validation and highly automated truth season bookie and by highly automated I don't mean completely automated you might have to support some extra facts on the side so that's sort of the research problem is how do you know about it and so we did talk to us on the study on how we check workflow policies and also check some critical infrastructure that we're using smart contracts in some of the products they have, services they have and this is sort of the governance protocols that underlie these things like Ethereum and Azure or Azure Blockchain service so basically they have implemented some of the governance in smart contracts and so it's very crucial because any bugs there compromise the entire infrastructure and so you have to ask questions like can I get into a deadlock can I remove the last administrator so that I would move from that issue and usually the work happened as it happens in K, you consult with a designer and you come to the specification to a boundary checking and then help them with the variants and repeat them while the code is being developed so we were very fortunate that we could actually do this and the program was being developed and tested on bugs, on bugs so that was a very nice experience and I think that would be nice to scale out such kind of experience so that's sort of the goal of the very sort who is to take it from research prototype where we have a few customers to help more people try to get a sense of the technology and enable the community either directly or like collaboration with folks early so I mostly glossed over the specifications that we have been adding to the code contract language so we need to check back in the two months for the section, these are sort of well known facts that are necessary for doing the program verification these will be always sufficient so pre-post conditions contract invades loop invades will mean various things created people so you have to read it there's a spectrum of what it can mean and so it's a library and then you have stops for loop conditions so for example you can follow very solid contract invades requires insurers invades and I know there's work in the Soliti compiler on supporting some of these specifications primitive waiting and watching to see once they come up we can sort of leverage some of that and so it's not tied to this standard in any way this is just a way to expose the facilities to developers and then again once you've pulled the search language you need to read about the whole value and mapping so the example I showed in PRC that you need to say that total choice total supply is equal to the sum of all the specifications very hard to do in a pool because you cannot even write such a specification because mappings don't allow iterating over all the indices and so on so there are some things that really get a lot of value just specifying it and even doing boundary checking and then we're working on other extensions part of the top that we have with RU is what kind of stuff so they have a lot of experience in those contracts specifications so one of the good high level so next three to six months we have a bunch of budget and then coverage of solidity so the reason I say high level reasoning I don't think we are going to go the parts of solidity that require low level reasoning that is known not to scale the higher and the SMT level so if you want automation then you have to sort of carve out sort of high level language what I've seen in enterprise people don't really care about so I shouldn't generalize they are happy to live with a fragment that they can be more sure of rather than getting optimized gas so there is a hope that a lot of the reasons can be reasoned to high level but there is more extension to make it easier and then going forward the discussion we are having is what kind of specifications can be added beyond assertions, temporal logic and a lot of research in the community also about how to improve in many different how to get users more control of how to be in many different and through the grand challenge how to push these tools all the way down to high level I think that's a very interesting problem so with that I'll conclude I know what it is I have a question I'm sorry, did what you just described use K as the back end? No, that's the future plan what's the level at the very solid level would like to convince K that it actually holds at the bytecode level that is the long term okay I'll talk about how to do this as well are you considering to have a a Linux version there is both a Linux and a Mac version of this okay, you can download it it's not a visualizer the visualizer now somebody else is going to visualize the rest of the tools there's a new category that I'm just going to do I'm going to work for more verification of consensus algorithms for what? consensus algorithms yes, I'm going to talk a bit more about that in the end let's get started so I, my name is Adrian, I'm going to talk about verification of the end bytecode we're more focused on the case study of in 2.0 we've recently made part the title contains verification but I'm not talking about that because it was already we talked about verification and we'll talk about more about the bytecode so those of you don't know about bytecode will learn something about bytecode there's no other way but bytecode will find this interesting so let me give a quick review of our bytecode verification tool so we have the ebm bytecode verification tool that builds on top k-frame model and kbm that we described and this tool has two inputs program bytecode and the specification and it checks that the program satisfies the specification or not this is very similar to very so at the high level the main difference is that it works at the bytecode level why the bytecode level is better I will talk about in a bit later but let me give you a quick example of how this tool works so this is the program very simple program that similar like ES and 20 transport types of program we have the receiver address from and to and value and update the balances of the from and to using the self-made so it any underflow or overflow isn't important so given this program you first compile down into the bytecode you put in the bytecode into our tool and then next you have to write down a specification which specify what the behavior that this program should have so here we are describing the behavior of the program in the success case which means that if the balance is enough I mean standard balance enough and there's no flow happens then the balances will be updated as a proper way I'm not going to do details about how this specification details but the thing is you have to write down the specification at the bytecode level in this way and some people say that specification is longer than program so yeah and this is why what Gregor mentioned it requires a simple network to write down the specification and also working at this level our future goal is to let you just write down the specification as an annotation language like insurers and inquires what we can post condition saying that okay this transfer function if the two preconditions satisfied like balance is enough and no flow happens then the post condition means that whenever this transfer returns safely I mean it comes through whatever or not it balances should be updated as expected so if you write down this specification at a high level our goal is automatically generate this bytecode level specification so you don't need to work on this level but that's the future work we are still working on that but we're hoping that provide this feature so using our this tool we verified some high profile smart contract so very recently we verified if 2.0 departs contract that I guess you will learn more about this in the later this if 2.0 talk but the main thing is that this contract is really important contract when you want to be a validator of if 2.0 because you want you need to project your subinterf to be validator sending your device into this contract if 1.x and then this contract is 1.8 function so you can't depart something but you cannot withdraw but the depart whatever made here will be claimed in the if 2.0 network so it's really important first make sure that you cannot withdraw the second whatever you depart here will be available in the if 2.0 network there's really important contract that needs to be correct so we recently verified this and this is our case study I will show you what we found while they found this contract and here we have other contract like notice is safe I think oh yeah it's a multi-sketched wallet and we also I mean the maker now we see a report was verified using our better vision tool that was by Denpov guy so we are very happy that our tool is used by another to verify their system so we are hoping to more people using our verifier to verify their system and then that's why we are working really hard to push ourselves to much more easier to use our tool and that's the more or less the Firefly tool to meet the final top we have some other bunch of the smart contract and now I'm going to explain why we are really obsessed in this viticolor verification the first reason is vitico is what really runs on the blockchain not the source code and then compilers, believe it or not unbox I'm not talking about the service compiler I mean usually compiler even the C compiler like GCC or LLVM which is really mature and develop long period of time as box new box reported even now today and some box actually hidden in for 20 years simply some guys are lucky to hit triggered the box hidden in 20 years and then report it so compiler as box actually more than you expected even if you verify your source code is correct if compiler as a as a problem then that doesn't mean that you are using verisor verifying just assume that compiler is correct if not in the case then your verification is does not actually guarantee what you want so this is really important that's why we are actually working at the vitico level and indeed we found some issues into the past contract while we verify vitico level that's I'm going to talk about so before explaining what box we found let me give a very quick background the how to read vitico how to read vitico or race so okay once you have a vital race like then 2 words 32 33 how that represented at the idiom level the single byte it's not a single byte it represents as 2 words essentially in each line a 32 byte line the very first 32 bytes you notice what the size of this byte is 0 0 0 1 and then you have an actual byte the data 1 bytes and then followed by 0 bytes so 31 0s if you have a byte of 2 then again size 2 there and then 2 bytes here and then 0 bytes and so on that's of course if you have a 32 bytes it's already online so you don't have any 0 bytes there that's how byte arrays are represented in the idiom level and if you have a 2 port of byte arrays like arguments or if you return something then they store as a 2 port of the bytes and if you have for example 2 port of 2 byte array then simply put these guys, put these 2 and then you put on top of these simple offset these offset points where the each element starts so since the offset is simply fixed so if you want to find some first element that's element, you find that you go to the offset find it by where start and you read the size and then from here read the size that's how you decode or encode this byte array that's you only to know the first bogey function the deposit function so this is one of the functions the deposit function called getDepositCount function which simply put on some sort of, oh by the way this is a viper those of you who are not familiar with this viper is similar to Soliti but simply Python each syntax so you can read so in return this deposit count this storage value but in the real Indian form so it's simply a reversed byte so assume that these two real Indians are simply correct then those of you who think that any problem with this seems to be too light how could it be wrong assume that this is correct assume that these two real Indians are correct can you think about something is wrong with this code yes so when you swap it around then you cut off to get the byte 8 you might cut off the wrong data that's very good yes but that's somehow handled by a very worthy storage in India yeah but yeah, very good yes, that's right it's, yeah actually I tortured you because you cannot get this because probably happens in the compiler byte compiler so this is the return the value when you call this function if this contract compiled by viper version 1.0 beta 11 which is the latest version of the time can you see any problem with this if you follow me the previous yeah yeah it's 20 years right this is all zero but here see something is non-zero there what happened what? what really happened is that so compiler first of all compiler what compiler does is that they need to prepare this beta in the local memory and then they return they first write down offset this is correct and write down this size and then they copy this the byte then it needs to be put all the zero like this is 8 bytes so 24 bytes of zero there but somehow compiler is involved they only put the zero on 8 bytes then leave whatever the 16 bytes there and the problem is these 16 bytes of the local memory garbage came from this function side effect so there is just garbage there which is non-zero so this is the example of the compiler book and then you cannot find this unless you work on the byte to the level and then I mean Ethereum foundation write down this code there is good job and no problem but just deploy this without verification and then you have these non-incorrect zero bytes and then somehow another contract they assume they expect that all zero may be exploited by some others this is really big that's why we really think a bit the byte puller verification is important I have another file which is more interesting but I even know but I even know let me group very quick so this is the main function, depart the depart function takes now three arguments or the byte array check that the length is whatever expected now this is the tuple of three elements like offset and the three arguments now what if this whole data input is not there for example by mistake or maliciously attractive for example like there is no actual bytes there is six words so this is a correct one 12 words what if you provide only six words you might think that oh there is no bytes you have the data so data length is zero but now we have a length check so it may devote but what really happens is this is a count example simply six bytes offset 96, 128, 160 32, 96 and from this size then no bytes this is all and then even this, the depart function simply goes through doesn't even work let me explain why first you try to read the pop key so the first argument you first go to the pop set size is 48 so you read 48 bytes so basically decode consider these garbage as a pop key move on next you go to the second option you go to this and its size is 32 and you simply read this guys which is actually another size the other garbage you read even more it points to this you read the size which is 96 and try to read beyond the core data which is 96 and what really happens is the yellow paper in the laboratory machine says that if you want to read something beyond core data you read that as if integrate zero are there already means that simply when you read this out of bounds says simply return all zero this is what happens even this in the core data the decoding process simply read three arguments for the garbage which for example signals as all zero you cannot claim later if you want to change whenever you have the positive so if you make any mistake somehow flip the bits in your core data then it happens this shouldn't happen but previously simply then go through and deposit this raw information and then you simply lost your so again the story is it's really important to work at the vital level if you want to full assurance of your contribution and then so far we are using our cable tool which requires some export and then some really high effort so far actually the large group like the foundation or some who can afford this large effort can use our tool to verify that but we are really hoping that even a small startup can use our tool to a full guarantee of their smart project and that's why we are working on this next to the vital any questions or any questions concerning the first issue so the non-zero bytes did they affect the correctness of that of that cost specifically because it sounds like did they affect the correctness of that cost first of all there is not ABI standard return value ABI standard specifically says that if you have a non-aligned return value you need to have full zero bytes and then that ABI is simply protocol whoever called this that contract would assume that there are going to be other zero bytes if somehow they exploit that fact like just copy those into another local memory assuming that there is always a zero may have different behavior but this is promise right you need to return something you need to follow some rules so yeah that's the problem thanks Kijun Kijun had some more interesting slides after that which I'm bummed we didn't get to but you should ask them about it afterwards okay so I'm here to talk about this is our attempt to take KBM which is kind of hard to use and wrap it up into an easy to use tool that just works in the background and does everything in an automated fashion as opposed to now where you basically need our team to use the K tool at all so this is the overview of my talk first I'm just going to talk briefly about the motivation of why we want something like this showing you our roadmap it's stretched out over about two years and then I'm going to talk about the individual tools that are on our roadmap which we have prototypes for all of the tools except this last one but the roadmap is still spread out over some time so you can do testing and internal stuff like that as a motivation we have tools like Verasol that are easy to use they're high level they're at the solidly sourced code level and developers can just basically use them out of the box but that problem as Dijun pointed out you don't get the guarantees that you would want at the EVM bytecode level but then we have KVM which is a low level EVM bytecode verification tool which really takes an expert to use and somehow there's not something bridging the gap between these two so that's what we're trying to trying to do so the ideal scenario for Firefly is that it's going to be a drop in testing client replacement for whatever you use as your testing framework right now we're kind of tardy in trouble so basically just replace ganache with Firefly and then you're off to the races and just a couple more bullet points one goal no user intervention required it should be that you just run the tool instead of your normal testing tool and it does the extra analysis and reports the results back to you another goal easy for us to roll out more features and reporting over time and then another goal is easy to dispatch large jobs in parallel to a service somewhere so these are kind of some long term goals of the project specifically tied to any of the actual tools we're developing but we'll develop the tools with this in mind here's the long term roadmap you can see it goes out to December 2020 we'll talk about these ones which are part of Firefly Suite a little bit more but then they're on the right hand side beyond Firefly where we get back into where you need a formal methods expert present again and we are doing some kind of prototyping of these tools out here so specification generation instead of having to write down a specification and verify your contract that you've generated for you proof of your generation is basically the highest level of assurance that you can get with anything formal methods usually usually not done except in like the most safety critical sorts of systems but we're toying with the idea of moving in that direction this is our more specific Firefly roadmap we have KS setups that we have like a front end which basically takes the K specification and translates it into multiple back ends the LLVM back end is our fast interpreters whenever we talk about the performance of K compared to other things we're talking about the LLVM generated interpreter for the K specification and then whenever we talk about symbolic execution or proving we're talking about the Haskell back end which is the one that actually does the proof steps needed to do symbolic execution I'm going to talk about each of these tools in a little more detail but we're starting off with just a test runner and just basically replace Ethereum.js VM with KVM and then everything looks the same as normal from the perspective of running truffle tests test case coverage reporting is next that's what we're working on right now is kind of integrating that coverage data back into truffle runtime monitoring which is like bounded model checking v1 and then assertion violation checker which is where we're going to actually integrate with the verisal tool to get kind of an assist from them bounded model checker v2 which is where we're going to do it symbolically and exhaustively and as opposed to here which is going to be concrete and only on the test cases that your actual test exercise and then finally at the end is test case generation that's the one tool we don't have a prototype for yet but we'll get there I'll explain why that's important or why it's nice and maybe it's not that's probably important okay so the test runner basically what we're doing is we're taking KVM or making it into a full client full client functionality so you can just use it anywhere where you could use a full client for doing testing so we're just going to support the web3.js on RPC and we already have it to where you can basically just run truffle test but where you're running the KVM server in the background instead of running the firefly server in the background instead of running ganache and it just works benefit of that is you get kind of more confidence in the test results because KVM is kind of regarded as a canonical spec of the EVM as one of the canonical specs so I mean some people like that you can take it or leave and then performance is comparable to the Ethereum JSVM but we actually have several performance enhancements coming out that are going to make us faster than Ethereum JSVM and then we have just kind of special purpose firefly under our web3.js on RPC command we're getting back reporting data from KVM about coverage analysis like the runtime monitoring stuff like that so this is just like the bare functionality we need this to exist right now but then everything is going to build on the fact that now this is just a drop in replacement you can switch over to using it not really notice any change and then we start rolling out features so the first feature we roll out is just test coverage we're specifically going to render off-code coverage so here's an example report that the road type spit out and you can see there's kind of this place after this jump by that's not tested by anything else everything else is tested by 73 different tests but these are tested by zero tests that's a whole in the coverage data and so as a developer if you were to see this report what you'd want to do is you'd want to say I promise to use a source map back for Solidity and write a test in your whatever testing partners you're using Solidity tests or not FireMuzer linking on trouble tests I swear I know the word trouble so write a test that targets these lines and then you add that to your test set and then the coverage report would come back green so every off-code at least is covered there's a lot of literature about whether the off-code coverage or branch coverage or path coverage or something is the correct thing to do but at least this is something right by the way there are tools that will do coverage already so if you look at browning that's an interesting tool that will do coverage already so common pitfalls in tests are you might not test something like the default or fallback function right you might only write test for the positive cases of your contract where things are going smoothly so you don't write test for the revert cases you might not write test that actually exercise the underflow overflow and then you might not even test the ABI decoding because at the Solidity level the ABI decoding is not apparent or obvious at all because the compiler takes care of that for you but as Dejun pointed out if you assume that the compiler is doing it correctly you could you know lose all the E1 funds when you're trying to move over to E2 okay so then the next tool on the roadmap is a runtime monitor so basically what the runtime monitor is is we instrument the semantics with specific events which basically can fire on any particular data from any particular point of execution during running the events that I have here are overflow and revert in this example so overflow we just detect are we doing an addition and the addition overflows or are we doing a subtraction and the subtraction underflows and revert is just that the revert opcode fires and then maybe like a little state machine dish thing here saying you know if we're doing execution and neither of these events are firing we're just in this normal mode if execution ends we go to this no violation found mode but if an overflow event fires then we know that we have to revert the end without reverting and it's a violation but if we end with revert right here then there's no violation because the overflow was followed by a revert so these properties are temporal in nature so it's not enough to just say there are no overflows because it's pretty common that you will have an overflow during execution but you have to instead say every overflow is eventually followed by a revert before the end of the contract so we're just going to build these events directly into the semantics and the users can supply more if you like but then the semantics has to be re-compiled but the properties themselves to check like this formula right here can be specified on the command line at runtime so we'll provide a preset bag of properties users can write their own properties and then basically we're just going to run the user's normal test set so you still just run trouble test but it runs it with this instrumented mode where it will check, you know, are these properties violated on these test runs and then if it finds a violation of one of these properties on this test run the user can inspect that violation and write an actual truffle test that would have caught that violation anyway so that you don't have to rerun this tool to get the same guarantee so you can kind of add tests that target these properties to your own test set so that's about the model checking B1 a search and violation checker from our roadmap is the where we would get into looking at the pre and post conditions that Dejunas talking about where that varisol will look at and so varisol checks this property at the Solidity level and there's lots of tools that do this right there's like the Solidity SMT checker there's Varex there's lots of tools that will do this we're going to do it by code level instead of the Solidity level for the higher guarantees and basically because of the way EVM is structured the way that Solidity compels EVM all we have to ask is, is it possible to ever reach the invalid off-code because the assertions the post conditions at the end of the functions are translated to the invalid off-code and if you don't find that it's possible then likely the user-stated pre-post conditions hold but if you do then we can provide a counter-example and once again that counter-example will turn into an actual trouble test so that it's quicker to catch next time without having to re-run this tool so then we are planning on getting an assist from Varisol so Varisol will first check it at the Solidity level and it's much quicker to run Varisol so they'll check it at the Solidity level if it finds a violation then you can just work with that directly but then Varisol gives you the green check mark then we run EVM and double check it at the white-foot level and that'll guard against bugs in compiler and bugs in Varisol so that's kind of the plan for this assertion violation checker okay, next is the bounded model checker which is basically the symbolic version of this runtime monitor so this runtime monitor only exercises checks for violations in the runs of your tests which are run on specific concrete inputs but you might want to know is it possible there's violations in any runs of a program and so to do that we basically will just run the same test but now on symbolic inputs instead and then run the bounded model checker out to some specific gap and see if we can find a violation there so it could be this example I have here is, you know, it could be that the test only exercises cases where X is less than 3 in which case you go directly from this exact state that no violation state and no overflow ever happens but when X is greater than or equal to 3 you get this overflow and then you get this end without a revert which is a violation of this property and so this wouldn't be caught by the runtime monitor because it only exercises the cases that are in your test set but then here we would catch it and provide a counter example saying oh, try X is 7 or something like that and then you would add that back to your test set and you wouldn't have to rerun this tool the next time you would just be part of your test set now sorry, I'm going so fast through this the other time finally the last tool that we're thinking of this is for each of these I've said, you know so once you find the violation add a test back to your test set that would catch it without having to rerun the tool and that's to kind of speed up your later development process and to guard against other issues but the very last tool we're planning on putting in Firefly is where we'll just generate that test for you so in the case of here, you know, we'll add for off-codes 12, 14 and 15 we'll do some analysis and find a test that will actually exercise that particular those particular off-codes basically or here, you know in the case where X greater than or equal to 3 we can ask the Microsoft tools they give us an example of an X that's greater than or equal to 3 I know that's really easy to do like 7 or something sometimes it's harder, you know and so then the tools can say, okay, here's an assignment variable that will exercise that particular execution panel so that's the overall roadmap for Firefly at the moment I'm going to put up this roadmap again you can see it kind of extends out of ways we are in this phase right now like I said, we have prototypes of all the tools going on here, not a test based generation but we've equipment ourselves in time to really make sure that the everything's ironed out and works well we just want it to be basically drop-in that's one of the goals users don't have to know that they're using the tool until it tells them, hey, here's the problem okay, we're always going to wrap up any questions for whatever you'd like yeah, you mentioned for the test you mentioned the case of the ABI and then you could go back in general, if there's missing test coverage, you can go back with the source mapping it may not be possible that you might have to actually wrap some specific transaction of quality like incorrect ABI coding yeah, I know, but I'm also saying like, so how are you experienced because often source mapping for this kind of generated code, it doesn't do any new code source mapping would be more just to assist the developer, it wouldn't be used probably in the automated test generation because there we can just generate exactly the call data we need to feed in to exercise that case and for overflows in our experience there's lots of overflows but the compiler likes to use them in more kinds of cases so do you have any featuring for that already well, so that's why you would you need the temporal property which says those are okay as long as they're followed by reverts in some cases obviously that's like a little too broad scope of a property I mean, because like sometimes the subtraction is optimized into like an addition with overflows and stuff like that but yeah, we can have a thing those already exist actually, that's just part of the case pool right now, I was just showing that's how the tool is made right now on the diagram so I wanted to ask about the two vacans after December 2020 why would you use the LLVM back end random, the other one because of the speed yeah, so the LLVM back end is basically take the semantics and generate LLVM big code that directly implements an interpreter for that language, but it can do any symbolic execution but there's a lot of overhead in doing symbolic execution that slows it down too much for actually making reference interpreter or something like that in about one minute I was just going to say, I think it's important to note that the technology is available, it works we use it internally it's simply actually modifying and extending it out for just to make it further so, please note this is not big for what you're buying internally we already use the K-frame for all these different variants of these formal methods, formal verification for analysis, just that the tools require experts, require us or others who have expertise in formal methods what we want to do literally is to leverage the capabilities of the tool through automation and make them available to everybody and automation comes from what we already do in our audience when we analyze them when we analyze people's code we know that something this can be automated we already do that, and that's what fireflies as much as we can and others come from the entire community and especially from all these amazing tools that Microsoft has and other tools in very short so, the key word here is automation and once the tools work then we make them available to you as push buttons analysis for the function I also wanted to add, we are looking for people who are interested in conversing with us and being kind of alpha users in like a closed alpha sort of thing so, grab a conference if you want to be a part of that all right so, again, I want to remind that the K-frame words is language parametric or language acoustic but none of the tools in the K-frame are what programming language you plug and play in these tools and that gives us opportunities to do the same with many other languages and we are already defining some of these for several other languages which will also take advantage of the tools that we support and we improve right now as part of Firefly and its integration in very short among others, besides what's working on several other languages maybe on instance, it is float based on your linear type and you sort of see something like boom and then you have out of the board formal analysis tools for this language to incentivize people to formalize their languages in K and thanks to the elegant interpretation which is pretty fast you can think of it as you implement your language in K and if you are happy with the speed then you may not even need another implementation there will be some customers who say that hey, I think that speed is good enough actually to have an aspect implementation of the language also somebody asked about protocols, if you want to formalize and analyze specific protocols yes, this is another advantage of the K-flamer that it doesn't even care whether what you are defining is a programming language or not, only you have a notion of transition from one stage to another and once you formalize that and that can be in particular a protocol or a token specification once you do that, then you can all the tools of the K-flamer to analyze your protocols and we are already doing it commercially we have formalized and analyzed several protocols actually right now we are working with some people and of course we see also the different chain you can think of K at that moment as a programming language say hey, I am implementing my protocols in K and then you can execute and run things the same as you implement them and run them with Python or more but at the same time you can also formally analyze them exactly as they are without any gap between what you run and what you verify and with this we would like to conclude and if you liked that what we saw, we strongly encourage you to attend the three other events that I am aware of that mentioned K in F1.5 come to our booth downstairs with the t-shirts and speakers go to the K-websites there are lots of languages defined for example K-flamer not only EVM, several others the Firefly Tools is available for download download language it and become an alpha tester and if you really like and you don't want to do it yourself we are here to help contact us at our booth and we will have you formally verify your protocols and we have a few minutes for questions one minute left now a real one minute yes Rust is a programming language if we have a formal semantics there are two formal semantics defined by other groups not by us we don't know who they are we haven't looked at them but there are two semantics one of them for sure but we are working on verifying some laws and code that is generated for us and you can define your language yourself so have to there are lots of examples and we believe that all the tools that K provides incentivize people to formalize if we don't have access to the formal verification in space for me I am just like a general manager in what kind of practice we can apply our data codes so that's why Firefly Firefly will be a tool that adapts with all the automation but then we need to formally specify the property that's unavoidable or work with other people that's the situation part of our bodies more than half the time in audit we spend with the customer understanding their properties to help them formalize the specifications of their properties I want to know what are some low-hex we can become nowadays I guess a lot of programmers use tasting for checking can we use some of the tools Firefly will do that we don't need to spend too much time to change the way we code it that's why we want it to have Firefly exactly for work does it mean that tasting could be automatic we don't really need to write you still need to write tests to get execution traces of Firefly but the point is that now you can get more out of writing your tests I mean there's some work that's unavoidable you need to somehow tell it this is what I want my contract to do either in the form of tests or in the form of formal specification which is more exhaustive for example if you're working with Solidity if you have Assertion Solidity write that song tell the tool what what I just want to