 Hey, everyone. So this time, I'm going to start my talk for real. No kidding around. We're going to hear about a crack-by-construction blockchain consensus protocol. And I hope you're all excited to be here. And I hope you're all aware that I'm starting to speak, and so are coming into the room. But I don't have time to wait. So here we go. So firstly, let's do a basic introduction. Consensus protocols are kind of fundamentally about deciding nodes on a distributed network, making the same decisions, even though they might have different views of the network. And those decisions, you can think of them as being on values. But normally, we think about, in our context, replicating a virtual machine or some kind of state machine. So traditional consensus protocols decide on one block of transactions at a time. Whereas they do it with irreversible finality, with every single block. You come to a Byzantine quorum, you sign the block, and every block is created, and then there's never any forks. And this usually requires, on the order of the number of validators, messages for every block to accomplish this, because you have to come to consensus before you make every block, essentially. Improve work consensus, on the other hand, we don't come to consensus on every block before we make it. Instead, we have this kind of forking phenomenon where we just try to put the block on there. But if we fail, it might leave. But somehow, if we wait long enough, we feel like, oh, it's never gonna leave the blockchain. And these protocols have a much lower overhead. So basically, the thing that I'm gonna show you today is this consensus protocol cast with a friendly ghost, which finalizes every block with a synchronous, Byzantine fault tolerant consensus safety, the way that traditional consensus protocols do, but is able to do this with the overhead of proof work blockchains. So it kind of represents some of the best of both worlds, right? Both finalized decisions and also hyper low network overhead. And the reason why this is cool is because it kind of provides us a unified way for thinking about both of these things. So I'm gonna give a little overview of my talk, and then I'm gonna go right into it. So the consensus safety proof is this thing that all of the consensus protocols that are cracked by construction process satisfy. And then I'm gonna specify some of these things, and then I'm gonna talk about maybe live this a little bit and then give some experimental observations where we're gonna see this kind of network efficiency that I'm talking about. So let's get started. So the consensus safety proof is probably the most important thing that you understand in order to grok this whole process. So we need a few things for this thing to be fine. We have protocol states and state transitions, and we have an estimator and this thing called estimate safety. I'm gonna define them in a second. And then we have this theorem, this result, that if nodes have a common future state in the protocol, then they have consensus safety on decisions that have estimate safety. So if I decide on something with estimate safety, then I'm gonna get consensus safety for free if we have a common future. And so basically protocol states are just like things in a group and state transitions are kind of paths between them, right? An estimator takes these protocol states and maps them to in the binary consensus, like a zero or one, and the blockchain consensus, like a whole blockchain. So it's kind of our fork choice rule mechanic, right? A little guess. An estimator's not a decision, it just guesses. Oh, looks like we have laggy. So an estimate is safe. Basically, if the estimator, for all future possible protocol states that can be accessed from that state, the estimator returns a value like one or has a blockchain that will always have that block in the chain. So a block is safe. If all future blockchain fork choices have that block in the chain and a bit is safe, if all future estimators have that bit. So if two estimates are safe at two states and those states have a state in common, then because those things are, the estimates hold for all future states and they have one in common, they both hold in that one state that they have in common, and that means that they have to be consistent because you can't have the block be in the fork and not at the same thing. So that was kind of the first part of the proof. Basically it says if nodes have a common future, then they have consensus safety. The next part basically is a construction to make sure that they will have a common future unless there's more than some number of Byzantine faults. And the way this works is basically we have protocol states which are messages. We have state transitions which corresponds to receiving messages or increasing like going to supersets of protocol messages. And if we count Byzantine faults and exclude the sets of messages that have too many faults, then we can guarantee that they have a common future unless there's that number of faults. Basically because first we allow all of them to have a common future no matter what, then we remove the states that have too many Byzantine faults. And so two states will have a common future unless there's too many Byzantine faults. And that gives us kind of our key result here which is that as long as there's not too many Byzantine faults and they have a common future and if they have a common future then they have consensus safety. And so this consensus safety proof basically says look as long as there's not too many Byzantine faults then we have consensus safety. And the cool thing about this is I had to say almost nothing about the consensus protocol, right? All I had to do was say it has protocol states and I mean I said nothing about the nature of the estimator, right? And so now we're gonna kind of see this, we're gonna see this thing happen. So I'm gonna define protocol messages and then the estimator, Byzantine fault detection, states and state transitions and then the end estimate safety for the binary consensus. And then we're gonna do a tiny change and generate the blockchain consensus and a tiny change and do validator rotation and a tiny change and remove fault tolerance thresholds from the protocol. And that'll kind of demonstrate the power of the approach here. So for the binary consensus we have protocol messages that are bits, sorry they have three parts, a bit for the estimate, sender which is like the node who sent it and a justification which is a set of messages. The protocol message is dependency of the other if it's like in the justification or in those justifications basically if there's pre-block pointers or pre-hash pointers to it then it's a dependency. The latest message is one which is kind of, there's no, there's nothing earlier, right? So a dependency is earlier, if you're not a dependency then you're later and a latest message is kind of extreme on there. So the way that the binary consensus works is you look at all the validator's latest messages, if more of them by weight have zero, then you pick zero and if more of them by weight have one then you pick zero, then you pick one. So basically if this, so the score of an estimate is the sum of the validator's who have latest messages who have that estimate. And if the score on zero is more than zero one, the estimator returns zero, otherwise it returns one, otherwise, and if it's a tie, then it returns this kind of exception. So that's kind of our definition of estimator and that's kind of gonna be necessary for us to have the concept of deciding on safe estimates. So Byzantine faults are kind of this interesting and tricky thing. Basically each of these nodes are making messages where they attest to what they've seen from other nodes. And if it's ever the case that I have a set of messages where different nodes attest to having seen some node make messages that could never have possibly been made by a single node, then we have a Byzantine fault. Because basically single nodes, the way they're characterized is that they follow protocol executions according to the protocol executions in the state. So here, equivocation is this property that two nodes that a node isn't actually a single threaded. And it means that they've made two messages, neither of which are in the dependency of the other. So rather than including their messages in each other and having a single thread, it's multi-threaded. And we basically are gonna be able to identify Byzantine nodes by their equivocations and we can count them all up. We can measure their weight. And then we can define protocol states as all of the sets of messages that evidence less than some of the faults. Then we can define protocol executions just as supersets in those things. And then we're basically done. The binary estimate safety basically means that the, so it's estimated is safe. It gives them fault tolerance in some state. If for all future states that estimate is also the estimator in the future states. So there's like a disinvariance, right? All future states will return zero or all future states will return one. And because of the way we constructed this, this satisfies our safety proof. They all have future states unless there's T faults. If they have future states, they have consensus safety. And so we have a binary consensus protocol with safety. And this is kind of where we can relax because that was most of the hard work. Now we're gonna find the blockchain consensus protocol and it's really gonna be a lot easier because we've already done most of the work. So the blocks kind of look very similar to the messages for the binary consensus only instead of having a bit, there's another block in the estimate spot. So every block kind of has a pre-block, a sender and justification. And there's this thing called the Genesis block. And this is kind of the heights of the blocks. So this is me defining all of the blocks, being all the blocks at all the heights. So there's a couple more definitions that we need. Basically, instead of having the estimate just be the same, we're gonna have the estimate be a member of the blockchain and that's what this represents. So like block one is in the blockchain of block two. And then the score of a block is the weight of validators whose latest messages have the block in their block chains. So if five weight of the validators have block A in their latest block chains and three of the weight have block B in the latest block chains and block A will have a higher score. So then we can have this fork choice rule. It's called the greedy heaviest observer subtree. You start at the Genesis block, you look at the children of the Genesis block which are the blocks that have a pre-block pointer to the Genesis block and you ask what are their scores and their scores are given by whether or not or how much of the weight of the validators latest messages are on those block chains and then you pick the best child and then you kind of repeat the process where again you calculate the scores for each of the blocks by looking at the latest messages and seeing if whose validators latest messages are on those blocks and then you pick the block with the highest score until at some point you get to a block with no children and that kind of represents the whole fork choice. The cool thing here is that protocol states and state transitions are defined exactly as before. There is literally no change, right? We just like the tech business defaults is exactly the same way. Define protocol states and sets of messages with less than some faults and state transitions as supersets and then we can define estimate safety for the block chain and basically it's very similar. It means that for all future protocol states, this block is in the block chain that we're gonna choose in those future states. So a block is safe. If in all future fork choice rules that you make, that block will be in there. So that and that was actually it, right? Cause all the other stuff and the specification is really kind of like done there because and we got to benefit from all of the work we did earlier from the safety proof and the binary consensus. We got to like pretty much reuse most of those, most of those, most of that work and only had to redefine, you only had to define the message type, the score of blocks, the fork choice rule and safety. So that's pretty cool. We were able to go from the binary consensus to the block chain consensus with a minimum change. Safety oracles, this is kind of important. It's kind of important that we are able to detect safety. I'm not sure we're having this slide. So an, no, like what I've shown so far is that if you were to make estimate decisions on safe estimates, then it would be consensus safe but you need a way, we might actually need a way to figure out if that's the case. So we have a couple of definitions here and this I'm gonna give like a simple way to do this. So a validator sees another validator agreeing on an estimate if the latest, in the latest message of validator I in their justification, the latest message they see from validator J has an estimate that agrees with the estimate that we're trying to say is safe. An agreement in the binary consensus means that they are the same bit and the blockchain consensus means that they are in the block chain of the other block. So VI sees VJ means that the validator I sees the latest message from validator J that agrees. And there's another definition here that's cropped out and it basically says that VI can't see VJ disagree which means that there's no new latest message from VJ that they could see that doesn't agree with the estimate. And so basically, if there's a set of validators who can't, who see each other agree and can't see each other disagree, turns out that if they have some number of weight, some amount of weight, and that weight is more than half the weight, then you can actually show that there's some amount of fault tolerance but our safety definition requires that we have T fault, we can tolerate up to T faults. However, there might already be some initial faults in our view, so we kind of subtract that. So basically, as long as the, this is kind of the overlap between two sets of validators that, like the overlap in weight of the validators in the clique, if that's more than the fault tolerance, then we have safety. It's very similar to a traditional kind of consensus safety proof, only it's working on these estimates instead. And the intuition's quite clear. Basically, in the clique, they can't convince each other that they've seen something else. And if they have more than half the weight, then whoever can convince them from outside the clique won't have enough weight to actually contribute. Great, so that actually was a hard part also, but we're getting through it, we're almost done. So, subjective fault tolerance thresholds is a pretty cool thing. Basically, if you have two protocols which have the same or are the same except for the fact that they operate at two different fault tolerance thresholds and one of them is small on the other, then consensus, the nodes who make decisions in these protocols will actually have consensus safety as long as the number of faults exhibited in their union and their views is less than the smallest of their fault tolerance thresholds. And this is really cool because having fault tolerance thresholds in the protocol makes it easier to attack and pose a bunch of economic problems that are kind of out of scope for this talk, but stuff about cartelization and focal points. Validator rotation, this is an exciting thing. So I'm gonna change the structure of the blockchain a little bit to add weights to the block, right? Before we had validator weights kind of via this global thing. Now we're gonna add weights to the justification. So the blocks are gonna be the same as before, like they have a block, a sender justification, and the justification now has this additional thing called the weights. And then we need to redefine the score of a block to use the weights of the parent's block in order to actually choose that block. If block A has weights, has a certain set of weights, then the children of block A, which are the blocks that have a pre-block pointed block A, need to be chosen according to those weights. And not a single one of the other definitions need to change. And so this is really like all that we need to do, to do consensus safety, sorry, to do validator rotation with consensus safety, basically because all of the definitions satisfy the abstract safety proof, because they have a common feature unless there's T faults. And they only decide on safe estimates. So liveness is an interesting thing. Liveness means a guarantee to eventually make a decision. This result in consensus literature called FLP and Possibility, that means you can't really be live and safe in an asynchronous network without using entropy or, oh no, without using entropy or cryptography. But actually we have kind of a bigger issue than just FLP and Possibility, which is that we never said anything about when validators were actually meant to send blocks. They actually, we just haven't specified that yet, because all we've really talked about is safety. And so, however, we're gonna see in our experimental results in just a second, that there are sets of messages that they could get to which are safe. And that means that as long as we can make those shapes, then we can have liveness. But, and we kind of know that in a synchronous network or in a partially synchronous network, nodes can use timeouts to make this kind of message shape, you know, up to some number of faults. And so if we were to pick one of these, then we could give a liveness proof for some strategy that says, oh look, you guys have to figure out how to make a tag of messages in this form. But, you know, we're not really gonna do that. We're just gonna leave liveness out of the story for now. So I'm gonna, this is kind of the, maybe the more exciting part of the talk. So this is, we're gonna look at actually and try to see what this looks like. So these are three validators running the binary consensus. And they, the color here means that they've reached safety with some amount of fault tolerance. And kind of, the numbers in here, you see there's ones and that one's a zero, it's maybe a little bit hard to see. The numbers are their estimates and they need to have the estimate be the max weight of the latest messages they've seen in the justifications. These arrows, these lines are the point to the messages that are included in the justification. Remember the data structure is estimate, which is a bit, sender, which is like validator one, validator two, validator three, and the justification which is encoded with this structure. Showing, you know, a few more nodes coming to consensus on a bit here. So the darker color means that they have more fault tolerance. This is kind of cool because it illustrates the flexibility that we have about the order of message passing because we haven't said really anything at all actually about that. Now the blockchain consensus protocol is a little bit more complicated to think about because it's got these things called pre-block pointers. So there's dotted lines, these thin ones are justification pointers, these are pre-block pointers, the blue ones are the latest fork choice by the validators, and the red one is our fork choice. So basically like, the way the fork choice works is okay, so validator one has the most weight, so from the Genesis block we go to validator one, and here it only has one parent, and here we could have went that way or this way, but actually the latest messages are here, there, and there, and all of them have their weight here, and so the weight is all on this fork and not on that one, and so we go left. And then here, you know, there's a pre-block pointer here and also another one there, and if you look at the latest messages, there's more of them here, and so that's why we're gonna go there. And what this kind of shows you here is that validator three has more weight than validator one. Right, so the reason why we initially chose this actually wasn't because validator zero had more weight, it's because all of their latest messages are on this one. So here we go. So this is kind of the blockchain consensus kind of in action for three notes. They're gonna use the greedy heaviest observed fork choice rule, where every choice in the road, like first we went that way, and now we went that way and more validators made blocks there. Here we fork, right, there's forking on the top, and then here we achieve safety, right? This kind of, when these things get colored, they have asynchronous Byzantine fault tolerance safety. And notice that they fork at the top and they have safety at the bottom, and the way we calculate safety here, you know, is using the safety oracle that I described earlier where you look at these cliques which see each other agree and can't see each other disagree. Now we can watch more notes kind of go faster, and here's where you see some interesting stuff where like a bunch of blocks will get finalized at the same time sometimes, right? Because every, this is the thing to kind of realize, like blocks can contribute to the finality of many, many blocks. And basically because every block you have will like agree and disagree with other blocks, large number of them, and which blocks you disagree with and agree with actually will determine how the safety oracle turns up. And so remember the colors, not forking, up here at the top, lots of forking, like look at the red line, it kind of goes and changes, right? Like you don't really know if these blocks are confirmed or if there are gonna be consensus, but the ones down here are, right? So it's very much like a forky thing, it's a kind of a blockchain like thing. But this is really what I'm talking about. This is kind of what I was saying when I say that I can achieve asynchronous business default tolerant consensus with the overhead of Nakamoto consensus. If they do manage to send the messages around Robin, then eventually you get to a point where every additional message finalized an additional block. And this kind of overhead for finality of consensus at every block, like one message per block for finality at any level of fault tolerant threshold is like theoretically optimal because you can't really get lower than one message per block. And this is kind of a little interesting here. You see that this validator has more weight than these two. This is why the fourth choice rule, this validator, even though you saw that block, would have preferred to build on his like initial block than these ones because these two guys have less weight than this one. And so this is kind of what I mean when I say that like we can achieve really, really efficient asynchronous consensus safety. So somehow, I know it's like a bit much, but I shared consensus safety proof that talks about these local determinations of estimate safety and how they're related to consensus safety. We were able to define a blockchain consensus protocol through like basically by first defining the binary consensus protocol that satisfies the safety proof and making a really like minor change that doesn't affect the safety proof at all. And then we were able to actually add validator rotation and remove finality thresholds also with very minimal changes to the protocol. And the reason why we can do this and why it's like so little work is because of the fact that the proofs that we're relying on are not changing while we're making these changes. We have like abstract safety proofs, concrete protocols which we can tweak without changing the safety proof. And we were able to achieve this really nice kind of latency numbers. So I think that this is hopefully gonna be educational, hopefully gonna lead to new consensus protocols. We actually have a bunch of consensus protocols that were mentioned here that were derived according to this process, replicating things like concurrent block structures instead of sequential ones or like an integer could be useful maybe for time snapping. So here's a little secret. Just now, I published the prototype for the binary consensus and the blockchain consensus and also like a specification in very much detail of all of this stuff. And more incomplete description of the correct by construction process given at a very, very high level of abstraction, much higher than I even hinted at today. So this kind of represents the state of the work today. And I'm excited for you guys to check it out and to take a look at the correct by construction consensus protocols and see if you find them as easy to reason about as I do. Some acknowledgments and then we're done, right? So Nate and Danny worked really hard to help me get ready for DevCon and get the code published, get the papers ready. Million thanks to them. Carl who just spoke last was the first person to really actually the first developer on the correct by construction Casper. Greg Meredith who is here somewhere but I haven't seen him yet has really introduced me to the correct by construction process and helped me really formalize and take this mathematical approach. Finally Vitalik has been working with me on this stuff for years, like three years more. So thanks a lot for everyone and thanks to you for listening.