 Right, okay, sorry about that, so I've been working at a company that designs processors for 13 years now and something that I'm more and more amazed at, the longer I work there, is this simple fact that processors are incredibly reliable. I mean, they're not completely reliable, right? I mean, they do occasionally fail, you know, we hear about bugs in processors, but it's such a rare thing that if you and I were to make a list of all the processor bugs we've ever heard of, we would probably have like nine out of ten things in our list would be the same. In fact, we probably wouldn't even get to ten, to be honest. So I'm going to be talking about how it is that processors are actually amazingly close to being, you know, completely correct. How do we actually achieve that? So if you want a more technical version of this, of the information in this talk, I've put links on the slides, I put my slides on my website afterwards, you can grab the links from there. Matty Sue's talk yesterday is also related to this and I wonder if anybody was in Johnny Austin's workshop yesterday where we had a bunch of humans pretending to be different parts of a computer modeling a five-stage pipeline that was also kind of quite similar and good fun. So processors always work and this is because there's a sort of tension going on. So is there an echo in this? So there's a tension going on in creating a processor. On the one hand, you want it to be correct and on the other, you want it to be fast and these things are completely in opposition to each other. It's really hard to make something correct and fast. And so this is a talk with a villain and a hero. And the villain in this, sorry, this is a maker festival, so bad news, the villain is in fact the designer. He's trying to make things faster and unfortunately as he tries to do that, he inevitably introduces bugs. Can you turn down the sound a wee bit? I'm getting a lot of echo, it's a bit distracting. Okay, well that could work too. So the designer is busy introducing bugs and making it go faster too. Meanwhile, the hero is the person trying to find those bugs and help fix them. So the verifier is pushing back against the designer. And in our commercial processor, the verifier usually wins because a fast processor is nice but a correct processor is critical. You don't have a, you know, no one's interested if it mostly works. So the verifier will usually win the battle over. Is this too complex a feature to add? And the verifier is helped by some useful tools which I'll be describing in this talk at a fairly high level. They're called sat solvers and bounded model checkers. Not very sexy names, but there you go. Okay, so what I'm going to do in this talk is I'm going to kind of look at this switch back and forth between correct and fast. And I'm going to start with a very simple processor. And so what I've done here is I've taken an early MIPS processor and I've simplified it down just by kind of pasting over some of the complicated features in it. So it's probably still more or less correct with my butchery applied to it. Let me just quickly go through the main bits of the processor. So working from left to right, before you can execute an instruction, you need to know what instruction you're going to execute. So you take your program counter from the green box, you hand it to the memory and it gives you back an instruction. You then have to figure out what the instruction means. So you decode it. That's traditionally, that block is shown in white in diagrams like this. And so I followed that tradition of the decode logic is shown in white so that you can't actually see it. It's the most complicated part of the processor. We always leave it out of the pictures. So once you've decoded the instruction, that gives you all the control signals to control all the rest of the circuit. So the next bit is from the green bit in the middle, the register file, you read some inputs and then you hand that to the ALU which can do arithmetic, addition, subtraction, comparison in the values. And you can also pass the values to the memory if you want to read or write memory. And then when you're all done, the results go in this big backward path back over to the register file where you can update any registers you've got. Okay, so that's very quickly what processor's like. Oh, I should also mention that there's these little gray boxes. I don't know if you can quite see them. They're labeled MUX or multiplexer. They're a switch which selects different inputs. And those are absolutely critical when you're worrying about a processor being correct because if something's already always connected to the same thing, there's a good chance you'll spot when you get it wrong if you've got the wrong information there. If you keep switching from one thing to another, it's very easy to switch to the wrong thing or to sometimes have the wrong information on an input and not really realize it because you're not selected. So the MUXs, again, little busy boxes don't look very significant. That's where a lot of bugs will occur is in those little boxes. But anyway, we've got a processor. I've simplified it down so it's really just one pipeline stage, one time set taken to go from wanting to fetch an instruction all the way around till you've executed the instruction and written back the results of the register file. So it's about as simple as you can make a processor. Now, having done that, remember the hero is the verifier and we're wondering, well, does it work? So how do we check if something works? Well, we test it. So what kind of tests do we run? I'm guessing most of you are more software people than hardware people, but it's the same either way. If you want to check something, you just kind of make a list of all the functionality you might have, all the weird corner cases you're going to worry about, and that gives you a bunch of cases that you ought to consider testing. And then once you've got that, you then decide, well, how many tests I want for each case? And if it doesn't matter much, whether it works or not, then maybe one test for every 10 cases is good enough, right? That's what I do in my hobby projects. If you're a bit more serious, maybe you want one test per case, 50, 1,000, do you need a million? How much is it going to cost if it goes wrong? How hard will it be to replace it? So you play around with this and you decide how much tests you want to do. So that's the traditional way of checking the processors, right? But before we get on to the less traditional, the modern way, let's think about we've got our tests, now we need to think about running them. So we've got a processor, and we can run some programs on them, but how will we know if the program did the right thing? Well, what we're going to want is a reference that we can compare against. So this might be a simulator, or it might be your friend is also implementing the same processor, and so he's got another implementation, and he's smarter than you. So maybe he's put less bugs in it, or maybe because he's smarter than you, he got too clever, and he put more bugs into it. But either way, you're hoping he's got different bugs, right? So you've got a reference to compare against, and then you can just take all the individual parts of the state of the processor and compare them. And so we can run tests through that, and we're quite happy. But instead of just running tests, there's another way using formal verification tools, which we can use. So there are tools which will take two circuits and compare them to tell you whether they're exactly equivalent to each other. Okay, so in the industry, those are called logical equivalence checkers. You'll also hear people talking about SAT solvers, which are a slightly more flexible version of the same thing. And what these do is they say, well, let's take every possible state we could start in, all possible instructions we could think of executing, all possible contents of memory, and then just check that these two circuits are going to do exactly the same thing, no matter what. So the beautiful thing about this is you don't have to think about testing, right? Because it just tries everything. It doesn't brute force it, that would be too hard. It uses various mathematical tricks to try to reduce the number of things it has to check. But it's doing something equivalent to checking for all possible inputs that these circuits are equivalent to each other. So you don't need to think of any tests, which is great, because I hate thinking about tests. It's just not fun, right? So this will make up the tests for you and think of all the other things you should have been thinking of testing that you forgot, but really nice tools when you can use them. Okay, so that's us at the start of this cycle. So we're on the end of we've got something that's correct but possibly slow. So now I'm going to look at a simple optimization, which people always do to this kind of processor, which will make it a bit faster. So the processor kind of did the work in two stages. Remember, it started off by fetching an instruction and decoding it. And then in the second phase of execution, it's kind of adding results, it's writing them to registers and so on. And whilst it's fetching, you don't know what the right hand side, the memory should be doing, so it's idle. And once you've fetched and decoded something, you can't do anything because, you know, you've got to wait until execute unit finishes. And so designers hate this kind of thing, hardware sitting idle, they hate that. So standard technique is instruction pipelining. And with instruction pipelining, what we're going to do is we're going to double the clock frequency. And then while we're executing the first instruction, doing the arithmetic or memory access, we're going to be at the same time fetching the second instruction. While we're executing that second instruction, we'll be fetching the third while we're executing the third will be fetching the fourth and so on. You get about double performance. So this is a technique IBM came up with in the six days, it hit mic processors in the mid eighties, a very common technique. And so here's the processor with a little bit more of it, a little bit of it restored. There's this green bar down the middle. And that green bar is a bunch of pipeline registers. It's remembering the first instruction it fetched while it's executing that. And that leaves the left hand side is now free to start fetching the second instruction. So instruction pipelining is a pretty common thing. And what we want to do now is check to see if we've introduced any bugs in doing this. So how are we going to do that? Well, let's start with the reference circuit. So we've got a reference circuit and the processor is going to be executing two instructions at a time. So I'm actually going to want to check both instructions. And so I'll put in two copies of the reference circuit and I'll just connect the output, the final state of the first circuit into the starting state of the next circuit. So that's just going to execute two instructions and sequence. And then I'm going to do the same thing with the processor that I want to check. So I'm going to make two copies of that. And again, I'm going to add in some data pass so that the final state of the first processor gets copied over into the final and intermediate state of the first processor gets copied into the next processor. So I'm going to forward that data over and then I'll add the checkers. So this probably feels like it might be a little complicated to do. Fortunately, there are tools which I'll mention a bit later on. But what I have now is something where I can feed two instructions into this processor and check that they're both going to execute correctly. And again, these formal verification tools will just check for all possible pairs of instructions, all possible initial states. So I don't have to come up with tests. But as I say, I can also use testing. I can just run one processor against one reference and check for bugs the usual way. Whichever one I do, it's probably not going to be very long before I run into a bug that looks a little bit like this. A branch instruction followed by, well, pretty much anything. Because that style of processor that I just showed you is prone to a bug that occurs after branch instructions. So what goes wrong in branch instructions? Well, let's think about how the pipeline is going to be executing this. So at the same time as it's executing the previous instruction, I was fetching the branch. Then next clock cycle, I move on and I start to execute the branch instruction to figure out where I should branch to, PC plus 100. And at the same time, I'm fetching the add instruction. Then in the next cycle, everything moves forward and I go to execute the add instruction. But why am I executing the add instruction? Because that branch, sorry, I shouldn't point at my screen. I shouldn't point there, perhaps. The branch instruction is telling me not to execute the next instruction. It's telling me to go off somewhere else and execute that and said. But what's happening is I will always execute the instruction immediately after a branch. So there's several ways of thinking about what's happened here. One is to say that this is a bug. That's a pretty common opinion these days, that you're after a branch, you always execute the instruction immediately after. At the time that people started pipelining mic processors, there was another opinion which was to say it's not a bug, it's a feature. And they gave that feature a special name so that you would know it was a feature, definitely not a bug. So it's called a branch delay slot. They were very popular at the time in the early risk processors. They kind of fell out of favor later. Because if you try to claim that every single bug you put in your processor is actually a feature, then it's very hard to make the second processor have exactly the same set of bugs. And of course, it's going to add some new bugs of its own. So then your third processor becomes really hard. You've got to get all the old bugs and then persuade people that the new bugs are features as well. So nowadays, people tend not to put this kind of thing in a processor architecture. They admit it's a bug and they fix it. And the usual way of fixing it is that when you see you're about to execute a branch instruction, you say, well, let's just hold off and fetching the next instruction until I know for sure whether I need it. That's one solution. So that's called stalling the pipeline. And another one is to go, you know what? I'm just going to go ahead and fetch it anyway. You know what? I might even start executing it a little bit. But I'll only go so far in executing it and hope nobody notices if I end up having to cancel it. So you guess that you might want to execute the instruction and then if you didn't need it, you cancel it quickly before anyone notices. And that's called speculation, which has been in the news a lot recently, well, this year because it turns out that some people can spot speculation and turn it into security holes. So we're less sure how to get around that one. Okay. So we added an optimization, various ways of testing and verifying it, and we've got a bug. And let's imagine I've put in a fix for it. Now having fixed that, I want to check if there's one did I fix it and two, are there any more bugs I should worry about? So what am I going to test? Well, I've got all my original tests that I started off with. Does it add, really add? Can it add big numbers and small numbers and so on? Obviously I'm going to add a whole load of tests for branches because I now know to be worried about that. But is there anything else I should be worried about in that processor? I'm not sure. My confidence is a bit shaken now. So I'm going to try and think of what makes pairs of instruction interesting. And then I'm going to test all of those. And this is black magic experience asking your friend what he tests. It's hard to know. You've actually tested everything you need to. And that, just to kind of emphasize it, that's where the formal verification techniques are really helpful because you don't have to come up with lists of pairs of instructions. It will just test a lot. Okay. So I've gone one round through this cycle, take a correct processor, make it faster, but broken, find some bugs and fix it. And I did that by going from a one-stage pipeline to a two-stage pipeline. Well, what would be better than a two-stage pipeline? For most people, the answer is a five-stage pipeline. So with five stages, we take that original diagram I had and it had about five major steps in it and we make each of those be one major step in execution. And what I'm going to do then is have my five pipelines, each row in this diagram is busy doing one of those five steps. And the idea is that once it gets up to speed, I'm executing five instructions at once. I'm working from the bottom upwards. I'm writing back the results of the first instruction while I'm doing the memory access for the second instruction. I'm doing the arithmetic for the third. I'm decoding the fourth and I'm fetching the fifth. So I'm keeping all the bits of the hardware busy. And this makes the designer really, really happy. And it tends to make the verifier a bit less happy. So let's just, here's the original circuit as I downloaded it off of Wikipedia with all the pipeline stages shown. So the pipeline stages are these big vertical green bars that just capture the bit that was just calculated and make it available in the next time step to the next part of the circuit. And what can I do to verify this? Well, I've got the same option as before, one copy of the reference circuit, one copy of the processor, and then add some comparison logic to just compare them. And then I can run all the tests I want in this. I can, yeah, I can run lots of tests on it. So I'm perfectly happy with that. But if I want to do the formal verification, then I'm going to do something similar to what I did previously when I had two copies of the reference, two copies of the processor. But I'm going to have, this time, it's a five-stage pipeline. So I'm going to have five copies of the reference and five copies of the processor, so I can model five instructions at once. Except this may not be quite enough, because if you remember the fix for that branch-related bug was that sometimes instructions could get stalled. And if I'm stalling instructions, they may not take five cycles to get all the way down to the end of the pipeline. They may sometimes take six cycles, so I'd better add another stage. And well, maybe I could have two instructions get stalled or one get stalled twice or something. Maybe I should add another stage. So I play around with that and try to get the right number of steps I'm checking. So on this side, I've labeled this with its official name. This is called bounded model checking. I'm not going to try to explain what model checking means, but the bounded bit is simply this number of how many copies I make. So I start off with a bound of five. I've increased it to a bound of seven. And as I say, there are tools that will do this little transformation of making the multiple copies and forwarding data between stages. They'll do it for you. There's tools like YOSIS, which if anybody is into doing open FPGA work, you'll probably know about the YOSIS tools. There's talks about using bounded model checking that you can get hold of. There's a paper that I mentioned right at the start that I wrote that you can get hold of. There's blogs. So there's a bunch of links there that you can follow. Okay, so we run some tests. We do some, oh, and let me just emphasize just once more that what this is going to do is it's going to test any sequence of instructions. In this case, any sequence of up to seven instructions are going to get checked and for all possible inputs. So it's going to run through the first check, then the second and the third and so on. And if it manages to get all the way to the end, then you're going to know that any seven cycle sequence, anywhere in execution, is going to behave correctly. And you don't have to come up with your own tests. So either through testing or through this bounded model checking, you will find another bug. And this is another well-known bug in the style of processor. It was actually fixed in the next generation of processor. The picture I'm showing is of the MIPS 1, the MIPS 2 fix this. So the bug involves a load followed by a store. So let's just think about what's going on when you have a load followed by a store. So I was kind of zoomed forward to the point where the load is in the memory stage and the store is behind it in the execute stage. And so the load is busy. It's got an address. It hands that to the memory and it reads back a value from the memory. So now you've got a value from memory. At the same time that that's happening, the store has finished reading some values from the register file and it's sitting in the execute stage waiting ready to execute. So now let's go forward one cycle. At this point, the load writes back the value it got from memory back to the register file and the store starts to write something to the data memory. But what is it writing? If you look at the program, you might expect I loaded into register zero and then I stored from register zero. So you would hope it's storing the value you just loaded from memory. But that's not what happens because the store had already read some values from the register file before it got to that point. So it's going to store the previous value from register zero to memory. So I would tend to label this a bug and the next generation, the processor, chose not to do this. But again, the first generation, they called it a feature and they gave it a fancy name. They called it a load delay slot and they said if you do a load, then you must wait at least one instruction before you actually try to use the result of that load. So that's one way of fixing it is to say it's a software problem, it's not hardware that's wrong, it's programmers need to deal with this. They need to find something else to do between the load and the store or they can maybe get a compiler to do it for them. But either way, it's not a hardware problem, definitely not a bug. It's a feature. So now if you disagree with that and you actually want to fix the problem, you could do as we did with the branch and just stall the processor, just wait a bit until the value has made it back to the register file and then you're all good. But the more common one is not to stall and try to maintain a bit of performance. And the reason for that is if you think about it, at the point when I went to execute the store instruction, I had actually already successfully got a value from memory. The only problem was I'd already got a value from somewhere else. And there was no way that the value I had, which by this point was in the right back stage, that little gray mux in the furthest to the right hand side, that's where the value was. And if only I could get it from there into the memory, everything would be good. So let's just add a data path that will do that. Right? So we just add a data path with add another mux to select, which whether to use this forwarded value, this bypassed value, or to use the value we got from registers. So we do that. But there's two inputs to the memory. Well, maybe we should add another forwarding path so that it doesn't matter whether we're using the value we read as an address or as data, either way it's going to work. So that's good. That's fixed it. But unfortunately, there's another bug. It's any use of the value will break things. So if I have a load followed by an add, that will also not work. Same fix works. Just a different data path. So we add another path going a bit further back. It could affect either input of the addition. So add another path. It turns out there's yet another bug, which is an add immediately followed by an add will have the same problem. It takes ages, it takes two whole cycles for the result of an add to get all the way around back to the register file and be ready for an operation. So we can fix that by adding another forwarding path and another forwarding path. And then one more final bug that I could spot in this diagram, which is if we have a load and then we branch to the address that we just loaded, you might do that when you're returning from a function, then you've got another bug. So we add another path. And in each of these cases, if we decide it's too expensive adding an extra data path, then we have the option of stalling in this circumstance. So for each of these potential bugs, we've got a choice of stalling or adding a forwarding path. And then we've got to make a list of all the times we want to stall, all the times we want to use a forwarding path, adding all the control logic to select the right input at the right time or to stall the processor at the right time. And what we end up with is a lot of new corner cases, a lot of different things we need to test. Because there's so many different situations in which this forwarding logic is meant to trigger or not meant to trigger. So back to what tests should we be running? Well, all our single-stage tests, all our two instruction tests, loads followed by pretty much anything, because we're not scared of loads going wrong. And then, well, how long does an instruction sequence have to be before it's guaranteed not to have any new bugs you haven't seen in a shorter sequence? Can you get four instructions to fail? I think you can get five instructions to fail, you know, a chain of five where the last one is the one that goes wrong. So you come up with your guess of how long that sequence can be, and then you try to figure out what would make a sequence interesting and write tests for it. Or again, you rely on formal verification if that's powerful enough to cope with the complexity of the design you're working on. Okay, so that's me. I've now done two rounds around this cycle. So that was something right. That's correct. Break it, but make it really fast. Fix it, then make it even faster and even more broken. Fix that. And hopefully, you now have a working processor because you've got good ways of testing it, good ways of applying formal verification to it. So I'm not going to do any more iterations on processor optimizations. I'm going to think about what's happening with formal verification. So what's going on here? Well, you can think of formal verification as being we start from one point and then we try all possible instructions that could happen next. And then we take another step, and we think of all possible instructions that could follow on from that. And then all possible instructions that could follow on from that. And we kind of keep going like this, doing this sort of bread first exploration. But if any of you have played around with bread first exploration, you'll know that the state space here is just exploding. And eventually, you're going to hit some limit where you just can't explore any deeper, right? You've been extremely thorough in what you've checked, but you just can't get any deeper because you run out of memory or it says it's going to take another day to run. It's going to take another week, another month, whatever to run. So you're kind of stuck at that point. So testing is a good technique as well. And I've been saying formal verification is great, but testing is really good because it has this nice property that you can take your starting point and then just run a big long program and a million cycles later you decide, okay, I've done enough of that. I'll try something else. I'll try something else. I'll just keep trying different things. And I'm not exploring very thoroughly, right? I'm not filling in the screen with solid blocks of color, but I am getting really deep. I'm getting into interesting complicated situations. At least I have my tests for any good. I hope I am. So I've got formal verification, which is bread first. I've got testing, which is depth first. Both really good for finding bugs. But more recently, what we've been doing is switching to our kind of mixed mode where we'll start off with the formal verification, doing that bread first exploration, seeing how deep we can get. But then once we reach whatever the limit is, we then switch to testing. But then we take some of the paths that we've found by testing, and we say, let's do a little formal verification around that. And we try to broaden out around each line. So we're kind of able to fill in more of that search space, get to these deep states, and then search around there. And so we keep doing that for a while. And we gain a lot more confidence, not just that we've searched short sequences really thoroughly, but that we've also seen a lot of depth as well. Okay, so I started off by saying that processors almost always work. And I hope I've given you a bit of a sense just in those simple optimizations I described of why this is actually an amazing thing. Because just going to a five-stage pipeline, things were getting, I think, kind of complicated. And the testing for them is getting complicated. So, and then I talked a lot about how we go back and forth between the designer and the verifier. And at the start, I said the verifier tends to win the fight over kind of how much complexity you're allowed in the processor because you want the thing to work. Using these formal verification techniques, we're able to check processors much more thoroughly. And this lets us shift the balance over so that the designer is able to push much harder towards what he wants to do, which is make the processor fast without making the verifier unhappy about what's going on. So let's see. So just in closing, I've been showing you processor designs from 1985. So just to bring it a bit more up-to-date, if any of you like playing with microcontrollers, this is one of ARM's more recent microcontrollers. It's Cortex-M7. So this is an eight-stage pipeline, so increasing from the five stages we saw earlier. And there's also five separate execution pipelines, instead just being able to do memory and arithmetic. And it can actually issue two instructions in parallel. So it can be two instructions can be executed in every step. And if you remember, from the five-stage pipeline added, what was it? Six, seven separate forwarding paths, so they would be nice and fast and wouldn't have to stall too often. Well, imagine for each pipeline, I'm going to add a number of forwarding paths, probably about maybe about four, six separate forwarding paths for each pipeline. And between pipelines, I'm going to add some forwarding paths as well. So let me just give you a quick glance at roughly what the forwarding paths in the Cortex-M7 looks like. So it's quite complicated. That joke is actually due to one of my colleagues, Will Keen, who came up with that joke. So this is not completely unrealistic. It really is incredibly complicated. I'm trying to make a process out of this complexity that's actually going to be reliable, this design for little embedded systems, or fairly big embedded systems actually. You want it to work. You don't want to have to go and replace it. So correctness really matters. So yeah, so I hope I've given you a bit of a sense of what designing and more importantly verifying cross is like. Thank you very much for your attention and yeah, if you've got any questions. Thank you very much for the talk. What methods do you use to decide when the verification is complete? What have you done your job? So there's a whole load of experience, right? So I think the first ARM process it was designed over 30 years ago. So there's 30 years of experience of just kind of like we did this much last time and we were happy with that or we were unhappy with it. So let's do more or the same. There's a whole load of people who think really hard about it, obviously. There's a classic thing of measure, you know, just the number of bugs and you look at the bug curve and as it starts to tail off, you think you're looking good. So there's a very long kind of soap testing period at the end. With the formal verification, you can also say you can try to come up with a property which when it's true will tell you that there's nothing more interesting to find that all the instructions have made it all the way down the pipeline, for example. And so you can kind of prove that that number is less than seven and then, you know, seven would be the right number to use. So there's a variety of techniques like that. I should actually say I'm actually in the research group. I help the people who were developing this. I did part of the work in developing this formal verification technique, but I'm kind of a fake. You know, the guys who actually do the verification of real processors, they're the heroes I was talking about, not me. Do you think that the recent speculative execution issues have put a greater focus on the formal verification stuff? Have you seen a shift in sort of day to day as a result? I think we'd already reached a point where the traditional testing was kind of at its limit, you know, and it's really important a processor works. If the processor in your phone has a problem, then, you know, we can't change all the software in the app store, right? And it's really expensive to recall phones. So they really have to work. So we're already kind of, you know, an old processor vendors are doing this, you know, pushing really hard for really high quality. It's maybe pushed it a little. I've got an idea of how to use formal verification techniques to look for particular kinds of security holes I'm thinking of applying. But yeah, it's maybe pushed us a little, but we were kind of already there. So how many cycles, subuse cycles, can you actually verify these days with bounded model checking techniques? It varies with which processor you're working on. So I was working on a little two and a half stage pipeline. And there we sometimes get like 80 cycles deep and then we'd kill it because it clearly was never going to find anything else. Others, when I was working in the project, we get stuck at about maybe 10 cycles, 12 cycles. It varies. And what does a cycle mean? Like this processor here is executing two instructions of a cycle. So if it gets, you know, if you can get 10 deep, that's you done 20 instructions. But then again, it's more likely to have problems because it's so complicated. So maybe you really do want to get 20 cycles deep. So in your experience, has a problem got out into the wild and having analyzed that problem when it's out in the real world? Have you had to modify your formal verification techniques as a result? Sort of what's the most embarrassing incident where this has actually happened? So I think we applied this to, we were doing it about three years ago that we were developing this kind of and then that was kind of tailing off over the next year or so. So it's maybe about two years ago that the earliest processors we did this to were finished as far as we were concerned, it then takes another year or so for something to make it in silicon, maybe another six to 12 months for it to actually kind of appear in a package that you can actually buy. So we're about to find out, I guess. Yeah, we do know it finds a lot more bugs. It finds them much earlier, which makes it cheaper to fix them because the cost of fixing a bug goes up exponentially, of course, with the length of time it sits in your code base. So we find them much earlier. We find bugs which we're not sure we would have caught by testing. So we're very happy with it as a technique. But yeah, I think we're still learning how to use it even now. I think that's all the questions. So thank you very much indeed. Else to read. Thank you.