 Determine this pleasure and the big honor here to introduce you to Clifford for the most of the guys who know him and the other guys who don't know him be warned because this man is like a bullet train. So the moment we press that button start he starts a race and you'll be careful. So fasten seatbelts ladies and gentlemen here's the man. Thank you. Yes I'm diffuse put it in play. So okay this is my talk about former verification with Joses SMT BMC and it's the this is my talk about former verification with Joses SMT BMC and it's a shorter version of my talk but I still need to speak very fast. So if you're interested in the full slide set go to this link the link will be also on the last slide of this presentation so you don't need to like write it down right now and you can also download all the example files for all the examples I'll show you during my talk from this page and my goal is really to just give you a rough idea of what you can do with with Joses SMT BMC and then you can download this example files reproduce the stuff that I've been I will be talking here and then modify these examples to fit your your verification needs whatever whatever you need to do. Okay but really quickly first a little bit about Joses SMT BMC as a whole because there are a lot of things you can do with Joses besides former verification so there are a lot of synthesis flows that are available in Joses so most prominently we can do synthesis for ICE40 FPGAs I had a talk last year at CCC about that and I had to reverse engineer the ICE40 bitstream format in order to enable this flow but there is also a synthesis flow for Xilinx 7 series FPGAs here you need to use Xilinx Vivado for place and route but the synthesis part can be done by Joses there are also a couple of ASIC flows that are enabled by Joses you can also do your custom synthesis flows which is something that some people do on the formal verification side we have Joses SMT BMC which is the thing that I will be talking about today but there are also some other verification flows available in Joses for example there are a couple of built-in commands in Joses directed that can be used for certain verification tasks there's a built-in SAT solver that you can use for some things there is an equivalence checking framework or you can use Joses just as a frontend to convert your design into something that can be understood by other tools for example you could use Joses as a frontend and combine it with ABC and do verification in ABC but I think I'll be talking about is the first one Joses SMT BMC a framework for a bounded model check and temporal induction enabled by Joses it can be combined with any existing SMT lib2 solver so that there is not a lock in here to one specific solver you can combine it with anyone because I'm using the SMT lib2 language to communicate with that solver and there are a couple of solvers that are well tested so usually when you read here supported solvers it means well tested solvers because in principle it should work with any solver that supports this file format and those well tested solvers are for example set free CVC for Yikes and so on good but before we dive into a former verification I have this slide and this is my why you should be excited about this slide so I have some thoughts here about availability of AIDA tools for students for hobbyists for enthusiasts who just would like to experiment with the technology all that kind of stuff and on the FPGA synthesis side there are a couple of free-to-use tools so if you're interested in playing with this kind of technologies you can just download the free stuff from Xilinx or Alterra or other vendor and install it and maybe within an hour or two if the installation goes without any major difficulties you can play around with it and you can start making your experiences with this kind of technology and with FPGA synthesis there are even some free and open source options so there is the user's flow with Project I-Storm for I-S40 for example but this is not the only one there is also VTR with Odin2 as verilog framework so there are a couple of options there. The same thing for HDL simulation if you would like to simulate a circuit or if you would like to make your own experiences with FPGA HDL simulation you can download some free-to-use tools like Xilinx Xim and there are even some open source tools that you can download and start learning about this kind of stuff just within minutes however it takes how long it takes to install Icarus verilog using upget or something like this. However on the former verification side we have a completely different picture there is practically no free-to-use tool besides the things I'll show you today no free and open source tool that you could use to do end-to-end former verification and interestingly at the same time people in the industry are complaining they can't hire anyone who has experience with former verification and I really wonder why why this is so why there are no people with former verification even though the industry demands them so I think this should be the the focus of that I can't really tell you stuff like how fast is my flow compared with the commercial flows because I don't have access to those commercial flows I have had no way so far to compare my stuff with the commercially available stuff because even though I do some research at the university institute and I have some commercial software there I have nothing that's comparable on the former verification front so I think it doesn't really matter that much if my tool is faster than the commercial tool if it supports exactly the same feature set the really important thing is that this is something that enables you to make experiences with this kind of technology and hopefully one day I will be able to compare my work with the commercial tools and then I will know if I'm better or if they are better or if they're about equal okay so when we're talking about former verification what are we actually talking about and in almost all our presentations about this kind of stuff you will find a slide that looks more like like this one where we have a couple of states your circuit can be in and the couple of simulation traces so the simulation traces are the I don't know what what color this is but this here is a simulation trace um oh you don't see it this here is one of those simulation traces um and you see we have three simulation traces here and they cover a subset of all the reachable states interestingly they don't cover this red state and the red state is a reachable state that violates an assertion so when we would like to verify this circuit we would like to be sure that we have caught this red state that we have shown that this state is actually reachable um one one way to improve the coverage here would be just to run more and more random simulations right um however if you look at this example really carefully you will see that there is only one possible path that goes to this one one red state and one of those traces almost got there but in the last minute it decided to go a different route within the state diagram um and if you just multiply the choices you have at each node on this path you will realize that you could run uh a thousand simulation traces on this really really small example and you still only have about 50 chance to uh to get to this one red state that violates your assertions so random simulation is not really a solution if you would like to make sure if an invalid state is reachable or not you have to use former methods and the former methods actually um like look at the whole state diagram as as as a whole so it's not simulating many many individual traces instead it will give you a definite answer if this state this red state is reachable uh or not in in this situation um and when you simulate it a lot you can be fairly certain you can be more certain if you simulate for many many hours uh but only with former verification you can absolutely be sure that no assertion can be violated uh in your design so that's that's what we're talking about when we're talking about former verification so why would we do this why would we want former verification and uh in almost all cases where people use former verification they have the first use case they would like to prove that this that that design is correct and in many cases this this is a really good reason to do former verification i just told you uh when you have to be sure this is the only way um how you can can achieve that um however proving that the design is correct even if you have a former verification tool is a pretty hot thing um so first of all um you need a formal specification of the correct behavior if you don't have a formal verification of the correct behavior how can the tool know which is the right behavior and which is the wrong behavior um and usually just this one step creating a formal specification of the right behavior can already be a showstop but this could be a project so large you you wouldn't think it's worth uh your your while in in in many applications so maybe if you uh in in some space applications you can't get around it uh but uh in many other cases people might say well if there is a problem in the field i'll just create a new fbj image and we will figure out how to how to update uh the devices in the field however there are two other big big application domains for a formal verification um there is back hunting and there is optimization um and i think usually uh we don't talk about these two uh that much and i think the reason is that the pricing on the commercial tool is so prohibitive so you're only uh buy those commercial tools if you really really have to and the only case where we really really have to is if you have the first case if we need to prove that the design is correct um and many projects might benefit from having formal verification around for back hunting optimization but they don't do it because it's not on the radar um so what is the difference between back hunting and proving that the design is correct so first of all for back hunting you only need a partial specification if you just have specified some part of the circuit you can use the formal tools to figure out if the circuit at least conforms to this partial specification maybe there is other stuff that is not specified with this partial specification but if there is a bug that is somewhere within the specified uh domain then then you can use this um the other thing is when you do back hunting you can do many different checks with different overlapping partial specifications um and you only need to have a really good feeling about that all these overlapping specifications really cover the entire uh behavior of the circuit whereas when you won't like to prove that the design is correct and you would like to fracture it into individual small checks then you have to have a second proof that proof that you haven't left any gaps with these partial specifications and there are cases where people do that and they claim they do the first thing but actually they don't because this these are questions that are left open if you do that so this is just back hunting but it can be really beneficial to have those tools for back hunting usually they will find bizarre bugs really really obscure bugs bugs that you would never find using handwritten test benches because nobody comes up with the crazy things they do with your design in order to trigger an assertion um so not only will it find bugs it will find those bugs that are really really really hard to find in the field the bugs where you have a processor and it works really fine and after a year that is in the field someone comes to you and says well there is this one dimp plugin and if you use this binary for this gimp plugin and you run it an entire day then it will somehow crash the processor at one point and this is like the thing you get as a bug report and then you have to figure out how to go from there to the one obscure thing in your instruction cache that went wrong and that was triggered by this one scenario um so the uh but if you find the same bug here the tool will give you a counter example will tell you well if i uh if i if i use the inputs of your circuit in this way then the circuit will go into this state and here you have said this state is an invalid state that should not be readable so you're in a much much better position to to figure out what's actually going wrong even if you could already reproduce the bug uh in hardware uh you would be on the very different standing here so so be the hunter don't be the hunted uh when it comes to two bugs um with optimization we essentially do um the same thing as we would do when we prove a design is correct but we just use the unoptimized design like a spec so we have already a design that works and we know this is the this the behavior we want and now we would like to optimize the core and usually when you optimize the core you'll make it a little bit more complex so if you would like to optimize for power for example you would add uh uh clock gating in certain cases and then uh you have to put additional thought into which uh flip flop stage can be clock gated gated in which uh cycle and if this if there is a problem somewhere then you would uh find this by formally verifying your optimized design using the unoptimized design um as specification um in this case this would be very very similar to formal formal equivalence checking however there are cases where your optimization reduces the number of cycles needed to produce the same result uh and then we are already beyond what formal equivalence checking would do because now we have two circuits that are formally not equivalent because one circle is circuit is producing the result one cycle earlier than the other one but we could still construct the formal proof using uh property checking to make sure that the result both circuits produce um the same okay i'm not really going to talk about the things on the slide here uh this is just uh um to to show you it's really easy to install this on upon to any db and also other distributions of course um so you really have no excuse uh not trying this out if you do anything where formal verification could be uh could be useful it's not like you have to spend an afternoon compiling fortran code from the 70s uh in uh in instead you just up get some prerequisites and then you build users and then you build a set uh an smt solar uh and after 20 minutes or however fast your uh your computer can build large c process bus projects um you have everything in start and you can um playing around with the examples um in this presentation um so uh the first link here is the same link we had on the first slide this is where i can download the slides and also some example files uh some example files uh that can also be interesting uh are the examples in examples slash smt bmc within the user's source code um and if you know my pico are within the two project it's a uh risk five um implementation uh there is a couple of formal stuff there as well and it could also um look look there for more examples and and crazy things you can do uh with this formal verification flow okay let's let's get right into it some code um this is a simple helovard example so we have a counter we initialized the counter to to zero in this case um and then we have a clocked block and we say whenever we get a rising clock edge when reset is high then counter will be set to zero and when reset is not high then we will increment the counter um and then we have some formal statements at the at the end of this input so we assume that the counter will never reach the value 10 and then we assert that the counter will never reach the value of 15 and since we are incrementing in steps of one we never can go to 15 without going over 10 so indeed this uh should hold um to verify this we call these three commands uh on on on the lower right uh the first command is users so we use users for synthesis of this circuit but we don't synthesize this for an fbj or an async architecture instead we synthesize it into an smt2 description of this circuit and then we use user's smt bmc to check if um if the assertions are right um and we have two calls here the first one does a bounded model check so we check if the circuit um um if all the assertions hold within the first n cycles the default value is 20 and the second call tries to perform an induction step to prove that in fact it doesn't only hold for the first n cycles it holds for infinitely many cycles it holds forever this is what we would like to do in most cases we would like to do that something holds forever however in in some cases with very complex designs this can be hard to achieve then you are in troubles if you would like to prove that your design is correct under all circumstances but if you are for example back hunting you could still say well a bounded model check for 100 cycles gives me enough competence uh confidence that uh that actually this design is is correct and works as expected because of my understanding of my design i know within 100 circles it should reach all the reachable states anyways so when you run it uh you you get this output um and the first one is for the bounded model check so we just check other assertions hold the assertions in time step zero then after one state transition do they hold in state one and so on until we reached state 19 and the other one is the uh induction step and the induction step counts backwards from from 20 and if it's very easy to prove something by induction i will talk a little bit more about this later in the presentation then this will maybe try to improve induction in step 20 and then in step 19 and we'll say well it passed everything is okay in this case it's a little bit harder and we need to go down all the way to step 15 before the induction um works and we will see a little bit later why this is okay um so who who here knows a little bit of verilog oh that's good it's at least at least maybe maybe 30 percent maybe maybe half of you that that's great um so for formal verification you need to have some additional statements uh in the verilog language um namely you need to have a way to create assertions and you need a way to create assumptions and we've seen those assert and assume statements already in in the hello world example i just showed you uh before um so when you have assert some expression it just says we say this expression must always be true and if there is a way to make this expression false then this is a bug and the tool should tell us how to get into the stage to demonstrate the reachability of this bug uh to us um and assume we also say this expression is always true but we don't say it's an inherent property of the design we say this is something we um we restrict our proof to so in some cases you might say we assume that this input signal is only always low or we assume that those two input signals are never high at the same time maybe because this is some kind of a protocol and we assume that the the other side of the the protocol the peer we talk this protocol to uh is uh is using the protocol correctly and maybe in this protocol those two signals can't be asserted at the same time uh restrict for verification essentially does the same thing we say we don't want to consider traces where this expression is is false we only want to consider traces where this expression is true um however there is a little uh difference in in the circumstances when you use assume and when you use uh restrict um and assume you use when your assertions depend on it when you say well if this assumption doesn't hold then i really don't care about the assertions because i know if for example these two lines are asserted both at the same time then my core will go into some crazy state because i wrote my core under the assumption that the peer i'm talking this protocol to will will actually adhere to the protocol specification with restriction we we we essentially say um this makes the proof easier uh if we assume this expression but it's not really necessary for our uh assertions uh to hold um and because of that when an assumption is violated simulation will also produce an error but when a restriction is violated in simulation then we don't care because we've said the assertions don't really depend on it it's just an additional thing for formal verification um that makes the proof a little bit easier there are immediate assertions and concurrent assertions in via haloc so uh immediate assertions are within a behavioral block like initial or always and there we can use assume and assert like like a normal statement um concurrent assertions are used in modular context um and essentially assume or assert that uh something will hold forever in all states whereas in the behavioral part we could use an if statement to uh to to add an addition a condition to an assertion or an assumption uh and then this assertion assumption would only be need to be through when the if statement um also works usually the concurrent assertions in veriloc are used with system veriloc properties that um can be temporal properties but right now yoses doesn't support any of that uh so and because of that uh we could say um when we are using yoses when we are using this flow then usually we will use immediate assertions within within an behavioral block like initial or always okay formal test benches um so in simulation we usually have a test bench that creates input signals for our circuit and stuff like that um with uh formal verification we also sometimes use test benches however those test benches don't uh uh play through one possible path with with our circuits like a normal simulation test bench does instead um we um um we just add some external restrictions or assumptions uh for stuff like well uh research should be lower in the first cycle things like that uh but other than that we we leave a lot of uh free room for the verification tool to actually explore different different parts and and uh and see if using the thing in the formal uh test bench it can can reach an invalid state so this is a variation of uh of the the sign we had before the hello world uh design but now using your formal test bench you see i've removed the initialization for the counter so now we only have these always blocks where we say well if reset is high then we set count to zero otherwise we increment count um and in our formal test bench on the right we assume that the reset signal will be high initially in the first cycle so because of that we we we reset the design from outside from the test bench in the first cycle um and then we have the same assumption and assertion combination we had before but now with a condition around it that we don't check it in the initial state um so in the initial state we don't care what count is because it can't be anything um we just wait for the reset to complete and after the reset has completed uh when we are not in the initial state anymore then we actually apply our assumption and assertion okay uh this is a slide i thought i removed yeah um so bounded and unbounded methods this is this is terminology we'll hear a lot when when when reading or talking about a formal method a bounded method essentially says we check that the property holds within the first n time steps um and we had this before here when I just call yours as SMTBMC without this dash i parameter then we run a bounded model check for 20 uh uh cycles and an unbounded method is we would like to prove that the property holds forever and not only for a limited number of cycles after the initial state um so BMC which is part of the yours dash SMTBMC name is a bounded model check that's a bounded uh method and temporal induction is a mechanism that uh that builds in a way on a bounded model check uh and extends uh the validity of the of the check for um forever um let's have a look how how we can do that that's the bounded model check uh so we create um uh a former circuit within our tool that more or less looks like the example uh at the top here we have an initial state in yellow and then uh three uh uh states that follow the initial state these are the green states here and for each of those four states we check if all the assertions hold so this is the the naive way of creating a bounded model check remember this now are like former states they're like variable they're like place holders for all the states that are possible the only thing that we constrain is the first one should be a yellow one it should be uh an initial state and the others uh don't need to be initial states they're only constrained that there must be a valid state that can follow after uh the an initial state in case of the second and so on and and this works so this is a correct way of making a bounded model check for for n equals three time steps uh but it's not very efficient and in in practice it turns out it's much more efficient to use the scheme on the bottom half of the slide we first just create a symbolic initial state so a symbolic state that can represent any initial state and we check is there any initial state that violates one of my assertions and if there is such an initial state then we are done i just output this state and say well obviously this initial state violates your assertions so this is your counter example um if uh this does not succeed in the way for the tool because the tool would like to find the bug uh so the tool doesn't find the bug here we figure out that all the initial states actually conform to the assertions then we create a formal model for two states um and now we can we can assume that all the assertions in the first state will hold because that's what we just proved in the first state um uh yeah and and and so on so if you find something here we output a counter example that's two cycles deep if it doesn't count well we move on to the to the next state uh next step and make it prove that's three uh uh cycle steep um and so forth and in practice this turns out to be much much more efficient than the naive approach where we would like to prove like everything at once especially if you would like to do a deep bounded model check there's like 100 cycles deep something like that so temporal induction now takes one of those bounded model checks so now we have one of those bounded model checks uh above it's the naive implementation but this way it doesn't matter here um and then we create the proof that you see in the middle of the slide here um so initially we say there is an initial state followed by three other states and none of them violates any assertions and we have proven that and now we would like to prove that when there are three states consecutive that don't violate assertion then there is no way for a first state to violate assertion and if this if this works if we can prove this we have obviously proven that the assertions will hold forever because this here kind of fits together so we can just say well we have proven one two three non-initial states uh so this exists and if this proof holds we know every state that follows the third state here must also uh conform with our assertions and then i just take this proof that i have made and i don't need to change this proof i don't need to rerun this proof just think of moving this one step to the right and i can apply the same argument again and again and again and therefore i have proven that my properties will hold forever so this is great however it doesn't always work um so if if this kind of thing works and tells us well your design seems to be fine um then uh uh then everything is good but this might tell us i can't i can't do this i can't do this proof i find a counter example and now this counter example can mean well maybe my assertions are violated somewhere deeper in the state space or it can mean well maybe um uh uh maybe everything is correct but i just need a deeper induction proof maybe it doesn't work with three time steps maybe it only works with 20 or 100 or maybe it doesn't work at all um and that's what we see in this picture here so um here again we have the states that violate assertions in red uh then we have all the reachable states in this bubble here so if you look at the direction of the arrows you see there is no error going out from this either bubble to the outer bubble so these actually are the reachable states in the system however usually we don't have an explicit representation an explicit picture like this of all the reachable states um and then we have this uh this intermediate zone here um where there are states uh that are okay with our assertions but nevertheless unreachable and when we look at this proof that we are trying to do this here we don't we don't know if these states are reachable or not um so we could for example find here one two three states that are okay with our assertions followed by a state that's not okay with our assertions so if we find something like that uh the easiest thing we can do we can increment the induction length so here we need at least five consecutive states because four are there but if you say we have proven for five consecutive states that are okay with our assertions um then then we won't get a count example here however we might get a count example here why because there is a loop and in this loop I can fit an arbitrarily long sequence of states that are all okay with my assertions and then in the end I break out and reach this state here so here I can't just make the induction longer I have to use something else uh for example I have to add an additional assertion to my design that invalidates one of those four states that built the loop and if I do this uh then I can can prove this example by induction because I don't have this this this loop there anymore um so uh when you would like to do this when you would like to make such a proof uh this is the entire flow you'd you'd be using you'd use you take your viagra lock design your assertions you put them through joses joses generates an smt tulip model then we put this into joses smt bmc together with an external constraints file uh pass it to the smt solver and then I get a pass fail information and if it failed I get a count example with a vcp file or a viagra lock test bench that reproduces the count example or a constraints file that I could put back in uh on on the left side for example if I would like to just easily reproduce uh um a thing that I found before so the typically workflow would be we first run a bounded model check that's the easy part we say for 10 steps for 20 steps see if you can find the violation of my assertions within 10 or 20 steps um and if this is uh uh if this fails then we know we have a bug we know we either have a bug or one of our assertions is bogus this is of course also um an option um so the solution is either fix the bug uh or add additional assumptions or loosen the assertions so that everything is right um if this passes then we move on to step two uh and step two is this this temporal induction step um and if that um fails then we have to count investigate the count the example we get and make a decision is this reachable what the solver is showing us or is this not reachable uh this is our judgment call essentially and if it's reachable then the same thing as before we have to fix the design we had have to add assumption or we have to loosen assertions however if it's unreachable then we have to add additional assertions usually to tell the solver that this is actually an unreachable state um and that's uh that's the kind of dialogue you have with this tool while formally verifying something the solver is giving you a count example and you have to explain to the solver well that's actually not reachable because this condition will never become true um and then the solver will probably give you another count example and so forth um the nice thing about this is um when it passes we know everything is right but we still have two options we can say we we are fine with the thing we have we have proven that our design passes verification we are done but we could also say I would like to have more assertions in my design and then I just reduce the induction length and I will get another count example now for uh induction with uh with uh a smaller induction length but now I will know that every count example I get is unreachable and it's much much easier to uh to work with this and if I make a mistake uh and then add an assertion while uh telling the solver that something is unreachable well the solver will then tell us here's an assertion that can be violated using this this path so there is really nothing you can inherently uh do do wrong um yeah the idea would be to talk about this flow again with this graphics but I will skip it so um we have a variation of the design we had before we have a counter again but uh this kind this time we have a mode uh input and the mode input tells us if we should count up or if we should count down um and uh we assume that initially count is five and we assert that it can never be 200 uh and if we stare at this for a little while we see this is true because when it's the count is 99 we will not increment it any further so there is no way to go from from five to 200 also there is no way to do it with another flow under flow because if the count is zero we also don't decrement it anymore um and if we run this uh the bounded model check will work however temporal induction will not work it will give us a counter example like this where it starts with uh 194 and then increment and increment and increment and decrement and so it will do some things here really really close to 200 and at the end of the count example we say and then I can increment from 199 to 200 of course the thing is uh the reason why we get a counter example like this is because we haven't really told the solver that count equals 194 is not reachable either so if we modify our design like this and we add an additional assertion where they say count is always smaller than 100 uh then the proof will work and we will have proven that all these assertions actually um hold and because in the example I had first I was only incrementing uh and I was saying we can't reach step 10 but we assert so we assume we don't reach step 10 and we assert we never reach step 15 because of that I had to have 15 steps in my temporal induction uh five steps in my temporal induction proof from uh 15 to 20 in order to prove that now that we have uh correctly described the entire reachable state of our design we can actually uh complete temporal induction with just within just one cycle okay I have some some some other examples that I just would like to mention quickly uh now um there is uh a special command called assert pmox that adds additional assertions to your design whenever there is a parallel case statement um and in this this example we have a parallel case statement in verilog so we as a designer tell the tool uh that those two cases and the parallel case statement are exclusive and the tool might use this information to do any optimizations um and if if we are wrong about this uh this thing that we say well this the case statement is parallel then we might run might run into evil problems uh because we might for example test our design in an FPGA for many many hours and everything works fine um and it turns out the reason is because maybe this FPGA tool in this instance has ignored the parallel case but then we go to an ASIC for example and we synthesize the same thing with the ASIC tool and maybe the ASIC tool will actually use this parallel case to do an optimization and then all these hours of testing we've done on an FPGA is completely worthless um but with a formal verification tool we can of course prove stuff like that um that's another uh example I have a memory design here then I have an other memory design that's kind of convoluted but sometimes code looks like this then you would like to force a synthesis tool to infer a special form of of hardware resource and we would like to prove that these two memories are are equal and if we if we do that we create this this kind of formal test bench that just instantiates both memories in parallel and then we add this additional constrained file on the upright where we assume that in the initial state the the memories in those two instances are actually the same um and then for all the other states uh we assert that they are still the same and we also assert that they output the same uh value for the same input and the input will be the same because the input parts are connected uh to the same top level input um this is an example for using multiple clock domains with your SMT BMC so just to show you that this is something that's also possible so here we have um a counter but we implement this counter just by using the uh the lower bit we've just created as a clock input to another toggle flip flop and then we go on so um if if you're building a counter from like TTL uh ICs then this would be a possible way to to build a counter usually you don't do this on a chip um because now we have uh one two three four different clock domains here uh so the first always block is like the regular counter and then the other uh for always blocks uh um the bits for for our other counter and then we assert these two counters they must always have the same value and uh indeed I can uh I can prove this uh I need the special command clock to ff logic to enable this uh but then uh this can be done uh similarly this is another design where I have a set reset flip flop uh never use something like that in the in the real design it's it's horrible but if if you have to use it for for some reason we can we can actually um check here um and of course and you this is actually what the xilinx tools do for a set reset flip flop because they don't have it in hardware resource they create this weird thing with two flip flops and the latch and the mux um and we can actually prove that this is the right thing to do that this still is uh flip flop like this can also do custom things I'm not going to talk about them um there is a larger project going on right now trying to do end to end former verification of uh risk five processes um it's called risk five former uh so I'm working on that right now and this is also of course using a lot of of of this infrastructure uh to do this kind of verification so if I would like to look into a more complex thing that can be done with former verification and with this former verification flow then this would probably be worth a look um future work well I would like to add more uh system where I look um assertion properties support um this is just something that you will find a lot of existing code for former verification tools and it would be interesting to be able to use some of that with this flow as well um then there is improved support for virolog x propagation so in some of my other former verification flows I can actually uh model virolog x propagation exactly the way the virolog lrm is specifying it uh it would be interesting to also be able to do that with this flow um I have on my to-do list a C back end for joses so you can take a circuit and conduct into it a flavor of c that is is is good for a former verification with c former verification tools like like cprover esb mc stuff like that um and also there is a project going on called symbiosis which is like a big front end to to use all this uh joses based former verification uh flows without having to learn a new thing whenever you switch from one flow uh to another that's it so what did we learn today Clifford what did we learn today what did I learn today that I can speak very fast no no you were really quick and something you do in the normally in half a day you did in an hour yeah it's fantastic yeah we have some questions in the audience I know there's one on the web I will take this right now no that's disappointing but we have some others this man please yeah I did not yet understand what is the difference between joses and just using like uh any kind of uh verilog simulator I mean the simulators also react on asserts and whatnot yeah but you have to write a test bench the triggers the assertion um and with formal verification you don't have to do that you can just say this is the set of valid inputs um and uh there is a better way to do that so uh when you go to that with simulation you just have individual traces that may or may not trigger your assertion with this formal verification you can I have a tool that for you looks at the big picture because I can't paint the big picture I mean this is a comic but the state uh graph for a real world jacket would probably have more states than the photons in the universe so you can't really yeah someone else this I'm wondering if in your experiments you have actually run into problems with vacuity um that that your assertions hold because you have restricted artificially the state space too far yeah I mean this is possible whenever you use uh restrict or assume you you have this risk that you you've restricted it in a way uh so it doesn't really work anymore so in the usual workflow you often have some verilog defines or whatever in your design that you can use to to artificially add bugs and then you always check that at least you still can find those bugs of course this is not a guarantee but at least it improves the situation a little bit okay a question there from out of space yes so um what kind of expressions does user support in these assertions um because essentially I only support immediate assertions you can use like any verilog expression but it's not the system verilog properties syntax that you could use to uh to express temporal properties in system verilog those are not supported right now one last question this sir can you give me a rough idea of how long the formal verification needs it really really depends on the kind of problem on the solver you have uh on on many other parameters so so here I have a couple of solvers three different benchmarks and you see there is quite quite as as a span um and always um maybe this is not really can can be seen here but uh but often you have an example that it works really really well with one solver but it takes a long time with another solver but with a different problem is just vice versa and because of that it's really good that we can use joses smt bmc with almost every smt2 solver thank you okay thank you Clifford it's really fantastic to have you here and really the will that you're sharing your knowledge your explorations with all these people thank uh give him another applause please