 So, let us get going, I am very happy to introduce Kulip, I hope you already know him, he is more alumnus and he also is visiting us regularly, he is working with us. So, Kulip is currently personal PhD at Price University, he has been doing a very good work on both sampling and counting, and he is going to talk about that today. He was also awarded the Vienna College of Law, he is best master's, this is a world by competition and some of the stuff here is part of that work, I believe he was talking about something more. Alright, thanks Superthick for hosting me here, I am really happy to be here back in the institute. So, I will be discussing work that I have been doing on sampling mostly, and before I get into why sampling in combinatorial species is important, let us start with a simple example. So, alright, so I guess one of the big problem that all of us designers face is how do we guarantee that the systems we design work correctly, and this is the part of what we now call as functional verification, where given a design we are trying to verify whether it satisfies the functionality that we desire. In functional verification, there are usually two parts, one is formal verification, where the idea is to model the system formally, and then use formal tools to verify the system. About 10 to 15 percent of the verification effort in industry goes through this paradigm, and key challenges that our prohibitive going from more than 10 to 15 percent are formal. So, writing formal specifications is hard, all of us are not really wide to write complicated mathematical expressions, and also the scalability of the formal tools. So, what in last 3 to 4 decades what has emerged as the dominant approach is dynamic verification. So, what is dynamic verification? Well, we have a design, and what we do is we simulate the design with test vectors, and these test vectors sort of represent different verification scenarios. So, once we simulate the design we have the results, we can compare the results with the specification. And if the results satisfy the specifications, and we try for large number of test vectors we get some confidence, and we say in industry someone says ok well we are now ready to go. However, the big challenge here is the space of test vectors can be really huge. How large it can be? Well, let us take a very simple toy example, our computers are far more complicated than this I guess all of us will agree with this. Here we have 2 inputs A and B both are 64 bit input C's let us say some output which is again 64 bit, and C some function of A and B. So, if you want to design, if you want to test this circuit how do we do it? Well, we can try for all combinations of A and B, how many combinations it will be, since this is 64 bit. So, we had to resist 64 values of A and similarly for B. So, we end up in 2 raise to 128 possible values, really huge number, without going too much into whether some will go no or not, but this is not scalable certainly. So, this was identified in early 90s, and the approach what was proposed is that the circuit is not going to fail on every all kinds of inputs. There are some kinds of inputs where the circuit probably has a more likelihood of failing, and these kind of constraints can come from different sources. So, here I have put some constraints the idea is for you not to read these constraints, but to just to say that these do get complicated. So, usually the constraints come from designers where the designer might say well you know I have designed this circuit, but these are the parts where I do not have fully confidence. From the past experience where we have designed similar circuits, and we have seen them failing over some values of A and B. So, once we have these constraints, now solutions of these constraints represent our test vectors. So, what we want to do, so these are our verification scenarios where we want to verify if I have a similar circuit, and so every solution to these constraints represent one of the possible test vector. However, the space of solutions represented by these constraints is still huge, in real world it gets more than 2 raise to 100 to 2 raise to 150, it is still very huge space. So, we cannot still try on all combinations, but anyway coming back to our problem. So, now we have these set of constraints, and what we want to do is we want to uniformly sample the values of A and B, we satisfy these constraints. The question is why uniformly? Well, since we do not have any more information where exactly the bug might lie, uniformity is our best bet, because that will ensure that in expectation we are guaranteed to hit a bug in certain number of steps, or certain number of tries of simulations. So, let me try to formulate the problem, we start with set of constraints which come from variety of sources, we can convert them into Boolean set formula, here I am sort of abusing the notation by set I mean Boolean formulas. And now the problem that we want to solve is given this set formula, how do we sample satisfying assignments uniformly at random? This is what we call as a scalable uniform generation of set witnesses, witness represents a satisfying assignment, we want it to be uniform, and of course the techniques that we design should be scalable, because we are going to deal with really large set of constraints. So, this is not a new problem, it has applications in wide variety of areas, I have just discussed one application in verification, there are applications in probabilistic reasoning, there are applications in multi agent planning, and so because it has a lot of applications, there has been a lot of work done over now since 1980s. What I am going to do now is to categorize the work along two axis, one is guarantees which means how good them, what kind of guarantees the method can provide, specifically how good the uniformity the method can theoretically guarantee. And on the x axis performance which sort of relays how large formula the method can handle. So, on one extreme we have techniques which have really good theoretical guarantee of uniformity, there are methods based on BDD which stands for binary decision diagram, which give you guarantees of perfect uniformity, however the problem is these techniques fail to scale beyond few hundreds of variables. There was a lot of theoretical work and we sort of culminated with really a seminal work by Bellaric, Goudrich and Petronc in 1998, which I am noting by BGP, which again gives you guarantee of uniformity, but again fails to scale beyond some 10 to 20 variables. On the other end of the spectrum we have techniques which have really good performance, these techniques can scale to formula with hundreds of thousands of variables, however they come if they give it that they do not have any guarantee of uniformity. So, they will give you distribution, but they cannot say much about what are the properties of this distribution. So, some of the techniques are set based techniques where you give some random input to the set solver and expect the distribution will be good, however theoretically you are unable to give any kind of guarantee. There are techniques based on Monte Carlo Markov chain, where again if you try to implement these techniques in practice you make some assumptions, you make some modifications and you look all the theoretical guarantees. So, how bad they are as by the EDA industries, so at the start of this project we contacted some of our partners in EDA industry in Bay Area and we asked what is it you want. So, one way to characterize that we settled on one style which is that let us take a simple set solver. So, if the run time for simple set solver is one unit, they are happy if the uniform generator takes about 10 times more time than a simple set solver. What I would like to emphasize is that this problem is in second level of polynomial hierarchy, so certainly far harder than just doing a set query. So, of course, some degradation is expected and this is what EDA industry seems to be happy with. In 2013 the state of art generator which had some sort of guarantees was XR simple prime and its run time was about 50,000. So, very far from what EDA industry expects. So, on typical examples, what is the ratio of the size of solutions of these constraints to the size of the entire set solver. So, that ratio is very small. It is very small, but is it like 2 to the minus 3 or something. It is even worse than that. So, you usually have formula with 100,000 variables and so the total space is 2 to 100,000, but you have solutions like to 200 to 200 people. So, the ratio is very small and that is why some of the techniques is just randomly pick one of the assignment will not work. So, over last two and a half years, we have been trying to get to what our colleagues in EDA industry design and we proposed a couple of techniques over last three years. We initially proposed algorithm unit width. We improved it last year in Unison and this year at TACA as Unison 2. I will discuss the whole journey and what has been sort of the theme across all these works is playing with the independence and I will discuss in more detail what exactly I mean by independence in different scenarios and also using parallelization to achieve the desired performance. And finally, I will conclude with what has been the lesson so far and some future work that I think will be excited. So, what is the main idea? So, let us consider this circle to be space of all possible assignments and do not represent satisfying assignments. So, if this set of satisfying assignment was small then I could have just enumerated everything and pick one of the satisfying assignment randomly. However, that is not feasible because we are talking about really huge spaces. So, what if I could partition this solution space into small cells such that every cell has roughly equal number of satisfying assignments and the number of satisfying assignment in every cell is small enough that we can enumerate all of them. If I can do that so what do I mean by how small it should be well we know that if it is too large then it will be hard to enumerate if it becomes very small when the number of satisfying assignment are just 1 or 2 then the variance gets high because you might either have a solution or not have a solution. So, to categorize small we have two parameters high thrash and low thrash which depend on how good uniformity you want. So, high thrash is the upper bound on the size of the cell low thrash is lower bound on the number of satisfying assignment the cell should have. So, for example, in our experiments low thrash terms are to be 11 high thrash is about 60 for the desired uniformity we want. So, what we assume is access to a set solar at the back end so we can just ask that solar you know if the number of satisfying assignment is less than 40 you just keep asking you say give me one more give me one more satisfying assignment. So, all of these techniques will be what I will be proposing these techniques you set solar at the back end. So, we already assume access to set solar. Yeah. So, that is our NPO. So, set solar returns to the entire set of. No, it returns only one. Only one. Yeah. But if you already got in time you can ask for any length that is different from the process. Yeah, that is very simple and you can just say that do not give me a satisfying assignment and it should not be one of these. So, going back to your idea what we want we want to partition this solution space into small cells we categorize the small cells by that the number of solutions should lie between low thrash and high thrash two parameters which depend on the uniformity we want. So, once we have this we can pick a random cell remember it has it does not have too many satisfying assignments. So, we will be able to enumerate all of them and then we can pick solutions randomly from this. How many to pick is going to be the key? So, here I sort of gave you the what is going to be the final but that is what we are going to discuss. So, this is how we end up finally, but I will go back in time and try to discuss different ideas of how many solutions we can pick from. But before we go there there is a key question which is how can we partition into roughly equal cells without knowing the distribution of solutions. There is a good question and the answer to that comes from the techniques of universal hashing that were proposed by Carter Beckman in 79. So, what is universal hashing? So, the traditional sense of hashing that we have is we have a hash function that maps from one space to another space. In this case we use n to denote the number of variables. So, our space of possible assignment is 0, 1, s to n and m is the 2 raise to m is the number of cells we want. So, this is the space of the cells. So, what we are doing is we want hash function that map from 2 raise to n to 2 raise to m. So, traditional hashing we know that if we have random inputs then all the cells are going to be roughly equal and but this is not going to be the case in our case in our problem. In our problem we want to have access we want to know what kind of formula that we are going to deal with. So, this was again observed in different context in cryptography and this led to the work on what is called universal family of hash functions. The idea here is to take randomness away from inputs and put it in the choice of hash function. So, what we do we start with a family of hash functions and we choose one of the hash functions randomly from this family. And this choice of randomness ensures that for any arbitrary distribution on inputs or on solutions all the cells are going to be roughly equal in expectation. However, this is still not good enough in our case because being roughly being equal in expectation does not guarantee that the number of solutions are going to be between the two bounds we were looking for. Again this was also observed in cryptography. So, this led to work on what is called as R universal hashing. So, for what is R universal hashing? So, this universal hashing guarantees that each solution is hashed uniformly. For R universal the guarantee that we have is that every R subset of solutions is hashed independently. So, what do I mean by that if we set R to be 2 and we take two solutions y 1 y 2 and two cells alpha 1 alpha 2 then the probability that our hash function that is randomly picked from this family maps y 1 to alpha 1 y 2 to alpha 2 is basically the independent event. So, this is just the product of the probability. What it intuitively means is that if I if you can pick a solution y 1 I will tell you where it gets mapped to you will still have no idea about any other solution where it is going to get mapped. To implement R universal hash functions we require polynomials of degree R minus 1. So, as you can see the stronger guarantee we are going to get we will also require to have higher degree polynomials. So, what is this independence matrix in our case? So, for this I am going to give you just a brief idea about some channel of bounds. So, what is going to be the theme in all these techniques is that remember we are going to pick a random cell and then we define some random variables. So, the random variables we define is we said to one if the solution if a solution is in the cell otherwise it is set to 0. And then we can use this channel of bound which is that if we have a set of R y's independent variables then let I be the sum of all of these variables. So, and then the probability that I is not too far from the mean value is. So, the probability that it is very close to the mean is lower bounded by some number well in this case the other way it is not too far off is upper bounded by some number which is c raise to minus r. So, what we are going to have is I is going to represent the number of solutions in our randomly picked cell and this is going to represent how much we deviate from the mean value. So, if we go back to your idea what we wanted if all the cells have roughly equal number of satisfying assignments. So, that is going to be mu minus delta mu and mu plus delta mu and. So, higher r gives us stronger probabilistic guarantees. So, I let everyone so now going back what we have we have high universality that gives a strong guarantees which is good, but at the same time it requires polynomials of a really high degree which is bad and this is going to be the trade of that we are going to play with. Because the idea of partitioning is not at all new this was initially initiated by Cipser in 83 and then for the work by Belare Gaudres and Petron. So, what finally Belare Gaudres and Petrarch short is that if you choose a n y universal hash function then you can guarantee that all the cells are going to be small. So, the number of solutions are going to be between the two bounds for all the cells and this is going to be true with high probability and this allows us to allows them to give the probability of uniformity. However, we want to deal with formula with and it is the number of variables. So, we are again going to deal with formula with hundreds of thousands of variables. So, it is just not feasible for us to have polynomials of degree let us say 100000. So, the question we are dealt with was what if we lower this requirement to simple three universal hashing can be just implemented by XOR simple linear hash functions which is something sets all those can deal with. But the problem it proposes is that all cells you can show that if you just use three universal hashing all cells are not guaranteed to be small and this is true with high probability. So, the question at no point all the cells will be small and this will be true with high probability. Some of the cells will be small, but at no point we will reach a stage where all the cells are small. That is a good observation and that is what we observed. While all the cells are not going to be small the majority of the cells are going to be small. So, what we did we said that well we are only interested in a randomly picked cell and this randomly picked cell is guaranteed to be small with high probability. So, I know the probability that all the cells are small at some point is very low, but since the at the same time majority of the cells are small with high probability. So, what you can do is that you just pick a cell and you check whether it is small and this event is going to be true with high probability and this allowed us to give the guarantees of almost uniformity. So, what are all what is almost uniformity? So, we started with uniformity what do we mean by uniformity? If y is the solution we want that probability y is output should be 1 or r f. What is r f? f is the input formula r f is the solution space cardinality of r f is the total number of satisfied cells. So, using any universal hashing it is possible to give the guarantee of uniformity. What we have been able to show is that using 3 universal hashing we have a guarantee of that probability divides within the 1 plus epsilon factor. This is still strong enough for almost all the practical applications because what it proves is that we are not going to miss some solutions trivially. If you run all the solutions have not revealed probability lower bound. Again the metal unigen lining the the number of set calls we wanted was polynomial number of set calls. So, one thing that I did not go too much into detail is what is a cell? Well a cell in this case is a conjunction of the original formula and since our hash function can be just represented by a set of XORs. So, it is conjunction of the formula with m random XOR constraints where 2 raise to m is the number of cells that we desire. This formula is since usually the input formula is given in the form of c n f. So, what we have is c n f and XORs and cryptominus set and the handles these kind of formulas that really allows us to get to large formulas. So, how well we did in performance to do this we compared unigen with XOR simple prime. I am presenting only for a subset of benchmarks we have companions over 200 plus benchmarks. So, on a Y axis I have the time on low scale on X axis a subset of benchmarks. What we see is that we beat XOR simple prime comprehensively even in the in the cases where it is able to almost since the time was 1800. So, and lot of cases is just time solved in the cases where it is able to do something unigen is still 2 to 3 orders of magnitude faster. What I would like to point out is that XOR simple prime has very big guarantees compared to unigen. So, where are we with respect to the scale in which we wanted to measure ourselves. So, the mean relative run time comes around to be 450. So, we did pretty well compared to the state of art as technique, but still not to the point where we wanted to go. So, how can we go to the or a holy grail of achieving something that EDM industry wants to do this we went to the next step which is independence among samples what do I mean by that. So, before that we have quick question. So, how many solutions are generated per sample? Remember the technique the answer is greater than loss phase what why because we pick a random cell we check if it is small and then we pick a solution randomly from. So, we generate up to loss phase number of solutions and we pick one solution randomly and number of solutions in a small cell is between low threshold and high threshold. So, we generate every such event at least require generation of at least low threshold number of solutions. We pick low threshold number of solutions instead of one solution from this set it is not so easy and why it is not we will have to go back to hashing. So, what we are using is 3 universal hashing. So, again what it means is that every input is hashed uniformly and every 3 solution set is hashed independently, but the same does not hold for a set of 4 solutions. Why does that matter? Well if you were choosing up to 3 samples then we could have guaranteed that there is full independence among solutions and the question is why do we first why do we care so much about independence. So, we care so much about independence because our event or motivating goal was to achieve coverage which is to hit a find a bug quickly. So, think about this being the set of all solutions dots again are satisfying solutions and the buggy inputs are within this circle the blue circle. So, if I if I pick this solution independently every time and uniformly then you can see that we are going to hit the buggy set in expectation which is going to be the number of solutions in the buggy set divided by total number of solutions. If we do not have independence and suppose I generate this solution first and then my next solution is not generated independently then we might go in some path while missing this completely for a long time. So, every time when we do any kind of coverage proofs we require consequent runs to be independent and in this case we would want our samples to be independent. What is interesting and what we were able to show is yes choosing low thrace number of samples makes you lose the full independence guarantees, but would as we were able to show you that it still has something called almost independence. So, while not complete independence the probability is still have lower and upper bounds with respect to. So, if probability of a and b is lower and upper bounded by again some epsilon factors and this still provides theoretical guarantees of coverage. So, I will not go into a proofs, but I will be very happy to talk about after the talk, but in that shell what we had if l is the number of samples that we generate then so again for a solution y belonging in the R f where i is the input formula R f is the solution space probability wise output is lower and upper bounded, but there is a constant in the numerator which is there. So, where l is the number of samples it is something to observe this is bigger than what we had earlier the earlier guarantee implies this, but not the other way. However, this is pretty strong enough and what is more interesting is that we can reduce the number of set calls per sample from polynomial to constant which is a big reason why well the our original motivation was being able to guarantee the coverage. So, we wanted to hit the bug with high probability. So, let us call that the bug frequency f which is the number of number of solutions or number of samples that are going to trigger bug b over the total number of samples. So, while going while doing all the computation we can arrive at two theoretical expressions which is the relative number of set calls we will have to do to hit the bugs with some probability where nu is its function of the probability that we want to hit with and nu prime again has some constant. I really instead of going very much into the what does every term mean to simplify the number of set calls that we would require for unison 2 is far less than the number of set calls that we require for unison. And this is very much important for us because we use sets over at the back end and sets calling sets over is both takes the most amount of the time. So, how do these number look in practice if bug frequency is 1 by 10 raise to 4. So, every one input in 10,000 every one sample in 10,000 samples is going to trigger a bug and suppose we want to find bug with probability at least half then expected number of set calls for unison is going to be 10 less 4.3 into 10 less to 7 for unison 2 it is 3.3 into 10 less to 6. The difference between these numbers increase as f decreases which is good because as we do more and more interest in our verification the probability that more samples are going to trigger bug is going to go at smaller. You get your smart phones not to have bug in every 5 classes you do. So, even in this case we have an order of magnitude difference and as it goes further and further in the verification cycles where the bugs are going to be less and less likely the difference goes more and more. So, unison 2 our performance is going to be less and completely by more orders of magnitude and number of set calls. Well that is what yeah what. So, we will go back and compare how well we do in terms you know we have the metric which is how much load on we have for unison 2 and it will be interesting to see where do we stand here. Before that there were few more modifications that I did not discuss, but what we were allowed what we were able to accomplish is improve unison by the performance of unison by 20 expected. So, again we perform 20 times better than unison in unison 2. So, now going back what we have ended up with is as unison 2 is about 20 times slower than just making a simple set call. We are getting there still not where we wanted to be. So, that brings me to the final part of the talk which is parallelization how we can use parallelization. So, before that I want to give a bit of our way of how verification works in industry and practice. So, what we have is we have simulators, simulators mimic the design you give a input you give a test vector they give you the output. So, we have simulators simulators can run in parallel they do not have to have any communication. So, currently we have a way of verification is that we have a test simulator and we have simulators. So, test simulator outputs a test vector this is run over simulator and we have lot of simulators in working in parallel. So, test simulator keeps giving them inputs. However, the current techniques that are used by industry they cannot have test generator in the parallel state because this test generator is maintained some sort of global state. What do they have is they say that well I have explored this part of the solution space and this information they use in the conjugate iterations of generation of test vectors. So, if you just parallelize all of them all of them are going to be done that the same test vectors which is completely useless. So, the current techniques indeed lose any if they have any theoretical guarantees of uniformity if you try to parallelize. What is bit interesting in our case is that we have some free processing which is to compute the number of cells which I did not discuss. So, we have some initial free processing state which is to compute the number of cells which is small part. Once we do this free processing the test generation can happen completely in parallel. No communication is required between test generators once the initial free processing state is done because none of our proofs depend on what was generated earlier in terms of uniformity or any kind of independence guarantees. So, this I remember this is not possible in the current state of our techniques that are used by the EDM industry. So, we need to do free processing only once we do not require any communication between different copies of test generator. So, we scale linearly with the number of cores in practice and this allows us to get where we wanted to be. So, with just two cores now we are able to achieve the performance that is what EDM industry wants while still giving the theoretical guarantees of coverage and uniformity. So, one of the thing that I did not talk about how well we do in practice in terms we have the theoretical guarantees of uniformity, but how well we do in practice. So, we took some benchmark that had low number of solutions because we wanted to compare with the ideal uniform generator. So, if it has small number of solutions we can enumerate all those solutions and then pick one of the solutions randomly. So, that was our ideal generator just enumerate all the solutions and pick one of them randomly. So, I am going to show results for one of the benchmark. So, this benchmark has 16000 solutions we generated 4 million samples for ideal unizentive and parallel version of unizentive. What we did then is to group solutions according to their frequency. So, every since we are generating 4 million times we are generating 4 million samples while the total number of solutions are the 16000 every solution is going to occur a lot of time. So, we group solution according to their frequency and then we plot number of solutions versus frequency. So, basically a point 200 and comma 250 says that 250 solutions appear 200 times each. If the underlying generators are uniform generators then in theory we can show that we should expect a percent distribution. And since we do not have the guarantees of uniformity. So, I guess it would be very easy for all of you to identify which one is ideal. One of them is parallel uniform generator one of them is unizentive on single core and one of them is ideal. If you think you are not able to do not worry because statistically they are indistinguishable and this is the observation we have on all the benchmark we are experimented. So, well if you are still wondering. So, the yellow one turns out to be the unizentive red is unizentive and the blue is parallel version of unizentive. So, in practice again while we have theoretical proofs, but the implemented implementation of these still have uniform the behaviors as if they are ideal uniform generators. This was setting epsilon to 16 1 by epsilon set to 15 1 plus epsilon. This was this. 1 plus epsilon. 16. What is epsilon after between 0 and 1? No, oh well epsilon can be greater than 1. So, not mean that probability can exceed 1 or moment is not. No. No, because it is 1 plus epsilon the R f is very huge. Yes, you can choose as such numbers, but then they would be completely uninteresting. You have all your experiments the same as epsilon. Yeah, these experiments we do different experiments for different things, but these experiments. Because your run time also depends on epsilon. Yeah. So, more uniformity you want I divided. It is 1 by epsilon square dependence on the epsilon. So, one of the thing that we have been sort of working on all the time is to improve because what we observe is 1 by 16 seems to be really bad, but this is what we get and this sort of indicates that our proofs are very conservative currently and they can be improved. So, we also have this graph for the example of the first color was taking points and. Well, not this graph we had some other graphs of uniformity comparison and it appears pretty well. So, now to conclude so as I told the story so far has been spreading of independence. So, initially we started with hashing where we wanted to relax independence. So, we had an universal hash functions which were behaving giving these annual independent events. Now, we went all the way down to 3 universal hash functions we lost uniformity and we ended up with almost uniformity which is still as good as uniformity for all our practical applications. But what we were really able to gain is 2 to 3 orders of magnitude improvement with respect to the state of art generators. Compared to the techniques the theoretical techniques this was a huge improvement because those techniques could scale only to formula 10 to 20 variables. Then we went and we said well why we are generating samples independently how about we relax some independence among samples. We got beacon guarantees of almost uniformity, but we still had theoretical guarantees of coverage we get 20 x improvement these techniques are parallelizable and we achieve desired performance. One of the lessons that has been for in the whole journey is that there has been techniques we lose independence completely and that is where the problem arises because once you lose complete independence then you cannot give any theoretical guarantees of uniformity or coverage. So, in the whole journey we have been training of independence for scalability but carefully. So, if afternoon lunch was too heavy here is what I think take away from this talk uniform generation is key problem has diverse applications. I discuss only one of them there are other applications what we propose is the first scalable parallel approach that provides strong guarantees and scans to formulas that arise in real world. It requires constant number of set calls per sample and scans linearly with the number of calls and we achieve what has been the desired performance given to us by our immediate partners and they are now working on implementing this in their tools. Thank you I will be happy to take questions and this is a final question. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . So, the first stage which is where verification engineers come in plane they write these corner cases where you want to test them, but the number of corner cases they have is still too large that you cannot try on all those corner cases. So, what you want to do you do not know where exactly now the bug lies for example, suppose you design a simple design circuit you say well test try around 0, but you do not know whether the bug lies in 0.001 or 0.002. So, in that case what you can do is you can only try on few test vectors, because usually you have some time bound you. So, of course we start with those corner cases these are the constraints that come from the engineers and from these corner cases you can try only on few samples. So, which ones you want to choose, since you do not know anything more after this you know that these are corner cases, but you do not know anything more you do not know anything more where exactly the bug might be formed. So, your best bet is that you randomly sample or uniform you sample. So, yes we the whole process starts with the corner cases that are or the design or the verification scenarios which are formula which are usually written by verification engineers. No louder please, I could not, oh so well this is something I did not talk about preprocessing is counting the number of cells you should divide into and we have a very simple linear way which is divide into one cell pick a random cell if it has if the number of solutions are between high thrash and low thrash then you said that this is the right number of cells to divide into if not you divide into. So, you start with one cell you divide into two cells you divide into four cells you basically just keep increasing the value of n and at one point you end up in a you basically end up in a for a value of n where one of the randomly chosen cell it happens to be small and you say well this is going to be my n from now on and you can show that this look dumb looking way gives all kinds of clarity. The number of cells is very important to fit the right number of cells that is the preprocessing, once you know that number then you make each one of them is divided independently into right number of cells, there are randomly different number of cells. Yeah well so the whole idea is that since they are not they do not have any sort of dependence. So, you could have thought of that you know one generating four samples or all four of them generating one sample each that we have very likely you expect because if you generate four random samples then you expect that you are covering most part. So, of course, theoretically there is still a very very very small probability that these have generated the same samples. We did a live demo at TACAS this time we were running on four cores for the whole demo 90 minutes and all of them generated about 100,000 samples and they did not repeat any sample. It is still not to say that there is theoretical problem that it is not but it is really just that. There is a theory which is that this three units of hatching generates near uniform near uniform cells, this and this for any arbitrary input problem. Yes. For any arbitrary input problem right probability this state of three units of hatching So, yes again to repeat I showed I kept showing circle because that was the easiest to draw but there is no assumption on the formula they start with and the spaces of