 Today's talk, of course, is Akshay, I am tempted to say there is a whole Akshay pantheram that is overflowing with the food of knowledge that we so crave, and so I propose that we study. All right, so thanks a lot Shivram for giving me this chance to present some of this work. So I think what I'll do is to try to give, so I don't know what is the background of the people in the audience, so let me start with really basics, and as it goes along things will get more and more interesting, and of course feel free to stop me at any time with questions, though I may decide that some of them will be bumped up after the talk. So the problem that I am interested in or the questions that I am going to be talking about are on some very basic kind of problems for a very basic kind of model, and my aim in this talk is to really convince at least some of you that not everything is known even about such a very basic, very simple model, and to try to motivate more people to maybe look at these problems and to take them seriously. So that is an interesting history why I started looking at this problem, but maybe I'll come to it after all the talk is over, so not right now. So the problem that we are looking at is the reachability problem for Markov chains. So what are Markov chains? So given that this is 530 in the evening, I am sure people are at various states of attentiveness in whatever in their current state. So suppose we wanted to say we want to really map the mood of all these people in some sense. So basically we have some people are interested, or even one of you. So let's take one person and we say currently maybe I hope when you start at least everyone is interested. But then as time goes on or maybe after a couple of minutes if I look at it, maybe some of you have become bored while some of you are still interested. So that's why there is a loop here, so I am just writing a transition system for it. And once you're bored some may decide to check your mail or you know check your whatever it is that you want and others may fall asleep. And of course when you at some point you may come back and so on. This is a very standard transition system and we want to put in probabilities in a very natural way. So if you have a single person, you can say what is the probability with which you are going to be bored or not bored. But if I am talking about a collection of individuals, so everybody in this room, I can say okay after 5 minutes, how many of you are going to be in this state and how many are going to be in the other state? So what is the fraction of you which is going to go from being interested to being bored? So the function of how good the talk is going to be but we'll ignore that at the moment. This is a transition system. So we can add probabilities as weights to these edges. So these are transition systems. So you have states and you have edges with weights on the edges. And the weights are the probabilities. And this is basically what a Markov chain is. And the crucial kind of assumption that we make or the definition or the crucial defining property is that the transition probabilities, the probabilities, the weights associated are only dependent on the last state that you have. Not at really the history. So of course you may question these assumptions at various points and say oh no but your model is not good enough to capture what I want to capture. That's a different question. So that is not something I'm going to be focused on though that's a very valid question and if you want to talk more about that we can talk to you. So really this is the basic model for probabilistic systems. So how many of you already know about Markov chains? So I can go faster. Almost everybody has lifted their hands except for few of you here. So I hope now it's clear what it is. So all we are saying that it's a transition system or an automaton with probabilities on the edges, weights on the edges which are. So of course you can notice that from one state if I look at the number of edges going out the sum of the probabilities on those edges adds up to one. That is, from some state you have to go somewhere either be here or go somewhere. If I add all this probability it better be, right? So in some sense your states could be, so there are two points here. I think this should have come in later. So your states though this is a very stupid example, in general this is a very, very, very descriptive model where your states could be anything. It could be for instance states could be web pages and your transitions could be probability that you click and go to some other web page. So you could actually, so you can think of the page rank, the Google page rank algorithm which defines all your, whatever, Googling searches and so on which is kind of, you know, core of many, many areas as being working on an underlying Markov chain. Of course there might be more complications but we are not interested in that. But at least we can think of a model as a Markov chain for this, for the page rank algorithm. Similarly you can think of states as being current weather outside and the probability as saying the probability that the weather will change from sunny to rainy and so on. So you can have various, various examples or applications and so this is a, which is why this is really a basic model. So you can always put in more stuff but this is a core model which occurs in lots of domains. Now, if you think about how to represent this, you can actually represent this as easily as a transition matrix where you have the current state here and the next state. So basically you're saying from, if I'm currently at state one and I want to go to state one in one step, the probability of that, that is remaining in state one is two by three. The probability of going from one to state two in one step is actually, there is no edge in this direction, right? So that probably is zero. And the other one is one by three and so on. So this is basically the transition matrix which describes my Markov chain. Is that fine? So the only property of this transition matrix is that it's a row stochastic. That means the row entries add up to one that we already discussed, right? So this is very simple and it has many, I mean it's not very difficult to see that it has many linear algebraic properties because it's a very core element of linear algebra, right? And you can really talk about, okay, so if I was in this distribution, so maybe when you came in half the people who are in the room were interested and half were already bored to begin with, right? Then after one step, what happens? What is the distribution? That is how many of you are in various status and that's really this. So that's why it's a transition system and this gives the transition matrix, right? So yet another slight variant of this, another way to see this is to say that, look, I'm basically what I'm saying is that every state, every configuration is a distribution over these small states. So half of them are here, half of them are there and so on. So it's really a distribution and what the Markov chain tells me is, how does a distribution get mapped in one step? To which distribution does it go to, right? So there are three in some sense slightly different views of the same Markov chain. A Markov chain can be either be thought of as a just a transition system with probabilities, it can be thought of as a matrix and pure linear algebra or it can be thought of as a linear transformer of distributions. And actually we will swap between these notions as and when we need them. So which is why it's important to see all these things, all right? Great. So now I think we are all set up to do a little bit more things. So what is the problem? What is the basic reachability question that one may ask? So one may be interested to find out, if I'm given an initial state or an initial distribution over the states, that is half of them are here and half of them are there, for instance. And I'm given a target state. Maybe my goal is to make sure that at the end of the talk everybody is asleep, right? So that would be 0, 0, 0, 1, right? And so that would be my target state. And my question that I ask is, okay, can you reach from a given initial state to a given target state with a probability equal to r, right? Because this problem is not very correctly posed. One question could be, I tell you in ten steps. So in ten minutes, can I go from here to here? So in each step this transition happens. And in ten steps, in ten minutes, can I go from here to here? Is this easy to see? Why, how would you do this? So in ten steps, suppose I want to ask, all you need to do is take the matrix, multiply it ten times and then check whether that entry is going to be satisfying this probability, is equal to r. It's very simple. I mean, everybody knows this. I mean, even if you didn't know about Markov chains, you can see that this is correct. There is nothing really fun happening there. Now, suppose I ask you, no, I don't want in ten steps, but I want to say that in the limit on an average, either on an average or eventually, at some point, eventually is it the case that is, as n tends to infinity, is it the case that I can go from the state to the state with probability r? What is the average, in other words, I can ask, what is the average amount of time spent at that state, right? So this requires a little bit more work, but most of you know about Markov chains who already know this, right? So this requires a little bit of basic undergraduate level probability theory. And what is this? So basically, the basic core property that you need is that, if a Markov chain is a little bit nice, then you're guaranteed that from any initial distribution, if you allow enough time, you will go to exactly the same final distribution. You will tend to the same final distribution, that is, the limit goes there. That probability and that's called, and it will tend to, in fact, a unique such stationary distribution, right? And what is nice mean here, nice means, for instance, it could mean that all the entries in your matrix are all strictly positive. So none of them must be zero. Or nice could mean that it's irredisposable and a periodic, that's kind of a much more general framework, right? So for those of you who know, it's, of course, obvious that this is the case, right? And this is really what is called the fundamental theorem, sometimes called the fundamental theorem of Markov chains. And the first thing that you would study in this area, which is that, if I take, for instance, suppose I take this simple Markov chain. Whose fixed point, if you can see is this. What is a fixed point? It's a vector such that when I multiply this with this, it gets back the same vector. So you can see that there is a unique such vector. And starting from anywhere, I will actually limit as n goes to infinity. I will reach to this vector when I raise the matrix to that power. Start from vector and I raise this matrix and I get to this vector. So this is the theorem. So I mean, again, it's not very difficult to prove, but it's probably a couple of one lecture that one would be. And this is true only if you have some nice assumptions. But even in general, even in general, you can do the similar kind of reasoning. So you can, instead of looking at just looking at the whole Markov chain, you can break it up into smaller parts which behave nicely and still answer the same question. So I'm not going to go into the detail. For those of you who know, you already know it. And for those who don't know, take my word for it, that this can be done for general Markov chains, okay? So now comes the interesting part, okay? So basically the question that we have asked is a basic reachability problem where we said, okay, reaching in, if I give you n steps and if I ask you what is the probability to go from here to there? Is the probability equal to r to go from initial to target state in n steps? This is trivial. In the limit, it's not trivial, but it's very easy. It's a first course, few lectures, a few, whatever, reading of lecture notes, and you will get it. So here is a third question that I would like to ask, which is very similar to this, is it the state that you can go from the initial, is it the case that you can go from the initial state to the final state? In, with the probability of equal to r, in some n steps, you see the difference between the first and the third? So n is given in the thing, the first, so then you could do m power n, right? And I asked you, okay, does there exist n such that m power n? So I can do m power n and I take the initial state and the target state. Then I get this number r. Is that, is the problem clear? Yeah, yeah, yeah, yeah, no, not eigenvalue one, eigenvalue Markov chains always have eigenvalue one. Yeah, yeah, in fact, sorry, rose to sum up to one. So that's always true, there is always one eigenvalue one and all the others are strictly less than one. What I'm asking is, is the stationary distribution, sorry, I'm asking the stationary, can you compute the stationary distribution? Right? Is the stationary distribution, does it have equal to r or something? Yeah, the niceness property basically says that, there will always be that unique stationary distribution. Otherwise, it may not be unique, but you can still figure all this out, that's essentially the idea. So, yeah, exactly, that's it, that's it, that's exactly it. So, so for instance, if I take, can you see this? This one, this one does not have a unique fixed point. Convince yourself that, that is the case, right? So, this is a very simple matrix and you can see that, basically what is this saying? It says, as a state, it says that, whatever, if you are here, you go there with probability one and go there with probability one. So, you're never going to be in the same state forever, right? So, you're never going to conduct. So, that's, that's a very, yeah, that's an easy part, I would say, right? So, trying to characterize that. We'll come back to the eigenvalue part, because that's really going to be crucial to the rest of the remaining part of the talk. But here, what I want to ask is, so does there exist a finite number n such that I can reach from initial to target with probability r in exactly instance? So, what do you think about this problem? Is it going to be as easy as this or harder or any comments for those who have not read the abstract? So, the crux of this talk, and I could actually stop the talk here if it were a three minute or whatever, a ten minute talk and so on, is that, oh yeah. So, before the crux of the talk, yeah, so this is, this problem is open. And again, what is this problem asking? The problem is asking, if I start with the vector in which I have one at the state s and I look, so start from state s and I apply n n times and reach the state t, is this probability? So, one, so this is just a symbol to say it's the vector which has one in the sth position and zero everywhere else, right? So, I'm just asking, is this equal to r? And this question is still open. So, what do I mean by it is still open? So, let's understand what it is. So, let me restate the problem so that everybody is clear that how easy the problem is, right? So, I mean to state. So, in other words, all I'm saying is, take any roast acoustic matrix, right? And I ask, and I give you some two indices of the matrix, i and j. And I give you a rational number and the question is, does there exist n such that, m power n i j is equal to r, this is it, all right? And what do I mean by this problem is open? Can you give an algorithm for this problem? So, can you give an procedure or an algorithm to check whether such an n exists or not? This is the question and actually this problem in its full generality is open, right? So, is that Aviram, is that fine, yeah, okay, yeah? No, so that is the point, so that's the next slide, so good, good. So, let's end this slide and you will get the answer to your question. So, it's open but exactly how hard is it? So, in other words, so suppose I, my r was zero or suppose I want to know whether it's going to be equal to zero or something, right? So, the probability of going from here to there, for that I don't even need the probability, I can just do some graphical analysis on the graph of the Markov chain and decide it. So, actually this problem, the hardness of the problem is not there everywhere. It's not uniformly hard, no matter what the r is, no matter what the this is. What I mean by it's being hard is there are some particular bad values of r where this problem is really, really hard. Everywhere else it can be solved. So, in particular, suppose I take this as my Markov chain and I ask, is it the case that we will go eventually to some value, let's say half, right? So, from, is it the case that I can go, does there exist a n such that I can go from state one to state two in n steps with probability equal to half? This problem is easy. What is the hard problem? It becomes hard when, oh, okay, that is, okay, fine. So, it becomes hard when, when this r is exactly the value of the fixed point, the coordinate of the fixed point at that point. So, if I ask you, is the probability, does there exist a n such that I can go from state one to state two in n steps, is the probability equal to 4 and 4 by 11, then it becomes hard. So, this is really the distinguishing thing. So, why is this the case? Because if it, if I am only asking whether it goes to half, I already know that there is a unique fixed point and I know by fundamental theorem that it's always going to converge to this point. So, there is a finite time very soon, in fact, in fact, you can do a very easy algorithm which will say that, okay, after this point it can never come closer to half. It must go only to, it will stay within a very small area around this point. Yeah. So, we don't know what monotonic convergence is there. So, that is exactly the, so the second hard question which again has got cut off is, what is the behavior around the limit point? So, is it, is there a monotonicity property, is the second problem and we don't know the answer to this. So, I'd like to, in this rest of the talk, actually convince you that there is no easy answer to that question of whether there is a nice monotonicity, can I say that it always approaches from above or can I say that it always approaches from below or can I say it keeps switching in a regular way. That is, you know, there is a regular automaton which can tell you how it behaves around the fixed point. All these questions are in some sense equally hard, not, well, I would, I don't know whether they are equally hard but all these questions are going, I am going to show various types of hardness for these questions, right, yeah, that's right. So, basically think of it this way. So, I mean if I, I have that picture coming in later but now itself we can talk about it, right. So, basically if you write in two dimensions or three dimensions, you each, I can think of this as a, each distribution as a point in that n minus 1 dimension, right, because whatever it has to add up to 1, so it's going to be in a smaller dimension and I can ask how does this point, so when I apply m, it's a linear transformation, so it takes this point and goes to some other point on this space of distributions and I am really asking how does this behave around the fixed point, that's the question, right. So, don't worry, there are nice pictures coming in which will illustrate this point. I just want to give an abstract notion of this and then give an idea of why this is interesting other than being an interesting theoretical problem and then go into exactly what the behavior is in that, so exactly what you ask for. In this simple case, yes, where it's a nice matrix, so in a nice matrix, the algorithm is very simple, the algorithm just says that you know that you are going to converge eventually to the fixed point and you are going to converge exponentially fast to the fixed point by rate of convergence. So, you just need to check for some amount of time and you know not only do you know that you are going to converge, because you know that you are going to monotonically converge in the sense that at all times you are going to come closer, you keep coming closer to this point, so the distance decreases in a, yeah, in a way, so therefore if my other point that I want to reach, if r was something else, I will be able to say that there exists some, I can compute actually a polynomial bound on the n, in fact, quadratic or something like that, a bound on the n such that after so many n steps, I will be in this ball and therefore I can never hit that. So, take any two points, if this is the fixed point and that is the r, I will just draw a ball separating it and I ask after which point is my trajectory always going to be inside this ball, right. So, that question is easy and in that sense monotonicity is there, but now suppose this r was exactly this, then no such ball can be drawn, then suppose I want to ask, is it the case that I always came from above that, I do not know, all right, okay, so I would like to, before going into the ideas of, so again we really want to understand how hard this is, so I mean I cannot just say hard, I cannot just say it is open and expect you to all believe me, right. So, I will try to motivate why it is hard or what is the hardness and that is going to be actually a bulk of the talk, but before that I would like to give some kind of motivation from where actually I got interested in this, these kind of problems, just to say what does formal verification have to do with this in some sense, right, yeah, I mean equal to r, I mean equal to r, but suppose you tell me, okay, I do not want, what do I care about equal to r, suppose I want to ask strictly greater than r, yeah, one, so is it the case for all n, is it the case that I was always above r, this problem is also hard and in fact that problem is also open, yeah, I wish it was that simple, so, but no, so that problem is also hard and we are going to say why that problem is also hard, so you are right, so that problem is in fact more interesting than asking just whether it is equal to r, so these two are two different problems and there are a complete different span of results on both of them and I would like to mention a few of those in the talk, okay. So, I will just give a two slide motivation for the kind of problems which got me interested in this, so I mean this is a, for instance a biological example, okay, so here basically what we have is that we want to actually look at the variation in the population of a yeast, yeast is some chemical, not chemical, whatever, some biological organism, right, so under some stress as shown by some marker, right, what is a marker, chemical marker, we do not care about all this, right, so essentially what the idea is is that there is a bunch of experiments that people do and they try to model what happens when yeast are affected with, are injected with some chemical marker, okay and they look at the population and this is a study that they have actually done, so for instance this particular one is taken from this paper and one can think of a, so basically you are looking at a population, so one can just as we started in the example in the beginning of the class, how many people are there who are at attractive state, I can talk about the states being the concentration of this marked yeast in the different populations, high population of concentration of marquees, medium level of population of them, low level of population of them and the transitions are really what is the proportion of yeast which move from here to there at one, all right, so again this is not necessarily the best model but this is a model and at least according to some of the biologists, well why not, this is another good enough model in the sense that there is no better information known, so this could be a very, there is a lot of thing which is getting cut, so I have not lot actually in this case only the page number got cut, so yeah, so what is the parameter, so that is another interesting problem, so it is not that that is the only interesting problem, there are cases where people are interested in this, there are cases where people are interested in that, so I will try to, I will give a list of problems and then you can see whether yours fits in here or you want another problem, so that, so all I am saying is that this is the model, so currently I am not talking about what is the problem, so currently this is the model and so I have states where this is high, this is medium, this is low and there are transitions which say with what probability it moves from whatever the concentration moves from here to there, so this is basically the idea and this is just actually to motivate the point that I am actually interested about the patterns which are formed by this, so if initially there is one third concentration in each of the states, then is it the case that the population with the high concentration always is above 5 by 12, this is a valid question, no matter what the parameter is given the current parameters under which they are working, they want to ask from the initial distribution which is in this some way of concentration, is it the case that the population with higher concentration is always above 5 by 12 or you could ask is the whatever the low population having high concentration, does it satisfy some nice structure, so is it the case that it was very low first and then became high and then became low, so you can ask any such question and all of these questions are interesting for different reasons, so again so we seem to be, now coming to your issue, we seem to be talking about these graphs in the air and so on, so let us try to make it a little bit more concrete, what is it that we mean, so really what we are saying is that there is some, so at least for the simple problems we are saying there is a threshold value and I am actually when I say it is above and below, I am actually saying it is above the threshold or below the threshold, so in other words I can actually talk about saying that I will take this space of all probability distributions and I partition it, saying that this is above and this is below, so it is just some partition and that partition is given by the biologist let us say, so maybe that biologist is interested really in whether it is above 5 by 12 or not, so 5 by 12 becomes the threshold, actually the biologist needs such a break up into high, medium and low because anyway they cannot do any better, so they do not, I mean usually they work with a very noisy framework, I mean there is a valid criticism which I will come to later but that one is actually from the biologist, the fact that there is a discretized interval which is high, low, medium or maybe it is not high, low, medium maybe it is high, very high, low, very low, medium, it is possible but they need a discretized interval because anyway all their experiments, all their data, everything actually, so for instance if they are looking at some plot, they are going to look at finally some discretization. So, I will mention this at the end of the talk, why, so yeah, so yeah, so no, go ahead, it depends on the model, sometimes they do, sometimes they have no clue, sometimes they do not even know whether this is the right model, so they do not know what the steady state is, 0.4 is true, yeah. I think in most cases they will be happy with the answer but I may want to also ask whether there is some pattern which is followed and with respect to that pattern maybe they are not happy with my saying whatever I want to say. In some sense, am I solving their problem or am I saying, I cannot really solve your problem, so let me solve some other problem and you better be satisfied with that. So, I mean there are both ways, I think currently in fact what we are saying is do exactly what you are saying, do not bother about the exact value, let us just give an approximate answer and that is good enough and I will show some results about that, that is finally our solution also, to say that anyway your biologists do not care about the things, so in terms of the practical application, we do not care, we do not really care whether it is exactly 0.5 or 0.42 and so on but, yeah but the crucial point to note here is that, I mean actually this is where the theorist in me comes out, right. So, this is where I say, okay look that is okay, I can solve it for them in this way but I would like to understand what is this point, what is this magic point that I am not able to solve, what is the hardness of that point, is there something really hard in it or is it just that I do not want to solve it, so that is where I go into more theoretical study of this, so one may question as to whether the practical applications are there and my answer is two-fold, in some cases they also need it, in other cases even if they do not need it, I as a theoretician, I am interested in, yeah, yes, yes, yes. We could, we could, no we could, we could, we could do all this, we can look at parameterized Markov chain, we can look at continuous time Markov chains, we can look at interval Markov chains, we can look at MDPs because why have deterministic structures, why not have decision making capability, so a better model would be all of this, I want to take the simplest model that I know of and already show these results and yeah, you can say okay but would simplify, so that is a valid question, so that could be one way, another way of tackling it, just as approximation is one way, parameterization is a yet another way, so these are all ways to tackle the problem but we can get straight at the problem in the setting, so let's understand the problem then we can have different discussion on various ways to actually tackle the problem, yeah. This, well I am in this talk only going to say how that results look but there is a student here who has the code for that, who can tell you exactly how it looks like, who is not sitting here actually, yes, oh this is, thank you for this question because that's exactly what is coming, so very good, I mean that's, if I end up saying everything in that talk I think I'll stop at that point, right, alright, so I think now everybody agrees this is a nice problem, this has nice theoretical components, this has some vague practical implications and yeah, so yeah, so in some sense this oscillation does the trajectory, this is your question, this is exactly my question, this is exactly the kind of question that I ask, is it the case that can I have an oscillating behavior or some regular behavior, so in fact I don't want to just talk about oscillating, I want to say is there some regular behavior, if I give you any regular behavior described by regular automaton or writable whatever you can write it in LTL logic or something like that, can I do everything that logic can say, can I answer every one of those questions, okay, yeah, so for instance this is a symbolic trajectory, this or is it B whatever, B A omega or something like that, is it below above, below above and so on, okay, so I'd like you to note at this point that these trajectories, this symbolic trajectories, once this partitioning is done is actually a word over a finite alphabet, so automata theory comes sneakily into the picture, right, so we are going to have basically what we are saying is that instead of looking at these real values, I'm looking at this partitioned alphabet and over this I can now for instance here I just say I'm only interested in whether it's below or above, so the trajectory I'm interested in is below below above below or whatever it is and the oscillation question is just asking is it the case that eventually it forms the pattern B A B A B A or something like that, right, so you can write these formally in some kind of logic, okay and so that's yeah, so okay, so this is about single trajectories but we were not interested only in single trajectories and this is again motivated from some biological motivation and where we say okay what if the initial distribution was not a single point but actually a parameter version, so there is a range in which the initial distribution is, so you know that whatever one of the, maybe you know one of the things exactly but the other things you know only in a range, then what you get is not a single word but you get a set of words and this, if this set of initial distributions is a big set, whatever if it's for instance if it can be anywhere between half and one-third, I get an infinite collection of trajectories which come above and I want to ask do these infinite collection of trajectories, symbolic trajectories that come here, do they all satisfy some property like do they oscillate forever, these kind of questions, alright, yeah, so for instance these are different trajectories so this one, so if you can see everything is completely heavier but you can see that actually all these three trajectories starting from here remain above B all the time, so is it the case that if I start from an initial collection, the whole set of trajectories remain above or not, so this question slowly becomes a question not only about a single trajectory but a set of trajectories and how they behave over time, right, so this is really the patterns that I am interested in which was in the abstract of the talk, right, so what is the way these things evolve over time and yeah, so this allows us to move from words to languages, so if we knew for instance, yeah so and I am going to use this notation, so let me just fix this notation, so this symbolic language we will denote as the language of a Markov chain with respect to an initial set of distributions and for my talk all the initial distributions I will think of as product of some initial intervals, actually I need only them, need them to be a convex set of initial distributions, then all the kind of things I talk about will be fine, right and if this language were regular then I can really ask, if this language was regular then I can ask, you know does it satisfy the oscillating property, does it satisfy something else, I can do pretty much whatever I want to do, is that clear, yeah and we can, so yeah, so now let me give the formal description of the problem and it is 6 o'clock, so I have to do it, perfect, so now, so this is, I hope this is the whatever the introduction of the talk but actually no, I will come to the outline after this talk, so this is all the problems that we are interested in and now you can ask me whether there are other problems of interest to you, so basically we start with a Markov chain, we start with some distributions and in it is the collection of distributions we start with the threshold value, some value which is going to distinguish what is interesting and so on and we start with the discretization of the space, so in general we will be interested in some random discretization but in this talk I am just going to be interested in whether it is above, so the only discretization is whether it is a or b, this is the only thing I am interested in, so in other words a or b with respect to this lambda, right, this lambda defines whether it is above the threshold or not and so the words are over this alphabet and the questions we are interested in are does there exist an integer n such that mu that is that initial distribution dot m power n dot sigma is equal to lambda, then for all integers n is it the case that this thing is greater than lambda, right, so these are basically the kind of questions you guys you are already asking me, so and so basically all these questions so is it the case that some an initial distribution does it satisfy some regular pattern or is it the case that the whole bunch of trajectories that I start from this initial convex set satisfy some nice pattern, so all these are very valid questions and one can generalize this to very nice things which automata theories have already developed if this language were regular, all right, all right, so that brings me to the outline, so I hope the technical problems are clear, so these are from now on I will be interested only in these problems and their links to various other problems, so we will again take lot of details but hopefully the idea is clear what these problems are and why they are interested, so this is going to be the real outline of the talk, so what I have explained till now is just the statements and why they are interesting, so what we are interested in this talk is to really understand how hard they are and the way we do that is by linking it to another well-known hard problem some of you may already know this, so how many of you know this problem called the Scholem problem by a show of hands, oh very good, so you are in for a good surprise let me say, so in fact then, so yeah I think that is going to be the interesting, one interesting part, so let me spend a lot of time on that and then depending on how much time is there, I will actually talk about the results that we have about the strategic trees and so on but even if we do not get here, I hope, I mean we are in the same department, so I hope all of you will be can come and ask me at any point of time, right and actually what I would like to do is also talk about links to various other problems and one particular problem which is of interest at least to some of us, so Prithik has been working on this also for a long time, is that right to say that, it is about program termination why is that linked to this problem, right, so I would like to, though I have written it this way and I will be presenting some results, I may push things under the carpet, so that I can get to telling you all the various links of this nice problem that we have looked at, alright, any questions at this point because now we are going to go into a, so it is like part two of the talk, so till, so if you have understood till here, I think you have got already the bulk of why this problem is interesting and what the problem is, so but next we are going into different direction which is about the links to this column problem, so if you have any questions this is would be the right time, alright, okay, so yes, it become undecided, two state MDPs, so no, no, no, okay, okay, sorry MDPs is different, so because actually for MDPs things become, okay, Markov chain with two states, no, these are not decided, these are not hard, in fact for Markov chain with two states all these become solvable, three they are already hard, that will come in this, so my counter examples which I will give for hardness will be, will require a dimension of three and that is really related to the Eigen values which that is already pointed out, right, so the structure of the Eigen values or is it what the values of the, whatever the, are they from, are they complex or real basically is going to decide whether these problems are easy or not, we will come to that, yeah, so this one, so this does not have multiplicity two but it has two, so zero one and one zero are both points which are, right, for which one, for this one or the yeah, so this one there is one and minus one, right, no, right, so that is okay, no this is not a nice matrix, so this one, so wait, sorry, I think I have drawn something wrong here, so this might be a nice matrix, it is the other one which, no no, so is this a nice matrix, okay, in some sense it is a nice matrix and in this case we will be able to solve it but, so for instance to interesting, so this one I just wanted to show, yeah in 3 by 3 I have an example, we will come to that, we will get non-trivial things, that is why his question is very valid, for 2 by 2 matrices things become too easy, things actually become too nice, so for instance in this one, if you look at, so what I mean is that this point, this does not have a unique limit for this one, right, so that is all I am talking about, so for me in this particular sense this is niceness, so this limit does not exist, so but I agree with you, I mean what is nice and which is exactly the nice matrix, 2 by 2 by 2, anyway I am not going to be interested in, I am going to be interested in more complicated matrix, yeah, it is beyond this that we are interested, that is right, so this is not an appropriate example for what I want to talk about, anyway what I want to talk about occur only after 3 by 3, so at 2 by 2 things are very nice and we will come to that if we have depending on how much time we have, we will come to that and even in the hardness you will see that you require such a blow up for things to become not so nice, all right, yeah so, okay in other words my niceness is not formal enough for me to be able to say whether this is nice or not because there are different things right, all right, okay, so this question I think all of you know, right, so this is like a new start, fresh start, talk 2 is beginning here, right, so this is a Fibonacci sequence, so this is the sequence, is there anybody here who does not know what a Fibonacci sequence is, I presume not, right, so it is a very standard Fibonacci sequence and it is given by the and with the initial conditions of u1 equal to 1 and u 0 to 1 and you can see that that describes what the multiplication of rabbits in psa or whatever it is, right, but this is not a very good recurrence, right, I mean for rabbits after all rabbits keep dying, so this recurrence relation that they it is just un minus 1 plus un minus 2 basically seems to require complicated example would be to take into account the fact that you know three generations on the line rabbits do die so you are reducing something so do not ask me exactly what the recurrence is I have no idea but all I am saying is that if I look at a more general recurrence relation with again some initial value and I ask you this question, I give you some more general recurrence relation like this from 0 alright, so is that easy enough problem? okay so before we go into that again yeah so the it does not matter you have not missed out anything here so this is in general called a linear recurrence sequence for obvious reasons so and basically if the nth value depends on only k whatever it is called k order depth is called k and the constants a1 to ak-1 are all some constants over some fields so basically we can take it to be integers we can take it to be more complicated things right now we are only interested in integers but we will worry about other things and that is it so this is actually means that such a simple problem whether a general linear recurrence sequence ever has a zero is already a very hard problem which is and this problem unlike the other one has received a lot of attention so in fact yeah so this is a little bit cheating to say it is 80 years old because 80 years back decidability itself was not really clear what this problem was and so on right but why is it called at 1934 yeah so in fact to tell you that this problem has received a lot of attention here is a quote by Terence Tau right in his blog in 2007 he has this quote which is that it's frankly outrages that this problem is still open it's like saying we don't know it it is currently not known whether it's decidable right for an arbitrary order k and now we can as usual look at variants of it right what if we are not interested in whether it's equal to 0 what if we are only interested in whether it's for all always is it greater than 0 actually that was that should have been the for all n is it the case that the nth entry is greater than equal to 0 the ultimate positivity problem asks whether there is a point after which this is true that is there's a point after which it's always greater than equal to 0 so these are related variants and just to tell you that this problem unlike the other one has been really well studied so this problem for instance was a piece of problem I think so this is originally there is this theorem of Scholem Mahler and Leck so Scholem proved it in 1934 and Mahler and Leck I think so he proved it for integers and it was raised to other things as well we don't need it to have integers as well so there are various variants of this problem it talks about basically what it says is that if you want to ask whether there are infinitely many points at which it's 0 or not that question is decidable whether it's finite or infinite the number of 0's whether it's finite or infinite is decidable it's whether it's 0 or non-zero which is not known it's not known could very well be true so that is actually very old result and then people in various so there are which showed the decidability of both the column and the positivity problem for various orders so now it should take just the order 2 so this really corresponds in our matrix case to the size of the matrix which really you know number of states in the matrix so this is exactly the same so what was shown is even for 2, 3, 4 there were 3 separate papers actually for recently that is about 2 years back there's been a rich hall of results on this topic and this is by these 2 people professors from university of Oxford Joela Vaknin and James Worrell and they have a whole bunch of papers and their best result which got I think the best paper award in iCalc 14 says that ultimate positivity for of so called simple LRS which have some nice eigenvalue properties is decidable and what about general LRS all they were able to show is that positivity for general LRS of order 5 so 5 is decidable with the complexity of co-NP power PP power PP power PP when for that paper and since that result on matrix iteration that's I think yeah oh no no no that's true it's completely different so this is talk number 2 so we have not linked up the 2 yet very good that's going to be my next slide so this is I am just saying that here is another problem which is open but which has received a lot of attention right and ultimate problem is perhaps this one which says that if you actually show decidability of this column problem for order 6 this would lead to some amazing breakthroughs in number theory and everybody knows number theory is something you know it's little bit untouchable right but there are some things called the diapantine approximation of some numbers become you have solved a really really hard problem right and the bottom line is their best results if you notice is for 5 6 and the simple restriction so in full generality this problem is still wide open right and now that brings me to why are we interested in this why this detour into this other thing it turns out so this is a alternative so this is a statement of the mark of reachability problem which we saw right so given a finite stochastic matrix with rational so now I chosen rational but I could choose rational algebraic numbers reals integers no integers doesn't make sense in mark of reachability you need at least rational but you can choose various other things there the 1 2th element is equal to r that's the mark of reachability problem and the Scholem problem asks if given a k cross k matrix does there exist n such that right so note this I have restated differently right so the sequence you can convince algebra right so with a little bit of linear algebra you can convince yourself that this is the same as the Scholem problem which is that I give you k cross integer rational real blah blah matrix does there exist n such that m power still it's not the same right and what we managed to show and is explicitly that actually you can easily prove and this is not a very even hard exercise to show that the mark of reachability problem is as hard as the Scholem problem alright so I did have a sketch so in fact yeah so in fact we show I put the first slide but I think I'm not going to describe it so because there are many other results I'd like to give an overview of rather than describe this thing but it will be put up on the web so those of you who are interested and it's there in my paper so it's a five page paper so you can or eight page paper so you can easily read this the core idea is to get rid of the negative values by the simple observation that any negative any value difference of two positive numbers right come on that's kind of I mean let's only restrict to rational numbers so in rational numbers any rational number can be written as a difference of two rational numbers and we use that observation and replace every entry of the matrix by a small two cross two matrix where the whatever the cross product actually gives you the value of the induction that actually this satisfies the property not only for the first value but no matter which power you take this property is still satisfied so this is the first step and once you do the step you've gotten rid of the negative entries in the matrix the second step consists of rescaling because the entries are to be stochastic you have to be values must be between 0 and 1 so you rescale so that everything becomes between 0 and 1 this you can do by adding it's not very this part is not very difficult to figure out and the third part so when you do all this what you get is that you do get exactly what you want you get the matrix is stochastic but this end vector that we want is not a very nice vector it looks it has entries not only in 0 1 but it has entry because you want it to be 0 0 1 in the tth place and how do you get rid of that requires a little bit of proof so this again I really don't want to talk about but this construction basically gives us that in the original matrix if in fact yeah in the original matrix this entry is 1 if and only if in the new matrix some entry is 1 by 4 so this basically becomes the proof sketch right and this actually very yeah post facto not very difficult proof shows that the column problem is reducible to the mark of reachability problem and you can see that when I do this actually the size of the matrix blows up in a quadratic way so I can get an exact bound on the blow up of the matrix and so the it doesn't become very large that's what I'm saying right so basically this is kind of shout out to people in formal methods who know various logics which have been defined over mark of change and it turns out there is a whole bunch of logics whose decidability was not known till now and now we can say that they were not known for good reason including a logic which we had come up with and this is actually the place of the approximation sense alright so and now we have a strong result to say that okay this is not no I mean it's not that we didn't know it because we didn't try to hard it's really a hard problem okay so model checking for all these logics is a column hard that's what we have managed to obtain about the trajectories and this should answer some of both Ritzitz and Abiram's questions alright and I break it up into two parts one is about single trajectories and another is about sets of trajectories and so I want to really rush through this so that I can tell you a little bit about program termination as well how hard are they are they always periodic do the patterns that trajectories from an initial point do they do can I describe them in a periodic way or are they regular and the first result is that the trajectories may not be ultimately periodic so it may be that they so yeah so and this is not true for two cross two three actually periodic and so the language so whatever the trajectories must be behaving very nicely and for already for three cross three matrix actually I think ah good so already for such a simple matrix and said note that this is my nice matrix so even for strictly positive even for this matrix if I start with this okay so there is a slide type of this should be a row vector and that take the transpose of the matrix so I switch notations a little bit there so if I take this matrix right or it's transpose this is a perfect Markov chain you can see the whatever the columns add up transpose but maybe for the row also it holds so essentially oh actually the reason is already given here I thought I won't give this but anyway so the reason is that if I start from this initial vector and if I look at this matrix it turns out that these are not going to be ultimately periodic so these are ultimately a periodic and I get ultimately a periodic word and the reason for this intuitively I mean not intuitively the exact reason for this is that the Eigen values of this are in a slightly bizarre way and what is this bizarre way well actually in this case they turn out to be the polar form complex numbers they are complex numbers having a certain slight complication property which is that they no power of them gives you a real number so they do not form roots of real numbers so in fact what we found is that if you take matrices in which all the Eigen values are indeed roots of real numbers that is eventually so it becomes so it starts repeating so a power n b power n is still ultimately periodic so you can have an automaton for it right so which is if and only if no it is not every is ultimately periodic then the roots should be real numbers no that is not yeah okay so therefore we get corollary that mark of reachability problems are decidable for this whole class of mark of chain whose Eigen values are roots of real numbers only the reachability problems from single trajectories right and next okay it is 630 so let me try to actually wind up without going into the results so if you want to not look at the exact value but approximately how it behaves even with languages we can show regularity right and this is the latest work we have in this and this was done with some collaborators in France as well as a BTEC student who is not here I think they have a tutorial today so at this time so this is Nikhil Vyas whose Eigen values are distinct positive real numbers then the symbolic language that language itself must be regular and now you may ask what if it is for the trajectories we had this result right if they are roots of real numbers then things are still nice right so if they are roots of real numbers things are still nice does that continue here so that is where the languages and the trajectories also start differing right and yeah so the source of difficulty I think we have seen enough sources of difficulty so let me quickly jump to one slide on program termination and then I stop is that fine alright so I have just one more actually one and a half slides so this is I didn't have time to go into this but I hope this part till here was clear and the last thing I want to do is kind of motivate completely different yet another third thread right so I could have again started the talk here so here is a third beginning for the talk right consider the problem of termination so everybody knows what is program termination so this is the classical problem this is of course undecidable whether this program terminates or not now consider a much simpler case and here is the case I consider I just look at linear programs so programs whose command is just updating by applying a linear function on top of it alright and the program here constraint so c transpose x is greater 0 update do this linear transformation right and question is as an instance of this program given b c and a is the termination problem decidable and now by now I hope nobody has any doubt that where we are going next this problem is as hard as the oh I have it good so can rewrite this as for all n is it the case that c transpose dot a power n dot b is greater than c right so really what we have been talking about is linear program termination right and that's not any difference from any of the other problems you have been looking right there is a b such that this is true so is there a initial vector such that the program terminates or does not terminate it turns out that this problem is in fact decidable and this was proved in two papers actually it was proved originally in curve 2014 over the integers by Breverman in 2006 and it was really again two very nice papers worth reading and the latest result was another small improvement in this in soda last year I think yeah I think that was again by Joe Charles and the problems here require an interplay of lot of theoretical ideas and they have lot of applications I hope you're all convinced of that at least and they have a lot of links one thing I didn't have time to talk about is the link with a completely different domain I mean it's not really of patrinets and other transition systems what touched upon is what happens if you have continuous time Markov chains yeah so either way what happens about them and so there are some recent results which I think are appearing last year and this year I don't know whether they have even been published yet by the same group of the people from Oxford who have shown that even in the continuous case results but very few and everywhere else it becomes as hard and so these are like really state of the art that I don't know exactly what the latest is because it's not yet been published I know it has been submitted yeah not yet so the only thing I'd be interested is are you interested in the problem of what happens in finite time does there exist yeah exactly this is the kind of thing I would like to do offline so currently we have as you can see we have found links to program I mean we this links are not found by me I mean the links were already there this particular Markov reachability to Skolom was perhaps the only link I was a part of but you can see that there are links between various things and I would like to add something like this so I'm not surprised because it does pop up and if you have such pointers I would be most happy I see okay we'll talk about this offline okay okay great so as you can see the last point is that this area has problems from puzzles to BTP MTP projects to PhD problems to a long standing open problem which can make your career so there you go