 So, I will continue the discussion about computing transition probabilities, one step transition probabilities I showed you and we had just started talking about two step transition probabilities. So, just consider a case, in the last lecture I consider the case x 2 equal to 1, x naught equal to 1. Now, let us look at x 2 equal to 2 and x naught equal to 1. So, in that case you see what are the possible roots of transitioning from the initial state of 1 to state 2 in two steps. So, you want to look at it and of course, the three possible paths would be see at state 0. So, this is time period 0, time period 1, time period 2 and these give you the states. So, therefore, at time 0 the system is in state 1, it transitions to state 1 that is a possibility, because in one step you may transition to 1, 2 or 3 and then you have to come back to state 2. So, therefore, one possible root would be from 1 to 1 and then 1 to 2. So, in one step transition in one period you go from 1 to 1 and then from 1 to 2. Similarly, you could go from 1 to 2 that means you transition to. So, this was our production and this is h r. So, production to h r and then h r to again h r and you can continue here or the third root would be 1 to 3 that means from production to sales and then sales to h r. So, these are the three roots and that I have written down the three roots in this way. These are the three possible ways no other root is possible of going from 1, 2, 2 in two steps. Now, we need to compute the probabilities of traversing these paths, because you want to compute two step transition probabilities. Now, look at the first path. So, the first path I know the probability of transitioning from 1 to 1, when I am initially in state 1 and I am now transforming or transitioning to state 1 in the next time period then it is p 1 1 I know that. Now, look at this part of the leg. So, there are two parts of the journey from 1 to 2 for example. So, this is this. So, first leg I know we already know these are the one step transition probabilities. Now, we use the Markovian property, because you see the probability that you want to compute is of going from 1 to 2 in that means you are actually looking for a probability x 1 to you are in 1 at time period 1 and you want to transition to 2 time period 2. So, then you see this is independent of the first leg y, because you see this is the Markovian property says that from here to here the transitioning probability is independent of where I was at time 0 the initial state. So, it is independent of the value of x 0 and therefore, I can write the probability of traversing this path as product of the first leg into the probability of the second leg. So, this is the idea and this is where I am using and otherwise I would not be able to write these transition probabilities, two step transition probabilities if I did not have the advantage of the Markovian property. So, it is clear that because as remember I said that the past history is not important for computing is not required for computing the transition probabilities. So, here it is where you are currently and where you want to transition. So, this is the only thing that we need to compute the probability I do not need to know where what was the value of x 0 and so therefore, the two are independent and since by our stationarity. So, the stationarity property I should say that stationarity property gives us says that probability x 1 equal to 1 and transitioning to x 2 equal to 2. That means the conditional probability of being in 1 and then transitioning to 2 this is same as probability x 0 equal to 1 transitioning to x 1 equal to 2. Remember because we said that stationarity says that probability x n plus 1 equal to j given x n equal to i is the same as probability x 1 equal to j given x 0 equal to i. So, the stationarity property says that these transition one step transition probabilities remain the same no matter which time period you are and therefore, this transition probability of going from 1 to 2 is the same as probability x 0 equal to 1. So, going from here to 2 in the initial stage. So, therefore, we are able to write down the probability of traversing these paths and so I have written them down here for all the three paths. So, this is for the first path this is for the second one because you are going from 1 to 2 and then 2 to 2 why have I written it as 1 p p 1 this was 2 1 to 2 and here I am using the fact that you are going from yeah what is this path x 2 equal to 2. So, what is the path I have written here x 0 equal to 1. So, 1 1 and 1 2 1 1 and 1 2. So, this is the path and then you have 1 2 and 2 2. So, that should be so this is that one and this is this one and this is the third one 1 2 3 and then 3 2 2. So, this is 1 1 and 1 2 and this is 1 2 and 2 2. So, that is the one which I have written here and then this is the one corresponding to this and then this is 1 2 3 and 3 2 2 p 1 1 see I do not know why am I writing this as x 0 is 1 and then you are going to what is this path this will be 1 3 and then this will be 3 2. All these 3 yeah I am sorry these 3 paths together give you this probability right of x 0 equal to 1 and x 2 equal to 2 yes this is. So, these are 3 possible ways of going from 1 to 2 in 2 steps right 2 step transition. So, all these 3 add up to this yeah I am sorry this is the one p 1 1 p 1 2 and then p 1 2 p 2 2. So, this is this is p 1 1 1 2 then this is 1 2 2 2 which is this path and then it is 1 3 and 3 2. So, 1 3 and 3 2 these are the things. So, now similarly when you want to when your x 0 is 1 and then x 2 is 1 that means in 2 steps you want to transition from 1 to 1. So, again you will have 3 possible such paths right may be I will just again repeat the whole thing. So, that there is no confusion. So, for example, you can go from 1 to 1 and then 1 to 1. So, this is here see the path corresponding to this will be this right. So, may be I make it this thing yeah just to and then what is the possible way you can go from here. So, 1 to 2 and then 2 to 1. So, 1 to 2 yes and then 2 to 1. So, this will be 1 right and then the third path would be when you want 1 to 3 and then 3 to 1 right 1 to 3. So, you would go from 1 to 3 and then 3 to 1. So, this will be 3 paths and then for each path you write down the probability. So, this corresponds to the 3 exclusive paths through which you can go from 1 to 1 in 2 steps and similarly this will be 3. So, now you have computed all the 2 step transition probabilities for when x 0 is 1 right and then in the in 2 steps you can be in 1 2 or 3. And so similarly you will have 6 more such transition probabilities when x 0 is 2 and then you want to transition to 1 or 2 or 3 in 2 steps starting from state 2 at time 0. And then finally it will be x 0 equal to 3. So, x 0 is equal to 3. So, because you do not know you could be in any of the 3 states at the beginning of the process. And so x 2 is equal to 1 you are transitioning to 1 from 3 in 2 steps from 3 to 2 in 2 steps and from 3 to 3 in 2 steps right. So, let p 2 denote the let p 2 denote the matrix of 2 step transition probabilities just as p we are not writing p 1 it is understood p 1 we said this is p. So, this is the transition matrix of 1 step transition probabilities. Now, let p 2 denote the matrix of 2 step transition probabilities then we will just quickly notice that p 2 is actually p square. Because here you have written the components of this let me just write down say for example, if you want to write. So, p 2 this here will be the first one will be p 1 1 2 you want to look at right. And so that will be going from. So, p 1 2 p 2 1 plus p 1 3 first for it will be 1 1 1 1 then it will be 1 2 p 2 1 plus p 1 3 p 3 1. So, this will be your first element right these are the 3 parts which I had drawn in the last lecture. So, then you see this is multiplying p 1 1 p 1 2 p 1 3 the first row of p and with p 1 1 p 2 1 p 3 1 yes. So, if you multiply the first row of p with the first column of p you get the entry p 1 1 2 right. And now you can also verify this for example, this is the first row p 1 1 p 1 2 p 1 3 you are multiplying with the second column p 1 2 p 2 2 p 3 2 right. So, to get the entry 1 2 in 1 2 in that means here if you want to get the entry 1 2 then you are multiplying the first row of p with the second column of p. And therefore, you can just verify yourself quickly that p 2 the second step transition matrix is nothing but the product of p and p. So, that makes life very easy and we will show that this is valid for all values of for higher powers of p also. That means if you want to look at yeah. So, let me just show you systematically that we would be really on the path of getting a very interesting and very useful result. Because to be able to compute these transition probabilities any step transition probabilities by raising the power of p is a very convenient way of getting the transition probability right. So, you are want to know that what is the probability that you will be in 10 steps you will be from i to j starting in state i will be in state j then the 10th the i jth entry of p 10 will give you the probability and so on. So, we will see that yeah. So, basically what we have done we have shown is that your two step transition probabilities can be expressed in terms of one step transition probabilities right. Because we just multiplying the one step transition matrix with itself and we are getting computing the two step transition probabilities right. Now, same way we can do it for any power and here again I just want to spend time it may look like a little repetition but it does not matter because you must get the ideas very clear. So, for example, now yeah so the whole target is now to compute p i j k that means k step transition probabilities we want to compute and so here again I will just start from x 3 equal to 1 and x naught equal to 1. So, suppose now the three step three step you want to transition from 1 to 1. So, this again I can break up like this x now starting with 1 you are in two steps in 1 state 1 and then you will be going again transition into 1 right. So, this is your two step transition probabilities and then this is your one step. So, again I can write down this and now these three paths are mutually exclusive and exhaustive right. There is no because you can go to x 2 you can be at after two steps after two transitions you can be in state 1, 2 or 3 right starting from x naught equal to 1 and then you have to transition finally, to state 1. So, therefore, these are the three paths right and here again we are using the Markov property because this one is this then this part of the path will be independent of the probability for this because here you have x 2 equal to 2 and x 3 equal to 1. So, the only the current state is needed to compute the transition probability and this is not dependent on where you were earlier either at x 1 or at x naught that is not important. So, therefore, we will write this and again using the stationarity I will simply be able to write this also again as a one step transition probability of going from 3 to 1 right. So, therefore, you see that these three probabilities can be written as p 1, 1, 2 yes in two steps you are going from 1 to 1 and then again in one step you are going from 1 to 1. So, this is p 1, 1, 2 and p 1, 1 right. Similarly, two step transition from 1 to 2, 1 to 2 p 1 and then 2 to 1 p 2, 1 plus p 1, 3, 2 and p 3, 1 right this is. So, and then I can write down x 3 equal to 2 when x naught is 1 and then of course, probability x 3 equal to 3 x naught equal to 1 right this will also be equal to. So, p 1, 1 p 1, 3 plus p 1, 2 p 2, 3 plus p 1, 3 p 3, 3. So, this is the first row of and you can see that since this is your element of p square. So, therefore, if you write this these are the entries of your p square and then you are again multiplying by p 1, 1, p 2, 1, p 3, 1. So, that means essentially yeah. So, we saw that p 3 that means if you want the three step transition probabilities this will be given by p 2 p square into p. So, again a third power of p and so in general we will should be we can now say that if you want n step transition probabilities then this will simply be raising the matrix p to power n that means multiplying p n times and the entries in p n will give us all the n step all the n step transition probabilities that we need right. Now, there is another method through walks on the on the transition graph, but you will soon realize that it is not a very efficient method. It is fine to see what is happening when your graph is small in a number of states are also not too many. So, for example, when you want to look at this probability of two step transition probability from going from 1 to 1 right. So, which would mean that you traverse this loop once and then again one more time. So, two times you traverse this loop and you have this then you can compute the probability which would mean that this is the probability of traversing the loop once is p 1 1. So, when you do it twice it will become. So, the probability would be so probability of this would be p 1 1 square fine. Then if you want to for example, compute the three step transition probability of going from 1 to 1 then let us see what are the. So, we will try to see all possible paths on this transition graph while of going from 1 to 1 right. So, of course, I have not written the detail for example, x naught equal to 1, x 1 equal to 1, x 2 equal to 1 and x 3 equal to 1. I have just you know made the notation simpler. So, here this tells you the you know one path which is you are traversing this loop four times right I mean 1 to 1. So, three times 1 to 1 then 2 and then then 3, three times because three step transition probability. So, this path you will traverse three times that means the loop you will traverse three times. Then the possibility is that you go from you traverse the loop once then you go to 2 and then from 2 you go to come back to 1 right. So, you traverse this loop twice then you go to 2 and then 2 to 1. So, this is one path and therefore, here again you can write down the probabilities as p 1 1 square into p 1 2 plus p 2 1 right. So, once you write down so the eight possible paths you can see that right three step transition and you have in this three possible states. So, therefore, eight possible paths and similarly like 1 to 2. So, you go from 1 to 2 then you traverse this loop 2 to 2 then 2 to 1. So, this will be p 1 2 into p 2 2 then p 2 1 right. So, this way I can write down all the probabilities and then compute the. So, therefore, they are all distinct paths and you can compute the probability of each path and as I told you each leg of the path is independent. So, we are multiplying the corresponding probabilities of traversing each leg of the path and so you can write down. But you see that this will become really cumbersome the moment your number of states become you know if you had 4 to 5 states even easily any Markov process that you consider may have 7 to 8 or 10 states. Then you see the possible number of the number of paths will really go up right exponentially and then you may it is since it is very likely that you will miss out on some of the paths because you have to enumerate all possible paths and here in this case only you know for x 3 equal to the 3 step transition probabilities for 1 you going from 1 to 1 you had to enumerate 8 paths. Now, if it becomes x 4 it will be 16 paths. So, the number just will blow up and so therefore, this is for large n and for large number of states that means if you want 10 step transition probabilities and your number of states is also 10 then it just not possible for you to enumerate all possible paths and then compute the probability. But for small cases and in fact to see actually what is happening this is a good way. So, I thought I will just talk to you about it and when you are working out small problems you can actually see this, but otherwise this is really a very efficient way of computing the higher order transition probabilities. Now, this is fine. So, therefore, we have now a method of computing any step transition probabilities provided we are given the one step transition probabilities. But again there is some more information that we need and that is see the value of x naught is not known with certainty. We do not know in which state the system was initially for where it started. So, this will be normally given by a probability distribution that means you will be given these values. So, p i naught is the probability that x naught is in state i. So, thus at time 0 that means at initial time the employee is in state i. So, this is there is a probability attached to it. Now, if you want to compute the probability that the employee is in HR in human resource section division at time period 5. So, you know at means you want to compute yeah. So, let us see you want to compute the probability that x 5 is equal to 2. So, the particular employee at time period 5 is in HR. Now, let us see the column of p 5 remember the 5 step transition probabilities are given by p 5. So, the column would be p 1 to 5 p 2 to 5 and p 3 to 5. So, actually I should have written this let me write this instead of immediately jumping to this. So, we want to say that probability x 5 is equal to 2 given that x naught is 1. Then you will multiply this by probability x naught equal to 1. So, I am writing this as a conditional probability then into the. So, then I will have to do it for all possible values of x naught. So, to get the probability that x 5 is equal to 2. So, this will be probability x 5 is equal to 2 given that x naught is 2. So, probability x naught is equal to 2 plus probability x 5 is equal to 2 given x naught is equal to 3 into probability x 3 equal to sorry x naught equal to 3. So, I break up this. So, again you see mostly what you have seen that you know the basic probability theory that we need for analyzing these two pastic processes. So, this is writing probability as a you know breaking it up into conditional events and then writing them down as the sum of these conditional probabilities. And so this one gives you this is your 5 step transition probability from 1 to 2. So, 1 to 2 5 and then into probability x naught is equal to 1 which is given to you from here. So, p 1 naught. So, this will be p 1 naught. Similarly, this will be 5 step transition probability of going from 2 to 2. So, this is p 2 2 5 into probability of being in state 2 at time 0. So, which is p 2 0 p 2 0 and this is the third one. So, this is how you can now write down the probability if you want to know that in time period 5 the employee will be in h r. And so you can now compute in general you can say that this is now p i n will be the probability that the system is in state i at time n. And see here just to make the presentation simple I am just taking all the time referring to our job assignment example, but you know that this can be made to whatever the number of states general you can have a symbol k here. So, k states and everything can be argued with respect to general number of states, but here I am doing it for this particular example just so that you can fix your ideas better. So, then p i n is probably x n equal to i that means the system is in state i at time period n. So, these are the probabilities and I have shown you how we will compute them. So, you can write them down right here. So, this will be your I am writing it as small p naught into p n. So, when you want the probabilities of the system being in a particular state at time period i n then this is the formula and these are called the state probabilities at time n. So, this is the important. So, now we have we do also have the transition probabilities for any time period and we also have the state probabilities at for any time period n right and yes. So, I have written it here that means I am saying that if p n is your row vector of probabilities. So, this was the i th component. So, then p n is yeah I should have written here this is the i th component. So, in general if this is a row vector is p 1 n p 2 n p 3 n. So, that means state probability of the system being in n being in 1 at time period n this is probability of the system being in state 2 at time n this is the probability of the system being in state 3 at time n. So, I denote this by a row vector p n and then this can be written as p naught into p n. So, this gives the state probabilities at time n right. Now, the Markov change is completely specified when you are given the first step transition matrix p and the initial probability initial probabilities of the system being in particular state. So, p naught is the vector which gives you p naught 1, p naught 2 and p naught 3. So, this will be the probabilities when this initially when the system is in state 1, 2 or 3 then you can compute these and of course, you can compute the transition probability n step transition probability also. So, let us consider an example the same example now with numbers and. So, we will just go through all the concept that we have talked about the transition probabilities and the state probabilities I mean higher order transition probabilities and the state probabilities suppose the transition matrix is given as this. So, then the corresponding diagrams you see for example, there is no arc from 2 to 1. So, therefore, this is missing here and similarly, you do not have a loop from 3 to 3. So, it is missing the probability is 0 right otherwise and you see that they add up to 1 all these probabilities the rows must add up to 1. So, this is a valid transition matrix entries are either positive or 0 and the row entries all add up to 1. So, this is a valid and the accompanying diagram is this the transition diagram is given by this right. Now, let us compute second order transition probabilities. So, I multiply p with p and I get these numbers. So, for example, here you can transition from 1 to 1 in 2 steps this will be 1 to 1. So, 2 steps therefore, again the same thing as I showed you that if you want to see it on the path it will be 1 by 4 this plus then you can go to you cannot go to 2 in 1 2 step transition because then you cannot come back to 1 right. So, then it will be you can go to 3 and then 3 to 1. So, this will be 3 by 4 into 1 by 4 right. See you can either stay with 1 to 1 and then or you can go from 1 to 3 and then 3 to 1 this is what you can do in 2 steps if you want to transition from 1 to 1 to 2 possible paths. And so, therefore, this is 3 by 16 and this is 4 by 16. So, 7 by 16 right similarly you can maybe look at this one here 5 by 16. So, this is from 1 to 2 1 to 2 you want to transition in 2 steps. So, then the possible path is 1 to 2 then 2 to 1 right. So, that will be what 1 to 2 is 1 by 4 into you transition from 2 to 2 will be 1 by 2 plus or you can stay from 1 to 1 and then go from 1 to 2. So, that will be 1 by 2 into 1 by 4 right. So, what how much will this be 1 by 8 plus or what I missed something out. So, this is you are going from 1 to 2 1 to 2 you are going in 2 steps. So, 1 by 2 this is 5 by 16 and you are computing p 1 2 right. So, you can go from p 1 1 into p 1 1. So, that was 1 by 4 and why did I multiply it by I want to go from 1 to 1 sorry this is not correct why should I say it 1 to this is simply 1 to 1. So, this into p 1 2. So, therefore, this is half that is I wrote this plus 1 by 2 into 1 by 4 or I can go from 1 to 2 which is 1 by 4 and then transition 2 to 2. So, that comes out to be 2 by 8 and I am getting it as 5 by 16. So, let us multiply this that means you want 1 and 2. So, that will be I am missing out on a path 1 by 8 plus 1 by 8 plus 1 by 16. So, 1 by if you want to go from 1 to 2. So, right I missed out on the path. So, this is 1 to 1 and then 1 to 2 then 1 to 2 and 2 to 2 and then there is another path 1 to 3 and 3 to 2. So, plus 1 to 3 is 1 by 4 and then from 1 into 3 to 2 is 1 by 4. See that is what I was trying to say that you know even in such a small diagram I was missing out on a path. So, just imagine if you had 5 or 6 states and then you had you know that many nodes. So, and then you had these so many arcs you certainly miss out it will be very difficult it will be very time consuming to enumerate all possible path. So, as it is you know such a small example I could miss out on the path 1 to 3 and 3 to 2. So, that will give you now 1 by 8 plus 1 by 8 plus 1 by 16. So, that will be 5 by 16 right 1 by 8 plus 1 by 8 which will be 1 by 4. So, that is 4 by 16 plus 1 by 16 5 by 16. So, anyway you can now interpret all these probabilities you know by looking at the path or by fine. Now, I made further computations took powers of p. So, this is p 3 comes out to be this and if you make compute the fourth power then these are the numbers. And another aspect that I want to point out here is that for example, in this particular case you are able to transition from any state to any state even though some of the arcs are missing, but even after that you can from go from 1 to 2, 2 to 3, 3 to 1 again and so on. So, you can transition from any state to any state may be not in one step always, but in fact here it is happening that you are able to go. For example, from 2 to 1 you cannot go in one step, but you will be able to go from 2 to 3 and then 3 to 1. So, in 2 steps you will be able to transition. So, here in fact the moment all your entries see at p 2. So, at p 2 all your entries are positive which shows see therefore that means your p i j 2 is positive for all i j. So, this immediately makes you conclude that you have a 2 step path from each state to from every state to every other state. So, that means the so the word for this is communicate that means all states communicate with each other and may be not in one step, but at least in 2. So, if all entries of p i j 2 are positive then that means all states communicate with each other. I will formally define that also, but essentially what I want to say here is that you can go from any state to another state in 2 steps in this particular example. And in some other example if it is you know there is some k for which this is positive then that means again there is a k step path from every state to another state. And yeah, so now when you look at the fourth power of p these are the numbers and you can now if you carefully look at the column numbers this is 1 0 3 upon 256, 1 0 2 upon 256 and 1 0 2 upon 256. So, the numbers are getting closer and you can say that almost converging to 1 0 2 upon 256. In fact, you might say that why not 1 0 3 or 1 0 2, but any one of these numbers. So, in other words that you know if you want to interpret these numbers that means now the probability say for example what is this number. So, 1 0 3 upon 256 is the probability of 1 1 4 that means 4 steps you are from 1 to 1 and then 1 0 2 upon 256 will is the probability 2 1 4 that means if you. So, here it says that if you are initially in state 1 then in 4 steps you will be back in state 1. So, this is the probability now this says that if you are in state 2 in the beginning and you transition to 1 in 4 steps then this is the probability and the third one is 1 0 2 upon 256. So, this is p 3 1 4. So, that means you are in state 3 and you are transitioning to 1 in 4 steps. So, you see that it is becoming almost immaterial to know from where you started because these probabilities are getting closer and we will show you then later on we will also formalize all this discussion. So, essentially the state in which you were at the initial time is becoming unimportant and similarly the same thing you can interpret for the second column because it is 85 by 256, 86 by 256 and 85. So, these numbers are also getting closer to 85 and here I have written because here it is 68 by 256, 68 by 256 and 69 by 256. So, you might say that why not 68, but then see remember the probabilities which when you finally say that all these are converging to they must also add up to 1 because the system has to be in one of the states. So, the same argument we will continue using and so here if I say that this converges to 1 0 2 this 2 upon 256 this is 85 by 256 then this should be 69 by 256. Of course, it is possible that they. So, essentially right now the probability of being in state 1 after 4 transitions is hovering between 1 0 2, 2 56. So, which is a more exact statement. This is a more exact statement. Similarly, here also I can say the probability of being in state 2 after 4 transitions is between is in the interval 85 upon 256 and 86 upon 256 which is a very small interval. So, difference of 1 upon 256 and similarly here it will be 68 upon 256 and 69 upon 256. So, this probability and if you take the fifth power then certainly you will see that the numbers are converging and we will talk about this formally in a way. And also while we were talking of this see I wanted to point out that here again these probability because it is now a row vector. This is a row vector remember 1 by 3 we are writing. So, this is 1 by 3 and this is 3 by 3. So, then again this is 1 by 3. So, 3 entries must again add up to 1 because at you know n after n transitions your system will be either in 1, 2 or 3. So, therefore, these probabilities also must add up to 1 right which you can see here also that. So, anyway these also add up to 1 and then now suppose for this job assignment problem if your initial probabilities are given by 1 by 4, 1 by 4, 1 by 2. That means probability of being in production is 1 by 4, probability of being in H R is 1 by 4 and probability of being in sales is half. Then you want to ask the question that what are the probabilities of being in state in 1, 2 or 3 after 2 transitions. So, that will be given by this. So, 1 by 4, 1 by 4, 1 by 2 multiplied by p square and so these are the probabilities of being in no. So, this gives you the probability of being in production after 2 transitions right and this is the probability of being. So, these are the state probabilities after 2 steps after 2 time period. So, this is 21 by 64 and 18 by 64 and these probabilities also must add up to 1 right. So, now we are trying to give you some more characteristics of the Markov process and we will do lot of analysis in terms of these transition probabilities and the state probabilities. So, let us just want to show you the limiting behavior of these steady state probabilities. So, let us just graph the values of p 1, 3, p 2, 3 and p 3, 3 for different values of n. So, you see the starting vector is 1 by 4, 1 by 2 and 0. So, p 1, 3 for example, 1 by 4 is 0.25. So, I am just starting. So, these are the time periods and these are your probabilities the numbers from 0.1 to 0.5. So, 0.25 see at time 1 it is in period 1 it is 0.25 then here also 4 by 16 is 0.25. So, in period 2 also this is the same then it very slightly goes up in period 3 to 17 by 64. So, I am just showing it like this and then it comes down to 68.256. Now, 68.256 is 0.26. So, therefore, it comes down to this is my value for 0.26. So, this is the graph for p 1, 3 as it goes to different periods. So, the values transition probabilities then if you look at p 2, 3 now p 2, 3 starts from half then it immediately comes down to 0.25 because this is 1 by 4. So, 0.25. So, this is where it is and then in the next one it goes to 18 by 64. So, actually my graph for p 1, 3 is this one because very slightly it goes up and then again it settles to here. This is the graph for p 2, 3. So, p 2, 3 the p 2, 3 graph is this. So, this is from 0.5 to 0.25 then it goes up to 18 by 64. So, this is above more than 17 by 64 this and then it again comes to 68 by 256. So, the same value as for p 1, 3 p 1, 3, 4 and p 2, 3, 4 are the same. So, this is how it is and for the for p 3, 3 see it is 0 in the beginning and in stage time period 1 then it jumps to 5 by 16 which is a little more than 1 by 4. So, little more than 0.25 and then it comes to 16 by 64 which is exactly 0.25. So, I have tried to just parallel it with this one here and then it will be 69.256. So, the line is slightly above this. So, therefore you can see that and then as you take higher powers all three will settle down to this number which we have to compute and we will do it when we find out the steady state probabilities. So, you see this is the limiting behavior and so it does not matter even though the three had different very different starts all the three, but finally, they match to the same. So, therefore the relevance of the starting state of the process is not at all relevant here. This is what you want to show and of course this will not always be true and we will now then find out during the course of next few lectures as to when this is valid or when this kind of limiting behavior is valid. So, what we are going to say is that rows of P n become identical to pi 1 pi 2 pi 3 all the rows because it does not matter the starting state of the system and so all the rows will be pi 1 pi 2 pi 3, but it is not necessary that the three probabilities are the same. So, that means it is being so essentially what we are saying this is now a long term behavior we are saying that the system will be in state 1. So, that is the probability then this is the probability of the system being in state 2 and this is the probability of the system being in state 3. So, the starting probabilities are not relevant, but the values the long run values will not necessarily be the same and so the definition for the steady state probability is that pi j is the limit of P j n as n goes to infinity remember we define this as the multiply by P 0 and P n. So, this is limiting value of P j n n goes to infinity which is actually probability x n is in j in the long run your system is occupying state j as n goes to infinity and what we are saying now is that this actually is equal to this conditional probability, but this part is becoming irrelevant. So, the I see that the probability finally is going to pi j. So, the I part is irrelevant here this is what we want to show you and so we will in the next lecture discuss the under what condition of course that will come in the due course of time, but first of all we would want to know how we go about computing these steady state probabilities when we know that they exist. So, right now we will assume that they exist and then we will find out the method of computing them and then we will continue with the discussion as to under what conditions they always exist.