 We started talking about this communicating classes in the Markov chains and in the last class we said this. So, the relations that i communicates to j and i and j communicates with each other that relations is an equivalence relation and that partitions my state space into classes which are equivalence classes. So, then we started looking into some properties of this communicating classes. What are the properties we studied about this communicating classes? We stated one class property theorem which says that if you have states in a particular class they must be all or of the same type either transient, positive recurrent or null recurrent. We looked into its proof today, building on that we will see some more properties of this class properties, some more properties related to this classes and then we will move on to something called invariant distribution. So, these are some more properties suppose let us say I have. So, what we are saying is suppose you know that j is a state which is recurrent and if you say that j communicates with i then is it obvious that j i from state i I have to come back to state j then i communicates with j. So, we are so they are saying j is recurrent by our notion of recurrence that means I should be hitting this state j again and again I should be coming back to this state with probability 1. But suppose I have from the state j I have reached some other state j in some finite time then if I have to again come back to that state then it must be a case from that state j I must return to state j. This also implies that f i j equals to 1. So, how we show this? So, since j goes to i I know that in some finite time there exists some n such that p i j of n is going to be greater than 0 since j plus that there exists n positive such that v of n is negative. So, the same condition also implies that I should be reaching. So, this is telling that at some point n I am going to reach state j with positive right. I mean I can also translate to the first time visit to state. So, this is j i first time visit to state i starting from j then that should be also happening to some positive probability. So, I can also take that to be maybe this need not be the same as this n it could be some different n when it is going to hit that particular state i for the first time. Now what I have? So, now let us try to prove this. Suppose let us say I am taking this to be suppose let us say f i j less than let us start with this negation of the statement given to me. Now let us look at this quantity f j j and this is not returning to state j right I am looking at 1 minus less than this. So, now it may happen that. So, I am asking about not returning to state j starting from j right. So, I am now can look at I hit some special state in between and from that I do not come back to j again and which state I hit in between could be many possible ways right. Now, I am going to take one particular possibility. So, that is why I am going to now look at the lower bound. So, let us say first I am going to hit this particular state i and after that I do not come back I from there I do not return to state j. So, this is 1 minus f j j is not returning to state j. So, what I am asking now looking at one particular possibility of that where first I go to state i and from there I again this is 1 minus f i j tells that if I start from i I never go back to j again right. So, this gives me one possibility of not going back to state j again. So, that is why this is going to be lower bound. Now, we know that this guy f j j of n is strictly positive and if this f i j is strictly less than 1 this guy is also positive. So, this product is positive and then is what we are saying is in this case if this is if we start with this assumption we are just proving that this f j j is less than 1. If f j j is less than 1 what does that mean it transient but we have initially assumed that this j is recurrent right. So, it contradicts. So, if j is recurrent this cannot happen. So, hence it must be the case that j is recurrent and if f i j equals to 1 it must be the case that starting from i I should be communicating to j at some point right. So, i is going to communicate with j. So, what we are saying is if my state j is recurrent if I leave state j and go to some other state at some point then it must be the case that from that state I should come back to state j again then only I am going to visit my state j again and again otherwise I am not going to visit state j again. Yes this is the whole contradiction right if I am going to take this f i j suppose let us say f i j is not equals to 1 but it is f i j is less than 1. Now what we are getting by using this assumption we are getting that f j j is strictly less than 1 that means j is transient but our hypothesis is what j is recurrent right. So, that is what is getting violated here this one. So, what is this? This is the probability of this you never coming back to state j in this can happen in many ways right. So, I am looking at a particular possibility where from j to you go to state I and from there you do not come back and this is just one possibility right that is why this probability is a lower bound on this event you are not coming back and may happen many ways. So, that is why this lower bound and I get greater than 0 here fine. Now we have already said that all my states get clustered into different communicating classes and now if I am going to take and these communicating classes can be either open or closed ok. Now suppose if I know this communicating class is open or this communicating class is closed can I further say that what kind of state it has. So, let us say I have a communicating class which is open then what kind of states possibly that communicating class will have right if that if my state j is recurrent I have to be hitting again and again. So, there should be a reverse path also, but in the open case that may not be the case right. So, that is why if I have a. So, let me also state it as a result. So, this is intuitively clear to us now how to show this. Suppose let us say I have a class here which has one particular state let us say I here and because this is an open communicating class I should have a state j outside this class. So, that the probability of reaching from this state to this should be positive that is the definition of our open communicating class right. So, if that happens how can I show that this state i is transient f i i equals to ok let us write what is f i f i is the probability of ever hit again hitting state i again starting from i right. So, I am going to now look at one step thing like ok I leave from that state i and then again I come back to that. So, I am now looking it in one step whereas, when I did this I looked at going to something in n step and then coming back to that state. So, what is the possibility in the first step I could go from the probability p i j I can go from state i to j and then I can look at f j from there I can look at coming back to i right. So, this is one possibility, but other possibilities could be instead of i I could be going to some other states and then if it could be f k right. So, there are many possibilities like maybe all this k not equals to j could be maybe contained within the same class what I am asking is in one step from the state a you go to some other state and from there you come back to state i. So, this is like this particular state which I know which is outside my class. So, I first go there and from there I will come back to state i again at some point and this is not just i yeah not just j I will go back to some other state k and from there I will come back at some point. Now, what I know? So, I know that p j i is what there exists a j outside if this is my my class is open communicating class right there exists a j outside my class. So, my there exists j which is outside my class such that p i j is what greater than 0 and then in this case this j is such that what should be f j i, why is that? So, in our comeback right if you are if f j i is not 0 then there is a positive probability that you will come back in that case means that actually i and j communicate they should be in the would have been in the same class not outside. So, what is this term is going to be with this term and now what I will do is I am going to I know that this f k i's are all upper bounded by greater than or equals to 1. So, I use this bound and get a bound like this. So, why this is an upper bound because f k i's I have put one for them because I know they have probability terms and they are up to at most 1. Now, look at this. So, now I am adding p i k's what is p i k probability of going from state i could i to k and now I am looking at all possibilities of states from where I can jump from state i except for j. So, this is what 1 minus now I have said that this p i j is what strictly positive. So, what this should be? So, this is what I wanted to show that state i that is inside this class is transient that is f i i is going to be strictly less than 1 fine. So, we already know that if I have a communicating class which is open that means every state in that should be transient because if I have one state which is transient in this communicating class. So, it must be the case that all the states in this communicating class must be transient. So, this is going to some other. So, but that is not like say the whole point is if at all you are going to from j you are going to come back i through some going to other class and other class that means with some positive probability in some finite n you are going to come back to this state. That means very i j is communicating with i. So, just what is the our definition of f i i sorry f i j f j i in this case this is the sum of f j i superscript n. So, if any term there is positive this term would have been positive, but then if this is 0 it must be the case that all the for all the term should be 0 in that case right. So, now what about close communicating class? So, fine open communication I just right away said that all the states there are transient what about close communication class. If I am going to say this is going to be a close communicating class everybody should be communicating with each other that is fine with within some finite time. And because of that is it required that they have to be either only positive recurrent or null recurrent anyway. So, what we have is it is not necessary that a close communication class if I look into generality like the arbitrary close communication class which could potentially have infinite many states in that then that could be any of the things transient positive recurrent or null recurrent I cannot in general. Where we said that S n equals to k plus summation x i i equals to 1 to n. And what was this i's i is such that it could take 1 and minus 1. So, what would we say about this communicating so this Markov chain. So, how many states it had? Accountably infinite and we said that is an irreducible. So, irreducible means we already kind of said that this is a communicating class one communicating class. So, one big communicating class right. So, if it is one big communicating class and is that going to be closed or open close right like just everybody is there in the same class. Now, what would we conclude each state there as what would we conclude? We said each state there is transient right. So, even though we could have a communicating class which is irreducible it could potentially have it could still be transient that we had an example. But it so happens that if we have a communicating class a closed communicating class which has only finitely many states in it then it is going to be going to be positive recurrent. So, finite see what we are saying is a set of states based on our communicating relation can be partitioned into many classes. One class may have finite number of states in it and it is a closed communicating class. That finite closed communicating class is such that each state in that class is going to be positive recurrent sorry positive yeah positive recurrent okay. How the proof of this? Okay, let us see we can decipher this proof. So, is this clear to you I am just saying that take any state in my communicating class and look at any step n. If you start from i the probability that you in the nth state remain in class C itself is going to be probability 1 that is that should happen right because this is a closed communicating class you are not going to escape from this. In any round it must be the class whatever you are going to start in that every n step you have to remain in the same C. So, this is same as saying that right you going from state i to j. So, I am looking at nth step this sum of this probability should be such that it should be equals to 1 it is just like rewriting this probability in this fashion. So, now look into this case now I am going to do is okay I started with some state i now what I am doing is now I am looking at another state in j only this is another state now I am going to look at this quantity here okay. So, what is this quantity it is just like I am looking at transition from i to j in different rounds and I am just taking average of this. So, I know that what I know about its limit as n goes to infinity if my state. So, if my state j is recurrent what I know about this where this this limit go. So, we had said right if it is I have said suppose my state j is recurrent we had a criteria to further distinguish whether it is going to be a positive recurrent and null recurrent. So, what is this limit is going to when it is a null recurrent it goes to 0. So, and what happens to this sum as n goes to 0 when my state j is transient. So, we said that that summation p if my state j is transient and if I start from initial state i let us say we had said that summation p i j script n what is that is going to be we say that that is finite right as n goes to infinity. So, if that is finite what happens if I take 1 by n if I divide it by n what happens to this this is going to 0. This is equals to 0 if c were transient or null recurrent right. So, if c is my communicating class is such that if the elements in that are either transient or null recurrent it must be the case that this limit happens right that is the property we have already shown. Now, this is true for any state in j right. So, if you state j you take any j if it if it is a transient or null recurrent this should happen. So, now what I will do is I will just add this over c this limit I am just adding both left hand limit both over j now and this is still going to remains 0 right the right hand also I am adding over all summation, but the summation contains only finitely many elements right. So, this is basically and this is going to be still 0 because this is a finite summation here that is just going to be 0. Now, what we are going to show that we are going to actually show by contradiction here if my elements the states in c are transient or null this has this quantity is supposed to be 0, but it is not actually the case the right hand side has turned out to be 0, but let us look into the quantity what this quantity is more about. So, let us look into this quantity at this point for each j I have a limit here can I interchange this limit and this finite summation here since this is a finite summation I am I can interchange right had it been c had been an infinite many state in this I may not so easily you have been able to interchange this, but now that I know that this c is finite I can blindly interchange this limit ok. Now, further these are two summations over a finite number of elements right this is c is finite and this is going from k to n I could easily again interchange this limits. Now, what is this quantity now? So, what is this quantity this is going from state i to state j in k steps and now I am summing it over all possible states. So, this is basically jumping from state i to another state within j in the kth step this is going to be 1 right because this is a communicating class close communicating class. So, this quantity is going to be 1 which we have already shown here. So, and this is 1 and this is what is this summation further is going to do this is going to be n and this is going to divide by n what is this quantity is going to be 1 and as limit this is going to be constant in this case right and what is this going to lead to you is. So, had it been any states here or either transient and null recurrent what we are ending up with is a contradiction here because 1 cannot be equal to 0 ok. See this is what like you have to be careful in interchanging the limit and the summation here. So, in this case we could interchange the limit because c is a finite, but had we blindly ignored it and even when c was infinitely many terms in this we have blindly ignored this that would have given us a result which was not at all consistent right ok. So, what we are saying is as if you just tell me my communicating class is closed I can tell you that it is going to be positive recurrent as long as this is finite if you tell me that I do not know it is finite it could be countably infinite my communicating class is countably infinite then any of the cases could be possible you need to prove me in that case what is that possibility. For example, we have said in this case we had a class where we had a communicating class with countably infinite things, but the states were all transient there.